Heisenbug at network level
Incident Report for Makersphere
This incident has been resolved.
Posted Nov 23, 2017 - 04:41 CET
Good news! Our ISP has finally configured our CPE correctly. πŸ‘
Most backend services will be online again in just a few hours.
We still have to wait for some DNS & PTR records to propagate, this can take up to 48h. πŸ‘¨πŸ»β€πŸ’»
Posted Aug 16, 2017 - 21:29 CEST
Our provider informed us that their internal ticket/management system has scheduled the deployment of RIPv2 for our account at sometime today, but no specific time was given. πŸ˜’
We should finally receive our /29 subnet tomorrow, if everything works out well. πŸ‘€
Posted Aug 15, 2017 - 19:01 CEST
After multiple calls with our ISP, we're still waiting for them to enable the new CPE with RIPv2 routing.
Apparently they are experiencing some issues with their internal ticket & provisioning systems.
But those should be fixed on Thursday and then, we finally can finish the setup. 😬
Posted Aug 15, 2017 - 00:42 CEST
Unfortunately we're not able to finish the setup because our ISP needs to enable RIPv2 mode on the new CPE.
This will probably happen on Monday. So no internet for us over the weekend. 😳
Posted Aug 13, 2017 - 03:30 CEST
We finally received the new CPE from our ISP. πŸ™
It will take us some time to get everything connected & configured but we'll keep you posted. πŸ‘©πŸ»β€πŸ”§
Posted Aug 12, 2017 - 14:45 CEST
We initiated the transition to a static IPv4 configuration.
In the coming days we will receive new CPE that supports the required protocols/speeds.
During this transition to our own IPv4 subnet, degraded performance is expected. πŸ‘¨πŸ»β€πŸ”§
Posted Aug 07, 2017 - 17:57 CEST
We're currently experiencing immensely degraded performance & reliability at our DE-RFN node.
After several hours of investigation, we have concluded that wrong routing policies on our side are part of the issue.
Based on our current understanding of the problem at hand, we believe that the ISP's DHCP setup (in combination with our routing policies) to be the root cause of this Heisenbug.
We work with our provider to replace the current (DHCP-based setup) with a "real" static configuration.
This transition (starting today) will take about a three to four day and also includes the assignment of a new IPv4 subnet. πŸ‘©πŸΌβ€πŸ’»
Posted Aug 07, 2017 - 02:50 CEST