Morning,
Apologies for the outage yesterday, there was an IPv6 multicast storm originating from the SA Digital Villages network yesterday morning that brought our core network down at 07H45. Storm control on the Cisco switch interfaces facing SADV did no detect this which it should have done, this has been escalated to Cisco. We immediately dispatched technicians to the data centre and had removed the cross-connects to SADV by 08H33, which then allowed the network to stabilise and start allowing clients to authenticate again.
SADV continued to work on the issue on their side and we reconnected the SADV network directly to a router to isolate the SADV network from our core network switches. SADV clients were back in service later in the morning.
With the change of the SADV network to Vumatel the SADV direct connections are due to be moved to the existing Vumatel network interfaces at the data centre, SADV is now prioritising this migration.
We have set up a new telephone system in the Amazon AWS cloud, we are busy porting our numbers away from our network to completely separate the telephone system so that it will be available to you regardless of our network, as there will be no dependency on our network. We will prioritise the completion of this.
Once again apologies for the outage.
Kind regards,
Mind the Speed - Management