Web sQuad ISP - Feedback Thread #2

Looney said:
Sure I get that, I haven’t logged with support yet, it’s just something I’ve noticed the last few days. I’ll continue monitoring and if it gets worse I’ll log a ticket.
Click to expand...
Thanks, happy to assist - let us know if you pick up something
 
@websquadza hi, I've been struggling with this ticket for a while (513980). I understand that it's limited to a particular route, and that is looks like multiple ISPs involved. but any more we can do?
 
websquadza said:
Evening all, apologies for the late reply. We had a team at Teraco until late this evening, and a configuration change led to a short outage just after 18:05pm, services were fully restored by 18:11. Apologies for the inconvenience caused.

We're still completing the post mortem on this morning's outage. Thank you for your patience and continued support.
Click to expand...
Thank you for the feedback (:
 
chrisGolf said:
@websquadza hi, I've been struggling with this ticket for a while (513980). I understand that it's limited to a particular route, and that is looks like multiple ISPs involved. but any more we can do?
Click to expand...
Morning, apologies missed this one. Yes we are aware of this one. It's been a bit hit or miss with NTT (Vodafone's preferred upstream). Let me sit with the team and review the trail, including with the B side ISP and see what can be done here. Thanks for your patience on this one.
 
Thank you for your patience on regarding Thursday's outage, we were awaiting feedback from Teraco on the cause of Thursday's outage - see below our Post Mortem:

On Thursday 4 December 2025, at around 11:00 we experienced an outage impacting our Teraco Isando core. Our investigation showed that a B-feed power circuit in our rack had tripped, causing two critical switches (Spine-01 and Leaf-02) to lose power. Both devices had unflagged A-feed vulnerabilities—Spine-01 had a failed PSU, and Leaf-02 had a loose A-feed power cable—so when the B-feed dropped, both switches went completely offline, isolating the remaining leaves and taking the fabric down.

Teraco has since completed their own investigation and confirmed that a fault on a PSU on a dual-corded device caused a circuit breaker trip on one of the PDUs, resulting in the feed loss and multiple devices powering down. Their technicians restored the breaker and verified the affected PDU’s stability once the device was reconnected.

Once services were restored at 11:51, our team remained onsite for the rest of the day to fully rewire the rack with new A/B power cabling, reseat all connections, replace our management stack, add full console access, and introduce our second spine into production for more resilient HA (this had been planned). Teraco is also installing an additional independent A+B rail for us to further strengthen power resilience with A1/A2, B1/B2 rails.

A tough outage, but a valuable one—we’ve addressed every contributing factor and implemented structural improvements to prevent a repeat. Thanks to everyone for the patience and support while we worked through it.
 
websquadza said:
Thank you for your patience on regarding Thursday's outage, we were awaiting feedback from Teraco on the cause of Thursday's outage - see below our Post Mortem:

On Thursday 4 December 2025, at around 11:00 we experienced an outage impacting our Teraco Isando core. Our investigation showed that a B-feed power circuit in our rack had tripped, causing two critical switches (Spine-01 and Leaf-02) to lose power. Both devices had unflagged A-feed vulnerabilities—Spine-01 had a failed PSU, and Leaf-02 had a loose A-feed power cable—so when the B-feed dropped, both switches went completely offline, isolating the remaining leaves and taking the fabric down.

Teraco has since completed their own investigation and confirmed that a fault on a PSU on a dual-corded device caused a circuit breaker trip on one of the PDUs, resulting in the feed loss and multiple devices powering down. Their technicians restored the breaker and verified the affected PDU’s stability once the device was reconnected.

Once services were restored at 11:51, our team remained onsite for the rest of the day to fully rewire the rack with new A/B power cabling, reseat all connections, replace our management stack, add full console access, and introduce our second spine into production for more resilient HA (this had been planned). Teraco is also installing an additional independent A+B rail for us to further strengthen power resilience with A1/A2, B1/B2 rails.

A tough outage, but a valuable one—we’ve addressed every contributing factor and implemented structural improvements to prevent a repeat. Thanks to everyone for the patience and support while we worked through it.
Click to expand...
awesome feedback :) , so sum it up ,a sketchy power cable..
 
Vuma trenched offline East Rand

It supposed to terminate 29/12/25

Are you on holiday. Logged ticket #905869. No response from the WEB.

Edit sorted ants in the outer box in street. 27/12/25
 
Last edited:
HowTo said:
Vuma trenched offline East Rand

It supposed to terminate 29/12/25

Are you on holiday. Logged ticket #905869. No response from the WEB.
Click to expand...
Morning I see our team did reply on your ticket. Logged with Vuma, seems to be a line fault. Will ask them to check in again.
 
Happy New Year!

Wishing you, and yours, an amazing 2026! Thank you as always for your continued support. We look forward to continuing to build and improve as we strive for the best possible experience for you this coming year.

Looking back on 2025, we learnt some hard lessons. Our team grew, found new ways of doing things and doubled down on improving our customer experience throughout your lifecycle with Web sQuad.

We also made some significant strides on the network and backend systems. We started 2025 with a few ambitious projects, which meant a complete network overhaul- no small feat on a production network.

As a group, we completed our Arista based Spine-Leaf deployment in all regions- transitioning entirely to EVPN-VXLAN, we migrated core routing to Arista's resilient and bulletproof IP capabilities, adding Tbps of capability in each region and allowing us significant room to expand reliably and resiliently. We also used the opportunity to split our wholesale and eyeball (Web sQuad) IP networks out entirely, we added 2 new Datacentres (bringing our presence to 11 DCs), scaled up our Datacenter Interconnect capacity with Dark Fibre in select regions, and bolstered our Dedicated Internet capabilities. Finally we started our very own Metro network builds in JHB and CPT while expanding our retail and residential fibre presence. We added no less than 6 new FNOs to our network. Finally our team worked tirelessly on implementing tools, and sometimes developing internal tooling, for monitoring and visibility on every part of the network, tightening up network policy and ensuring sustainable practices as we continue to grow. It was a busy year.

Thank you to our dedicated team for taking the challenge by the horns and making the very best of it.

2026's to do list is equally long, if not longer- and we can't wait to get started.

We appreciate our community's constant feedback and do value your input- so as always please continue to let us know what we are doing wrong, what we are doing right and what you want from Web sQuad during the year ahead!
 
Last edited:
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X