Web Squad ISP

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
694
Update: We've turned down transit Cape Town, so International traffic will no longer get blackholed there without a route to JHB or Durban. JHB/KZN traffic should be back to 100% now.

PS. Cape Town, unfortunately the flapping is wreaking havoc with routing tables there. We are working to find a dampening solution there. Vendors are also working on resolving the flapping (this is the same switch that was adversely affected by a software update recently).

PPS. Our new Cape Town - JHB circuits are ordered and we have been eagerly awaiting their delivery - which will prevent issues like this in the future.
 

Meester

Active Member
Joined
Feb 28, 2007
Messages
80
Seems it's flapping for international still from CapeTown... my skype in and out to a friend also on Websquad in CapeTown.
 

Napalm

Expert Member
Joined
Jun 22, 2006
Messages
2,266
Yeah. Youtube, netflix and most things international not usable. Infinite loading screen :(

This site can’t be reached
www.youtube.com unexpectedly closed the connection.
 

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
694
Update: Our Vendor is still working on a resolution to this particular issue. We’re in constant contact with them awaiting resolution.
 

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
694
Update: Ok Cape Town, our JHB-CPT route is still extremely unstable. We've had to get creative re-routing traffic but we've come up with a solution which should work for now
 

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
694
First let me start off by apologising to you for last night's outage - we pride ourselves in having built a resilient, independent network, but a small part of it has let us down in a big way yesterday.

What happened?
Last night's (28 August 2019) issue was a result of our upstream's switches going into a constant flapping (service turning up and down) state between 20:30 and 00:45. The vendor is still investigating the cause.

Both our protected route between JHB and Cape Town as well as International Transit from this vendor are provided via a redundant link at our Cape Town node, from these affected switches. This meant that some transit traffic bound for our network was couldn't make it to where it needed to go, both in Cape Town and in some cases, nationally. In addition, services like AWS which we pick up in Cape Town primarily, were affected for a short while until we forced these to JHB.

What are we doing about it?
We are bringing up new National routes between Cape Town and Johannesburg that we will manage entirely. Web sQuad are one of the most diversely connected ISPs locally, with multiple transit providers, and we are restructuring this design so that a single provider can't adversely affect a users' experience to the extent it did yesterday. We have also terminated all services with the affected vendor.

What's the way forward?
Our upgrade path to ensure you're not affected again is as follows:
  • We are establishing a new, fully protected JHB-CPT circuit with immediate effect. We are awaiting the installation of a few nuts and bolts today and tomorrow and we will immediately migrate the L2 service away from our current vendor - this will completely eliminate any risk associated with the switches in question. We are working to have this process completed by early next week
  • Our largest international transit provider will switch their handoff to Cape Town instead of Johannesburg later next week
  • A new transit provider will be brought up in both Cape Town and Johannesburg during September
In closing, I want to assure you that ensuring optimal connectivity and the highest standards of uptime and support are a core principle at Web sQuad. We know that the above upgrades will provide you with the best possible experience going forward and we look forward to remaining your preferred service provider in the future!
 

ISP cash cow

Executive Member
Joined
Feb 10, 2011
Messages
5,329
I absolutely love the honesty and transparency of @websquadza
This..

I understand that I am not going to get 100% uptime as things will occur on both Websquad and Vumatel's network (this is technology after all), but the fact that you actually take the time to keep people up to date with the bugs and fixes but then even go as far as to actually give a report back on what happened as well as a company fix you are trying to implement is absolutely amazing.

Keep up the good work Websquadza and team
 

lestoran

Senior Member
Joined
Dec 30, 2007
Messages
788
31st is my last day with existing ISP. What's the best way to make sure the transition to WebSquad is smooth?

Vumatel btw.
 
Top