Web Squad ISP

[KaMoS]

Down again?

Yip. I've been down twice today.

 

[Concentric]

Done here too.
Norwood JHB

 

[daelm]

down again.

 

[Zodiak]

Down again half way into my meeting.... what is going on all day today?!

 

[websquadza]

We've isolated the issue. Please allow a few minutes for a fix. Will communicate a RFO later today

 

[Looney]

@websquadza My internet just went dead. Vumatel Aerial, Glenhazel. ONT is fine.

 

[daelm]

We've isolate the issue. Please allow a few minutes for a fix. Will communicate a RFO later today

thanks. i need to know when you're stable before i reconnect, as i'm in the middle of a pitch presentation and can't really get kicked out of it again

edit: tethered to phone for the time being

 

[KaMoS]

Can't get hold of support. Line seems to be down and I can't get the the support page either. https://my.websquad.co.za/knowledgebase.php

 

[mystic007]

Can't get hold of support. Line seems to be down and I can't get the the support page either. https://my.websquad.co.za/knowledgebase.php

I also tried to log a call but can't open the support page either from my LTE

 

[HowTo]

Down Springs East-Rand Vumatel trenched.
Need to connect to Telkom. My cell data is loading.

Up again.

 

[abudabi]

Back

 

[ijacobs3]

Down Springs East-Rand Vumatel trenched.
Need to connect to Telkom. My cell data is loading

mine just came on

 

[Concentric]

Back up

 

[Looney]

Yup back up. @websquadza, don’t do that again

 

[rvanwyk]

These drops are becoming very concerning. What is the long-term solution to this?

 

[Looney]

First time I’ve experienced a drop in a very long time and this was obviously something on Websquad’s side. I’m also concerned this will become a regular thing…

 

[websquadza]

thanks. i need to know when you're stable before i reconnect, as i'm in the middle of a pitch presentation and can't really get kicked out of it again

edit: tethered to phone for the time being

Can't get hold of support. Line seems to be down and I can't get the the support page either. https://my.websquad.co.za/knowledgebase.php

Apologies here, broken peering and routes tend to do this. We'll look at alternate plans in this regard.

Back

Back up

Yup back up. @websquadza, don’t do that again

These drops are becoming very concerning. What is the long-term solution to this?

Apologies all for the inconvenience caused here today; short of it - one of our primary peering interfaces was dumped into a quarantine VLAN at one of the exchanges without us being notified. This caused a fair amount of knock on issues. Once notified, we were able to isolate the issue and resolve it. We are investigating why/how this happened and how/if it's related to this morning's issues.

 

[daelm]

Apologies here, broken peering and routes tend to do this. We'll look at alternate plans in this regard.





Apologies all for the inconvenience caused here today; short of it - one of our primary peering interfaces was dumped into a quarantine VLAN at one of the exchanges without us being notified. This caused a fair amount of knock on issues. Once notified, we were able to isolate the issue and resolve it. We are investigating why/how this happened and how/if it's related to this morning's issues.

stable?

 

[websquadza]

These drops are becoming very concerning. What is the long-term solution to this?

First time I’ve experienced a drop in a very long time and this was obviously something on Websquad’s side. I’m also concerned this will become a regular thing…

We haven't had a drop at core level in a very long time - uptime on some of the bgp sessions that were dropped today was well over a year.

That said, no, these incidents are not common. We are investigating what led to it and will definitely learn from it and make necessary changes to ensure we don't see something like this again. A combination of unlikely events led to this outage today.

Once again, apologies to all for the inconvenience caused.

 

[websquadza]

stable?

We're running tests, but yes.