Web Squad ISP

websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
We've picked up on an edge router in JHB that's flapping BGP sessions. It's been taken offline; please allow a few mins for services to reconverge fully. Will have affected a large proportion of traffic from NAP JHB, which would affect clients accesing services here. Apologies for the inconvenience caused here. The affected device is now out of service while we escalate to the vendor.
 
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
Update: We're picking up on lower than normal traffic volumes across NAP in JHB. Our team is looking into this.
 
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
Update (and technical explanation for those who want it): Thanks all for the updates from your end. Our investigation picked up on an edge router flapping bgp sessions repeatedly. This affected device has been removed from service (we have 5 separate edges, between two DCs in JHB for redundancy).

The issue that caused the drops tonight related to the rapid flapping of the affected device which is directly connected to NAP (hence the number of paths affected) - which created a massive processing overhead for our cores and route reflectors in all regions. Flaps also result in ungraceful withdrawals of prefixes from peers - they keep sending traffic to the affected edge until they detect a timeout (this can be up to 3 mins).

Our route reflectors process changes across the routing table (nationally) in their entirety when they occur (which is usually sporadic and infrequent). The rapid flapping (loading and ungraceful withdrawal of routing tables) created a bit of a processing nightmare across these routers as paths required a timeout to recalculate. These flaps also resulted in stale routes taking time to clear from peer routers (they also need to compute our prefix changes every time they occur).
 
Last edited:
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
IdlePhaedrus said:
No service in Melkbos on Octotel at the moment. Rebooted ont and router. still nothing. Anyone else in the western seaboard area?
Click to expand...
Just received a notification of an outage on Octotel's Melkbosstrand network - will update our network status page:

Dear ISP Partner,

Customers connected to the below node are currently impacted.

Node: Melkbosstrand

Our teams are actively working to identify the fault and restore connectivity, Due to the nature of the fault we are unable to provide an ETA for resolution at this time.

We apologize for the inconvenience, Further updates to follow.
Click to expand...
 
HowTo

HowTo

Expert Member
Joined
Aug 7, 2006
Messages
3,728
I submitted a ticket query on the 24/11/21.
Ticket number #858510 up today no reply.
Static IP was added somhow.
Invoice for sep nil static ip
Incoice for oct. and nov. static ip charges all of sudden included. This is my query?
Never requested this only for Ipv6.
"
Thank you for your ticket.

We have received your request and someone will be looking at it shortly.

Feedback will be shared as soon as possible.

Thank you"
@websquadza pls look into this. It got lost in translation some how.
 
Last edited:
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
HowTo said:
I submitted a ticket query on the 24/11/21.
Ticket number #858510 up today no reply.
Static IP was added somhow.
Invoice for sep nil static ip
Incoice for oct. and nov. static ip charges all of sudden included. This is my query?
Never requested this only for Ipv6.
"
Thank you for your ticket.

We have received your request and someone will be looking at it shortly.

Feedback will be shared as soon as possible.

Thank you"
@websquadza pls look into this. It got lost in translation some how.
Click to expand...

Apologies here. Will get the team to look into this and ensure it’s quickly resolved.
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
This is some impressive performance for 1000/200:



NNTP and international in general are pretty damn slow.
 
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
Seeyou said:
This is some impressive performance for 1000/200:



NNTP and international in general are pretty damn slow.
Click to expand...
Sopbeen said:
yep, same here. Getting exact same speed too
Click to expand...

We're investigating this; seems some routes are affected in London (we're seeing good performance to services like Digital Ocean) which use the same peering and paths. But return paths from CoreIX are particularly questionable - so we've asked our upstreams to see what they can do.
 
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
websquadza said:
We're investigating this; seems some routes are affected in London (we're seeing good performance to services like Digital Ocean) which use the same peering and paths. But return paths from CoreIX are particularly questionable - so we've asked our upstreams to see what they can do.
Click to expand...
Issue seems to have cleared between our last tests and posting the above. We'll continue investigating the cause though.
 
K

Krazee

Active Member
Joined
Nov 28, 2019
Messages
41
@websquadza any luck with Vumatel?. Getting a ton of disconnects in game and Twitch is impossible to watch, the only way I can use my line now is through a VPN.

12439629663.png


Code: 
iperf3 -R -u -b 10M -O2 -t 30 -p 17001 -c ter-jhb-iperf-1.cisp.co.za
Connecting to host ter-jhb-iperf-1.cisp.co.za, port 17001
Reverse mode, remote host ter-jhb-iperf-1.cisp.co.za is sending
[  5] local 192.168.2.85 port 53555 connected to 154.0.2.42 port 17001
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  1.24 MBytes  10.4 Mbits/sec  0.319 ms  0/889 (0%)  (omitted)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.453 ms  0/857 (0%)  (omitted)
[  5]   0.00-1.00   sec  1.20 MBytes  10.0 Mbits/sec  0.060 ms  0/859 (0%)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.477 ms  0/856 (0%)
[  5]   2.00-3.00   sec  1.19 MBytes  10.0 Mbits/sec  0.176 ms  0/856 (0%)
[  5]   3.00-4.00   sec  1.19 MBytes  10.0 Mbits/sec  0.137 ms  0/857 (0%)
[  5]   4.00-5.00   sec  1.19 MBytes  10.0 Mbits/sec  0.105 ms  0/856 (0%)
[  5]   5.00-6.00   sec  1.19 MBytes  10.0 Mbits/sec  0.210 ms  0/856 (0%)
[  5]   6.00-7.00   sec  1.19 MBytes  9.99 Mbits/sec  0.670 ms  1/856 (0.12%)
[  5]   7.00-8.00   sec  1.19 MBytes  10.0 Mbits/sec  0.204 ms  0/856 (0%)
[  5]   8.00-9.00   sec  1.19 MBytes  9.99 Mbits/sec  0.199 ms  0/855 (0%)
[  5]   9.00-10.00  sec  1.07 MBytes  8.99 Mbits/sec  3.830 ms  88/857 (10%)
[  5]  10.00-11.00  sec  1.19 MBytes  10.0 Mbits/sec  0.247 ms  0/857 (0%)
[  5]  11.00-12.00  sec  1.19 MBytes  9.96 Mbits/sec  1.265 ms  2/856 (0.23%)
[  5]  12.00-13.00  sec  1.10 MBytes  9.25 Mbits/sec  0.125 ms  65/856 (7.6%)
[  5]  13.00-14.00  sec  1.17 MBytes  9.81 Mbits/sec  0.290 ms  16/856 (1.9%)
[  5]  14.00-15.00  sec  1.21 MBytes  10.2 Mbits/sec  0.198 ms  -16/856 (-1.9%)
[  5]  15.00-16.00  sec  1.19 MBytes  10.0 Mbits/sec  0.302 ms  0/857 (0%)
[  5]  16.00-17.00  sec  1.19 MBytes  10.0 Mbits/sec  0.242 ms  0/855 (0%)
[  5]  17.00-18.00  sec  1.19 MBytes  10.0 Mbits/sec  0.494 ms  0/857 (0%)
[  5]  18.00-19.00  sec  1.19 MBytes  10.0 Mbits/sec  0.134 ms  0/856 (0%)
[  5]  19.00-20.00  sec  1.19 MBytes  9.99 Mbits/sec  0.285 ms  0/855 (0%)
[  5]  20.00-21.00  sec  1.19 MBytes  10.0 Mbits/sec  0.294 ms  0/857 (0%)
[  5]  21.00-22.00  sec  1.07 MBytes  8.97 Mbits/sec  0.380 ms  88/857 (10%)
[  5]  22.00-23.00  sec  1.19 MBytes  10.0 Mbits/sec  0.143 ms  0/856 (0%)
[  5]  23.00-24.00  sec  1.19 MBytes  10.0 Mbits/sec  0.557 ms  0/856 (0%)
[  5]  24.00-25.00  sec  1.19 MBytes  10.0 Mbits/sec  0.057 ms  0/856 (0%)
[  5]  25.00-26.00  sec  1.19 MBytes  9.98 Mbits/sec  0.291 ms  0/855 (0%)
[  5]  26.00-27.00  sec  1.19 MBytes  10.0 Mbits/sec  0.329 ms  0/857 (0%)
[  5]  27.00-28.00  sec  1.19 MBytes  10.0 Mbits/sec  0.232 ms  0/857 (0%)
[  5]  28.00-29.00  sec  1.19 MBytes  10.0 Mbits/sec  0.225 ms  0/856 (0%)
[  5]  29.00-30.00  sec  1.19 MBytes  9.99 Mbits/sec  0.447 ms  0/854 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.04  sec  35.8 MBytes  10.0 Mbits/sec  0.000 ms  0/25689 (0%)  sender
[SUM]  0.0-30.0 sec  339 datagrams received out-of-order
[  5]   0.00-30.00  sec  35.4 MBytes  9.91 Mbits/sec  0.447 ms  244/25686 (0.95%)  receiver

iperf Done.
 
websquadza

websquadza

WebSquad
Company Rep
Joined
Mar 26, 2018
Messages
3,322
Krazee said:
@websquadza any luck with Vumatel?. Getting a ton of disconnects in game and Twitch is impossible to watch, the only way I can use my line now is through a VPN.

12439629663.png


Code: 
iperf3 -R -u -b 10M -O2 -t 30 -p 17001 -c ter-jhb-iperf-1.cisp.co.za
Connecting to host ter-jhb-iperf-1.cisp.co.za, port 17001
Reverse mode, remote host ter-jhb-iperf-1.cisp.co.za is sending
[  5] local 192.168.2.85 port 53555 connected to 154.0.2.42 port 17001
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  1.24 MBytes  10.4 Mbits/sec  0.319 ms  0/889 (0%)  (omitted)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.453 ms  0/857 (0%)  (omitted)
[  5]   0.00-1.00   sec  1.20 MBytes  10.0 Mbits/sec  0.060 ms  0/859 (0%)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.477 ms  0/856 (0%)
[  5]   2.00-3.00   sec  1.19 MBytes  10.0 Mbits/sec  0.176 ms  0/856 (0%)
[  5]   3.00-4.00   sec  1.19 MBytes  10.0 Mbits/sec  0.137 ms  0/857 (0%)
[  5]   4.00-5.00   sec  1.19 MBytes  10.0 Mbits/sec  0.105 ms  0/856 (0%)
[  5]   5.00-6.00   sec  1.19 MBytes  10.0 Mbits/sec  0.210 ms  0/856 (0%)
[  5]   6.00-7.00   sec  1.19 MBytes  9.99 Mbits/sec  0.670 ms  1/856 (0.12%)
[  5]   7.00-8.00   sec  1.19 MBytes  10.0 Mbits/sec  0.204 ms  0/856 (0%)
[  5]   8.00-9.00   sec  1.19 MBytes  9.99 Mbits/sec  0.199 ms  0/855 (0%)
[  5]   9.00-10.00  sec  1.07 MBytes  8.99 Mbits/sec  3.830 ms  88/857 (10%)
[  5]  10.00-11.00  sec  1.19 MBytes  10.0 Mbits/sec  0.247 ms  0/857 (0%)
[  5]  11.00-12.00  sec  1.19 MBytes  9.96 Mbits/sec  1.265 ms  2/856 (0.23%)
[  5]  12.00-13.00  sec  1.10 MBytes  9.25 Mbits/sec  0.125 ms  65/856 (7.6%)
[  5]  13.00-14.00  sec  1.17 MBytes  9.81 Mbits/sec  0.290 ms  16/856 (1.9%)
[  5]  14.00-15.00  sec  1.21 MBytes  10.2 Mbits/sec  0.198 ms  -16/856 (-1.9%)
[  5]  15.00-16.00  sec  1.19 MBytes  10.0 Mbits/sec  0.302 ms  0/857 (0%)
[  5]  16.00-17.00  sec  1.19 MBytes  10.0 Mbits/sec  0.242 ms  0/855 (0%)
[  5]  17.00-18.00  sec  1.19 MBytes  10.0 Mbits/sec  0.494 ms  0/857 (0%)
[  5]  18.00-19.00  sec  1.19 MBytes  10.0 Mbits/sec  0.134 ms  0/856 (0%)
[  5]  19.00-20.00  sec  1.19 MBytes  9.99 Mbits/sec  0.285 ms  0/855 (0%)
[  5]  20.00-21.00  sec  1.19 MBytes  10.0 Mbits/sec  0.294 ms  0/857 (0%)
[  5]  21.00-22.00  sec  1.07 MBytes  8.97 Mbits/sec  0.380 ms  88/857 (10%)
[  5]  22.00-23.00  sec  1.19 MBytes  10.0 Mbits/sec  0.143 ms  0/856 (0%)
[  5]  23.00-24.00  sec  1.19 MBytes  10.0 Mbits/sec  0.557 ms  0/856 (0%)
[  5]  24.00-25.00  sec  1.19 MBytes  10.0 Mbits/sec  0.057 ms  0/856 (0%)
[  5]  25.00-26.00  sec  1.19 MBytes  9.98 Mbits/sec  0.291 ms  0/855 (0%)
[  5]  26.00-27.00  sec  1.19 MBytes  10.0 Mbits/sec  0.329 ms  0/857 (0%)
[  5]  27.00-28.00  sec  1.19 MBytes  10.0 Mbits/sec  0.232 ms  0/857 (0%)
[  5]  28.00-29.00  sec  1.19 MBytes  10.0 Mbits/sec  0.225 ms  0/856 (0%)
[  5]  29.00-30.00  sec  1.19 MBytes  9.99 Mbits/sec  0.447 ms  0/854 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.04  sec  35.8 MBytes  10.0 Mbits/sec  0.000 ms  0/25689 (0%)  sender
[SUM]  0.0-30.0 sec  339 datagrams received out-of-order
[  5]   0.00-30.00  sec  35.4 MBytes  9.91 Mbits/sec  0.447 ms  244/25686 (0.95%)  receiver

iperf Done.
Click to expand...

We’ve been back and forth with AEX. The first engineer who was on it pretty much didn’t do anything for two weeks. We had a meeting with them yesterday; and we’ve got a commitment that it’s been escalated and our engineers are now finally getting together to try resolve this for once and for all. We’ve picked up on other issues from your region; and using your results and these other ones- pushed back hard to show them the issue is definitely L2. Thanks for your patience here. We’re hoping to see some finality to this over the next few days. Will update you on your ticket as well.
 
ijacobs3

ijacobs3

Expert Member
Joined
Oct 15, 2009
Messages
4,139
Krazee said:
@websquadza any luck with Vumatel?. Getting a ton of disconnects in game and Twitch is impossible to watch, the only way I can use my line now is through a VPN.

12439629663.png


Code: 
iperf3 -R -u -b 10M -O2 -t 30 -p 17001 -c ter-jhb-iperf-1.cisp.co.za
Connecting to host ter-jhb-iperf-1.cisp.co.za, port 17001
Reverse mode, remote host ter-jhb-iperf-1.cisp.co.za is sending
[  5] local 192.168.2.85 port 53555 connected to 154.0.2.42 port 17001
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  1.24 MBytes  10.4 Mbits/sec  0.319 ms  0/889 (0%)  (omitted)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.453 ms  0/857 (0%)  (omitted)
[  5]   0.00-1.00   sec  1.20 MBytes  10.0 Mbits/sec  0.060 ms  0/859 (0%)
[  5]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec  0.477 ms  0/856 (0%)
[  5]   2.00-3.00   sec  1.19 MBytes  10.0 Mbits/sec  0.176 ms  0/856 (0%)
[  5]   3.00-4.00   sec  1.19 MBytes  10.0 Mbits/sec  0.137 ms  0/857 (0%)
[  5]   4.00-5.00   sec  1.19 MBytes  10.0 Mbits/sec  0.105 ms  0/856 (0%)
[  5]   5.00-6.00   sec  1.19 MBytes  10.0 Mbits/sec  0.210 ms  0/856 (0%)
[  5]   6.00-7.00   sec  1.19 MBytes  9.99 Mbits/sec  0.670 ms  1/856 (0.12%)
[  5]   7.00-8.00   sec  1.19 MBytes  10.0 Mbits/sec  0.204 ms  0/856 (0%)
[  5]   8.00-9.00   sec  1.19 MBytes  9.99 Mbits/sec  0.199 ms  0/855 (0%)
[  5]   9.00-10.00  sec  1.07 MBytes  8.99 Mbits/sec  3.830 ms  88/857 (10%)
[  5]  10.00-11.00  sec  1.19 MBytes  10.0 Mbits/sec  0.247 ms  0/857 (0%)
[  5]  11.00-12.00  sec  1.19 MBytes  9.96 Mbits/sec  1.265 ms  2/856 (0.23%)
[  5]  12.00-13.00  sec  1.10 MBytes  9.25 Mbits/sec  0.125 ms  65/856 (7.6%)
[  5]  13.00-14.00  sec  1.17 MBytes  9.81 Mbits/sec  0.290 ms  16/856 (1.9%)
[  5]  14.00-15.00  sec  1.21 MBytes  10.2 Mbits/sec  0.198 ms  -16/856 (-1.9%)
[  5]  15.00-16.00  sec  1.19 MBytes  10.0 Mbits/sec  0.302 ms  0/857 (0%)
[  5]  16.00-17.00  sec  1.19 MBytes  10.0 Mbits/sec  0.242 ms  0/855 (0%)
[  5]  17.00-18.00  sec  1.19 MBytes  10.0 Mbits/sec  0.494 ms  0/857 (0%)
[  5]  18.00-19.00  sec  1.19 MBytes  10.0 Mbits/sec  0.134 ms  0/856 (0%)
[  5]  19.00-20.00  sec  1.19 MBytes  9.99 Mbits/sec  0.285 ms  0/855 (0%)
[  5]  20.00-21.00  sec  1.19 MBytes  10.0 Mbits/sec  0.294 ms  0/857 (0%)
[  5]  21.00-22.00  sec  1.07 MBytes  8.97 Mbits/sec  0.380 ms  88/857 (10%)
[  5]  22.00-23.00  sec  1.19 MBytes  10.0 Mbits/sec  0.143 ms  0/856 (0%)
[  5]  23.00-24.00  sec  1.19 MBytes  10.0 Mbits/sec  0.557 ms  0/856 (0%)
[  5]  24.00-25.00  sec  1.19 MBytes  10.0 Mbits/sec  0.057 ms  0/856 (0%)
[  5]  25.00-26.00  sec  1.19 MBytes  9.98 Mbits/sec  0.291 ms  0/855 (0%)
[  5]  26.00-27.00  sec  1.19 MBytes  10.0 Mbits/sec  0.329 ms  0/857 (0%)
[  5]  27.00-28.00  sec  1.19 MBytes  10.0 Mbits/sec  0.232 ms  0/857 (0%)
[  5]  28.00-29.00  sec  1.19 MBytes  10.0 Mbits/sec  0.225 ms  0/856 (0%)
[  5]  29.00-30.00  sec  1.19 MBytes  9.99 Mbits/sec  0.447 ms  0/854 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.04  sec  35.8 MBytes  10.0 Mbits/sec  0.000 ms  0/25689 (0%)  sender
[SUM]  0.0-30.0 sec  339 datagrams received out-of-order
[  5]   0.00-30.00  sec  35.4 MBytes  9.91 Mbits/sec  0.447 ms  244/25686 (0.95%)  receiver

iperf Done.
Click to expand...
12439900279.png



mines not too bad :)
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
@websquadza - sorry to post this here but it's been 24 hours with no response to my emails, nor PMs, after the initial cancellation query went unanswered for several days and meanwhile the clock is ticking till the end of the month.

All I want to know is whether I can (a) cancel this month and have the line released without being charged for January and (b) since the 1st of Jan falls on a Saturday (and because it's also new year's day etc) whether or not the line could be released earlier the previous week to make sure there are people around to address any possible migration issues.
 
G

Gimli_

Senior Member
Joined
Feb 8, 2005
Messages
945
Man, some good deals to be had there!

MyBroadband: RSAWEB – Summer Fibre Sale.
mybroadband.co.za

RSAWEB – Summer Fibre Sale

RSAWEB is running an amazing sale that provides you with massively reduced prices on its fibre products.
mybroadband.co.za mybroadband.co.za
 
You must log in or register to reply here.
Top