Web Squad ISP

Status
Not open for further replies.
Seeyou said:
@websquadza - since yesterday's blip, I'm noticing in-game pings in Apex have gone up. Amsterdam has gone from 154ms > 164ms, and Frankfurt from ~180ms to over 200ms. London still seems normal. As a result I'm seeing the high ping warning in game quite a bit more frequently as servers are randomly assigned all over Europe. Are these changes permanent?
Click to expand...

Morning, APEX is an odd one - it's hosted on AWS as far as I know. So their traffic is completely unaffected by yesterday's issue on our upstream's DECIX port (this is currently being sorted out and should be back online ASAP).. I could be wrong though - could you see if you can find an IP that you are connecting to so we can trace?
 
This is the current latency situation. Interestingly when the weird routing was in place yesterday, both Frankfurt servers were down to 154ms and Belgium was lower too - but packet loss was through the roof.

Screenshot (35).png
 
websquadza said:
Morning, APEX is an odd one - it's hosted on AWS as far as I know. So their traffic is completely unaffected by yesterday's issue on our upstream's DECIX port (this is currently being sorted out and should be back online ASAP).. I could be wrong though - could you see if you can find an IP that you are connecting to so we can trace?
Click to expand...

You're correct, they're hosted on AWS, but it was definitely affected by yesterday's issue, packet loss was >25%, completely unplayable for a while. I just got some work in, but I'll try grab a wireshark capture in a bit during the startup/server ping phase so you can get some IPs.
 
websquadza said:
Morning, APEX is an odd one - it's hosted on AWS as far as I know. So their traffic is completely unaffected by yesterday's issue on our upstream's DECIX port (this is currently being sorted out and should be back online ASAP).. I could be wrong though - could you see if you can find an IP that you are connecting to so we can trace?
Click to expand...
This should still be accurate:

display "London"

name "west europe"

host "217.147.89.101"

port 37015

display "Amsterdam"

name "west europe"

host "64.95.100.189"

port 37015

display "Belgium - GCE 1"

name "west europe"

host "146.148.120.92"

port 37015

display "Belgium - GCE 2"

name "west europe"

host "104.155.80.155"

port 37015

display "Belgium - GCE 3"

name "west europe"

host "130.211.51.110"

port 37015

display "Frankfurt 1" // AWS

name "west europe"

host "52.58.81.34"

port 37015

display "Frankfurt 2" // AWS

name "west europe"

host "52.59.121.244"

port 37015
 
Soul Assassin said:
This should still be accurate:

display "London"

name "west europe"

host "217.147.89.101"

port 37015

display "Amsterdam"

name "west europe"

host "64.95.100.189"

port 37015

display "Belgium - GCE 1"

name "west europe"

host "146.148.120.92"

port 37015

display "Belgium - GCE 2"

name "west europe"

host "104.155.80.155"

port 37015

display "Belgium - GCE 3"

name "west europe"

host "130.211.51.110"

port 37015

display "Frankfurt 1" // AWS

name "west europe"

host "52.58.81.34"

port 37015

display "Frankfurt 2" // AWS

name "west europe"

host "52.59.121.244"

port 37015
Click to expand...

The ping results do seem to line up with the in-game latencies displayed, thanks.
 
@websquadza - you can see the IPs Soul Assassin posted - they seem to be accurate.

Would also appreciate the instructions for enabling IPV6 when you get a moment, thank you!
 
EU connectivity going up and down. Just me?

Code: 
|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                             192.168.1.1 -    0 |    5 |    5 |    0 |    0 |    0 |    0 |
|             core.as-01.cp1.za.ws.net.za -    0 |    5 |    5 |    0 |    0 |    1 |    0 |
|             core.cr-01.cp1.za.ws.net.za -    0 |    5 |    5 |    0 |    0 |    1 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider
 
Seeyou said:
EU connectivity going up and down. Just me?

Code: 
|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                             192.168.1.1 -    0 |    5 |    5 |    0 |    0 |    0 |    0 |
|             core.as-01.cp1.za.ws.net.za -    0 |    5 |    5 |    0 |    0 |    1 |    0 |
|             core.cr-01.cp1.za.ws.net.za -    0 |    5 |    5 |    0 |    0 |    1 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider
Click to expand...

What IP are you tracing to? We're not seeing this on the access router from our side.
 
Seeyou said:
@websquadza - you can see the IPs Soul Assassin posted - they seem to be accurate.

Would also appreciate the instructions for enabling IPV6 when you get a moment, thank you!
Click to expand...

Looking at these IPs and also return routes from AWS regions and will revery shortly.

I'll send you an IPv6 guide.
 
websquadza said:
What IP are you tracing to? We're not seeing this on the access router from our side.
Click to expand...

Was tracing to speedtest.coreix.net, but pretty much any EU site / service I try to access when it happens times out or is unreachable. Any tests you want me to run if/when it happens again?

Nevermind, happening again right now. To api.rubyonrails.org:

Code: 
|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                             192.168.1.1 -    0 |    4 |    4 |    0 |    0 |    0 |    0 |
|             core.as-01.cp1.za.ws.net.za -    0 |    4 |    4 |    0 |    0 |    0 |    0 |
|             core.cr-01.cp1.za.ws.net.za -    0 |    4 |    4 |    0 |    0 |    1 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|                   No response from host -    0 |    0 |    0 |    0 |    0 |    0 |    0 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider
 
Seeyou said:
Was tracing to speedtest.coreix.net, but pretty much any EU site / service I try to access when it happens times out or is unreachable. Any tests you want me to run if/when it happens again?
Click to expand...

We've set up a bunch of active probes right now. Just let me know next time you see it so we can compare from our end. Then we can trace back to you?
 
I'm wondering if it's the DDoS mitigation? I'm testing NNTP with 100 connections right now - would that trigger it and result in me getting cut off for a few mins?
 
Seeyou said:
I'm wondering if it's the DDoS mitigation? I'm testing NNTP with 100 connections right now - would that trigger it and result in me getting cut off for a few mins?
Click to expand...

Yep just happened again after 150 connections. It's currently happening if you caught anything @websquadza ?
 
Seeyou said:
I'm wondering if it's the DDoS mitigation? I'm testing NNTP with 100 connections right now - would that trigger it and result in me getting cut off for a few mins?
Click to expand...
Seeyou said:
Yep just happened again after 150 connections. It's currently happening if you caught anything @websquadza ?
Click to expand...

Apologies, didn't check our DDOS tool - so this explains yesterday's blip at 8:30 as well. Checking limits there and will revert asap

I'm waiting for a report on the limits setting alarms and blackholing your IP, and we can work from there to resolve. As soon as our DDOS platform sees an offending breach, it will blackhole that IP (hence the MTR).

With regards to the APEX IPs provided - I think some of them are GCP Hosted, some are IOMART - not seeing any AWS - so I think they've changed away from that. Our routing changes are temporary regarding the DECIX (That's seems to have been faulty hardware) - and should be back in play anytime soon.
 
Last edited:
websquadza said:
Apologies, didn't check our DDOS tool - so this explains yesterday's blip at 8:30 as well. Checking limits there and will revert asap

I'm waiting for a report on the limits setting alarms and blackholing your IP, and we can work from there to resolve. As soon as our DDOS platform sees an offending breach, it will blackhole that IP (hence the MTR).

With regards to the APEX IPs provided - I think some of them are GCP Hosted, some are IOMART - not seeing any AWS - so I think they've changed away from that. Our routing changes are temporary regarding the DECIX (That's seems to have been faulty hardware) - and should be back in play anytime soon.
Click to expand...

Cool- definitely seems to be triggered by the multi-threaded downloads. I've been testing a new Usenet service which offers more threads and it seems any time I exceed 50 or so I'm getting blackholed for a few minutes.
 
Seeyou said:
Cool- definitely seems to be triggered by the multi-threaded downloads. I've been testing a new Usenet service which offers more threads and it seems any time I exceed 50 or so I'm getting blackholed for a few minutes.
Click to expand...

Yup, TCP packet volumes exceeded all threshold levels (which is what these multithread apps do very well) - we've relaxed these these for you a little. Can you test and see if you still hitting limits?
 
Soul Assassin said:
This should still be accurate:

display "London"

name "west europe"

host "217.147.89.101"

port 37015

display "Amsterdam"

name "west europe"

host "64.95.100.189"

port 37015

display "Belgium - GCE 1"

name "west europe"

host "146.148.120.92"

port 37015

display "Belgium - GCE 2"

name "west europe"

host "104.155.80.155"

port 37015

display "Belgium - GCE 3"

name "west europe"

host "130.211.51.110"

port 37015

display "Frankfurt 1" // AWS

name "west europe"

host "52.58.81.34"

port 37015

display "Frankfurt 2" // AWS

name "west europe"

host "52.59.121.244"

port 37015
Click to expand...
Thank you for this aw how i have been searching for this
 
Seeyou said:
Yep just happened again after 150 connections. It's currently happening if you caught anything @websquadza ?
Click to expand...
Just wondering why you would need 100-150 connections for NNTP? I can pretty much max out my 1Gbps connection to Astraweb's servers with 50 connections:
1594907261321.png
Perhaps performance differs per usenet provider? Also, disclaimer: I usually limit to 50MB/s as to not hog all the internets :ROFL:
 
Status
Not open for further replies.
Back
Top
Sign up to the MyBroadband newsletter
X