Xneelo MTU problems

Deckert

Deckert

Well-Known Member
Joined
Jan 13, 2004
Messages
425
Reaction score
38
Location
Centurion, South Africa
Hi,

As of late., we're experiencing significant disruptions from the Xneelo datacenter to/from international destinations for packets larger than 1476 bytes. The default is 1500 bytes, but at the time of this writing, any packet larger than 1476 bytes is simply dropped.

Is anybody else experiencing this issue?

As a simple test, from your (Linux) Xneelo server, ping any international destination:

This works:
$ ping -n speedtest.london.linode.com -s 1448 -M do
PING speedtest-1.lon1.gb.prod.linode.com (176.58.107.39) 1448(1476) bytes of data.
1456 bytes from 176.58.107.39: icmp_seq=1 ttl=50 time=168 ms
1456 bytes from 176.58.107.39: icmp_seq=2 ttl=50 time=167 ms

This does not:
~$ ping -n speedtest.london.linode.com -s 1449 -M do
PING speedtest-1.lon1.gb.prod.linode.com (176.58.107.39) 1448(1477) bytes of data.
^C
--- speedtest-1.lon1.gb.prod.linode.com ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5145ms


Note, the increasing packets size from 1476 to 1477 in the above two examples.

Normally, with MTU path discovery, this does not cause an issue (although it still causes degradation due to packet fragmentation), but for customer-facing VPNs this is a disaster. Further more, we cannot control the maximum MTU size on the customer's end, so we need to contact every customer and convince them that they need to reduce their MTU or add MSS clamping.

This is not an issue we see with any of the other datacenters we host in. Frustrating to say the least.

--deckert
 
We can reproduce this on an Xneelo subnet however it does not appear to impact traffic over peering and may be isolated to a specific region/upstream.
 
Last edited:
Rackzar said:
We can reproduce this however it does not appear to impact traffic over peering and may be isolated to a specific region/upstream.
Click to expand...

It's affecting us in a fairly big way. All the cloud providers that offer IPSec VPN services (e.g. GCP - Google Cloud) have issues connecting to our services.

It affects all international destinations. Local (i.e. within the ZA routing/peering space) things work perfectly fine. We've raised a ticket with Xneelo and the worst part is how dismissive they are about the issue. They acknowledge the problem and state that we should set our interface MTUs to 1476. Since when is this an acceptable solution?

None of our other locations have this issue and testing from my personal VPSs at Absolute Hosting and Datakeepers, they do not exhibit this behaviour either.

--deckert
 
Its likely designed by them for some reason, likely they encapsulate to transit somewhere, perhaps.
Set the MTU on your side of the VPNs?
 
Packet-Kollector said:
Its likely designed by them for some reason, likely they encapsulate to transit somewhere, perhaps.
Set the MTU on your side of the VPNs?
Click to expand...

It's a bad design then.

The issue is with inbound traffic, makes no difference if we set it for outbound.

In any event, we've since learned (from another forum member) that Xneelo sends their international traffic to a "scrubbing" company that helps them deal with DDoS traffic. I can only assume that this means they tunnel all of their international inbound traffic via these scrubbers and the tunnel as a 24-byte overhead.

Look, it's good that they at least try to protect their DC, but the customer-service side on the side-effect is down right atrocious. We've had to spend an enormous amount of time trouble shooting this, testing, finding out on our own. If they just let us know before hand, we could have planned and immediately been able to start interacting with our clients.

I still think that it's a technically poor solution - because we don't see this anywhere else, even the big hyper scalers such as AWS, Linode, etc. They have DDoS mitigation too, but it does not affect packet MTU sizes.

--deckert
 
Deckert said:
It's affecting us in a fairly big way. All the cloud providers that offer IPSec VPN services (e.g. GCP - Google Cloud) have issues connecting to our services.

It affects all international destinations. Local (i.e. within the ZA routing/peering space) things work perfectly fine. We've raised a ticket with Xneelo and the worst part is how dismissive they are about the issue. They acknowledge the problem and state that we should set our interface MTUs to 1476. Since when is this an acceptable solution?

None of our other locations have this issue and testing from my personal VPSs at Absolute Hosting and Datakeepers, they do not exhibit this behaviour either.

--deckert
Click to expand...
If you want we can force the protection group that sits in front of your VPS at Absolute Hosting into upstream scrubbing to see if you are able to duplicate the issue on our environment and this may give you some leverage in getting Xneelo to resolve the issue on their end?

Feel free to DM me and I'll get this setup for you.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X