Networking Wizards - Weirdness with Fiber via WAN PPPoE

K

KillerX

Expert Member
Joined
Jul 4, 2005
Messages
1,109
PsyWulf said:
I'd say run fiddler with HTTPS decryption and see what's happening,sounds like some kind of MITM
Click to expand...

Ok, got that up and running with HTTPS debugging enabled. See alot of outbound 'tunnel to' with plenty of information. What exactly am I looking for?
 
K

KillerX

Expert Member
Joined
Jul 4, 2005
Messages
1,109
RoganDawes said:
Or you could try this: https://elifulkerson.com/projects/tcproute.php

Disclaimer: I've never used it
Click to expand...

Thanks, good idea.

Tried the windows version. Runs fine on port 80 - shows all the hops like a tracert. But when I try 443 I just get a loop going forever:

Tried 2 different sites.

Using the following values:
---------------------------
Local IP: 192.168.1.110
Remote IP: 0.0.1.187

Tracing route to 443:80
1 2009 ms timed out
1 2001 ms timed out
1 2001 ms timed out
2 2001 ms timed out
2 2001 ms timed out
2 2001 ms timed out
3 2002 ms timed out
3 2001 ms timed out
3 2001 ms timed out
4 2002 ms timed out
4 2001 ms timed out
4 2001 ms timed out
5 2002 ms timed out
5 2001 ms timed out
5 2002 ms timed out
6 2002 ms timed out
 
Sinbad

Sinbad

Honorary Master
Joined
Jun 5, 2006
Messages
81,151
KillerX said:
Thanks, good idea.

Tried the windows version. Runs fine on port 80 - shows all the hops like a tracert. But when I try 443 I just get a loop going forever:

Tried 2 different sites.

Using the following values:
---------------------------
Local IP: 192.168.1.110
Remote IP: 0.0.1.187

Tracing route to 443:80
1 2009 ms timed out
1 2001 ms timed out
1 2001 ms timed out
2 2001 ms timed out
2 2001 ms timed out
2 2001 ms timed out
3 2002 ms timed out
3 2001 ms timed out
3 2001 ms timed out
4 2002 ms timed out
4 2001 ms timed out
4 2001 ms timed out
5 2002 ms timed out
5 2001 ms timed out
5 2002 ms timed out
6 2002 ms timed out
Click to expand...

You're using it wrong.

look at the IP address it's trying to get to.

Check the switches/command line arguments.
 
R

RoganDawes

Expert Member
Joined
Apr 18, 2007
Messages
1,259
Sinbad said:
You're using it wrong.

look at the IP address it's trying to get to.

Check the switches/command line arguments.
Click to expand...

Agree he is using it wrong. But I would still expect to see the packets making it out of his network, which they do not seem to be doing.
 
K

KillerX

Expert Member
Joined
Jul 4, 2005
Messages
1,109
Sinbad said:
http://packetlife.net/blog/2008/aug/18/path-mtu-discovery/

try that path mtu discovery tool, and make sure the MTU on your linksys is set at or below that number.
Click to expand...

DUDE! You're a genius. It worked! ;)

So looking at my router, it was set to AUTO - and this picked it up as : 1492.

I ran the tool to google with the following result

C:\Users\admin\Desktop>mturoute.exe -t www.google.com
mturoute to www.google.com, 30 hops max, variable sized packets
* ICMP Fragmentation is not permitted. *
* Speed optimization is enabled. *
* Maximum payload is 10000 bytes. *
1 +- host: 192.168.1.1 max: 1500 bytes
2 -++++++++-++- host: 155.93.139.129 max: 1492 bytes
3 +- host: 154.0.2.61 max: 1492 bytes
4 ...-+++++++.-++++ host: 196.60.8.166 max: 1480 bytes
5 ++--+---+++---- host: 72.14.237.239 max: 1492 bytes
6 +- host: 216.58.223.4 max: 1492 bytes

I wasnt sure if it is only the first hop that matters, or the lowest number in that whole trace. So I set it manually to 1480 and voila. All sites I was struggling with now work without issue.

I don't understand how an Auto MTU (which ive been using forever) would let TLS handshakes fail?

Repped Sinbad! If I could do it x10 I would. Been struggling with this for days now, and my ISP and fiber providers both think I'm nuts.
 
Last edited:
Sinbad

Sinbad

Honorary Master
Joined
Jun 5, 2006
Messages
81,151
KillerX said:
DUDE! You're a genius. It worked! ;)

So looking at my router, it was set to AUTO - and this picked it up as : 1492.

I ran the tool to google with the following result

C:\Users\admin\Desktop>mturoute.exe -t www.google.com
mturoute to www.google.com, 30 hops max, variable sized packets
* ICMP Fragmentation is not permitted. *
* Speed optimization is enabled. *
* Maximum payload is 10000 bytes. *
1 +- host: 192.168.1.1 max: 1500 bytes
2 -++++++++-++- host: 155.93.139.129 max: 1492 bytes
3 +- host: 154.0.2.61 max: 1492 bytes
4 ...-+++++++.-++++ host: 196.60.8.166 max: 1480 bytes
5 ++--+---+++---- host: 72.14.237.239 max: 1492 bytes
6 +- host: 216.58.223.4 max: 1492 bytes

I wasnt sure if it is only the first hop that matters, or the lowest number in that whole trace. So I set it manually to 1480 and voila. All sites I was struggling with now work without issue.

I don't understand how an Auto MTU (which ive been using forever) would let TLS handshakes fail?
Click to expand...

Someone else pointed to MTU before me ;)
That box with the smaller MTU seems to be a google interconnect. So not sure why it would affect your other stuff as well, unless there's a piece of equipment at your ISP that's ACTUALLY the cause - a peering router or something.
 
K

KillerX

Expert Member
Joined
Jul 4, 2005
Messages
1,109
Sinbad said:
Someone else pointed to MTU before me ;)
That box with the smaller MTU seems to be a google interconnect. So not sure why it would affect your other stuff as well, unless there's a piece of equipment at your ISP that's ACTUALLY the cause - a peering router or something.
Click to expand...

Right, Nuke said something about MTU clamping, but I'm not too clued up on the network side. I needed someone to just dumb it down a little for me with an example.

Maybe the routers AUTO function itself isn't too accurate and it might have been adjusting it too high?
 
Sinbad

Sinbad

Honorary Master
Joined
Jun 5, 2006
Messages
81,151
KillerX said:
Right, Nuke said something about MTU clamping, but I'm not too clued up on the network side. I needed someone to just dumb it down a little for me with an example.

Maybe the routers AUTO function itself isn't too accurate and it might have been adjusting it too high?
Click to expand...

Quite possible, it probably sets the MTU to the max of its own upstream connection.
Run that trace to a few other SSL sites you've had issues with and see if there's a common hop that restricts to 1480?
 
aybbleek

aybbleek

Expert Member
Joined
Sep 9, 2013
Messages
1,611
Glad that you got it sorted. Will remember this... I'm sure you won't be the last person to experience this problem :)
 
R

RoganDawes

Expert Member
Joined
Apr 18, 2007
Messages
1,259
That doesn't actually make sense. MTU only refers to the largest packet that can be transmitted without being fragmented. Any router on the patch that has a smaller MTU will simply have to fragment the packet and send it in two or more pieces. This should not be a problem, it is just a performance thing.

However, this article (see Path MTU discovery) https://en.wikipedia.org/wiki/Maximum_transmission_unit has a possible explanation, where it talks about ICMP filtering. Since your router was set to auto, it would have been doing MTU discovery to figure out its MTU.If your ISP equipment is filtering ICMP responses, then the MTU discovery would fail. And since https sends large packets (e.g. certificates, etc), it would explain why some protocols work (e.g. http), while others do not e.g. SSL.
 
Stratt

Stratt

Well-Known Member
Joined
Jan 10, 2014
Messages
162
RoganDawes said:
That doesn't actually make sense. MTU only refers to the largest packet that can be transmitted without being fragmented. Any router on the patch that has a smaller MTU will simply have to fragment the packet and send it in two or more pieces. This should not be a problem, it is just a performance thing.

However, this article (see Path MTU discovery) https://en.wikipedia.org/wiki/Maximum_transmission_unit has a possible explanation, where it talks about ICMP filtering. Since your router was set to auto, it would have been doing MTU discovery to figure out its MTU.If your ISP equipment is filtering ICMP responses, then the MTU discovery would fail. And since https sends large packets (e.g. certificates, etc), it would explain why some protocols work (e.g. http), while others do not e.g. SSL.
Click to expand...

Actually no, almost all encrypted traffic has its DF (Don't fragment) bit set. Therefore no router will fragment the packet and instead drop it, should the packet size be above its MTU. This explains why TLS/SSL traffic was the only sessions affected. Path MTU discovery does not mean that his router will "discover" the MTU. This standard relies on the router dropping the packet to send a ICMP packet to the original sending host with a response "ICMP fragmentation needed but DF bit set". Check the RFC, I think its only like 4 pages in total.

The other option to have fixed this would have been to adjust the MSS (Maximum send size - the actual data payload size excluding headers) on TCP session setup.
 
R

RoganDawes

Expert Member
Joined
Apr 18, 2007
Messages
1,259
Stratt said:
Actually no, almost all encrypted traffic has its DF (Don't fragment) bit set. Therefore no router will fragment the packet and instead drop it, should the packet size be above its MTU. This explains why TLS/SSL traffic was the only sessions affected. Path MTU discovery does not mean that his router will "discover" the MTU. This standard relies on the router dropping the packet to send a ICMP packet to the original sending host with a response "ICMP fragmentation needed but DF bit set". Check the RFC, I think its only like 4 pages in total.

The other option to have fixed this would have been to adjust the MSS (Maximum send size - the actual data payload size excluding headers) on TCP session setup.
Click to expand...

Hmmm, it seems that this is not a standard (i.e. not documented in an RFC anywhere), but may be practice nonetheless. I learned something new, thanks!
 
You must log in or register to reply here.
Top