Speed - optimal settings: RWIN, MTU etc...

[michael.bytz]

On the link i posted above you can download TCPOptimiser - it finds the max mtu by pinging a specified address with different size packets until it fragments.
It's the first link.

Hi MaD,

Thanks for the info. Tried the link before my previous post and it points to a Windows executable. I'm running a Mac

Regards,

Michael

 

[Jerrek]

Windows default settings are more than fine. I get 1 MB/sec out of the box.

 

[ic]

Please give this a try, I want to know if anyone gets different results

Scroll through the following which I have just run, you will see that the highest possible MTU value for a PPPoE connection (UTD via ethernet) is still 1392, which is 1364+28:

Microsoft Windows 2000 [Version 5.00.2195]
(C) Copyright 1985-2000 Microsoft Corp.

D:\Documents and Settings\...>ping -f -l 1400 196.30.31.100

Pinging 196.30.31.100 with 1400 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>ping -f -l 1392 196.30.31.100

Pinging 196.30.31.100 with 1392 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>ping -f -l 1390 196.30.31.100

Pinging 196.30.31.100 with 1390 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>ping -f -l 1380 196.30.31.100

Pinging 196.30.31.100 with 1380 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>ping -f -l 1370 196.30.31.100

Pinging 196.30.31.100 with 1370 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>ping -f -l 1360 196.30.31.100

Pinging 196.30.31.100 with 1360 bytes of data:

Reply from 196.30.31.100: bytes=1360 time=430ms TTL=255
Reply from 196.30.31.100: bytes=1360 time=1072ms TTL=255
Reply from 196.30.31.100: bytes=1360 time=220ms TTL=255
Reply from 196.30.31.100: bytes=1360 time=1152ms TTL=255

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 220ms, Maximum =  1152ms, Average =  718ms

D:\Documents and Settings\...>ping -f -l 1361 196.30.31.100

Pinging 196.30.31.100 with 1361 bytes of data:

Reply from 196.30.31.100: bytes=1361 time=771ms TTL=255
Reply from 196.30.31.100: bytes=1361 time=821ms TTL=255
Reply from 196.30.31.100: bytes=1361 time=1101ms TTL=255
Reply from 196.30.31.100: bytes=1361 time=1202ms TTL=255

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 771ms, Maximum =  1202ms, Average =  973ms

D:\Documents and Settings\...>ping -f -l 1362 196.30.31.100

Pinging 196.30.31.100 with 1362 bytes of data:

Reply from 196.30.31.100: bytes=1362 time=521ms TTL=255
Reply from 196.30.31.100: bytes=1362 time=291ms TTL=255
Request timed out.
Reply from 196.30.31.100: bytes=1362 time=221ms TTL=255

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
    Minimum = 221ms, Maximum =  521ms, Average =  258ms

D:\Documents and Settings\...>ping -f -l 1363 196.30.31.100

Pinging 196.30.31.100 with 1363 bytes of data:

Request timed out.
Reply from 196.30.31.100: bytes=1363 time=901ms TTL=255
Reply from 196.30.31.100: bytes=1363 time=942ms TTL=255
Reply from 196.30.31.100: bytes=1363 time=591ms TTL=255

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
    Minimum = 591ms, Maximum =  942ms, Average =  608ms

D:\Documents and Settings\...>[B][COLOR=Green]ping -f -l 1364 196.30.31.100[/COLOR][/B]

Pinging 196.30.31.100 with 1364 bytes of data:

Reply from 196.30.31.100: bytes=1364 time=291ms TTL=255
Reply from 196.30.31.100: bytes=1364 time=420ms TTL=255
Request timed out.
Request timed out.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 2, Lost = 2 (50% loss),
Approximate round trip times in milli-seconds:
    Minimum = 291ms, Maximum =  420ms, Average =  177ms

D:\Documents and Settings\...>[B][COLOR=Red]ping -f -l 1365 196.30.31.100[/COLOR][/B]

Pinging 196.30.31.100 with 1365 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.100:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum =  0ms, Average =  0ms

D:\Documents and Settings\...>

I suggest that you verify this for the base-station that you are connected to by running the following at a command prompt:

ping -f -l 1365 196.30.31.100
ping -f -l 1364 196.30.31.100

If 1365 says it needs to be fragmented, and 1364 works then your MTU is in fact 1364+28==1392.

 

[Gatecrasher]

IC, that is very odd. When I first tried just now, it fragmented above 1324.

But since my MTU was set at 1352, I guessed that was why. So I reset my MTU to 1500 and re-tested. Here are the results:

>ping -f -l 1405 www.iburst.co.za

Pinging www.iburst.co.za [196.30.31.120] with 1405 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 196.30.31.120:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

>ping -f -l 1404 www.iburst.co.za

Pinging www.iburst.co.za [196.30.31.120] with 1404 bytes of data:

Reply from 196.30.31.120: bytes=1404 time=294ms TTL=126
Reply from 196.30.31.120: bytes=1404 time=359ms TTL=126
Reply from 196.30.31.120: bytes=1404 time=296ms TTL=126
Reply from 196.30.31.120: bytes=1404 time=312ms TTL=126

Ping statistics for 196.30.31.120:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 294ms, Maximum = 359ms, Average = 315ms

I got the same with 196.30.31.100 (what is that?)

Which would make for an MTU of 1432.



I am certain I ran the same exercise 2 days ago and got to 1352.

Am I going nuts? Or are WBS continuing with their "experiments"?

 

[ic]

...
I got the same with 196.30.31.100 (what is that?)

Which would make for an MTU of 1432.



I am certain I ran the same exercise 2 days ago and got to 1352.

Am I going nuts? Or are WBS continuing with their "experiments"?

196.30.31.100 is our closest IP peer - maybe the 1st visible router or what I don't know, call it our iBurst gateway into the Internet.

Very odd results, but an indication that either WBS are playing games again, or each base-station its own router(s) with potentially different MTU settings.

 

[Gatecrasher]

196.30.31.100 is our closest IP peer - maybe the 1st visible router or what I don't know, call it our iBurst gateway into the Internet.

Very odd results, but an indication that either WBS are playing games again, or each base-station its own router(s) with potentially different MTU settings.

Do you get 1432? You have to set your MTU higher before you test, otherwise you will, of course, get back to your current MTU setting. I've kept mine at 1500, and I'm still getting 1432, confirmed by the broadbandreports tweak test.

 

[ic]

GC, the way I understand it, the MTU setting on your PC is the default to use when, however the MTU can be overriden by applications, in this case ping with the -f (don't fragment) and -l (small L MSS value for buffer size). What this means is that running these ping tests will tell you the highest MSS value that you can use, when you are using PPPoE MTU==MSS+28.

 

[Gatecrasher]

Yes, then how do you explain getting 1324 when I had MTU set at 1352, and 1404 when I had the MTU set at 1500? The -f command doesn't appear to overule the MTU setting.

Here's what I get using TCPOptimizer.

Pinging [63.217.30.70] with 40 bytes ->bytes=40 time=385ms TTL=48
Pinging [63.217.30.70] with 750 bytes ->bytes=750 time=514ms TTL=48
Pinging [63.217.30.70] with 1125 bytes ->bytes=1125 time=467ms TTL=48
Pinging [63.217.30.70] with 1312 bytes ->bytes=1312 time=482ms TTL=48
Pinging [63.217.30.70] with 1406 bytes -> ..fragmented
Pinging [63.217.30.70] with 1359 bytes ->bytes=1359 time=652ms TTL=48
Pinging [63.217.30.70] with 1382 bytes ->bytes=1382 time=529ms TTL=48
Pinging [63.217.30.70] with 1394 bytes ->bytes=1394 time=529ms TTL=48
Pinging [63.217.30.70] with 1400 bytes ->bytes=1400 time=591ms TTL=48
Pinging [63.217.30.70] with 1403 bytes ->bytes=1403 time=529ms TTL=48
Pinging [63.217.30.70] with 1404 bytes ->bytes=1404 time=513ms TTL=48
Pinging [63.217.30.70] with 1405 bytes -> ..fragmented
The largest possible non-fragmented packet is 1404  (1432 - 28 ICMP & IP headers).
You can set your MTU to 1432

I did the same test 2 days ago, setting my MTU to 1500 first, and then testing. I got 1324 and 1352.

Well, I'm going to use 1432 for a few days and see how it goes. Wish WBS would make their minds up.

 

[ic]

Yes, then how do you explain getting 1324 when I had MTU set at 1352, and 1404 when I had the MTU set at 1500? The -f command doesn't appear to overule the MTU setting.

Here's what I get using TCPOptimizer.

...
Pinging [63.217.30.70] with 1405 bytes -> ..fragmented
The largest possible non-fragmented packet is 1404  (1432 - 28 ICMP & IP headers).
You can set your MTU to 1432

I did the same test 2 days ago, setting my MTU to 1500 first, and then testing. I got 1324 and 1352.

Well, I'm going to use 1432 for a few days and see how it goes. Wish WBS would make their minds up.

If we had more iBursters doing this test & posting the results along with the base-station(s) they are likely to be connected to, we would be able see a trend. What I want to find out is whether each base-station has it's own IP router with a potentially different MTU value to other base-stations. I was under the impression that base-stations transmitted IP packets using a non-IP based proprietary protocol using microwave links between base-stations. If more iBursters participate we should be able to make an educated guess, right now it's anyone's guess.

Now, about TCPOptimizer, notice that it is pinging 63.217.30.70, and it will be measuring the MTU for the router(s) associated with 63.217.30.70. I haven't tried TCPOptimizer, but it would be better if it could test against 196.30.31.100 instead, is there a config option anywhere that allows you to use 196.30.31.100 instead of 63.217.30.70?

If you use an MTU higher than that allowed by 196.30.31.100 (or that of intermediary base-stations) then your packets will be fragmented along the way, which will affect throughput, and therefore also speed due to the increased number of ACKs.

Anyway I am not an IP expert, so don't take my word for it, I would rather Dorris or Kuberkoos commented as they are far more knowledgable than I am.

More info: http://dslnuts.com/pppoe.shtml

 

[ic]

More about RWIN/TcpWindowSize...

Another setting that will affect your connection's throughput/speed is TcpWindowSize (on Windoze boxes, not sure how/where you set it on Unix/Linux boxes). TcpWindowSize is the same as RWIN (AFAIK, Receive Window). The default value for TcpWindowSize on Windoze is usually way too small, and not optimized for "broadband" connections. You can calculate the optimal value for TcpWindowSize, but it is tricky bcos you first need to know what your latency is, and latency differs from one IP host to the next, and also if different routes are used.

So, if your are an online gamer or you have other latency dependent applications, your best bet is to measure latency "under stress" to the IP hosts that you need to optimize for.

If you have an application A that you need optimized over & abover any other applications, then measure the latency for that IP host, and then use the RWIN calculator to get the optimal TcpWindowSize for that application.

If you have applications A, B, C, .. Z and you want to optimize for all of them, you will have to measure latency for all of them, and then use the highest latency value of all the IP hosts, feed that into the RWIN calculator, and then set TcpWindowSize.

More info:
http://dslnuts.com/ping.shtml
http://www.dslnuts.com/bitsbytes.shtml

 

[Gatecrasher]

Now, about TCPOptimizer, notice that it is pinging 63.217.30.70, and it will be measuring the MTU for the router(s) associated with 63.217.30.70. I haven't tried TCPOptimizer, but it would be better if it could test against 196.30.31.100 instead, is there a config option anywhere that allows you to use 196.30.31.100 instead of 63.217.30.70?

You can use any IP address...

Pinging [196.30.31.100] with 40 bytes ->bytes=40 time=80ms TTL=255
Pinging [196.30.31.100] with 750 bytes ->bytes=750 time=264ms TTL=255
Pinging [196.30.31.100] with 1125 bytes ->bytes=1125 time=280ms TTL=255
Pinging [196.30.31.100] with 1312 bytes ->bytes=1312 time=327ms TTL=255
Pinging [196.30.31.100] with 1406 bytes -> ..fragmented
Pinging [196.30.31.100] with 1359 bytes ->bytes=1359 time=247ms TTL=255
Pinging [196.30.31.100] with 1382 bytes ->bytes=1382 time=233ms TTL=255
Pinging [196.30.31.100] with 1394 bytes ->bytes=1394 time=217ms TTL=255
Pinging [196.30.31.100] with 1400 bytes ->bytes=1400 time=249ms TTL=255
Pinging [196.30.31.100] with 1403 bytes ->bytes=1403 time=279ms TTL=255
Pinging [196.30.31.100] with 1404 bytes ->bytes=1404 time=248ms TTL=255
Pinging [196.30.31.100] with 1405 bytes -> ..fragmented
The largest possible non-fragmented packet is 1404  (1432 - 28 ICMP & IP headers).
You can set your MTU to 1432

TCPOptimizer warns that you set MTU to 1500 before running the test. I guess because you will fragment above your current MTU setting. I still think that is why you are getting 1392, while I'm getting 1432, rather than there being differences from station to station.

But I do agree it would be great if other Ibursters tried this out, and posted their results.

 

[ic]

GC, dude, I am flumaxed , I think I will just wait for more test results from other iBursters & more clued up people than myself to comment...

 

[MaD]

The joys of technology.. ahhhhh...

 

[Gatecrasher]

A minor tweak, after some research:

I bumped up my RWIN size because I had misread how to determine the ideal setting. The mistake was using a normal 32 byte ping instead of a big ping to determine latency, ie latency in the calculation is determined by sending a full data packet. This works out about 300ms local, rising to 500ms for international. I figured on an average of 420ms. So the calculation is:

420ms x 1.5 x 128K/s = 80640.

Then divide by 1392 (MSS) gives 57.93. Round up to the nearest even number = 58.

RWIN = 58 x 1392 = 80736.

Window scaling on.

It seems to have helped with browsing speed and single threaded downloads. I haven't done any other testing. I'll see how it goes.

Interestingly, With MTU at 1432 I'm only sending/requesting packets of size 1392 through the WBS server, even though the ping -f -l goes up to 1404. Hence the use of 1392, not 1404 in the RWIN calculation. However this seems to consitent with the following:

http://www.broadbandreports.com/faq/2053

 

[WickedWeasel]

Does you distance from the tower not affect the MTU ? size ? as I have seen this on normal 802.11 ?

 

[Gatecrasher]

Does you distance from the tower not affect the MTU ? size ? as I have seen this on normal 802.11 ?

It will directly affect the latency, and if your latency goes up, your "ideal" RWIN setting should rise proportionally. It has no effect on MTU.

Having the wrong MTU has the potential to cause web pages not to load and downloads to stall, but having the "wrong" RWIN will only reduce the efficiency of your connection, rather than being a cause of errors.

 

[minkukel]

hhhmm , IPCOP gave me MTU = 1432 on pppoe the last week or so , and just now after a disconnect its 1352.

 

[ic]

I have decided to leave my MTU @ 1352 until (if ever) the network "stabilises", then I will go optimizing again

 

[Gatecrasher]

hhhmm , IPCOP gave me MTU = 1432 on pppoe the last week or so , and just now after a disconnect its 1352.

Yikes. We are iburst, resistance is futile...

I don't know whether to laugh or cry. I will check when I get home in an hour or so. But it may well be that MTU varies from station to station, day to day, hour to hour, minute to minute, second to second, milli.... nah. But I wish they'd make their minds up!

 

[Mayh3m]

hhhmm , IPCOP gave me MTU = 1432 on pppoe the last week or so , and just now after a disconnect its 1352.

My IpCOP has always set it to 1432. That is the highest you can use on a pppoe connection as far as I know. It has never changed, and I don't think they can set it to change on their side