Broadband speed tests: Speedtest.net concerns in SA

I just ran a speedtest now and checked the URLs... they're something like
http://speedtest.dne.vodacombusiness.co.za/speedtest/latency.txt?x=1298841160857
http://speedtest.dne.vodacombusiness.co.za/speedtest/random350x350.jpg?x=1298841163345-1
http://speedtest.dne.vodacombusiness.co.za/speedtest/random1500x1500.jpg?x=1298841164201-1

Cybersmart needs to flick their cache administrator on the ear - if they're ignoring parameters in their caching, then it's Cybersmart's problem... not speedtest.net. This will cause UNTOLD issues on SO many ajax-type websites. I suspect they're ignoring parameters on data types they deem to be static, ie. the jpg's - I can understand how a newbie cache admin would think this is right, but it's not. A lot of analytics programs, including google analytics, use a dummy .gif or .jpg with parameters, and the badly configured cache will be robbing these sites of the hits. This practice goes back as far as I can remember, it's pretty common practice now.

According to Fialkov the reason for the random image being cacheable may be because the image itself does not change sufficiently to avoid caching.
Click to expand...

Umm... wut? Content is irrelevant. You'd only know if the content changed if you download it, decoded both the old and new version, compared it to the old version with a complex algorithm, then served the old version up. The whole point of caching is that the image isn't retrieved from the source in the first place!

Fialkov added that their speedtest.net results improved by 15% after employing their caching servers, a sign that caching has a direct influence on these results.
Click to expand...

No. it's a sign they need to fix their cache server configuration!
 
Failkov explained that they tested the caching of Speedtest.net images, and found that they served the ‘random Speedtest.net images’ from their cache multiple times in an hour.
Click to expand...

That typo has strangely become quite common in articles.. :p
 
I, too, thought this was pretty hilarious and saw it coming as soon as I started reading the article.

For shame, cybersmart.
 
:/

Mmm I dnt understand . Are they saying that me doing a speedtest to the WA server in cpt is going to be faster coz the file is on the server?
I have done a few speed tests and most of them were slower than what i usualy get while downloading ,which is 3.1 mb/s.

Which might also be seen as a problem as im downloading faster int than local?

Im using a 3.6 VodaModem
 
The Cybersmart clowns are still in business? Amazing.

Laurie, if your proxy servers are caching content marked non-cachable, they are configured incorrectly.

There are statistical methods for excluding outlier data (e.g. van Zyl's argument), speedtest.net should employ those.

Speedtest.net data also suffers another major problem: selection bias. The average tech nerd salivating over his new 10Mbit ADSL connection is much more likely to visit speedtest.net in the first place than grandma with her 384 line, for example. Their data should be taken with more than a gigantic rock of salt.


ChrisGeo said:
Mmm I dnt understand . Are they saying that me doing a speedtest to the WA server in cpt is going to be faster coz the file is on the server?
Click to expand...

What Van Zyl is saying is that if you, for example, work in the WA datacenter or are networked close to an exchange where WA peers, that you could log a speedtest at speeds that are insanely higher than an actual WA customer will ever get, e.g. hundreds of Mbits/sec, and when Speedtest averages test values these will skew the results.

What Failkov is saying is that if the file (for say an overseas test) is cached on a WA proxy, then you as a customer might download the file at local speeds because the file is on the WA proxy server. However this is unlikely to happen to anyone, and usually implies an incorrect caching setup.
 
Last edited:
Assume the analysis is correct:
- 15% is not much (better boosts can be obtained by using multi-segmentation tools)
- A 1 Gb/s speedtest is averaged out to below 10 Mb/s within 100 tests.
- Speedtest.net does not average per ISP but averages out per advertised prefix. When you cycle through the address pools different averages are presented. Core equipment would be in a common prefix.

Allowing techies unrestricted throughput off core equipment is questionable - they should be rate limited as every time they do such a test subscribers are impacted - thus not speedtest.net's problem.

Can you publish how these cacheable results were obtained and what metric and measurement was used? Something does not add up? e.g. all the ads and flash on the site is cacheable?
 
Another problem I noticed, is that both my IP and ISP are IS, but the actual connection tested is SAIX. I'm route splitting, so this makes sense - the "speedtest.net" server is international, so it's routed over my blended account.

I'm not sure how much of this is happening, but speedtest should get it's ISP info from the actual test, not the website loading.
 
“If we look at the logs of one of the Vodacom speedtest.net servers, the average speed of the top 100 entries of a local mobile network is 80Mbps, with the top entry in excess of 400Mbps! These speeds are added to the ISP results and the end result does not reflect a single access network anymore,” said Van Zyl.
Click to expand...
How much of this is done with the sole purpose to inflate the results?

If I want to use speedtest.net to show I've got the fastest network, all I have to do is get my engineering department to run a backbone speedtest a few times a day. And suddenly I go from 3Mb/s to 5Mb/s.
 
Averages? Who cares, I use it to see if my line does with it is supposed to do. I could not care less for some punk sitting at UCT getting 80mbps.
 
As the Administrator of that 'midrand server' I should mention:

What is often forgotten is that even 'techies' use speedtest.net for assessing network performance.

In this case it was to test a peering link that was brought online. If Vodacom peered publically they'd also be able to attain those speeds to the 'midrand server'.

;-)
 
I use pingtest.net more than speedtest, anyone know whether this article has bearing on pingtest accuracy?
Been having issues with IS bandwidth lately so I use it mainly to check for packet loss.
 
regardtv said:
As the Administrator of that 'midrand server' I should mention:

What is often forgotten is that even 'techies' use speedtest.net for assessing network performance.

In this case it was to test a peering link that was brought online. If Vodacom peered publically they'd also be able to attain those speeds to the 'midrand server'.

;-)
Click to expand...

Would also be nice to hear your view re: cached content as I'm inclined to agree with elvis_presley and others that mentioned Cybersmart's possible configuration mistake that allows for this to happen
 
r00igev@@r said:
- A 1 Gb/s speedtest is averaged out to below 10 Mb/s within 100 tests.
Click to expand...
The effect is actually substantial. Let's use a real-world, practical example from one of the logs to show how results can be skewed.

A local broadband provider operates a national wireless consumer network that can deliver a maximum speed of 1Mb/s.

In the speedtest.net server log there are currently 6994 entries with an average download speed of 828Kb/s. On the surface this would appear as delivering more than 80% of maximum speed.

However, if you look at the logs, it seems that 584 results log speeds in excess of 1Mb/s with the highest speed 144Mb/s and the average of the top 100 entries at 14.7Mb/s. Clearly these speed could not have been measured from the radio network.

If one removes all speeds above 1Mb/s, to be left with the 6410 results, the average download speed now drops to 522Kb/s.

So, what appears on the surface to be a 828K/s 'averaged network' actually is a 522Kb/s one.

A small number (8%) of the entries pushed the average speed up by a massive margin (58%).

Again, it is important to note the numbers are not wrong. But if some use the Ookla numbers and they tend to think of the specific network as just a 1Mb/s access network, they'll believe the average performance is much higher than it actually is.
 
Last edited:
r00igev@@r said:
So which one are you talking about? Vodacom or MTN?
Click to expand...
Ok, I'll bite. :)

Maximum radio speeds on local mobile broadband providers:

Vodacom - 42Mb/s (A few LTE sites @ 150Mb/s are up)
MTN - 42Mb/s
Cell C - 21Mb/s
Telkom - 7.2/14.4Mb/s (I'm actually not sure on this one, but it's at least this.)
Neotel - 3.1Mb/s
iBurst - 1Mb/s

So which network do you think I referred? ;)
 
jannievanzyl said:
Ok, I'll bite. :)

Maximum radio speeds on local mobile broadband providers:

Vodacom - 42Mb/s (A few LTE sites @ 150Mb/s are up)
MTN - 42Mb/s
Cell C - 21Mb/s
Telkom - 7.2/14.4Mb/s (I'm actually not sure on this one, but it's at least this.)
Neotel - 3.1Mb/s
iBurst - 1Mb/s

So which network do you think I referred? ;)
Click to expand...

Didn't someone run to the ASA complaining about those speeds being used as a metric?

http://www.speedtest.net/result/1177580227.png

Speedtest.net does not report iBurst as an ISP and the ISP that it does report has more than a thousand pairs of radios capable of true sustained broadband speeds that does equate to the averages you mentioned earlier.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X