Charting Fibre speed over time.

[Marz]

Thanks Jacof for all the hard work. Following this thread with eager anticipation.

 

[jacof]

Paul at Cool Ideas asked me to test against their server in Randburg. I have now changed the code to only test against that server. Lets see what we get.

 

[fragtion]

Paul at Cool Ideas asked me to test against their server in Randburg. I have now changed the code to only test against that server. Lets see what we get.

You already know all of this I'm sure, but assuming you get full throughput to the new server, I suggest then testing to more/random servers afterwards as well. Perhaps even round-robin the tests, with each ISP. At the end of the day, the simple fact is that a fast line/isp isn't fast because you got good speeds to a preferential/optimized speedtest server that your ISP prescribed to you - it's fast when you get good speeds to just about ANY half-decent speedtest server that your ISP need not even know about! My reason for saying this is that I really won't be surprised to see a growing trend of ISP's who purposefully exempt OOKLA's speedtest.net servers from their shaping regimens. Of course I highly doubt that this is what CI are doing in this case, but it's something we should all be weary of.

 

[Seeyou]

... I really won't be surprised to see a growing trend of ISP's who purposefully exempt OOKLA's speedtest.net servers from their shaping regimens. Of course I highly doubt that this is what CI are doing in this case, but it's something we should all be weary of

That is already happening, at pretty much every ISP, since speedtest has become the default test of performance even though its results have almost no bearing on real world usage/performance.

 

[PBCool]

That is already happening, at pretty much every ISP, since speedtest has become the default test of performance even though its results have almost no bearing on real world usage/performance.

Yeah we don't do that at all.

 

[Seeyou]

Yeah we don't do that at all.

Not pointing at you specifically - just pointing out that it's common practice.

 

[jacof]

You already know all of this I'm sure, but assuming you get full throughput to the new server, I suggest then testing to more/random servers afterwards as well. Perhaps even round-robin the tests, with each ISP. At the end of the day, the simple fact is that a fast line/isp isn't fast because you got good speeds to a preferential/optimized speedtest server that your ISP prescribed to you - it's fast when you get good speeds to just about ANY half-decent speedtest server that your ISP need not even know about! My reason for saying this is that I really won't be surprised to see a growing trend of ISP's who purposefully exempt OOKLA's speedtest.net servers from their shaping regimens. Of course I highly doubt that this is what CI are doing in this case, but it's something we should all be weary of.

We already know the speed is all over the place if we test to the servers in Pretoria. What I did was to give it the list of servers in Pretoria and then it would find the server with the fastest ping to run the test against (the same way the web browser version does). I have to give it the list else it will pick a server in Cape Town because of the IP I get when I connect to the CI network. I agree testing to the ISPs own server does not prove your speed as we are not using the link to move data between the ISP test server and your network. I just want to see if it is stable or not. If stable then it will tell us the outbound link of the ISP is not stable.

 

[Mr Scratch]

Jaco, care to publish your script? I've built a similar one using speedtest-cli that outputs the results with a timestamp to a textfile, but graphing is not something I know how to do easily.

 

[PBCool]

We already know the speed is all over the place if we test to the servers in Pretoria. What I did was to give it the list of servers in Pretoria and then it would find the server with the fastest ping to run the test against (the same way the web browser version does). I have to give it the list else it will pick a server in Cape Town because of the IP I get when I connect to the CI network. I agree testing to the ISPs own server does not prove your speed as we are not using the link to move data between the ISP test server and your network. I just want to see if it is stable or not. If stable then it will tell us the outbound link of the ISP is not stable.

Well yes and no, if you are testing to a server on our network we have full visibility to that server. If you had to test to another ISP via our network which would then use a peering point, it is up to us to get the traffic to the peering point it is then the respective other ISPs responsibility to carry it on-wards. Also testing to random servers will give you random results. But the current test would be more relative to how the Openserve network is performing.

 

[ponder]

Jaco, care to publish your script? I've built a similar one using speedtest-cli that outputs the results with a timestamp to a textfile, but graphing is not something I know how to do easily.

https://mybroadband.co.za/vb/showth...-over-time?p=20179910&viewfull=1#post20179910

 

[Mr Scratch]

@ponder thanks!

 

[rawoke]

Very interesting thread ! +1 for publishing of script

 

[jacof]

Jaco, care to publish your script? I've built a similar one using speedtest-cli that outputs the results with a timestamp to a textfile, but graphing is not something I know how to do easily.

You can use EMONCMS.org if you don't have your own emon server on your network https://emoncms.org/


I have made some changes to the script. Please remember this script is going to eat your data LOL

import speedtest
import socket
import paho.mqtt.client as mqtt
import time
import os
import sys
import traceback
from datetime import datetime
import httplib
import psycopg2

download = 0.0
upload = 0.0
ping = 0.0
source_ip =" "
source_isp = " "
server_name = " "
server_id = " "
location = " "

# On connect Callback
def on_connect(client, userdata, flags, rc):
    print("rc: " + str(rc))


mqttc = mqtt.Client()

# Assign event callbacks
mqttc.on_connect = on_connect

# Connect
mqttc.connect("10.64.9.34", 1883,50)

#Start the mqtt loop
mqttc.loop_start()


# Write apikey of emoncms account
apikey2 = "write API key"

run = True

#Pretoria Server List
#servers = [6158,1284,7430,11580,11912,9355]
#Cool Ideas Server
servers = [6591]
#servers = []
source = "10.64.9.42"
speedtest.SOURCE = source
socket.socket = speedtest.bound_socket

while run:

    try:
               
        os.system('cls')       
      
        print("Set source ip : " + source)
        s = speedtest.Speedtest()
        source_ip = s.config['client'].get('ip')
        source_isp = s.config['client'].get('isp')

        print("Source ISP ip : " + source_ip)
        print("Source ISP : " + source_isp )

        print("Get the server list")
        s.get_servers(servers)
        print("Find the best server")
        s.get_best_server()
        print("Start download test")
        s.download()
        print("Start upload test")
        s.upload()
 
        #print (s.results)

        download = s.results.download / 1000.0 / 1000.0
        upload = s.results.upload / 1000.0 / 1000.0
        server_name = s.results.server['sponsor']
        server_id = s.results.server['id']
        location = s.results.server['name']
        ping = s.results.ping

        print("Server : " + server_name)
        print("Server id : " + server_id)
        print("Location : " + location)
        print("Ping: {0:.2f}".format(ping))
        print("Download: {0:.2f}".format(download) + " Mbit/s")
        print("Upload: {0:.2f}".format(upload) + " Mbit/s")
        
        try:
		
            #Declare the db vars
            conn = psycopg2.connect("dbname='db' user='user' host='host' password='password'")
            cur = conn.cursor()

            #Write the values to the DB
            print("Write vales to the DB")
	    cur.execute("insert into speedtest(record_timestamp , source_isp , source_ip , server_name , server_id , location , ping , download , upload) values (now(), %s , %s , %s , %s , %s  , %s , %s , %s  )" , (source_isp, source_ip, server_name , server_id ,  location , ping , download , upload ))
            conn.commit()
            cur.close()
            conn.close()
			
        except:
            traceback.print_exc()

        print("Send Values via MQTT")
        mqttc.publish("Internetspeed/Download", str(round(download , 2)))
        mqttc.publish("Internetspeed/Upload", str(round(upload , 2)))

        try:
            print("Send Values via EMONCMS")
            domain = "10.64.9.34"
            emon_conn = httplib.HTTPConnection(domain,timeout = 1)
            emon_conn.request("GET", "/emoncms/input/post.json?node=internetspeed&json={ping:" +  str(round(s.results.ping , 2)) + ",download:" +   str(round(download , 2))  + ",upload:" +   str(round(upload , 2)) + "}&apikey="+apikey2)
            emon_response = emon_conn.getresponse()

            print emon_response.read()
			
        except:
            traceback.print_exc()
			
        print("Waiting for 10 minutes")
                

        time.sleep(600)


    except KeyboardInterrupt:
        print "End"
        run = False

    except:
        traceback.print_exc()
        time.sleep(10)

 

[jacof]

I use Emon to plot my solar and water system. I have 3 of them on my network and also the emon.org one.

There is an API that you log the data with and then you build your own Dashboards to plot what ever you want.

This is my emoncms.org Dashboard.

http://emoncms.org/JacoFourie/Power

 

[jacof]

Well yes and no, if you are testing to a server on our network we have full visibility to that server. If you had to test to another ISP via our network which would then use a peering point, it is up to us to get the traffic to the peering point it is then the respective other ISPs responsibility to carry it on-wards. Also testing to random servers will give you random results. But the current test would be more relative to how the Openserve network is performing.

Paul I will let this test run today and then tomorrow I will pick 1 server that I will test against from the CI network and the VOX network that is not on ether's network and compare the 2. Then we have everything constant accept the ISP. But I can show you on the database side that most of the tests happened against the Cell C server. From VOX and from the CI side. So that proves this point already to some degree. As this test server was chosen most of the time based on the ping. Only once was another Pretoria server selected.

 

[PBCool]

Paul I will let this test run today and then tomorrow I will pick 1 server that I will test against from the CI network and the VOX network that is not on ether's network and compare the 2. Then we have everything constant accept the ISP. But I can show you on the database side that most of the tests happened against the Cell C server. From VOX and from the CI side. So that proves this point already to some degree. As this test server was chosen most of the time based on the ping. Only once was another Pretoria server selected.

Cool thanks for the feedback.

 

[jacof]

OK so I got my Call Ref from ICASA. I would suggest everybody who has speed issues log a complaint.

 

[Seeyou]

I have a feeling they will fall back on their ever present "best effort service" clause.

 

[PBCool]

Paul I will let this test run today and then tomorrow I will pick 1 server that I will test against from the CI network and the VOX network that is not on ether's network and compare the 2. Then we have everything constant accept the ISP. But I can show you on the database side that most of the tests happened against the Cell C server. From VOX and from the CI side. So that proves this point already to some degree. As this test server was chosen most of the time based on the ping. Only once was another Pretoria server selected.

Still something on the OS network side as capacity to that server isn't an issue:

 

[jacof]

I have a feeling they will fall back on their ever present "best effort service" clause.

If only I complain then probably yes. But if everybody logs a complaint at ICASA they have to do something. Can Mybroadband not do something about it. I thought that is why the site is here?