Cool Ideas Fibre ISP – Feedback Thread 4

Status
Not open for further replies.
Tinuva

Tinuva

The Magician
Joined
Feb 10, 2005
Messages
12,475
Seeyou said:
Same issue on a 155.xxx address as a 102.xxx address. I suspect an arbitrary upload rate throttle is in place somewhere on Google's side for whatever reason.
Click to expand...
Changing IP wont change google drive performance.

You need to have googleapis.com resolve to a different cluster. One way to do that, is using different DNS servers.

I have a config in adguard home to do this:
Code: 
[/googleapis.com/]8.20.247.20
[/googleapis.com/]8.26.56.26

What this does, all *.googleapis.com requests goes to the dns servers on 8.26.56.26 and 8.20.247.20

This causes my google drive traffic to use AMS instead of JNB.

Example time.

Local 1st:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @1.1.1.1
216.58.223.144

~ % mtr 216.58.223.144 --report-wide -c4
Start: 2023-01-27T12:21:15+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- router-vlan20.heaven.za.net   0.0%     4    0.8   1.7   0.5   4.9   2.2
  2.|-- ujk1-cust.coolideas.co.za     0.0%     4    2.9   3.2   1.9   4.5   1.1
  3.|-- um5g-cust.coolideas.co.za     0.0%     4    3.9  12.0   3.9  32.7  13.9
  4.|-- c3h-backbone.coolideas.co.za  0.0%     4    1.7   5.0   1.7  14.4   6.3
  5.|-- 100.98.0.3                    0.0%     4    1.8   3.2   1.8   5.4   1.7
        100.98.0.1
  6.|-- 100.98.1.19                   0.0%     4   19.9  23.4  18.7  28.4   4.8
        100.98.1.1
        100.98.1.2
  7.|-- 100.99.1.250                  0.0%     4   24.9  22.1  19.1  24.9   2.9
        100.99.0.235
  8.|-- 196-60-9-113.ixp.joburg       0.0%     4   21.3  23.2  20.2  30.7   5.1
  9.|-- 172.253.65.181                0.0%     4   20.1  20.1  19.7  20.7   0.4
 10.|-- 172.253.65.255                0.0%     4   19.8  19.9  18.8  21.0   0.9
 11.|-- jnb02s01-in-f16.1e100.net     0.0%     4   18.4  18.7  18.0  20.0   0.9

Using the above dns:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @8.26.56.26
216.58.212.240

~ % mtr 216.58.212.240 --report-wide -c4
Start: 2023-01-27T12:22:29+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- router-vlan20.heaven.za.net   0.0%     4    0.8   1.2   0.5   3.1   1.2
  2.|-- ujk1-cust.coolideas.co.za     0.0%     4    2.0   2.9   2.0   5.8   1.9
  3.|-- um5g-cust.coolideas.co.za    75.0%     4    5.4   5.4   5.4   5.4   0.0
  4.|-- c3h-backbone.coolideas.co.za  0.0%     4    1.8   6.3   1.7  19.9   9.1
  5.|-- 100.98.0.3                    0.0%     4    2.5   2.6   1.9   3.9   0.9
  6.|-- 100.98.1.17                   0.0%     4   20.1  19.2  17.6  20.1   1.1
        100.98.1.3
        100.98.1.1
  7.|-- 100.99.0.235                  0.0%     4   20.8  19.4  17.6  20.8   1.4
        100.99.1.250
  8.|-- 196-60-9-113.ixp.joburg       0.0%     4   19.3  20.8  18.6  25.0   2.9
  9.|-- 74.125.245.209                0.0%     4   20.6  19.7  18.6  20.6   0.9
 10.|-- 74.125.245.214                0.0%     4   18.9  19.3  18.8  20.4   0.7
 11.|-- 209.85.250.190                0.0%     4   18.4  26.1  18.0  50.0  15.9
 12.|-- 142.251.236.115               0.0%     4  172.7 173.0 172.3 174.1   0.8
 13.|-- 108.170.246.161               0.0%     4  176.1 176.7 173.4 179.6   2.6
 14.|-- 216.239.41.241                0.0%     4  173.6 176.5 173.1 183.1   4.6
 15.|-- ams16s22-in-f16.1e100.net     0.0%     4  171.7 174.6 171.7 180.5   4.0

Bypassing the horrible JNB cluster. The performance in JNB sucks, sorry to say it.
 
PBCool

PBCool

Cool Ideas Rep
Company Rep
Joined
Jan 11, 2016
Messages
13,304
Seeyou said:
Same issue on a 155.xxx address as a 102.xxx address. I suspect an arbitrary upload rate throttle is in place somewhere on Google's side for whatever reason.
Click to expand...
Yeah then it's related to profile more than likely which is arb.
 
PBCool

PBCool

Cool Ideas Rep
Company Rep
Joined
Jan 11, 2016
Messages
13,304
Tinuva said:
Changing IP wont change google drive performance.

You need to have googleapis.com resolve to a different cluster. One way to do that, is using different DNS servers.

I have a config in adguard home to do this:
Code: 
[/googleapis.com/]8.20.247.20
[/googleapis.com/]8.26.56.26

What this does, all *.googleapis.com requests goes to the dns servers on 8.26.56.26 and 8.20.247.20

This causes my google drive traffic to use AMS instead of JNB.

Example time.

Local 1st:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @1.1.1.1
216.58.223.144

~ % mtr 216.58.223.144 --report-wide -c4
Start: 2023-01-27T12:21:15+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa Loss% Snt Last Avg Best Wrst StDev
1.|-- router-vlan20.heaven.za.net 0.0% 4 0.8 1.7 0.5 4.9 2.2
2.|-- ujk1-cust.coolideas.co.za 0.0% 4 2.9 3.2 1.9 4.5 1.1
3.|-- um5g-cust.coolideas.co.za 0.0% 4 3.9 12.0 3.9 32.7 13.9
4.|-- c3h-backbone.coolideas.co.za 0.0% 4 1.7 5.0 1.7 14.4 6.3
5.|-- 100.98.0.3 0.0% 4 1.8 3.2 1.8 5.4 1.7
100.98.0.1
6.|-- 100.98.1.19 0.0% 4 19.9 23.4 18.7 28.4 4.8
100.98.1.1
100.98.1.2
7.|-- 100.99.1.250 0.0% 4 24.9 22.1 19.1 24.9 2.9
100.99.0.235
8.|-- 196-60-9-113.ixp.joburg 0.0% 4 21.3 23.2 20.2 30.7 5.1
9.|-- 172.253.65.181 0.0% 4 20.1 20.1 19.7 20.7 0.4
10.|-- 172.253.65.255 0.0% 4 19.8 19.9 18.8 21.0 0.9
11.|-- jnb02s01-in-f16.1e100.net 0.0% 4 18.4 18.7 18.0 20.0 0.9

Using the above dns:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @8.26.56.26
216.58.212.240

~ % mtr 216.58.212.240 --report-wide -c4
Start: 2023-01-27T12:22:29+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa Loss% Snt Last Avg Best Wrst StDev
1.|-- router-vlan20.heaven.za.net 0.0% 4 0.8 1.2 0.5 3.1 1.2
2.|-- ujk1-cust.coolideas.co.za 0.0% 4 2.0 2.9 2.0 5.8 1.9
3.|-- um5g-cust.coolideas.co.za 75.0% 4 5.4 5.4 5.4 5.4 0.0
4.|-- c3h-backbone.coolideas.co.za 0.0% 4 1.8 6.3 1.7 19.9 9.1
5.|-- 100.98.0.3 0.0% 4 2.5 2.6 1.9 3.9 0.9
6.|-- 100.98.1.17 0.0% 4 20.1 19.2 17.6 20.1 1.1
100.98.1.3
100.98.1.1
7.|-- 100.99.0.235 0.0% 4 20.8 19.4 17.6 20.8 1.4
100.99.1.250
8.|-- 196-60-9-113.ixp.joburg 0.0% 4 19.3 20.8 18.6 25.0 2.9
9.|-- 74.125.245.209 0.0% 4 20.6 19.7 18.6 20.6 0.9
10.|-- 74.125.245.214 0.0% 4 18.9 19.3 18.8 20.4 0.7
11.|-- 209.85.250.190 0.0% 4 18.4 26.1 18.0 50.0 15.9
12.|-- 142.251.236.115 0.0% 4 172.7 173.0 172.3 174.1 0.8
13.|-- 108.170.246.161 0.0% 4 176.1 176.7 173.4 179.6 2.6
14.|-- 216.239.41.241 0.0% 4 173.6 176.5 173.1 183.1 4.6
15.|-- ams16s22-in-f16.1e100.net 0.0% 4 171.7 174.6 171.7 180.5 4.0

Bypassing the horrible JNB cluster. The performance in JNB sucks, sorry to say it.
Click to expand...
Yeah good idea, we don't manage how the clusters work obviously. All we know is they currently peak at about 40% capacity.

And then transit wise they access the peer fabric to Google the same way you would.
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
Tinuva said:
Changing IP wont change google drive performance.

You need to have googleapis.com resolve to a different cluster. One way to do that, is using different DNS servers.

I have a config in adguard home to do this:
Code: 
[/googleapis.com/]8.20.247.20
[/googleapis.com/]8.26.56.26

What this does, all *.googleapis.com requests goes to the dns servers on 8.26.56.26 and 8.20.247.20

This causes my google drive traffic to use AMS instead of JNB.

Example time.

Local 1st:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @1.1.1.1
216.58.223.144

~ % mtr 216.58.223.144 --report-wide -c4
Start: 2023-01-27T12:21:15+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa  Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- router-vlan20.heaven.za.net   0.0%     4    0.8   1.7   0.5   4.9   2.2
  2.|-- ujk1-cust.coolideas.co.za     0.0%     4    2.9   3.2   1.9   4.5   1.1
  3.|-- um5g-cust.coolideas.co.za     0.0%     4    3.9  12.0   3.9  32.7  13.9
  4.|-- c3h-backbone.coolideas.co.za  0.0%     4    1.7   5.0   1.7  14.4   6.3
  5.|-- 100.98.0.3                    0.0%     4    1.8   3.2   1.8   5.4   1.7
        100.98.0.1
  6.|-- 100.98.1.19                   0.0%     4   19.9  23.4  18.7  28.4   4.8
        100.98.1.1
        100.98.1.2
  7.|-- 100.99.1.250                  0.0%     4   24.9  22.1  19.1  24.9   2.9
        100.99.0.235
  8.|-- 196-60-9-113.ixp.joburg       0.0%     4   21.3  23.2  20.2  30.7   5.1
  9.|-- 172.253.65.181                0.0%     4   20.1  20.1  19.7  20.7   0.4
 10.|-- 172.253.65.255                0.0%     4   19.8  19.9  18.8  21.0   0.9
 11.|-- jnb02s01-in-f16.1e100.net     0.0%     4   18.4  18.7  18.0  20.0   0.9

Using the above dns:
Code: 
~ % dig chromedriver.storage.googleapis.com +short @8.26.56.26
216.58.212.240

~ % mtr 216.58.212.240 --report-wide -c4
Start: 2023-01-27T12:22:29+0200
HOST: 909c4ace21ef.xxx.aaaaaaaaaa Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- router-vlan20.heaven.za.net   0.0%     4    0.8   1.2   0.5   3.1   1.2
  2.|-- ujk1-cust.coolideas.co.za     0.0%     4    2.0   2.9   2.0   5.8   1.9
  3.|-- um5g-cust.coolideas.co.za    75.0%     4    5.4   5.4   5.4   5.4   0.0
  4.|-- c3h-backbone.coolideas.co.za  0.0%     4    1.8   6.3   1.7  19.9   9.1
  5.|-- 100.98.0.3                    0.0%     4    2.5   2.6   1.9   3.9   0.9
  6.|-- 100.98.1.17                   0.0%     4   20.1  19.2  17.6  20.1   1.1
        100.98.1.3
        100.98.1.1
  7.|-- 100.99.0.235                  0.0%     4   20.8  19.4  17.6  20.8   1.4
        100.99.1.250
  8.|-- 196-60-9-113.ixp.joburg       0.0%     4   19.3  20.8  18.6  25.0   2.9
  9.|-- 74.125.245.209                0.0%     4   20.6  19.7  18.6  20.6   0.9
 10.|-- 74.125.245.214                0.0%     4   18.9  19.3  18.8  20.4   0.7
 11.|-- 209.85.250.190                0.0%     4   18.4  26.1  18.0  50.0  15.9
 12.|-- 142.251.236.115               0.0%     4  172.7 173.0 172.3 174.1   0.8
 13.|-- 108.170.246.161               0.0%     4  176.1 176.7 173.4 179.6   2.6
 14.|-- 216.239.41.241                0.0%     4  173.6 176.5 173.1 183.1   4.6
 15.|-- ams16s22-in-f16.1e100.net     0.0%     4  171.7 174.6 171.7 180.5   4.0

Bypassing the horrible JNB cluster. The performance in JNB sucks, sorry to say it.
Click to expand...

Tried it using a similar setup in Pihole. Unfortunately, performance just completely tanked - barely 1MB/sec. Perhaps a combination of my line and some limit on Google's side, even though I can saturate an upload speed test.
 
cavedog

cavedog

Honorary Master
Joined
Oct 19, 2007
Messages
22,657
You need to use the flag --drive-chunk-size 256M

If you don't the chink sizes are too small causing slow speeds
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
cavedog said:
You need to use the flag --drive-chunk-size 256M

If you don't the chink sizes are too small causing slow speeds
Click to expand...
Yeah went down that rabbit hole on google. Tried chunk sizes from 128M all the way up to 1G. Speeds it up a bit when it starts, but always drops right back down to around 30-40mbps. Caching options for mounts also ineffective.
 
PBCool

PBCool

Cool Ideas Rep
Company Rep
Joined
Jan 11, 2016
Messages
13,304
Seeyou said:
Yeah went down that rabbit hole on google. Tried chunk sizes from 128M all the way up to 1G. Speeds it up a bit when it starts, but always drops right back down to around 30-40mbps. Caching options for mounts also ineffective.
Click to expand...
Are you able to test with more than one device possibly?
 
cavedog

cavedog

Honorary Master
Joined
Oct 19, 2007
Messages
22,657
Seeyou said:
Yeah went down that rabbit hole on google. Tried chunk sizes from 128M all the way up to 1G. Speeds it up a bit when it starts, but always drops right back down to around 30-40mbps. Caching options for mounts also ineffective.
Click to expand...

Okay I think you need to look closer at your setup or line. This is single threaded on ipv4 so Google drive is fine. You can try changing the www.googleapis.com IP by specifying it in the host file but I don't think Google is your issue.

fdhfdghfg.png
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
PBCool said:
Are you able to test with more than one device possibly?
Click to expand...

cavedog said:
Okay I think you need to look closer at your setup or line. This is single threaded on ipv4 so Google drive is fine. You can try changing the www.googleapis.com IP by specifying it in the host file but I don't think Google is your issue.

View attachment 1464985
Click to expand...

Ok that's interesting - installed rclone on my development machine which is running Ubuntu, and that maxes out upload at 200mbps. The machine struggling is a Windows 10 machine on the same network which has no problems with download/upload/hardware. This is a speedtest result on it:

 
PBCool

PBCool

Cool Ideas Rep
Company Rep
Joined
Jan 11, 2016
Messages
13,304
Seeyou said:
Ok that's interesting - installed rclone on my development machine which is running Ubuntu, and that maxes out upload at 200mbps. The machine struggling is a Windows 10 machine on the same network which has no problems with download/upload/hardware. This is a speedtest result on it:

Click to expand...
Could be window scaling setting on that windows box.
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
PBCool said:
Could be window scaling setting on that windows box.
Click to expand...

Code: 
TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State          : enabled
Receive Window Auto-Tuning Level    : normal
Add-On Congestion Control Provider  : default
ECN Capability                      : disabled
RFC 1323 Timestamps                 : disabled
Initial RTO                         : 1000
Receive Segment Coalescing State    : enabled
Non Sack Rtt Resiliency             : disabled
Max SYN Retransmissions             : 4
Fast Open                           : enabled
Fast Open Fallback                  : enabled
HyStart                             : enabled
Proportional Rate Reduction         : enabled
Pacing Profile                      : off

Anything look out of order there?
 
jannier

jannier

Expert Member
Joined
Jul 31, 2005
Messages
2,075
Seeyou said:
Code: 
TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State          : enabled
Receive Window Auto-Tuning Level    : normal
Add-On Congestion Control Provider  : default
ECN Capability                      : disabled
RFC 1323 Timestamps                 : disabled
Initial RTO                         : 1000
Receive Segment Coalescing State    : enabled
Non Sack Rtt Resiliency             : disabled
Max SYN Retransmissions             : 4
Fast Open                           : enabled
Fast Open Fallback                  : enabled
HyStart                             : enabled
Proportional Rate Reduction         : enabled
Pacing Profile                      : off

Anything look out of order there?
Click to expand...

Mine; Only difference I see it "RFC 1323 Timestamps"

Code: 
TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State          : enabled
Receive Window Auto-Tuning Level    : normal
Add-On Congestion Control Provider  : default
ECN Capability                      : disabled
RFC 1323 Timestamps                 : allowed
Initial RTO                         : 1000
Receive Segment Coalescing State    : enabled
Non Sack Rtt Resiliency             : disabled
Max SYN Retransmissions             : 4
Fast Open                           : enabled
Fast Open Fallback                  : enabled
HyStart                             : enabled
Proportional Rate Reduction         : enabled
Pacing Profile                      : off
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
jannier said:
Mine; Only difference I see it "RFC 1323 Timestamps"

Code: 
TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State          : enabled
Receive Window Auto-Tuning Level    : normal
Add-On Congestion Control Provider  : default
ECN Capability                      : disabled
RFC 1323 Timestamps                 : allowed
Initial RTO                         : 1000
Receive Segment Coalescing State    : enabled
Non Sack Rtt Resiliency             : disabled
Max SYN Retransmissions             : 4
Fast Open                           : enabled
Fast Open Fallback                  : enabled
HyStart                             : enabled
Proportional Rate Reduction         : enabled
Pacing Profile                      : off
Click to expand...

Thanks - tried the RFC1323 setting but made no difference. My dev PC in Windows 11 doesn't go quite as fast as Ubuntu, but still outperforms this other PC too. Tried a different NIC, different switch, no switch, different cable, same thing. Quite confounding.
 
Last edited:
PBCool

PBCool

Cool Ideas Rep
Company Rep
Joined
Jan 11, 2016
Messages
13,304
Seeyou said:
Thanks - tried the RFC1323 setting but made no difference. My dev PC in Windows 11 doesn't go quite as fast Ubuntu, but still outperforms this other PC too. Tried a different NIC, different switch, no switch, different cable, same thing. Quite confounding.
Click to expand...
Defo something to do with the way that device is scaling its window size would be my thoughts.
 
T

tardomatic

Well-Known Member
Joined
May 3, 2006
Messages
305
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
tardomatic said:
Windows provides an API for filtering IP traffic called WPF (https://learn.microsoft.com/en-us/w...stem/net-sub-performance-tuning-nics#bkmk_wfp) - Virus scanners / 3rd party firewalls / hypervisors etc use this API for packet filtering. Some of these have an impact on throughput. Try uninstall 3rd party virus scanners, firewalls, WinPCAP devices (Wireshark), Hyper-V virtual nics and see if that helps
Click to expand...

None of that running on this machine. It's somehow restricted to Google Drive, as a local speed test saturates 200mbps on the same machine.
 
S

Seeyou

Expert Member
Joined
May 1, 2007
Messages
2,705
OneDrive saturates the upload with the default desktop app and costs exactly the same. I'm just going to cancel Google One and sub to OneDrive. Thanks to everyone who pitched in with ideas.
 
CrypticZA

CrypticZA

Expert Member
Joined
Sep 21, 2019
Messages
3,045
afrihost.co.za.png
Is this packet loss on your gusy side @PBCool ?

I know it shows as a little but i ran this in my Dota 2 game since yesterday and today saw massive spikes going to JHB wanted to avoid using 8.8.8.8 so found Afrihosts site hosted in JHB
 
Status
Not open for further replies.
Top