Afrihost Microsoft Connect Cache feedback

[AfriNatic]

Hi

I have been very busy building a server and deploying a server to cache microsoft content for faster delivery.

The POC cache is in with only a 20Gbps link but want to see how well it performs.

This is the content that is supposed to cache and serve. The server we deployed is in JNB (169.1.1.132 afrihost-mcc-jnb1.afrihost.co.za) feedback if you see any improvement would be great.

 

[Pulserider]

Hi

I have been very busy building a server and deploying a server to cache microsoft content for faster delivery.

The POC cache is in with only a 20Gbps link but want to see how well it performs.

This is the content that is supposed to cache and serve. The server we deployed is in JNB (169.1.1.132 afrihost-mcc-jnb1.afrihost.co.za) feedback if you see any improvement would be great.
View attachment 1788565

Do we need to do any changes on our side like DNS entries added?

 

[cavedog]

Do we need to do any changes on our side like DNS entries added?

The documentaion isn't very clear but I would assume that you need to use a DNS server that sends your IP like Afrihost DNS, Google 8.8.8.8 or Quad9 9.9.9.11 for example

 

[Rickster]

Is a one of a kind for ISP's in SA? AKA is this unique in SA?

 

[Pho3nix]

The documentaion isn't very clear but I would assume that you need to use a DNS server that sends your IP like Afrihost DNS, Google 8.8.8.8 or Quad9 9.9.9.11 for example

I’ll turn on my Xbox for some updates and advise if I finally get sustained full line speed.

Anyway to check on PC if we are hitting the cache?

 

[cavedog]

Is a one of a kind for ISP's in SA? AKA is this unique in SA?

No any ISP can host one from what I can see in the documentation.

I’ll turn on my Xbox for some updates and advise if I finally get sustained full line speed.

Anyway to check on PC if we are hitting the cache?

Best way would be to open task manager and click resource manager and under the networking tab see if you see the IP 169.1.1.132 then you know the data is from the cache.

 

[Rickster]

No any ISP can host one from what I can see in the documentation.

Sorry, I meant is this the 1st of its kind in SA that you know of?

 

[Jason-ZA]

It looks like Afrihost's DNS is still returning a fastly CDN IP, perhaps download.windowsupdate.com is a bad domain to test against, but I know some windows update content uses that domain.

 

[cavedog]

Sorry, I meant is this the 1st of its kind in SA that you know of?

Other ISPs might also be hosting it not sure but I wanted it to be deployed because of slow Xbox Pc Game pass downloads.

It looks like Afrihost's DNS is still returning a fastly CDN IP, perhaps download.windowsupdate.com is a bad domain to test against, but I know some windows update content uses that domain.

View attachment 1788620

Microsoft Connected Cache content and services endpoints

List of fully qualified domain names, ports, and associated content used by Microsoft Connected Cache.

learn.microsoft.com

The documentation lists the end points supported. I'm slightly disappointed that it seems pc game pass downloads are not triggering the cache it seems. Could also be that currently the POC server is sweating maxing the 20Gbps and I had to reduce egress a bit.

Looks like a 100G server or at least 2 x 50G servers per region is needed.

 

[Budza]

I assume it's in AH's interest to cache this and other content- popular PC games etc?

Means you don't need to get it from the international source for all users that can access the local cache.

Are there any metrics available here?

IE: How many TB of cached content served locally does it take to justify caching that content in the first place?

 

[AfriNatic]

I assume it's in AH's interest to cache this and other content- popular PC games etc?

Means you don't need to get it from the international source for all users that can access the local cache.

Are there any metrics available here?

IE: How many TB of cached content served locally does it take to justify caching that content in the first place?

Not really. Hosting a cache is a costly exercise even the "free" appliances we get from content providers. The actual machines are free but the hosting fees at Teraco are not free. We spend a lot on electricity and connectivity in the cabinets but we do that because it brings content closer to our clients and we believe the closer the content is to end users the better the experience.

Same principle for the Microsoft Connected Cache. The hardware and hosting fees are expensive and will likely be more than transit to EU which is happening now but it means our clients gets served the content from a source much closer.

We are one of the very few ISPs that have big 400Gbps+ clusters of caches from Netflix, Akamai, Google, Apple and now Microsoft in each region JHB, CPT and DBN.

 

[Xtm]

Not really. Hosting a cache is a costly exercise even the "free" appliances we get from content providers. The actual machines are free but the hosting fees at Teraco are not free. We spend a lot on electricity and connectivity in the cabinets but we do that because it brings content closer to our clients and we believe the closer the content is to end users the better the experience.

Same principle for the Microsoft Connected Cache. The hardware and hosting fees are expensive and will likely be more than transit to EU which is happening now but it means our clients gets served the content from a source much closer.

We are one of the very few ISPs that have big 400Gbps+ clusters of caches from Netflix, Akamai, Google, Apple and now Microsoft in each region JHB, CPT and DBN.

Don't you think these servers would overload and become congested on long run, should instead divert your efforts toward direct source international contents?

 

[cavedog]

It looks like Afrihost's DNS is still returning a fastly CDN IP, perhaps download.windowsupdate.com is a bad domain to test against, but I know some windows update content uses that domain.

View attachment 1788620

So, after a lot of digging and testing it seems on Windows 10 you need to enable it as it's disabled by default. Windows 11 has it enabled by default.

Settings > Delivery Optimization > Click PCs on my local network & PCs on the internet.

Now traffic will come from the ISP hosted MCC if you use DNS that sends the subnet info.

 

[Jason-ZA]

So, after a lot of digging and testing it seems on Windows 10 you need to enable it as it's disabled by default. Windows 11 has it enabled by default.

Settings > Delivery Optimization > Click PCs on my local network & PCs on the internet.

Now traffic will come from the ISP hosted MCC if you use DNS that sends the subnet info.

View attachment 1794479

From my experience, if you are on windows 11 and the setting is off, it still pulls from the MCC.

I have it disabled and my windows updates and microsoft store downloads pull from the cache.



The highlighed IP at the bottom is CISP's cache.

 

[cavedog]

From my experience, if you are on windows 11 and the setting is off, it still pulls from the MCC.

I have it disabled and my windows updates and microsoft store downloads pull from the cache.

View attachment 1794534

The highlighed IP at the bottom is CISP's cache.

View attachment 1794537

How often do you get game downloads from the cache on Win11. The testing I did only some of the store games and apps gets pulled from the cache but on my win10 pc every single game, app and update comes from the cache.

 

[Jason-ZA]

How often do you get game downloads from the cache on Win11. The testing I did only some of the store games and apps gets pulled from the cache but on my win10 pc every single game, app and update comes from the cache.

I haven't really tried many games, but noticed this mostly with apps, I first noticed it when I installed whatsapp from the store.

I also noticed that office updates install from the cache as well.

 

[cavedog]

Some stats for the last 24 hours

 

[Jason-ZA]

Some stats for the last 24 hours

View attachment 1795748

Damn, flatlining at 20gbps for a long time, are you guys going to be setting up more servers for it?

 

[cavedog]

Damn, flatlining at 20gbps for a long time, are you guys going to be setting up more servers for it?

Yes I got a server build already laying in our Noc office. 25G x 2. So JHB will have 70Gbps then will get DBN and CPT sorted.

 

[cavedog]

Uncached content is extremely slow from this cache and it's disappointing because it's only ssd and nvme drives but uncached content is served at like ~35Mbps

I think it's because when I look at traffic stats it's filling from Fastly Marseille and it's a single thread fill. After a while microsoft sees downloads are slow and stop serving that from the cache and starts throwing in their CDN's and dropping the cache connection.

I managed to bypass that a bit by adding akamai, fastly and liquid cdn ip ranges to my firewall forcing it to use the cache and cache the content.

Feels like the old usenet. Who is caching what on which server...



Funny thing is just last week Friday 60G of Avowed was cached now it's just like 10G the other portions already expired due to windows defender, office and store apps...