Understanding the FTTH ISP business model

J

JFT96

Well-Known Member
Joined
May 17, 2016
Messages
153
Reaction score
62
Location
Pampoenkraal
Can anyone refer me to any articles or forum posts explaining how the ISP / Fibre infrastructure business model works?
What I’m trying to understand is what costs are associated with who in the FTTH business environment.

Here are my specific questions/queries in case anyone can answer them specifically:

In my case, I use Afrihost as my ISP and Vuma as my fibre infrastructure provider.
  1. When looking at performance issues experienced by people on this forum, they differentiate between local and international traffic. What does this actually mean? How does using a VPN influence better performance?
  2. Are there any fees associated with the ISP to use what we call the local network? So when testing on speedtest.net, we always get the exact line speed being paid for but not the case when testing internationally. Also not always the case when testing traffic from CPT to JHB. Surely all local traffic travels over fibre networks between ISP's?
  3. Who provides and pays for international traffic? Is it a capacity-based model where, in my case, Afrihost buys capacity on one or more of the international cables (WACS, BRICS, Seacom, Google etc). Or is it a case of Vuma knowing that all my international traffic is to be charged to Afrihost because I chose Afrihost as my ISP in the Vuma "shop" and Vuma buys the international capacity and routes my traffic? So one could say some infrastructure providers are better than others for performance rather than saying one ISP is better than the other for international performance.
  4. I pay Afrihost R1197 per month for 200/200. Let' say I were to stream FHD video content internationally 24/7, who picks up that bill? What would their costs be? (I know I would likely be flagged for exceeding fair use policies but if I were not flagged, who carries the cost?

Reasons for these questions:


I’ve been around the block with a few ISP’s in my FTTH lifetime, currently I'm with Afrihost and very happy with the service. But this is not so for many of my friends and family who are not on Vuma / Afrihost. They have chosen other combinations of FTTH and consistently experience very different performance levels. Of course service and support across the board seems to be consistently pathetic (and that includes Afrihost / Vuma).


But here is what I’m getting at when it comes to the FTTH business model.


If ISP A is in fact liable for costs associated with my personal international traffic preferences, it would make business sense for them to shape and throttle my traffic if I am within the top 10 users of international traffic. As in my scenario above, excessive traffic usage must be a serious business risk. So who monitors this, the ISP or Vuma?

traceroute.jpg
Above image is the traceroute of my traffic when streaming my IPTV. I have no idea where traffic moves from Afrihost to other traffic providers. I'm assuming 10.189.30.10 is where the international traffic starts. Is 196.25.26.65 Vuma? (I'll google that later EDIT: osnet is SAIX).

Let’s look at an example. I use an internationally based IPTV streaming service which offers loads of great high-quality content. Currently, on Afrihost / Vuma I can stream most FHD content over fibre, including live TV and all sport even on Wifi. However, I have family and friends on other combinations of networks who cannot stream even SD quality content on the same IPTV service as mine without constant buffering, even when connected directly to the CPE over ethernet. Why is that? My thinking is that this is a well-optimised traffic shaping implementation of cost management by the ISP or infrastructure provider. Of course, the ISP always denies shaping and leaves it to the customer to prove inadequate performance, which for most (even on this forum) is pretty difficult to do.

Please see this post as the general question basically being why is there such a massive variance of performance quality across different ISP’s and different networks? I also apologise upfront for any misunderstanding and incorrect assumptions. I’m trying to understand and willing to learn from anyone with better insight than me.
 
Last edited:
No simple answer but to simplify:

Vumatel or any fibre provider provide the "plumbing" from a home to a central location or pop. In most circumstances it is a Teraco datacentre.

From there it is handed over to a particular ISP, how that ISP interfaces with other "local" providers is typically via a peering location or fabric. In SA the two major facilities for this are dubbed NAP Africa or an INX. IE traffic between us (Cool Ideas) to Afrihost would traverse said fabric. Then return to an FNO if it were to be traffic from ISP to ISP.

Beyond that internationally most ISPs pickup what's called transit from "carrier" networks, which then get capacity on a selection of undersea cables.

These carriers do exactly the same thing as above with our local peering but would buy undersea capacity to London or Marseille and then interconnect to peering fabrics there or transit services there, and so on and so forth.

The majority of impact on an international service throughput wise is packet loss and what's called a TCP window and scaler.

If you pickup loss *anywhere* along a path it will affect performance in a drastic way on higher latency traffic (like your IPTV), which could be wireless related, fibre last mile, router, ISP, an interconnect, congestion etc etc

Basically there is no simple answer to your question as it's very dependant, typically not just ISP A is good and ISP B is crap. But that can also apply considering there are hundreds of ISPs.
 
Last edited:
Just another note at scale it's cheaper to buy capacity vs implement systems to shape traffic these days. It's almost always actually loss related.
 
PBCool said:
Just another note at scale it's cheaper to buy capacity vs implement systems to shape traffic these days. It's almost always actually loss related.
Click to expand...
Thank you @PBCool. Much appreciated.

Just a comment on the response above.

I've always maintained that Afrihost and Cool Ideas were (and to date remain) the two ISP's with whom I've had the best performance regarding IPTV streaming. From your response, I take it that the answer to my question would be that yours and Afrihosts network implementation and monitoring is just a lot better than the other ISP's in avoiding packet loss.
 
Packet loss on the FNO affects all ISPs in a similar manner. The causation is down to faulty equipment or poor plant maintenance.

Another big factor impacting experience is what is generically know as bufferbloat. This is down to equipment choice. The best end user equipment uses either fq_codel / CAKE.

ISPs generically select cheap equipment to provide for free that lacks the ability to mitigate bufferbloat. The higher the end point device count for a site the worse the experience.
 
PBCool said:
Vumatel or any fibre provider provide the "plumbing" from a home to a central location or pop. In most circumstances it is a Teraco datacentre.
Click to expand...
More on the actual physical network side. Let's say MFN, right. I can see my area's GPON splitter cabinet. Does MFN themselves run a line to the Teraco building which is 12.5KM away line of sight, or would they pay DarkFibre Africa to connect to their backhaul that's literally less than a 200 metres down the road, and use that to get to the Teraco building in Riverhorse Valley, Durban, let's say.
 
@PBCool Related to your capacity vs shaping, does CI monitor total usage at all? I’m curious about my usage at times, but I see no way to monitor it on the client zone.
 
Vorastra said:
More on the actual physical network side. Let's say MFN, right. I can see my area's GPON splitter cabinet. Does MFN themselves run a line to the Teraco building which is 12.5KM away line of sight, or would they pay DarkFibre Africa to connect to their backhaul that's literally less than a 200 metres down the road, and use that to get to the Teraco building in Riverhorse Valley, Durban, let's say.
Click to expand...

They would have a POP nearby with OLTs/Equipment that then goes back to teraco via a Metro provider, IE liquid, DFA, etc, essentially POP to street cabinet is Fibre backbone then from cabinet to end users homes, just because GPON can go 20km most FNOs would have shorter runs less than 10KM in most cases to keep losses to a minimum

Most FNOs would purchase a Bulk Layer 2 service or reticulate their own dark Fibre back to the Data Centre
 
[XC] Oj101 said:
@PBCool Related to your capacity vs shaping, does CI monitor total usage at all? I’m curious about my usage at times, but I see no way to monitor it on the client zone.
Click to expand...
CI does not monitor usage, uncapped, unthrottled, unshaped 100% of the time
 
JFT96 said:
Can anyone refer me to any articles or forum posts explaining how the ISP / Fibre infrastructure business model works?
What I’m trying to understand is what costs are associated with who in the FTTH business environment.

Here are my specific questions/queries in case anyone can answer them specifically:

In my case, I use Afrihost as my ISP and Vuma as my fibre infrastructure provider.
  1. When looking at performance issues experienced by people on this forum, they differentiate between local and international traffic. What does this actually mean? How does using a VPN influence better performance?
  2. Are there any fees associated with the ISP to use what we call the local network? So when testing on speedtest.net, we always get the exact line speed being paid for but not the case when testing internationally. Also not always the case when testing traffic from CPT to JHB. Surely all local traffic travels over fibre networks between ISP's?
  3. Who provides and pays for international traffic? Is it a capacity-based model where, in my case, Afrihost buys capacity on one or more of the international cables (WACS, BRICS, Seacom, Google etc). Or is it a case of Vuma knowing that all my international traffic is to be charged to Afrihost because I chose Afrihost as my ISP in the Vuma "shop" and Vuma buys the international capacity and routes my traffic? So one could say some infrastructure providers are better than others for performance rather than saying one ISP is better than the other for international performance.
  4. I pay Afrihost R1197 per month for 200/200. Let' say I were to stream FHD video content internationally 24/7, who picks up that bill? What would their costs be? (I know I would likely be flagged for exceeding fair use policies but if I were not flagged, who carries the cost?

Reasons for these questions:


I’ve been around the block with a few ISP’s in my FTTH lifetime, currently I'm with Afrihost and very happy with the service. But this is not so for many of my friends and family who are not on Vuma / Afrihost. They have chosen other combinations of FTTH and consistently experience very different performance levels. Of course service and support across the board seems to be consistently pathetic (and that includes Afrihost / Vuma).


But here is what I’m getting at when it comes to the FTTH business model.


If ISP A is in fact liable for costs associated with my personal international traffic preferences, it would make business sense for them to shape and throttle my traffic if I am within the top 10 users of international traffic. As in my scenario above, excessive traffic usage must be a serious business risk. So who monitors this, the ISP or Vuma?

View attachment 1368599
Above image is the traceroute of my traffic when streaming my IPTV. I have no idea where traffic moves from Afrihost to other traffic providers. I'm assuming 10.189.30.10 is where the international traffic starts. Is 196.25.26.65 Vuma? (I'll google that later EDIT: osnet is SAIX).

Let’s look at an example. I use an internationally based IPTV streaming service which offers loads of great high-quality content. Currently, on Afrihost / Vuma I can stream most FHD content over fibre, including live TV and all sport even on Wifi. However, I have family and friends on other combinations of networks who cannot stream even SD quality content on the same IPTV service as mine without constant buffering, even when connected directly to the CPE over ethernet. Why is that? My thinking is that this is a well-optimised traffic shaping implementation of cost management by the ISP or infrastructure provider. Of course, the ISP always denies shaping and leaves it to the customer to prove inadequate performance, which for most (even on this forum) is pretty difficult to do.

Please see this post as the general question basically being why is there such a massive variance of performance quality across different ISP’s and different networks? I also apologise upfront for any misunderstanding and incorrect assumptions. I’m trying to understand and willing to learn from anyone with better insight than me.
Click to expand...

Hi,

As explained by @PBCool there is quite a number of moving parts.

We don't shape or throttle at all. If a client wants to use only international they are free to do so we have enough capacity. We monitor trends and usage and determine what capacity we need and where. We make extensive use of high capacity cache servers in all regions. This saves quite a bit on international and local transit for the ISP but also improves the experience for everyone because the most frequently accessed content is just a few ms away from clients. A win win situation.

The last mile provider also plays a very big role. If they are are experiencing issues the issue will filter through and degrade the over all experience and the same goes for our network. IF there is any issues everyone on our network will experience issues.
 
Vorastra said:
More on the actual physical network side. Let's say MFN, right. I can see my area's GPON splitter cabinet. Does MFN themselves run a line to the Teraco building which is 12.5KM away line of sight, or would they pay DarkFibre Africa to connect to their backhaul that's literally less than a 200 metres down the road, and use that to get to the Teraco building in Riverhorse Valley, Durban, let's say.
Click to expand...
It depends on what's available, if they have backhaul nearby they would use that. If not they would use someone else. Often the FNOs use the likes of a DFA to start with whilst they build their own backhaul, then just move the DFA circuits to the next deployment.
 
[XC] Oj101 said:
@PBCool Related to your capacity vs shaping, does CI monitor total usage at all? I’m curious about my usage at times, but I see no way to monitor it on the client zone.
Click to expand...
It varies, on layer2 FNOs this is done naively with radius and PPP. But as far as "monitor" it takes more resources to do this kinda thing vs just buying more capacity.
 
PBCool said:
It varies, on layer2 FNOs this is done naively with radius and PPP. But as far as "monitor" it takes more resources to do this kinda thing vs just buying more capacity.
Click to expand...
Lemme put this theory to the test.

200 Mbps line, gets around 23 MB/s flat out
23 * 60 = 1,380 MB/min
1,380 * 60 = 82,800 MB/hour
82,800 * 24 = 1,987,200 MB/day
1,987,200 * 31 = 61,603,200 MB/month, or 58.75 TB/month

Mind me pulling 705 TB over the next year to see if it's really cheaper? :ROFL:

Na, I was asking out of curiosity from my side. I know some days I hardly hit 1 GB while others I do a few hundred gigs, I wanted to see my own trend.
 
[XC] Oj101 said:
Lemme put this theory to the test.

200 Mbps line, gets around 23 MB/s flat out
23 * 60 = 1,380 MB/min
1,380 * 60 = 82,800 MB/hour
82,800 * 24 = 1,987,200 MB/day
1,987,200 * 31 = 61,603,200 MB/month, or 58.75 TB/month

Mind me pulling 705 TB over the next year to see if it's really cheaper? :ROFL:

Na, I was asking out of curiosity from my side. I know some days I hardly hit 1 GB while others I do a few hundred gigs, I wanted to see my own trend.
Click to expand...
We have users that consume over 40TB per month.
 
Vorastra said:
At what point is it too much and unsustainable, forcing you to act? Or is it easier to not act due to the possible "backlash".
Click to expand...
At that level we typically investigate why, what would the backlash be :).
 
PBCool said:
At that level we typically investigate why, what would the backlash be :).
Click to expand...
You know what people can be like.
"Oh but it's meant to be uncapped" rabble rabble rabble
It's happened to a bunch of ISPs on this forum when everyone still had ADSL.

And next thing you know, MyBB is running an article about how it's not uncapped or whatever.

That sort of backlash. ¯\_(ツ)_/¯

Unless they're breaking TOS with line sharing or whatever.
 
Vorastra said:
You know what people can be like.
"Oh but it's meant to be uncapped" rabble rabble rabble
It's happened to a bunch of ISPs on this forum when everyone still had ADSL.

And next thing you know, MyBB is running an article about how it's not uncapped or whatever.

That sort of backlash. ¯\_(ツ)_/¯

Unless they're breaking TOS with line sharing or whatever.
Click to expand...
During the ADSL days the significant majority of traffic was international. Nowadays it has flipped so the transit costs are lower.

Additionally, in those days you had tossers like Internet Solution, Telkom, Vodacom, MTN, et. al who would not peer. Nowadays peering is easy, except for Seacom.
 
r00igev@@r said:
During the ADSL days the significant majority of traffic was international. Nowadays it has flipped so the transit costs are lower.

Additionally, in those days you had tossers like Internet Solution, Telkom, Vodacom, MTN, et. al who would not peer. Nowadays peering is easy, except for Seacom.
Click to expand...
Yet the point of the conversation is that someone doing 40TB per month warrants an investigation, as said by @PBCool

I want to know how much is too much and if an account would be terminated, or if it's easier to just allow it to continue.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X