Cool Ideas has stopped taking orders for TT Connect because of consistent network problems and poor support.
On 6 November 2018, Cool Ideas posted a message on its website saying it cannot take new orders until “TT Connect can resolve their network issues”.
“We want to provide our loyal customers with a certain level of service and unfortunately on the TT Connect network this is not possible at the moment,” Cool Ideas said.
MyBroadband has learned that other Internet service providers are experiencing the same problems with TT Connect.
Why Cool Ideas stopped taking TT Connect orders
Cool Ideas, which was crowned South Africa’s ISP of the Year in October, offers services over more than 30 fibre network operators (FNOs).
In an interview with MyBroadband, Cool Ideas said it has a good relationship with all FNOs and work closely with them if problems occur.
TT Connect, however, is the exception. Cool Ideas said TT Connect has caused endless support and network quality issues for ISPs.
Cool Ideas said it has engaged with TT Connect management and technical staff for over two years to try to resolve the problems without real progress.
It has happened before
Cool Ideas also stopped processing orders from TT Connect in July 2017 to get a response to service delivery and quality issues.
“Some issues were resolved in October/November 2017, after which Cool Ideas resumed taking orders for clients on the TT Connect network,” Cool Ideas said.
“Resolution required Cool Ideas to monitor TT Connect POP switches to prove to TT Connect that packet loss and regular micro-outages (120 seconds and more) were indeed a real occurrence.”
However, these problems cropped up again this year which led to the decision to stop sign-ups.
Cool Ideas said they have engaged with TT Connect’s management on many occasions with constructive suggestions, but to no avail.
“Our suggestions fell on deaf ears and our customers and staff suffer from the lack of service from TT Connect,” Cool Ideas said.
TT Connect network problems
Cool Ideas highlighted numerous problems on TT Connect’s network, including packet loss, point-of-presence (POP) power problems, and poor technical support.
“TT Connect’s network infrastructure and backhaul network are regularly congested, with significant packet loss between TT Connect POPs,” Cool Ideas said.
“Packet loss, combined with 180ms international latency, results in severely degraded TCP connection performance causing our clients to have sub-3Mbps download speeds to international speed test servers.”
Cool Ideas said there have been several network loops and broadcast storms on the TT Connect network, which affected all ISPs.
“When there are issues with our NNI (interconnect) there is no urgency from TT Connect to restore the service,” Cool Ideas said.
Cool Ideas added that TT Connect also has issues in keeping their road-side POP installations powered, sometimes “resorting to generators and battery banks to keep POPs powered for weeks on end”.
The pictures below, provided to MyBroadband, allegedly show TT Connect POPs powered by generators.
Problems go even deeper
Cool Ideas said the challenges which they experience go even further and include problems with placing orders, ISP migrations, installations, and fault resolutions.
“Migrations between ISPs take an extremely long time to complete. What takes other operators a few hours can take TT Connect weeks,” it said.
A particular challenge, Cool Ideas said, is that technical support tickets are not answered and often go missing.
“When phoning in to follow up on tickets we are told the support team is too busy to attend to us now, and they will get to it when they get to it,” Cool Ideas said.
TT Connect responds
TT Connect’s Peter Jeffrey told MyBroadband they were not aware of Cool Ideas’ statement until they received an email from MyBroadband.
“We will address Cool Ideas statements with them direct during the course of this week,” said Jeffrey.
He said they are “aware that Cool Ideas have at times made incorrect statements to their customers about the reason for their customers perceived poor international throughput on their network as being a TT Connect issue due to packet loss”.
“When we became aware of these allegations, we conducted extensive Iperf testing on the customer link,” Jeffrey said.
“The results over the last few weeks have shown that if there is packet loss it is well below the above specification and any issue the customer may have must be taken with Cool Ideas.”
Jeffrey added that network loops and broadcast storms are not common on the TT Connect network.
He said only one broadcast storm has occurred which affected ISPs, and they have only experienced one network loop event within the network over the last quarter.
Installations and ISP migrations
Jeffrey also dismissed concerns about installations and ISP migrations, saying they are not aware of any issues around free-standing home installations or customer migrations.
He explained that due to the nature of multi dwelling units (MDUs), an estate buildout can only occur once HOA approval has been obtained.
“Currently our MDU buildout teams commence working on the MDU within two weeks of receiving the final approval from council and the HOA,” he said.
Commenting on technical support problems, Jeffrey said historically and contractually TT Connect’s support hours were Monday to Friday 07:00 until 22:00.
“We have received requests from ISPs for 24/7 support which we unofficially commenced on 1 September 2018.”
“Because of the above we will be making the 24/7 now a standard part of our SLA going forward,” he said.
Jeffrey also told MyBroadband that they only have five sites running on generator at the moment – four within the Boksburg municipality and one within the Germiston municipality.
The company explained that it sometimes needs to run a POP in a temporary hybrid mode due to the rate of its network expansion.
“Due to the fast pace of our network expansion of 2-3 suburbs a month, we do need to sometimes run some of our POPs in a hybrid mode when we initially go live for a period of 4-12 weeks,” TT Connect said.
It said this only happens while it waits for the relevant council to process and install its power order for the POPs.
While it waits for these installations to occur, maintenance teams conduct regular site inspections to ensure that the POPs remain powered.
Sometimes a POP may go down the day of a scheduled generator run, but it is usually brought back up fairly quickly, TT Connect said.
He said no power outages have been recorded at the POPs which are fed by generator power.
He added that there was, however, a major power outage in Edenvale recently which lasted for 20 hours and affected several suburbs.
“The power outage was due to the Eskom connection feeding Edenvale failing. Due to the duration of the power outage some of our POP UPSs were depleted prior to the power being restored,” he said.
Big improvements coming soon
Jeffrey added that they are hoping to complete their implementation of an automated OSS/BSS system and have the first ISP live on the system by 1 December 2018.
“Once rolled out to all ISPs in the first quarter of next year, we are confident it will provide service providers with greater access to our network so that they can support their customers more directly,” he said.
Logging of tickets, migrations, upgrades, and terminations will all be managed automatically by the OSS/BSS system once fully rolled out to all ISPs.
For the last month, TT Connect has also conducted “intensive network testing” of its new core network equipment.
“As such we will be moving all ISPs and links onto the new core equipment in stages over the next two weeks,” he said.
“The new Arista equipment was procured to enable us to launch our new 1Gbps services schedules which will be commercially available early next year.”