ICASA hits back at network quality report criticism

MyBroadband recently published an article titled “What ICASA is hiding about network quality”, which questioned whether the regulator did enough to correct the faulty information published in its Western Cape Quality of Service Report (see What Icasa is hiding about network quality).

Icasa said that its original report inadvertently contained “some corrupt retainability measurement data, which indicated an incorrect number of drop calls”.

MyBroadband argued that ICASA should have been more pro-active in highlighting all the errors which corrupt data caused in its original report.

It further questioned some of ICASA’s testing routes and methodology, which included testing in areas such as a nature reserve where coverage is a challenge.

ICASA hit back, saying that this article cannot go unnoticed. The full reply from ICASA is provided below.

The Independent Communications Authority Of South Africa would like to respond to the above article which we believe lacked facts and conveyed a wrong impression of ICASA and undermining the readers who have interest in network quality in South Africa.

The article purports that ICASA is hiding something and is not willing to answer questions posed on network quality; but the article fails to acknowledge that ICASA openly took responsibility of the error that was made particularly with the Western Cape report and corrected it.

The updated results and the commentary is included in the updated report. The press release that ICASA issued only highlighted the area where the data was incorrect. The writer failed to go through the report to have a clearer understanding of the correction and the QoS report itself.

The overall results did change based on the Worcester data. The intention of the press release was a quick summation of the problem of the previous report, not to address the findings of the new report.

It was and certainly not ICASA’s intention to hide any findings, as they are clearly visible in the updated report which is a public document. It should be noted that ICASA had to redo the drive test in the Worcester route again and the updated report contains an entirely new data set.

It is furthermore incorrect to state that the charts on the article which were produced by ICASA were not made public to show the extent to which the incorrect data influenced overall results. If they were not made public, how did the writer get hold of these charts or information?

The reports are shared with all licensees concerned before they are published for public consumption. When the Authority issued the erratum yesterday, the report automatically became a public document and uploaded on ICASA website.

In terms of ICASA using strange routes, it is perplexing to read such statement because the report is based on widely used routes and areas that we have received complaints from the public. The Western Cape route included the CPT CBD, Cape Town International Airport, Somerset West, Stellenbosch and Worcester.

How can these areas not reflect areas where citizens are using mobile services? ICASA has previously tested QoS in the Western Cape and will continue to do so on different routes.

Perhaps MyBroadband needs to quantify how much of a “nature reserve” route formed part of the entire drive test? This statement implies that the entire drive test was conducted in a nature reserve, which is patently and blatantly incorrect. Moreover, operators are licensed on a national scale and that means the public must be able to pick up the network anywhere within the Republic.

It is important to note that ICASA provides the operator with so called “logfiles” (and the report), which is the raw data from the drive tests before these reports are made public. The operators use their own software programs to simulate the entire drive test for themselves, with the idea to identify areas where their network require attention.

ICASA could not be more transparent that this, as these logfiles provide full information on the routes, signal levels, test setup, test methodology and results for virtually dozens of tested parameters (and we only report on 2).

And lastly, ICASA puts it on record again that the IVR system in question was MTN’s – not ICASA’s. MTN reported the problem to ICASA just days before the report was due to be published. In fact, MTN went to an extent of thanking ICASA for picking this problem up for them after ICASA provided them with a copy of the report before it was published for the general public.

The IVR system used is owned by each operator. ICASA dials the operator’s own internal test number, to ensure that the test is always on-net (i.e. from MTN to MTN). This means that the quality of the call is measured totally within the operator’s network.

ICASA still has to liaise with MTN to determine the nature of the problem and the extent to which (if any) it affects the results. ICASA has not yet verified that the problem existed, save for a letter from MTN before the results were to be published. ICASA could not hold off on publishing the report, but we will investigate this further with MTN.

It is ICASA’s view that the article is bias, lacked facts and misleading the public.

More on the ICASA quality of service reports

Icasa messes up network quality report

Is SA’s mobile network quality good enough?

ICASA QoS results: Vodacom vs MTN vs Cell C