Government has dismissed a World Economic Forum (WEF) report which ranked South Africa’s mathematics and science education quality as the worst in the world (see SA has worst mathematics, science education in the world: WEF).
The department of basic education (DBE) said that the report is “not a credible or accurate reflection of the state of education in South Africa”.
Science and Technology Minister Naledi Pandor added that the “WEF report was based on perceptions, it wasn’t a test of our learners”.
Nic Spaull, researcher at the Stellenbosch University’s economics department, said that South Africa’s education system is in crisis. However, he said it is not as bad as the WEF report makes it out to be.
Spaull added that the WEF report’s education quality measure can be dismissed, as it does not match other empirical studies on educational quality.
He said that “the methods used to calculate these WEF education rankings are subjective, unscientific, unreliable and lack any form of technical credibility whatsoever”.
Pandor and the DBE referenced the Trends in Mathematics and Science Studies (TIMSS), which showed an improvement in the scores of grade 9 students in South Africa.
What the TIMSS report really says
Many people accept the TIMSS report as a good measure of the quality of mathematics and science education in the country. Unfortunately it does not tell a good story about SA’s education system.
It is indeed true that the TIMSS survey shows an improvement in the performance of mathematics and science education at grade 9 level in SA.
However, what the DBE and Pandor failed to mention is that the TIMSS results show that South Africa’s performance is very close to the bottom of the surveyed countries.
South Africa’s poor performance in Mathematics and Science education quality was highlighted in the first three TIMSS reports – 1995, 1999 and 2002.
What happened in 2002 is that grade 9 students in South Africa started to write the grade 8 tests.
The reason is simple – the grade eight TIMSS assessment was too difficult for eighth grade students in South Africa.
In 2011 South Africa was only one of three countries – with Botswana and Honduras – where grade nine students did the grade eight tests.
This did not do much to improve SA’s overall ranking. South Africa’s performance was so poor that the grade 8 students of all the participating countries, except Ghana, outperformed South Africa’s grade 9 students.
“A striking feature of the mathematics and science scores is that the average scale score of the top seven countries exceeds South African performance at the 95th percentile,” the Human Science Research Council stated.
“This means that the most proficient learners in South Africa approached the average performance in Singapore, Chinese Taipei, Republic of Korea, Japan, Finland, Slovenia and Russian Federation.”
To put South Africa’s latest TIMSS results in context – our grade 9 students performed worse than grade 8 students in other developing countries such as Morocco, Indonesia, Tunisia and Lebanon.
The TIMSS results therefore show that South Africa is far closer to the bottom of the list than the top of the list – something which should concern the country’s education department.
DBE explains “decline” from 1995 to 2002
The TIMSS scores declined between 1995 and 2002, but the DBE explained that the trend in both mathematics and science achievement over these years was flat.
“Although the exact point estimates in average score fluctuated slightly up and down, these changes were not statistically significant,” the DBE said.
“Given that TIMSS tests a sample of learners the changes were so small that it was not possible to conclude that there was any change in the full population.”
Why are grade 9 learners in SA taking grade 8 tests?
The DBE said that the primary goal of the TIMSS assessments is to research the determinants of learning. For this reason it is important that the level of the test is well matched to the performance level of most learners.
“Therefore, the researchers from the International Association for the Evaluation of Educational Achievement (the international body that administers TIMSS) recommended that several countries with previously low performance participate with grade 9 learners so that the test would be more appropriate,” the DBE said.
“In essence it was a technical research imperative that led to the decision to test at the grade 9 level.”
The department said that this decision was “trialled” in TIMSS 2002, when South Africa also tested a representative sample of grade 9 learners as well as grade 8 learners.
“This allows us to make accurate comparisons of performance between grade 9 achievement in 2002 and grade 9 achievement in 2011,” the DBE said.
“Importantly, there was a substantial and statistically significant improvement in both mathematics and science between 2002 and 2011. The improvements were largest amongst schools in poor communities.”
“By testing at the grade 9 level in 2002 and 2011, we now have an appropriate benchmark against which to measure future improvements.”
Why are SA’s grade 9’s doing worse than grade 8’s from other countries?
The DBE pointed out that SA’s students did not finish last in the TIMSS findings. “South Africa beat grade 9 students from Honduras in mathematics and also beat grade 8 students from Ghana,” the DBE said.
“However, it remains true that amongst the TIMSS participants, which are mainly developed countries, South Africa is a low performer,” it added.
Is South Africa’s mathematics and science education of the worst in the world?
The DBE said that the notion that South Africa’s mathematics and science education is the worst in the world, as purported in the World Economic Forum’s Global Competitiveness Report, is unambiguously incorrect.
“That report is based on the perceptions of business executives, and at best should be regarded as a satisfaction index,” the DBE said.
More rigorous assessments of education quality that actually test nationally representative samples of learners show that South Africa is not bottom of the world,” the department said.
“In TIMSS we are near the bottom, and in SACMEQ we are 8th out of 15 amongst a more comparable group of countries in the region.”