HERE is an interesting puzzle to consider. There have been, in the last decade or so, literally dozens of reforms, large and small, that have been announced and implemented in the area of school education across all of the provinces of the country. Yet, at the end of it all, learning outcomes of children have virtually not improved. Government schools still have a pretty poor reputation. What could possibly explain this?

Reforms in the education area have been quite significant, deep and in almost all sub-sectors of education. Teacher entry requirements, modes of selection, salaries, posting, transfer and promotion systems have been changed. Teacher and school monitoring systems have been completely revamped. Student testing has been changed. Curriculum and books have been overhauled several times. Education budgets, for all provinces, have more than doubled in the last decade. We have documented more than 70 or 80 major reforms that have been implemented in school education departments across the country in the last decade or so. Think of any recommendation that you could want to make, large or small, and it is very likely that it has been tried in the education sector.

 Data from the Annual Status of Education Report (ASER), based on testing of children from a very large sample from across all districts of the country, shows that there has been little or no improvement in student learning over the same period. There have been some gains in enrolment, especially at the primary level, but it is not clear if we will be able to sustain these gains as in recent times enrolment gains have also plateaued. It is not just ASER data that shows no gains in learning, even government examination results data, such as matriculation examination results and Punjab Examination Commission results of grade 5 and 8 examinations, also do not show any learning gains.

Strong data gathering and monitoring of results is a must for improving education policies.

So, what has all the reform done and what will we have to do to improve learning?

If reforms had not been implemented at all or had not been implemented properly, the above mentioned situation would have made sense. Though it is hard to check on all of the reforms and there is some unevenness in how they have been implemented, a large percentage of the reforms mentioned have been implemented quite well. Teacher entry requirements, recruitment processes and salaries have increased significantly, teacher attendance has improved, school monitoring is better, student testing is being done regularly, and textbooks are being delivered in schools. We need to look elsewhere for a fuller explanation.

It is clear that some of the reform that has been done, across provinces, has little or nothing to do with student learning. There is no evidence, anywhere, that just distributing laptops or tablets to students or even teachers, or opening a small number of very expensive schools for a few children while the majority of children continue to attend regular government schools, improves student learning. But a number of provincial governments have continued with laptop distribution schemes and there are still schemes being implemented that are distributing tablets to teachers.

We need to build a much better feedback mechanism in our policy-making: policies should be based on some evidence and once policies are implemented, we need to know if they are achieving their objectives and if they are not, they need to be altered and/or tailored to ensure better results. Does giving laptops make any difference to learning? If not, should this be continued as an education programme?

Each reform could also have unintended consequences and these could work against getting the results we have been looking for. In the case of some reforms the unintended consequences are quite clear. As we raise teacher entry requirements from matriculation/intermediate and teacher certificate to bachelors/masters with an education degree, the base for teacher recruitment will become more urban, teachers from local community, in some communities, will be harder to recruit and the social distance, between teachers and students who attend government schools, will increase. This could work against the objectives on learning enhancement. As we introduce more tablets and internet-based material, it could work against teacher motivation.

It is, usually, not possible to work out all the consequences of a policy/intervention. We only know about them once an intervention has been implemented and there is data on consequences that can allow us to look at what the results were. The government needs to collect a lot of data on results. Once this data is available, learning from it can be organised. This brings us back to the issue of feedback loop and learning from experience. It does seem that bureaucracies find it difficult to be learning organisations. They need to figure out how to do course correction after initial action.

If raising teacher entry requirements is leading to higher social distance between teachers and students, can we alter pre-service and in-service teacher training to ensure better teacher socialisation and preparation? If traditional models of providing continuous professional development of teachers are not working, how can we re-design them to make them more effective? Policy change is a complex process and needs to be recognised as such. Strong data gathering and monitoring of results, based on initial interventions, is a must for effecting improvements in subsequent rounds.

Reforms have been pretty deep and extensive in the education area, and across provinces. But, so far, we have little to show for all the effort and expense that has gone in. This offers an interesting puzzle for us. It does seem that ‘systems’ level thinking might be a must to unravel this puzzle but this area requires a lot more thinking before we can be comfortable with the answers.

The writer is a senior research fellow at the Institute of Development and Economic Alternatives and an associate professor of economics at Lums, Lahore.

Published in Dawn, July 14th, 2017