14
RECENTLY, I reviewed a file of a social sciences academic from a public-sector university for possible appointment at a more senior position. I noticed that the gentleman had listed 11 publications that had come in the same year and in the same journal as well. Some of these were single-author publications, and in others he was a co-author.
Eleven publications in one year, and in social sciences! This gentleman was either a genius or there was something funny going on. Nor is there any journal of repute, in the social sciences, who would take 11 articles from the same author in the same year, however frequently they might bring out their issues.
I did some quick investigation. It appeared that the journal in question was recognised by the Higher Education Commission a year or so before the gentleman started publishing in it; that the HEC raised its ranking around the time the gentleman published in that journal; and a year or so after that, the journal was taken off the HEC’s list of recognised journals.
If this was a coincidence, it was a very lucky break for this academic. He was able to get published 11 times, with HEC recognition, by one journal in one year. He was all set to meet the HEC set requirements for promotion and senior appointments.
Pakistani academics know all too well how easy it is to game the system set up by the HEC.
So is there ‘gaming’ going on in the system? And, if so, who are the players who are gaming the system? Equally importantly, is this gaming producing any research that is of good quality? If there is gaming, but we are still getting good quality research, one could live with some amount of gaming.
This was not an isolated incident. Once alerted to the fact, I started looking at the issue a bit more carefully. I have found a number of such instances now.
Another academic, who works for a private-sector university and is even the acting head of his department, had 17 publications in one journal over an 18-month period. And the story was no different in this case too. The journal was, around this period, recognised by the HEC — hence, the incentive to publish in it. But 17 articles, and in one journal! I wonder if Guinness works on world records in such areas as well.
Academics in Pakistan are clearly responding to incentives that have been set for them. HEC wants to have ‘recognised’ publications from academics on tenure track, and these publications, impact factors and citations become the basis for promotions as well as appointments as deans, vice chancellors and so on. Academics have responded to the call. Research output from Pakistani academics has gone up over the last decade or so as a response to these incentives. But — and here is the rub — what has been the quality of this output?
I am making a distinction between social sciences and humanities on one hand and physical sciences on the other. Academics do get a lot more publications in some areas of pure sciences, especially those that are experiment-based like chemistry, and science journals, in some cases, do have more issues that come out every year. But social sciences tend to be quite different.
Empirical work in the social sciences tends to take long as it is usually based on surveys and/or other data collected by the researcher in the field by him/herself. Field experiments, if possible at all, also take years to set up and complete. Qualitative work tends to take even longer in most cases. Hence the expectation, in terms of frequency of publication, tends to be lower.
But the HEC has worked on setting incentives very differently. There has been a very high premium on quantity, almost at the complete cost of quality.
Of the 28 articles that have been mentioned above, not even one of them was worth reading or talking about. They did not add to any knowledge in their respective areas, did not give any new insights, did not give any interesting examples, did not extend theory or even empirical understanding in any way, and did not give any interesting policy insights as well. What is the point of counting such articles, and basing promotions and appointments on them?
Even the citation numbers are being ‘gamed’. Authors cite every article they have ever written, groups of authors cite each other to raise their citation numbers and so on.
Policy has to evolve with time and has to have a feedback loop from implementation and implications to policy change and reform. A focus on quantity might have been an important goal when HEC was set up and in its initial years. It was probably a poor policy choice even then but, even if it was a necessary step, it is no longer working. We need to realign incentives for faculty to ensure better quality research.
This is not the place for detailed design of policy, but some ingredients are quite clear. Beyond numbers, there has to be some premium on quality. And judgement of quality has not only to be based on whether an article has been published or who a book has been published by, but also must have some ingredient of peer evaluation eg, has the research created some engagement in the relevant area and/or has it attracted the attention of peers? Peer judgement has to be contextual as well. Not only do we have to ensure that international peers are engaged, local ones have to be engaged in relevant dialogue as well.
Will the HEC be able to bring about a more nuanced incentive policy for academics in the country? If it does not, the current games will continue, and the quality of educational quality in the country will remain poor.
The writer is a senior research fellow at the Institute of Development and Economic Alternatives and an associate professor of economics at Lums, Lahore.
Published in Dawn, June 29th, 2018
