TCA Srinivasa-Raghavan: The death of peer reviewing?

OKONOMOS

An eminent economist told me he reviewed as many as 40 papers on international flights, before meals!

In 1975, when I joined Macmillan as the economics editor, I was told that after I had decided to proceed with a manuscript "at my level" I would have to get it peer-reviewed. I was as brash then as I am now, and wanted to know why my judgement was not good enough. The boss said a peer review helped to establish without a shadow of doubt that the manuscript was up to Macmillan's "exacting" quality standards.

But I soon realised that I could influence the reviewer through a variety of means. The best way, of course, was to send it to someone I knew who would approve or disapprove. That always guaranteed the desired result. But I indulged in this subterfuge only twice.

I left Macmillan in 1980, and that decade saw a proliferation of journals. To face the competition, book publishers gradually stopped using peer review as a necessary condition for publication.

The journals filled that space, in my view, more in order to indemnify themselves against plagiarism than to ensure quality. After all, as many as 10,000 papers get published every year in journals. How can they all be of "exacting" standards and, more importantly, how do you detect plagiarism? (But now there is a software for detecting it).

Publishing in journals had become simply too tiresome. This obsession of the publishers with peer reviews led to a slowing down of the journals publication process in the 1990s. But it speeded up book publishing and most book publishers are now just printers with distribution chains or access to them.

Then in the latter 1990s the Net came along. Peer reviewing, as this paper* by Glenn Ellison, an economist at MIT, points out, may well have received its final death blow.

Not a day too soon, if you ask me, because much of it was an elaborate fraud anyway. One very eminent economist once told me that he reviewed as many as 40 papers on international flights. One way or round trip, I asked. "Oh, round trip," he said earnestly. "I read them before they serve the meal."

Ellison says he has documented the decline in the number of peer-reviewed papers in top economics journals written by economists from the highest-ranked economics departments. "Several observations are consistent with the hypothesis that the Internet improves the ability of high-profile authors to disseminate their research without going through the traditional peer-review process."

The result is that "" using the "up-yours, Jack" principle "" top economists from the best departments are publishing less in peer-reviewed journals. Can't say I blame them.

"The citation benefit to publishing in a top general-interest journal," says Ellison, "appears to be fairly small for top-department authors now. Another reason is that Harvard authors appear to be quite successful in garnering citations to papers that are not published in top journals."

In other words, it is the big boys who are breaking the rules of the game while insisting that the smaller fry observe them. That, too, is par for the course.

Ellis is bothered by the fact that "more top economists may realise that the publication hassles they have been enduring are not necessary." He thinks as more of them vote with their feet, publication in high quality journals may not signal what it is intended to signal: top quality "...and this may lead to further withdrawals."

What does all this matter for economic research which has reached a cul de sac anyway? The same as it did for books, I think, namely, how good or bad research is should not be made a function of publication. That leads to coteries and other sorts of vested interests. Nonsense between covers, even if certified as kosher by pals, is nonsense nevertheless.

But Ellis takes a dimmer view. "...If only the top general-interest journals maintain their stature, then more economists may concentrate on 'general interest' research and decline to make the kinds of incremental contributions to sophisticated literature that appeal relatively more to those who are experts on a topic."

Someone should have told that to Ricardo, Marshall, Keynes, Sraffa and that whole bunch which thought more, and published less.

*Is Peer Review in Decline? NBER Working Paper No. 13272, July 2007