Thursday, July 30, 2020

Dirty Secrets of Scientific Peer Review

Is peer review a joke?  Surely not.  Peer review often makes good papers better and makes bad papers go away.  Of course we all wish that "good" and "bad" papers could be so cleanly and cavalierly classified.

Conversely, is peer review something to rely upon as firmly establishing scientific credibility?  Now THAT'S the joke.  Blind acceptance of "peer reviewed" as "trustworthy" is like blind acceptance of myriad other naive administrative check-the-box "solutions" -- a dubious strategy at best.  I fear, however, that significant parts of the public have been fooled into thinking that "passing peer review" equals "trustworthy", and that "not yet having passed peer review" equals "not yet trustworthy". The real test is whether a paper influences the course of thought in the medium and long run. Who decides that? The fiercely competitive market for ideas/researchers is highly, if not perfectly, efficient, and it usually sorts things out correctly.

For young researchers:

Immediately put your new paper in a visible working paper series like SSRN or arXiv, and let the market take over. But of course work hard simultaneously to get your paper published in a top place.  The refereeing process will (hopefully) improve it, and having the imprimatur of Top Journal X will send a valuable signal to the profession. Just don't be too sad, or too worried, if it doesn't work out as hoped with Top Journal X. 

No comments:

Post a Comment