Peer
review simply isn't what it is advertised to be; it is not only little more than editing, most of the time
it is not even competent editing:
As
a scientist with a 15 year career behind me so far, I am afraid that my
experiences reflect this. Peer review is excellent in theory but not in
practice. Much of the time, the only vetting the papers get are two relatively
junior people in a field (often grad students or postdocs) giving it a thumbs
up or thumbs down. That is absolutely it. In theory, the editors should make
the decisions with the recommendations of the reviewers, but the editors rarely
have the time or the expertise to judge the papers and often automatically
defer to reviewers. Also, the papers should be reviewed by luminaries of the
field, but these folks rarely have the time, and either decline invitations or
bounce the work to a student or another trainee. It's not just bad papers that
get through, but also good, rigorous, papers that are bounced by this system.
Many if not most of the people in academic science today, at least in biology (my field), are overwhelmed with the need to publish in such high volumes, few people with the needed expertise can afford the time to go over the results in detail. All this while, at the same time and for the same reason, the volume of papers that needs to be reviewed goes up. I've heard of (and had myself) papers havve lingered for 4+ months before they even went out for review.
And, in our rush to publish, we often don't read this literature carefully ourselves but start citing papers anyway, which weaves these potentially weak or erroneous papers even more tightly into the fabric of their field.
It's difficult to care a lot about the quality of your work when you know the extra effort often doesn't help something go through this fickle review process, and when you know people will cite it without really reading it closely. There is little incentive to spend longer on a paper to make sure everything is right and the results are reproducible because there is very little accountability for errors and huge rewards for being prolific.
Many if not most of the people in academic science today, at least in biology (my field), are overwhelmed with the need to publish in such high volumes, few people with the needed expertise can afford the time to go over the results in detail. All this while, at the same time and for the same reason, the volume of papers that needs to be reviewed goes up. I've heard of (and had myself) papers havve lingered for 4+ months before they even went out for review.
And, in our rush to publish, we often don't read this literature carefully ourselves but start citing papers anyway, which weaves these potentially weak or erroneous papers even more tightly into the fabric of their field.
It's difficult to care a lot about the quality of your work when you know the extra effort often doesn't help something go through this fickle review process, and when you know people will cite it without really reading it closely. There is little incentive to spend longer on a paper to make sure everything is right and the results are reproducible because there is very little accountability for errors and huge rewards for being prolific.
The ironic thing is that True Believers and
the I Fucking Love Science crowd genuinely believe that "peer reviewed
science" is the gold standard for evidence. But there is a reason scientific evidence is not
automatically allowed in a court of law, let alone considered conclusive, and
the more we learn about the defects of peer review, the better we understand
that science's credibility is limited.
We have a word for science that is trustworthy, and that word is engineering. Until science can be applied, it cannot be fully trusted to be correct.
All peer review is really designed to do is to reassure the reader that the information presented fits safely within the confines of the consensus status quo.
We have a word for science that is trustworthy, and that word is engineering. Until science can be applied, it cannot be fully trusted to be correct.
All peer review is really designed to do is to reassure the reader that the information presented fits safely within the confines of the consensus status quo.
