What is Peer Review?
History of Peer Review
Peer review was largely born out of the Royal Society of London, where their motto “Nullius in verba” means ‘on no ones word”
So members gathered at the Royal Society and reviewed each others’ work. Members helped the editors choose which manuscripts to include in their “Philosophical Proceedings” based on relevance, not strictly quality.
Then American Journals in the early 1900s (Science, Nature, JAMA) started quality-checked peer review.
Remember: creating reliable copies of manuscripts as very difficult pre-1950 – then photocopiers were invented and peer review exploded
Why Do Peer Review?
Assisting in editorial decisions
Improving the paper.
Systems of Peer Review
- Single Blind: reviewer is redacted
- Double Blind: reviewer and author is redacted
- Triple Blind: reviewer, author, and associate editor is redacted
- Open Peer Review: No one is redacted.
- Rolling Review
Single Blind
Advantages: Reviewers are free from reprisal
Disadvantages: Well-known authors get benefit of the doubt, detrimental to junior authors, discrimination possible.
Double Blind
Advantages: Same as single blind. May protect authors from reviewer bias
Disadvantages: Line-of-work is unknown. Authors may escape plagiarism detection.
Some small studies have shown little difference in quality nor decision to publish between Double and Single Blind Review.
Often times easy to guess who the authors are: Why? Use of first-person, excessive citations from a single source, citations of in-press articles, reuse or general-style of writing or experimentation.
Open-Peer Review
Advantages: Increase transparency, reviewers are more constructive in their critiques.
Disadvantages: Higher decline rate from reviewers. Critical reviewers may decline.
Disadvantages of Peer Review
- Your paper can be rejected… even if you think its good. You can get unlucky.
- Journal-hopping.
- Reviewers sometimes ask for silly things
- Scientists are asked to review a lot, review quality can suffer.
- Authors write for the reviewer, not for science. (I strongly dislike this)
Does Peer Review Even Work?
http://blog.mrtz.org/2014/12/15/the-nips-experiment.html
New Models of Peer Review
Conference Review (Computer Science)
Quick, but no revision possible
Rolling Review
New model by ACL
“Overall the peer review process is slow, expensive, prone to bias, open to abuse, prevents innovation, and unable to detect fraud.” – Dr. Richard Smith (Editor of BMJ)
Bad system, but maybe the best we have created so far.
Other models?
PlosONE and ScientificReports models of no judgement of innovation
ArXiV only model: publish everything
Groups involved in Peer Review
Authors: Publish or Perish attitude creates pressure to publish at all costs and encourages fraud.
Editors: Under pressure to publish hits and improve impact factor and citation scores.
Reviewers: Try to fit in paper review into the insane demands of publish or perish pressures.
Publishers: Compete to get the best work to sell to libraries.
What does it look like on the inside?