Read on Twitter
I’ve had the privilege of reviewing many papers over the years.
I’ve also been on the receiving end of reviews.
The difference in the quality of feedback is often striking.
So here are some of my suggestions on how to review manuscripts effectively (thread):
I’ve also been on the receiving end of reviews.
The difference in the quality of feedback is often striking.
So here are some of my suggestions on how to review manuscripts effectively (thread):
1\\ Begin a review by clearing your mind and setting an intent to be helpful.
Think of the person who spent hours doing this work. Think of the privilege you have of reviewing their work.
Respect their effort by not rushing yours.
Resolve to be kind. Aim to help. Not hurt.
Think of the person who spent hours doing this work. Think of the privilege you have of reviewing their work.
Respect their effort by not rushing yours.
Resolve to be kind. Aim to help. Not hurt.
2\\ Read the entire paper (incl appendices) in entirety at least twice (pref thrice) before you begin your review.
Your second and third read are critical as they focus your mind on the issues and help frame the problem clearly.
Don’t skip the repeat reads!
Your second and third read are critical as they focus your mind on the issues and help frame the problem clearly.
Don’t skip the repeat reads!
Then, list 2-3 MAJOR Concerns - issues that substantially weaken the manuscript. Things like design flaws, measurement problems, residual confounding, inadequate adjustment, threats to inference live here.
Major concerns threatens the conclusions of the authors.
Major concerns threatens the conclusions of the authors.
Now - for every major concern, list 2-3 potential solutions. This should feel hard. Even if some flaws are fatal, suggest how a retake/redesign could help. Or how you would have done it differently.
Explain why your suggestion is better. Grow authors, even if the news is bad.
Explain why your suggestion is better. Grow authors, even if the news is bad.
Then, identify 2-3 minor concerns. Here’s where you can go to town on syntax, grammar, language, layout, font, headings, style.
For each minor concern, follow with suggestions on how to fix.
When you start to focus on solutions, you become more thoughtful about problems!
For each minor concern, follow with suggestions on how to fix.
When you start to focus on solutions, you become more thoughtful about problems!
Don’t forget to look at the tables and figures (and appendix or supplemental material).
Do they stand alone?
Do they all have legends, labels, axes? Do the tables make sense? Are imp columns missing?
Are all the tables/figures necessary? Are key ones missing?
Do they stand alone?
Do they all have legends, labels, axes? Do the tables make sense? Are imp columns missing?
Are all the tables/figures necessary? Are key ones missing?
If you follow this approach, your review should be no more than 1-2 pages.
But it should take you 1-2 hours to write one of these.
If you can’t commit this time to a review, don’t agree to review.
I can’t stress this enough. Do it well. Or don’t do it at all.
But it should take you 1-2 hours to write one of these.
If you can’t commit this time to a review, don’t agree to review.
I can’t stress this enough. Do it well. Or don’t do it at all.
Try to close with words of praise. Writing is not easy. Writing well is V V hard. Applaud the effort and help grow papers and authors.
And after you have done this, smile for your act of kindness to others.
Your karma and the authors will thank you.
-FIN-
And after you have done this, smile for your act of kindness to others.
Your karma and the authors will thank you.
-FIN-
If you have suggestions for how to do better reviews, pls add here!
