Peer-review instructions

Download: Word · PDF · Back to assignments

Foundations of Clinical Research · Level 1: From Clinical Question to Publication

Organisation: Neudata (#ClearDataClearImpact) When: the 2 weeks after Session 8 (homework) Who reviews whom: each participant reviews one classmate’s draft, assigned by the instructor and sent by email or through the online course folder. Everyone reviews one draft and receives one review. This is individual homework. Use with: manuscript_brief.md, the 10-question appraisal checklist from Session 8 (“Be the reviewer”), the STROBE checklist, and Course/Templates/Response_to_Reviewers_Template.docx

1. Why we do this

Every paper you submit will be peer reviewed, and sooner or later you will be asked to review. In Session 8 you practised on two published papers. Now you review one classmate’s draft and receive a review of your own, exactly as a journal would. The aim is to improve both papers, not to judge the people who wrote them.

2. Timeline

When Who What
Day of Session 8, 23:59 Each participant Submits their draft manuscript (Surname_Manuscript_QN_draft.docx)
Within 2 days Course lead Sends each participant the one draft they will review (without changes). The course lead makes a simple list so that every draft is reviewed exactly once, and where possible pairs drafts on a different question
Within 1 week after Session 8 Reviewer Sends the review report (Surname_Review_of_[author's surname].docx) to the course lead, who forwards it to the author
Within 2 weeks after Session 8 Author Sends the revised manuscript (tracked changes) and the response letter to the course lead

Plan about 2 hours for the review: about 45 minutes to read the draft, 45 minutes to check it against the results pack and the checklists, and 30 minutes to write the report.

3. How to review (reviewer)

  1. Read it once first. Read the whole draft once, without commenting, and write down in one sentence what the paper found.
  2. Check against the results pack. Open the results pack for the author’s question (Course/Data/results_packs/QN/results_pack_QN.md) and check the main numbers in the abstract, text and tables.
  3. Apply the 10-question appraisal checklist (below).
  4. Apply the writing checks (below).
  5. Sort your comments into major (would change the conclusions or the reader’s trust) and minor (clarity, wording, formatting). Keep only the comments that matter: about 3 to 6 major and 5 to 10 minor.
  6. Write the report using the template in section 5.

The 10-question appraisal checklist (applied to a classmate’s draft)

Answer Yes / No / Can’t tell and give the page or line.

# Question What to look for in the draft
1 Question: Can you write the study’s PICO in one line? Is the aim in the Introduction the same as the analysis in the Results?
2 Design: What is the design, and is it right for the question? Cohort stated in the title or abstract; no trial language.
3 Participants: Who was included and left out? Eligibility; the flow of patients; number analysed for this question (e.g. survivors only in Q5).
4 Comparison groups: Were the exposure groups similar at the start? How was this handled? Table 1 by exposure group; adjustment explained and justified.
5 Bias: Could knowledge of the exposure, or how data were collected, change the outcome? Confounding by indication discussed; information bias in fluid charting; missing data.
6 Outcome: One pre-specified primary outcome, measured the same way? Outcome defined (e.g. KDIGO for AKI); the SAP mentioned.
7 Numbers: Main result as an effect size with 95% CI? Narrow or wide? Crude and adjusted estimates; CIs; denominators.
8 Words vs design: Do conclusions match the design? “Associated” vs “caused”; no “trend”; abstract conclusion no stronger than the Discussion.
9 Limitations: Stated honestly? Name one they missed. Residual confounding, missing data, simulated single dataset, generalisability.
10 Use: Would this change practice? Why / why not? Are the implications proportionate to an observational result?

Writing checks

Check Yes / No Page / line
Title names population, exposure, outcome and design
Structured abstract of 250 words or fewer, with the main effect and 95% CI
Introduction ends with a clear aim
Methods: variables defined, statistical methods (including missing data) described, simulated-data statement present
Results: participants first; numbers with denominators and 95% CIs; no interpretation
Numbers agree between abstract, text, tables and the results pack
Tables and figures stand alone (title, units, n, abbreviations)
Discussion: main finding, comparison, explanations, limitations, implications, conclusion
STROBE checklist completed with page numbers
References consistent (Vancouver); about 2,000 words; CRediT statement, course acknowledgement and AI-use statement present

4. Tone rules for reviewers

  1. Comment on the paper, not the people. Write “The Results do not report the crude OR”, not “You forgot…” or “The author clearly doesn’t understand…”.
  2. Be specific. Give the page and line, say what the problem is and why it matters.
  3. Be constructive. Every major comment ends with a suggestion: “Consider reporting both the crude and the adjusted OR so readers can see the effect of adjustment.”
  4. Start with what works. Open the report with two or three genuine strengths.
  5. Do not rewrite the paper or impose your own style. Suggest, do not dictate.
  6. Stay within the data. Do not ask for analyses that are not in the results pack. You may suggest them as a limitation.
  7. Keep it confidential. Do not share the draft with anyone else, and do not paste it into an AI tool or translation service.
  8. Be kind and honest. Write as you would like to be reviewed: clear about problems, respectful in tone.

5. Review report template

Copy this into Surname_Review_of_[author's surname].docx.

Manuscript reviewed: author ______________, Question Q_, working title: ____________________________ Reviewer: ______________ Date: ____________

1. Summary (3 or 4 sentences). What the paper asks, what it did and what it found, in your own words.

2. Strengths (2 or 3 points).

3. Major comments (numbered; each with page/line, the problem, why it matters and a suggestion).

  • M1.
  • M2.
  • M3.

4. Minor comments (numbered; page/line and suggestion).

  • m1.
  • m2.

5. Appraisal checklist and writing checks. Attach the two completed tables from section 3.

6. Recommendation (tick one, as a journal reviewer would):

☐ Accept as it is ☐ Minor revision ☐ Major revision ☐ Reject (explain why)

6. How authors respond (author)

  1. Read the review twice, then wait a day before replying. Every reviewer comment is an opportunity to make the paper clearer.
  2. Use the response template (Course/Templates/Response_to_Reviewers_Template.docx). Open with one sentence thanking the reviewer.
  3. Answer point by point. For every comment, give: (a) the reviewer’s comment (copied), (b) your reply, (c) what you changed, with the page and line in the revised manuscript.
  4. Make the changes in the manuscript, with tracked changes switched on, and make sure every change you claim is really there.
  5. Disagree politely, with reasons. You do not have to accept every suggestion. If you disagree, say why, citing the data or the literature: “We agree this would be informative. However, this analysis is not in the pre-specified SAP. We have added it as a limitation (page 9, line 12).”
  6. Never answer just “Done”. Say what was done and where.
  7. Keep the tone neutral and grateful, even if a comment feels unfair.

Example

Reviewer comment Response Change in manuscript
M1. The Results give only the adjusted OR; readers cannot see how much adjustment changed the estimate. Thank you. We now report the crude OR alongside the adjusted OR and comment on the difference in the Discussion. Results, p. 6, lines 4 to 7; Discussion, p. 8, lines 15 to 18; Table 2 updated.

7. Marking

The quality of the review you give and of your response is criterion 10 of manuscript_rubric.md (10 points).