← All articles

Track Exclusion Reasons for Review Teams: Pilot 50–100, Lock Codes

Track Exclusion Reasons for Review Teams: Pilot 50–100, Lock Codes

Hand-drawn exclusion tracking title card

When you exclude a study, record one specific, pre-defined exclusion reason mapped to your protocol. Do this at full-text review and preserve the reviewer ID and timestamp. This single habit keeps your screening consistent, gives peer reviewers a transparent trail, and makes your PRISMA reporting defensible months later when nobody remembers why a study was dropped.

TL;DR:

  • Recording predefined exclusion reasons at full-text review ensures transparency and reproducibility, especially for peer review and PRISMA reporting.
  • The exclusion vocabulary must be drafted beforehand, mapped to protocol criteria, and locked before screening to prevent inconsistencies and biases.
  • Assigning exactly one primary reason during independent double review, with real-time logging of reviewer ID and timestamp, minimizes bias and maximizes audit trail clarity.
  • Piloting the reason list with small batches helps identify ambiguous categories and reviewer disagreement patterns to refine the process early.
  • Using structured exclusion records with fixed fields allows automated generation of PRISMA flow diagrams and disposition tables, streamlining systematic review reporting.
Papersynapse
Streamline Your Literature Review
PaperSynapse helps researchers extract, normalize, and analyze paper data in one platform, supporting more consistent systematic reviews.

Table of Contents

What exclusion reasons are and when to record them

An exclusion reason is the specific, protocol-linked justification for removing a study from your review: not a vague note, but a category tied directly to an eligibility criterion. At title and abstract screening, a short label is usually enough since most exclusions are obvious. At full-text review, the standard tightens: you need a defensible, specific reason because these decisions face the closest scrutiny from peer reviewers and readers.

Common categories include:

  • Wrong study design (for example, a case report when the protocol requires randomized trials)
  • Wrong population (participants outside the specified age, condition, or setting)
  • Outcome not measured or not reported in a usable form
  • Language or publication date outside the protocol’s specified limits

The Cochrane Handbook treats predefined eligibility criteria as a prerequisite for a reproducible review, not an optional nicety. Skip the granular labeling at title and abstract stage unless your protocol specifically demands it there. Save the precision for full text, where it earns its keep.

Set up and customize your exclusion-reason list

Your exclusion vocabulary should exist before screening starts, and it should map one-to-one with your protocol’s eligibility criteria. Every reason on the list needs a home: a PICO element, a design requirement, an outcome, or a language or date limit. Guessing at categories mid-screen is how teams end up with a dozen overlapping labels that mean slightly different things to each reviewer.

  1. Draft one reason per eligibility criterion, phrased as a short human-readable label.
  2. Attach a machine-readable code to each label (for example, “outcome_missing” for “Outcome not measured”).
  3. Assign one person, usually the lead reviewer or methodologist, to approve any addition, deletion, or reordering of the list.
  4. Lock the list before full-scale screening begins, once your pilot round confirms it works.

Reordering is fine early on. Deleting a reason mid-screen after it has already been applied to excluded studies is not, since it breaks your audit trail. Reserve list changes for the pilot phase and document any that happen afterward.

Pro Tip: Keep the human label and the reason_code visible side by side in your screening tool so reviewers see both the plain-language meaning and the code that will later group it in your tables.

Paired exclusion labels and code workflow

Choosing and recording the reason during screening

Every excluded study gets exactly one primary reason, chosen from the locked vocabulary, at the moment of exclusion. If a study fails on two criteria, pick the one that would have excluded it first in your screening order (design before population before outcome, for instance) and note the secondary issue only if your protocol calls for it.

  • Use free-text notes sparingly, reserved for genuine edge cases the controlled vocabulary does not anticipate.
  • Run screening as independent double review, so two reviewers assign reasons without seeing each other’s choices first.
  • Reconcile disagreements directly between the two reviewers before escalating.
  • Send unresolved conflicts to a third reviewer for adjudication rather than letting one person override the other.
  • Capture reviewer ID and timestamp automatically wherever your tool allows it, and log the reason in real time rather than assigning it retrospectively from memory.

Retrospective assignment is where most inconsistency creeps in. A reviewer who excludes 40 studies in an afternoon and labels them the next day is guessing, not recording. Real-time logging, even a few seconds slower per study, protects the integrity of the whole dataset.

Piloting, granularity, and monitoring for drift

Pilot your exclusion vocabulary before committing to it at scale. Screening 50 to 100 records with the draft reason list is enough to surface ambiguous categories, missing labels, and reviewer disagreement patterns while the fix is still cheap.

  1. Screen a pilot batch and have both reviewers apply reasons independently.
  2. Compare results, refine any reason that produced repeated disagreement, and retrain reviewers on the revised codebook.
  3. Set a firm rule for the “other” category: it should never exceed a small fraction of exclusions, and every “other” entry gets reclassified into the finalized vocabulary before main screening proceeds, a practice the Cochrane Handbook recommends explicitly.
  4. After the pilot, review the distribution of reasons periodically throughout screening, watching for a reviewer whose pattern diverges sharply from the team’s.

Predefining and documenting eligibility and exclusion criteria in advance reduces bias and is described as a fundamental prerequisite for systematic reviews by the Cochrane Handbook. That single habit, done before screening starts rather than reconstructed afterward, is what separates a defensible review from a disputed one.

Documenting exclusions for reporting

Your exclusion log is the raw material for two PRISMA-facing outputs: the flow diagram and the list of excluded studies with reasons. PRISMA 2020 recommends citing studies that might plausibly appear eligible to a reader but were excluded, and stating why, under item 16b.

  • Map each locked reason_code to its corresponding box in the PRISMA flow diagram.
  • Include a category for “records removed for other reasons” whenever automated deduplication or format filtering removes records before human screening starts.
  • Prepare a short excluded-studies list covering studies a reader might reasonably expect to see included, each with one explicit protocol-mapped reason.
  • Keep every excluded study linked to its unique identifier and the screening step where it was removed, so the decision can be traced later.

A minimal disposition table, built straight from your logged records, looks like this:

Structured records like these, built on reason plus reason_code plus timestamp, are exactly what the lineager exclusion-tracking approach uses to make disposition tables reproducible and traceable back to individual records.

Common pitfalls and troubleshooting

Vague reasons are the most common failure. “No relevant data” tells nobody anything; “outcome not measured” or “ineligible study design” does. The PRISMA explanation and elaboration document specifically warns against ambiguous labels because they block later verification of your decisions.

  • Never delete or silently overwrite a reason once it has been applied. If a study needs reclassification, keep the original entry and add a dated reclassification log entry.
  • Document and justify in your methods section any change to the exclusion reason list made after screening began.
  • Hold short daily consensus calls during the first week of full-text screening, while patterns of disagreement are still forming.
  • Build a codebook with a real example for every reason, not just a label, so reviewers stop guessing at edge cases.
  • Set a firm rule to escalate any unresolved disagreement to a third reviewer rather than letting it linger.

Pro Tip: If two reviewers keep disagreeing on the same reason pair, the problem is usually the definition, not the reviewers. Fix the codebook before you fix the people.

Automation and structured exclusion tracking: a practitioner checklist

Structured capture works best when every exclusion record holds the same fields: reason, reason_code, reviewer_id, timestamp, screening_step, and an optional short free-text justification. Locking these fields early, whether in a spreadsheet or dedicated screening software, is what makes later PRISMA exports and disposition tables possible without reconstruction.

  • Capture all six fields at the moment of exclusion, never after the fact.
  • Run pilots on small batches, from 5 up to 100 records, before committing to a final vocabulary.
  • Use programmatic aggregation to build disposition tables directly from logged codes instead of manually recounting.
step action
Pilot Screen a small batch to test the reason list
Lock codes Finalize reason_code definitions before full screening
Double screening Two reviewers apply reasons independently
Periodic review Check reason distributions for drift
Export Generate PRISMA-ready tables from logged codes

Certain systematic review software supports this structure directly: papers imported from reference managers carry through a screening workflow where exclusion labels stay tied to reviewer and timestamp metadata, ready for export.

Author perspective: why we prioritize protocol-first exclusion tracking

The reviews that hold up under scrutiny are the ones where the exclusion list was locked before screening, not patched together after a peer reviewer asked an awkward question. Piloting feels slow in week one and saves weeks three through six.

— Ubada

PaperSynapse: a practical option to implement structured exclusion tracking

Building a protocol-mapped exclusion vocabulary by hand in a spreadsheet works, but keeping reason, reviewer_id, and timestamp consistent across hundreds of records is where teams lose time. PaperSynapse imports references directly from Scopus or Web of Science and lets you customize extraction fields, including exclusion reason codes, before screening starts.

Papersynapse

  • Import reference lists and screen abstracts with structured, exportable fields.
  • Pilot a batch of 50 to 100 records on an initial plan before locking your vocabulary.
  • Export screening results in formats ready to feed a PRISMA flow diagram.

Start a pilot on the Free plan at PaperSynapse and see how your exclusion vocabulary holds up before committing to full screening.

Sources

FAQ

What are some examples of exclusions?

Common exclusion reasons include wrong study design, wrong population, outcomes not measured, and language or publication date outside the protocol’s limits. Each should map to a specific eligibility criterion rather than a vague catch-all label like “not relevant.”

What is the purpose of exclusion checks?

Exclusion checks confirm that a study fails at least one predefined eligibility criterion before it is removed from a review. Recording the specific reason, as recommended in PRISMA 2020, keeps the process transparent and reproducible for readers and peer reviewers.

What is an exclusion status?

An exclusion status is the recorded outcome of a screening decision, typically the reason_code, reviewer ID, and timestamp attached to a study once it is removed from further review. This structured record, as described in the lineager exclusion-tracking approach, allows the decision to be traced and aggregated later.

What are exclusion criteria?

Exclusion criteria are the predefined conditions, drawn from the review’s protocol, that determine which studies are removed during screening. The Cochrane Handbook recommends defining these criteria before screening begins so they can be applied consistently across all reviewers.

Track Exclusion Reasons for Review Teams: Pilot 50–100, Lock Codes | PaperSynapse