← All articles

Pilot 50–100 Records Before Locking Eligibility Criteria

Pilot 50–100 Records Before Locking Eligibility Criteria

Eligibility criteria pilot screening title card

Inclusion and exclusion criteria, together called eligibility criteria, are the pre-specified rules that define who qualifies for a study and who does not. They exist to protect participant safety, control confounding, and set the boundary between how well findings apply to the exact sample studied versus the wider population a researcher hopes to reach. Every criterion must be locked into the protocol before recruitment begins, not adjusted after the fact.

TL;DR:

  • Strictly differentiate between absolute and relative exclusions, ensuring each criterion is appropriate for safety or scientific validity without overly limiting generalizability.
  • Carefully justify every eligibility criterion based on safety, confounding control, or data availability, avoiding arbitrary thresholds or defaults.
  • Finalize and document the eligibility criteria in the protocol before recruitment begins, and distinguish between clarifications and substantive changes to meet ethical standards.
  • Use structured data extraction tools and pilot screening to ensure consistent application of criteria across large datasets or multiple reviewers.
  • Question the necessity of each exclusion criterion by asking what bias or harm it prevents, and remove non-essential rules to improve the study’s relevance to real-world conditions.

Table of Contents

Inclusion Vs. Exclusion Criteria: What’s the Real Difference?

Inclusion criteria are the positive attributes a person, record, or data unit must have to enter a study: an age range, a confirmed diagnosis, a care setting, a language spoken. Exclusion criteria strip out otherwise-eligible candidates for reasons tied to safety, confounding, or missing data. A trial might include adults aged 40 to 75 with confirmed type 2 diabetes, then exclude anyone with a history of pancreatitis because the drug under study carries that specific risk.

The distinction sounds simple until you draft it. Two traps show up constantly:

  • Absolute exclusions eliminate a candidate outright (pregnancy in a teratogenic drug trial, prior enrollment in a competing study).
  • Relative exclusions flag a factor that increases risk or noise without automatically disqualifying someone (mild renal impairment, concurrent low-dose medication).
  • The single-list principle: a rule belongs on one list, never both. Writing “must be 18 or older” as inclusion and “excludes anyone under 18” as exclusion is redundant and invites boundary confusion during screening, a point the CASRAI eligibility guide flags directly.

Get this framing right early, and screeners downstream will thank you.

Why Eligibility Criteria Shape Validity, Ethics, and Recruitment

Every criterion you add is a trade. Tighten enrollment and you gain internal validity: cleaner data, fewer confounders, an easier statistical story. But you lose external validity, the degree to which results generalize past your sample. There is no criterion that gives you both for free.

Regulators have started pushing back on criteria that trade too much generalizability for convenience. The FDA’s workshop report on eligibility criteria urges scientifically justified, inclusive design and warns that unnecessary exclusions, an arbitrary upper age cap, a blanket comorbidity ban, leave evidence gaps for exactly the patients who will eventually receive the treatment in practice.

The consequences are not hypothetical. A multicenter COPD study that excluded people with comorbid respiratory disease found adherence split sharply by risk group, 20% in the low-adherence group versus 51% in the high-adherence group. That kind of exclusion protects the trial’s internal signal, but it also means the published results say very little about the more complicated patients most clinicians actually see. Every added criterion should answer one question honestly: does this exclusion serve the science, or just the convenience of running the study?

Inclusion and Exclusion Criteria Examples You Can Adapt

Seeing a worked example beats reading another abstract definition. Here is a short clinical trial eligibility set, annotated for the reasoning behind each line:

  1. Inclusion, age 40–75 — narrow enough to control for age-related physiology, wide enough to stay clinically relevant.
  2. Inclusion, confirmed diagnosis via lab test within 90 days — a measurable, dated threshold, not “recently diagnosed.”
  3. Inclusion, willing and able to complete follow-up visits — a feasibility filter that protects data completeness.
  4. Exclusion, pregnancy or breastfeeding — an absolute safety exclusion tied directly to known drug risk.
  5. Exclusion, prior enrollment in a related trial within 6 months — controls for carryover confounding.
  6. Exclusion, missing baseline lab values — a data-availability rule, applied before anyone looks at outcomes.

A systematic review eligibility set works the same way but starts from PICO: Population (adults with major depressive disorder), Intervention (cognitive behavioral therapy), Comparison (waitlist or usual care), Outcome (validated depression scale score change). Layer in study-type filters, randomized controlled trials only, published in a peer-reviewed journal, English or French language, and you have criteria ready to paste into a PRISMA eligibility table. The Cochrane Handbook’s chapter on defining criteria frames this as a synthesis decision as much as a search decision.

How to Write Eligibility Criteria That Hold Up

Start from your research question, not from a template. Every PICO element should translate into at least one measurable criterion:

  • Turn “adults with hypertension” into “systolic blood pressure ≥140 mmHg on two readings, age 18 and older.”
  • Turn “recent diagnosis” into a specific test, date window, or documented threshold.
  • Check data availability before you finalize a criterion; sourcing reliable primary antibodies and research reagents can be critical for lab-based inclusion decisions. If a lab value is not routinely collected at your sites, requiring it will gut recruitment.
  • Justify each line in the protocol itself. A reviewer or IRB member should be able to read one sentence and understand why it exists.
  • Limit the list to what the research question actually needs. Extra criteria feel rigorous but often just shrink your sample without adding scientific value.

Pro Tip: Pilot-screen a sample of 50 to 100 records or candidate participants against your draft criteria before locking the protocol. If ambiguity or feasibility problems show up in that pilot, you fix them now instead of mid-recruitment, when a criteria change can trigger an IRB amendment.

Set data-quality thresholds in advance too, something like “exclude records missing more than 20% of required extracted fields”, and apply them mechanically, without peeking at outcomes first. That single habit prevents a subtle form of selection bias that is easy to introduce with good intentions and hard to detect afterward.

Common Mistakes and a Quick Eligibility Checklist

The same errors show up across disciplines. Redundant rules across inclusion and exclusion lists create the boundary confusion mentioned earlier. Arbitrary age caps (“must be under 65”) often reflect old habits rather than actual physiological rationale. Comorbidity exclusions get added by default rather than because the comorbidity actually threatens safety or validity. And data-quality exclusions applied after seeing results, rather than pre-specified, quietly bias findings.

  • Scan both lists for duplicate or overlapping rules.
  • Ask whether each exclusion has a stated scientific or safety rationale, not just tradition.
  • Confirm every threshold is measurable, not descriptive (“recent” is not a threshold; “within 90 days” is).
  • Run a feasibility check: will realistic recruitment numbers actually clear these filters?
Pitfall Why it happens Fix
Redundant criteria across lists Copy-pasted from an old protocol Keep each rule on one list only
Arbitrary age caps Convention, not justification Tie age limits to physiological rationale
Unjustified comorbidity exclusion Default caution Require a stated safety or confounding reason
Post-hoc data exclusion Missing pre-specification Set completeness thresholds before screening

Locking the Protocol: Timing and IRB Implications

Criteria should be finalized and written into the protocol before recruitment opens, with the rationale for each line documented alongside it. That documentation matters later, both for reviewers and for your own team when someone asks why a rule exists six months into enrollment.

Not every change afterward needs a full amendment. A wording clarification, spelling out what “recent” meant all along, usually counts as an operational clarification. A substantive change, adding a new exclusion or loosening an age range, generally requires IRB notification or amendment before it takes effect.

  • Finalize eligibility criteria in the protocol before the first participant is screened.
  • Distinguish clarifications from substantive changes when documenting amendments.
  • Report eligibility criteria transparently: CONSORT guidelines for trials, PRISMA guidelines for systematic reviews, so readers can judge generalizability themselves.

Making Eligibility Criteria Operational at Scale

Writing good criteria is one problem. Applying them consistently across hundreds of abstracts is a different one entirely, and it’s where most screening teams lose time and introduce disagreement between reviewers. Manually checking each record against six or eight criteria invites drift, especially when three people are screening in parallel over several weeks.

Structured extraction tools change that math. Instead of a reviewer rereading an abstract for every criterion, the extraction step pulls the relevant fields (population, intervention, study design) into a table your team can screen consistently. PaperSynapse applies this to systematic reviews specifically: import references from Scopus or Web of Science, let the extraction step read abstracts against your defined fields, then pilot-screen a batch before committing to the full set. Draft criteria, pilot-screen 50 to 100 records, revise, then screen at scale, the same sequence recommended for manual protocols, just faster to execute and easier to keep consistent across reviewers.

Four-step eligibility criteria pilot workflow

Where the Real Discipline in Eligibility Criteria Comes From

Most guidance on eligibility criteria treats them as a compliance step, boxes to fill before a protocol gets approved. That framing misses the actual difficulty. The hard part is not writing a criterion; it’s resisting the urge to add one.

Where the Real Discipline in Eligibility Criteria Comes From — overview diagram

Every exclusion feels protective in the moment. It removes a variable, tidies the dataset, makes the statistics cleaner. But the COPD adherence data makes the cost obvious: a study that excludes comorbidities to get a cleaner internal signal ends up with results that barely apply to the patients who show up in an actual clinic. That is not a minor tradeoff buried in a limitations section. It is the study quietly answering a narrower question than the one it claims to answer.

The fix isn’t more criteria review meetings. It’s a habit: for every line in your eligibility list, write down the specific harm or bias it prevents. If you can’t name one, cut the line. Reviewers rarely reject a protocol for having too few exclusions. They reject it, or worse, publish it and let readers discover the gap later, when exclusions accumulate without anyone asking whether each one still earns its place.

— Ubada

Sources