Pilot Study Progression Criteria: A Worked Example

Ask what could prevent the main study from working: slow recruitment, unacceptable burden, poor retention, delivery failures, unreliable outcome measurement or a process that cannot be sustained at scale.

For each uncertainty, specify the evidence needed. “Recruitment was good” is not a measure. “The proportion of eligible people consenting during a defined recruitment period” is more useful, provided eligibility and the denominator are clear.

The Eldridge et al. CONSORT extension to randomised pilot and feasibility trials. 2016. recommends transparent reporting of objectives and criteria used to judge progression for randomised pilot and feasibility trials. Use standards appropriate to your design if the pilot is not randomised.

Imagine a fictional trial requiring recruitment at each clinic. The main study budget supports a limited recruitment window. The pilot therefore needs to estimate the rate of eligible participants recruited per clinic per month, not simply the total number enrolled.

Set thresholds using the main study’s requirements, available resources, uncertainty and stakeholder input. A published threshold from another project may have no relevance to yours.

If using green, amber and red categories, define what each means. Amber might trigger a modification and additional feasibility work, not an automatic proceed decision. Explain who will review the evidence and how conflicting criteria will be handled.

A small pilot estimate can be imprecise. A single percentage may conceal large differences between sites. Examine denominators, uncertainty and context rather than treating a threshold as a mechanical switch.

For example, strong recruitment at one highly supported clinic does not establish that the same rate will occur across less-resourced sites. Document what support was necessary and whether it is available for the main study.

A small pilot is generally not designed to provide a definitive test of intervention effectiveness. An encouraging p-value should not compensate for an infeasible recruitment process, and a non-significant effect should not automatically end a programme whose pilot objective was feasibility.

Use outcome data according to the pilot’s objectives and acknowledge their uncertainty. Avoid presenting exploratory estimates as confirmed benefits.

Show the original criteria, observed results, modifications and final decision. If the team departs from the prespecified criteria, explain why. A progression decision can involve judgement without becoming opaque.

Fictional recruitment requirement: 240 people, 4 clinics, 6 months.

Here is a fictional planning example, not a recommended universal threshold. A main study needs 240 recruits across four clinics in six months. The required average rate is 240 ÷ (4 × 6) = 10 recruits per clinic-month, assuming all four clinics recruit throughout.

A team could prespecify: at least 10 supports the planned recruitment schedule; 8 to fewer than 10 requires a costed change and reassessment; below 8 does not support proceeding with the current plan. At eight per clinic-month, the same plan recruits only 192 people. The team must explain why that shortfall is remediable, not simply label it amber.

Use the pilot’s actual clinic-time denominator, examine uncertainty and site differences, and combine recruitment with other feasibility findings. These thresholds illustrate a decision linked to resources. They do not constitute a statistical hypothesis test or guarantee future recruitment.

The strongest pilot conclusion is not always “proceed”. It may be “modify the follow-up process and test it again” or “do not proceed with this design”. Either can save substantial time and resources. Start with question–methods alignment to ensure the pilot is answering a feasibility question rather than imitating a smaller effectiveness study.