Implementation Outcomes vs Service and Health Outcomes
Distinguish implementation, service and health outcomes, and specify the actor, denominator, time point and evidence for each measure.
“The programme was successful” can mean several different things. Staff may have adopted it, users may have found it acceptable, appointments may have become easier to obtain, or health outcomes may have improved. Those are related possibilities, not interchangeable findings.
Separating implementation, service and health outcomes helps you choose measures that answer the study question—and prevents a favourable result in one category from standing in for evidence in another.
In this guide
Name the level of the outcome
Implementation, service and health outcomes require different evidence.
Proctor et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. 2011. distinguishes implementation outcomes from service and client outcomes. The framework includes concepts such as acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration and sustainability.
These labels are starting points for measurement, not self-explanatory variables. “Acceptability was high” is incomplete until readers know acceptable to whom, at what stage, measured how and in relation to what.
Apply the distinction to one programme
Consider a fictional clinic appointment-reminder programme.
| Question | Example measure | Outcome category |
|---|---|---|
| Did clinics begin using the system? | Clinics initiating use among eligible clinics | Implementation: adoption |
| Was it delivered as intended? | Eligible appointments receiving a reminder within the specified window | Implementation: fidelity to a defined component |
| Did the service become more timely? | Time from booking request to appointment | Service outcome |
| Did participants’ health improve? | A prespecified health measure at follow-up | Health/client outcome |
The same programme can perform well on one measure and poorly on another. A clinic may adopt a system but use it inconsistently. Reminders may be delivered correctly while appointment capacity remains insufficient.
Define the denominator and the actor
“Eighty per cent adopted the programme” means little without knowing whether the denominator is clinics, staff, eligible participants or organisations invited to join.
Specify the unit, eligible population, time frame and operational definition. Initiating use once is different from sustained routine use. Acceptability among programme managers may differ from acceptability among frontline workers or users.
Where several perspectives matter, measure them separately before combining them into a conclusion. An average across unlike actors can hide a serious implementation problem.
Match the evidence to the construct
Attendance at a training session does not by itself establish competence, acceptability or fidelity. A positive satisfaction item may not fully capture whether staff consider a programme appropriate for their setting.
Use measures with relevant supporting evidence where available, and justify adaptations. Qualitative work can help explain how people understand an outcome, why it varies and what routine indicators overlook.
Do not assign framework labels after data collection simply because they sound more rigorous than the variables originally collected. Explain what the available evidence can reasonably represent.
Specify the time point
Early enthusiasm may not predict sustained use. A feasibility problem during start-up may be resolved later—or concealed by temporary external support. Plan measurement around meaningful implementation stages.
If reporting sustainability, make the observation period and conditions explicit. Continued activity while a funded team is actively supporting delivery does not necessarily show that the programme will continue after that support ends.
Draw a bounded conclusion
For the fictional reminder programme, a defensible conclusion might be: “Most participating clinics initiated use, but delivery records showed inconsistent timing. The study therefore supports initial adoption, not reliable delivery across all sites.”
That is more informative than calling implementation successful. It identifies the next problem to solve without claiming evidence the study did not collect. Use the research design guide to align each outcome with its question, data source and analysis.
Continue your research
Three useful next steps for this topic.
Research notes, once a week
Practical research guidance, useful tools and funding opportunities in your inbox.