Missing Data Analysis: Audit Patterns Before Choosing a Method

Start by distinguishing data that are genuinely missing from values that are not applicable, structurally skipped or outside the measurement schedule. A participant who was never eligible for a question is not necessarily a non-responder to that question.

Check special codes and import conventions. Blank strings, zeros, 99 and “unknown” may have been handled differently across files. Reconcile these with the instrument and data dictionary before calculating missingness.

For each analysis variable, report the number expected, observed and missing. Then examine missingness by time point, study group, site and relevant observed characteristics.

In a fictional study of 200 participants, 20 lack a follow-up outcome. If 18 are from one site that lost access to its records, the problem differs from 20 scattered non-responses across all sites. The overall 10% does not convey that distinction.

Also count the participants who would remain in the proposed complete-case model. Modest missingness across several different variables can result in a much smaller complete-case sample.

Use contact logs, collection notes and instrument logic where available. Distinguish refusal, inability to contact, equipment failure and administrative omission if the data support those categories.

Do not assign a reason based on assumptions about the participant. “Lost to follow-up” describes an observation status; it does not explain the mechanism that produced it.

Compare observed characteristics of those with and without outcome data, but recognise the limit: observed data cannot by themselves establish that missingness is unrelated to unobserved values.

Complete-case analysis is not automatically acceptable below a particular missingness percentage. Multiple imputation is not an automatic repair above it. Validity depends on the estimand, model, missingness process and assumptions.

Sterne et al. Multiple imputation for missing data: potential and pitfalls. 2009. explains both the potential and pitfalls of multiple imputation. Hughes et al. Accounting for missing data in statistical analyses: multiple imputation is not always the answer. 2019. further shows why choosing between methods requires more than assuming imputation is always preferable.

If imputation is used, its model should be justified and compatible with important features of the analysis. Do not treat software defaults as a methodological rationale. Complex settings, including clustering and longitudinal data, may need specialist input.

Ask how conclusions might change if participants with missing outcomes differed from those observed in ways the main analysis does not capture. The sensitivity analysis should address a meaningful assumption, not simply run a second convenient model.

Document decisions before interpreting results where possible. If the plan changes after examining the data, explain the change and its reason.

Describe missingness for key variables, the analysis population, the method used and the assumptions on which it relies. State the number of participants contributing to each main analysis rather than implying that everyone recruited contributed equally.

The first question is not “Which command handles missing values?” It is “What information is absent, and what would need to be true for this analysis to answer our question?” Start there, then use the statistics and results collection to place the decision within the wider analysis plan.