Analysis plan

Aim

To promote structured targeted data analysis.

 

Requirements

  • An analysis plan should be created prior to the data analyses.

 

Documentation

 

Responsibilities

Executing researcher: To create the analysis plan prior to the data analyses, containing a description of the research question and what the various steps in the analysis are going to be.
Project leaders: To inform the executing researcher about setting up the analysis plan before analyses are undertaken.
Research assistant: N.a.

 

How To

An analysis plan should be created prior to the data analyses. The analysis plan contains a description of the research question and what the various steps in the analysis are going to be. The analysis plan is intended as a starting point for the analysis. It ensures that the analysis can be undertaken in a targeted manner.
However, both the research questions and the analyses may be revised during the data analysis. It may also be that certain options are not yet clear before the start of the data analysis. Even explorative data analysis is possible. The findings and decisions made during the analyses may be documented at a later stage in the analysis plan, meaning the analysis plan becomes a dynamic document. However, there is also the option of documenting findings and decisions made during the data analysis in SPSS syntax (see guideline 1.4-05 Documentation of data analysis). In this instance the analysis plan only serves as the starting point.
The concrete research question needs to be formulated firstly within the analysis plan; this is the question intended to be answered by the analyses. Concrete research questions may be defined using the acronym PICO: Population, Intervention, Comparison, Outcomes. An example of a concrete question could be: “Does frequent bending at work lead to an elevated risk of lower back pain occurring in employees?” (Population = Employees; Intervention = Frequent bending; Comparison = Infrequent bending; Outcome = Occurrence of back pain). Concrete research questions are essential for determining the analyses required.
The analysis plan should then describe which statistical techniques are to be used to analyse the data. The following issues need to be considered in this process and described where applicable:
  • Which (subgroup of the) population is to be included in the analyses?;
  • Data from which endpoint (T1, T2, etc.) will be used?;
  • Which (dependent and independent) variables are to be used in the analyses and how are the variables supposed to be analysed (e.g. continuous or in categories)?;
  • Which variables are to be investigated as potential confounders or effect modifiers and how are these variables supposed to be analysed? There are different ways of dealing with confounders. We recommend the following:
    1) correct for all potential confounders (and do not concern about the question whether or not a variable is a ‘real’ confounder). Mostly, confounders are split up in little groups (demographic factors, clinical parameters, etc.). As a result you get corrected model 1, corrected model 2, etc.
    2) if the sample size is not big enough relative to the number of potential confounders, than it is better to only correct for those confounders really of importance. To select the relevant confounders, mostly a forward selection procedure is performed. In this case the confounders are added to the model one by one. Subsequently, there will be considered to what extent the effect of the variable of interest is changed. Then first choose the strongest confounder in the model. Subsequently, repeat this procedure till no confounder has a relevant effect (<10% change in regression coefficient);
  • How to deal with missing values?;
  • Which analyses are to be carried out in which order (e.g. univariable analyses, multivariable analyses, analysis of confounders, analysis of interaction effects, analysis of sub-populations, etc.)?;
  • Do the data meet the criteria for the specific statistical technique?
A statistician may need to be consulted regarding the choice of statistical techniques.
It can be quite efficient to create a number of empty tables to be included in the article prior to the start of data analysis. This is often very helpful in deciding which analyses are exactly required in order to analyse the data in a targeted manner.

 

Appendices/references/links

 

Audit questions

  1. Has an analysis plan been created prior to the start of analysis?
  2. Has a concrete research question been formulated in the analysis plan?
  3. Have the points described under ‘How To’ been considered?
  4. Has a stepwise description of the analyses to be applied been provided in the analysis?

 

V3.0: 23 Jan 2017: Adaption of the example of analysis plan and minor revisions
V3.0: 13 Oct 2016: Minor revision
V2.0: 12 May 2015: Revision format
V1.2: 1 Jan 2010: English translation
V1.1: 21 Jan 2008: Text in guideline has been re-written with more emphasis on a flexible approach