Evaluating measurement instruments (questionnaires)

Aim

To evaluate the measurement properties of the measurement instrument(s) you are considering to use and to choose the most appropriate one.

 

Requirements

  • Knowledge about reliability, validity, responsiveness and interpretability of the finally chosen measurement instrument(s) as well as the considered alternatives.

 

Documentation

List of the known reliability, validity, responsiveness and interpretability of the finally chosen measurement instrument(s), including references and clear argumentation why you choose this particular instrument.

 

Responsibilities

Executing researcher: Search for scientific articles providing information about the important measurement properties of the measurement instrument(s). When this is inadequate, chose different instrument or develop new instrument (See guideline ‘developing measurement instruments’). When information on various measurement properties is lacking, perform clinimetric studies to fill the information gaps.
Project leaders: Discuss important measurement properties the measurement instrument(s) and consider alternative measurement instrument(s) when the properties are inadequate
Research assistant: N.a.

 

How To

The measurement properties of an instrument determine the quality of that instrument. It is important to use an instrument of which reliability, validity, and responsiveness are evaluated in the population that you are going to use it for.
Important measurement properties are (Mokkink et al., 2010):
  1. Reliability
    • Internal consistency, i.e. the degree of interrelatedness among the items.
    • Reliability, i.e. the proportion of the total variance in the measurements which is due to ‘true’ differences between patients.
    • Measurement error, i.e. the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.
  2. Validity
    1. Content validity, i.e. the degree to which the content of an health related patient-reported outcome (HR-PRO) instrument is an adequate reflection of the construct to be measured.
    2. Structural validity, i.e. the degree to which the scores of an HR-PRO instrument are an adequate reflection of the dimensionality of the construct to be measured.
    3. Hypotheses testing, i.e. the degree to which the scores of an HR-PRO instrument are consistent with hypotheses based on the assumption that the HR-PRO instrument validly measures the construct to be measured.
    4. Cross-cultural validity, i.e. the degree to which the performance of the items on a translated or culturally adapted HR-PRO instrument are an adequate reflection of the performance of the items of the original version of the HR-Pro instrument.
    5. Criterion validity, i.e. the degree to which the scores of an HR-PRO instrument are an adequate reflection of a ‘gold standard’.
  3. Responsiveness
    • Responsiveness, i.e. the ability of an HR-PRO instrument to detect change over time in the construct to be measured.
  4. Interpretability
    • Interpretability, i.e. the degree to which one can assign qualitative meaning – that is, clinical or commonly understood connotations – to an instrument’s quantitative scores or change in scores.
Tip: A database of more than 300 systematic reviews of measurement properties of various measurements instruments can be found at http://www.cosmin.nl/ [2]. This website also contains relevant and up-to-date information on how to evaluate measurement properties of measurement instruments.

Appendices/references/links

  • [1] Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HCW. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidem 2010;63(7):737-745.
  • [2] http://www.cosmin.nl This website ontains relevant and up-to-date information on how to evaluate measurement properties of measurement instruments.

Audit questions

  1. Have the measurement instrument been translated into Dutch?
If yes:
  • Have there been 2-3 forward translations by translators who are native speakers of Dutch and 2-3 back translations by translators who are native speakers of the source language?
  • Have the measurement properties of the instrument been assessed after the translation?
  1. Have the measurement properties of the instrument been assessed (by the researchers themselves or others)?
If yes:
  • Was the validation study carried out in a similar sample to that in which the questionnaire will be used?
  • Was de sample size adequate (N>50)?
  • Were the appropriate methods and statistics applied?
  • Have all relevant measurement properties been sufficiently evaluated?

 

V3.0: 12 Aug 2016: Revision guideline
V2.1: 12 May 2015: Revision format
V2.0: 27 Mai 2011: Guideline 1.1B-08 rewritten and divided into 3 guidelines: 1.1B-08 a, b and c