Aim

Evaluating measurement properties

Description

The measurement properties of an instrument determine the quality of that instrument. Since some of the measurement properties are highly dependent on the population in which they are examined, the measurement properties need to be evaluated in the population in which you want to use the instrument.

Validity and interpretability are always important. When an instrument is used in a discriminative application, reliability is important, and when an instrument is used in an evaluative application, measurement error and responsiveness are important. When instruments are used to detect very small changes, or on individual level, it is important that the measurement error of the instrument is extremely small. Internal consistency is important for multi-item (sub) scales measuring unidimensional constructs.

Important measurement properties are (Mokkink, 2010):

1 Reliability

• Internal consistency (i.e. the degree of the interrelatedness among the items)
• Reliability (i.e. the proportion of the total variance in the measurements which is due to ‘true’ differences between patients)
• Measurement error (i.e. the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured)

2 Validity

• Content validity (i.e. the degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured)
• Structural validity (i.e. the degree to which the scores of an HR-PRO instrument are an adequate reflection of the dimensionality of the construct to be measured)
• Hypotheses testing (i.e. the degree to which the scores of an HR-PRO instrument are consistent with hypotheses based on the assumption that the HR-PRO instrument validly measures the construct to be measured)
• Cross-cultural validity (i.e. the degree to which the performance of the items on a translated or culturally adapted HR-PRO instrument are an adequate reflection of the performance of the items of the original version of the HR-PRO instrument)
• Criterion validity (i.e. the degree to which the scores of an HR-PRO instrument are an adequate reflection of a ‘gold standard’)

3 Responsiveness

• Responsiveness (i.e. the ability of an HR-PRO instrument to detect change over time in the construct to be measured)

4 Interpretability
(i.e. the degree to which one can assign qualitative meaning - that is, clinical or commonly understood connotations – to an instrument’s quantitative scores or change in scores)
Two important issues that tell you about interpretability of the score, are minimal important change (MIC) value of an instrument (De Vet et al. 2006) and the occurrence of response shift (Sprangers and Schwartz, 1999).

We refer to www.cosmin.nl for design requirements and preferred statistical analyses for studies on each of these measurement properties, and to De Vet et al. 2011 for more information on evaluating measurement properties. For additional help with e.g. determining if a measurement instrument is appropriately validated for your purpose, or for help with validating the instrument of your interest, you can request for a consultation at the Knowledge Center Measurement Instruments (email: cb.terwee@vumc.nl)

 

 

• Beaton, D. E., C. Bombardier, F. Guillemin, and M. Bosi Ferraz. 2000. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 25:3186-3191.
• Van den Brink WP, Mellenbergh GJ. 1998. Testleer en testconstructie. Boom , Amsterdam.
• De Vet, H. C. W., R. W. Ostelo, C. B. Terwee, N. Van der Roer, D. L. Knol, H. Beckerman, M. Boers, and L. M. Bouter. 2006. Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. J Clin. Epidemiol.
• De Vet, H. C., C. B. Terwee, L. B. Mokkink, and D. L. Knol. 2011. Measurements in Medicine. A practical guide. Cambridge University Press.
• Fayers, P. M., D. J. Hand, K. Bjordal, and M. Groenvold. 1997. Causal indicators in quality of life research. Quality of Life Research 6:393-406.
• Mokkink, L. B., C. B. Terwee, D. L. Patrick, J. Alonso, P. W. Stratford, D. L. Knol, L. M. Bouter, and H. C. de Vet. 2010. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J. Clin. Epidemiol. 63:737-745.
• Sprangers, M. A. and C. E. Schwartz. 1999. Integrating response shift into health-related quality of life research: a theoretical model. Soc. Sci. Med. 48:1507-1515.
• Terwee, C.B., Jansma, E.P., Riphagen, I.I., de Vet, H.C.W. 2009. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual. Life Res. 18:115-1123 (open access).
  Wilson, I. B. and P. D. Cleary. 1995. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA 273:59-65.
• World Health Organization. 2011. International Classification of Functioning, Disability and Health (ICF). www.who.int/classifications/icf/en

Term	Explanation
Concept	Global definition and demarcation of the subject of measurement.
Construct	A well-defined and precisely demarcated subject of measurement. By psychologists used for unobservable characteristics, such as intelligence, depression, or health-related quality of life.
Conceptual model	Theoretical model of how different constructs within a concept are related, e.g. the Wilson and Cleary* model of health status.
Patient-reported outcomes (PRO)	A measurement of any aspect of a patient’s health status that comes directly from the patient, without interpretation of the patient’s responses by a physician or anyone else.
Non-PRO measurement instruments	All other types of measurement instruments, e.g. clinician-based reports, imaging techniques, biochemical analyses, or performance-based tests.
Health-related quality of life (HRQL)	An individual's perception of how an illness and its treatment affect the physical, mental, and social aspects of his or her life.
HR-PRO	Health related patient-reported outcomes (HR-PROs): Modes of data-collection for patient-reported outcomes (PROs) include interviewer-administered instruments, self-reported instruments, or computer-administered instrument. With the restriction of health-related outcomes we exclude constructs such as quality of care, or patient satisfaction. Examples of HR-PROs are questionnaires assessing symptoms, functional status, and health-related quality of life.

V2.0 27 Mai 2011 Guideline 1.1B-08 rewritten and divided into 3 guidelines: 1.1B-08 a,b and c

Have the study questionnaires been translated into Dutch?
if yes: - have there been 2-3 forward translations by translators who are native speakers of Dutch and 2-3 back translations by translators who are native speakers of the source language?
- have the measurement properties of the questionnaire been assessed after the translation?
Have the measurement properties of the questionnaires used been assessed (by the researchers themselves or others)?
if yes: - did the validation study use a sample that is representative of the population in which the instrument will be used?
- was the validation study carried out in a different sample to that in which the questionnaire will be used?
- was the sample size adequate (at least 50)?
- were the appropriate methods and statistics applied (see checklist)?
- have all relevant measurement properties been sufficiently evaluated?
if no: will attempts be made to evaluate these properties?
- do the measurement properties of the questionnaires used satisfy the guidelines described in the checklist?
if no: have attempts been made to improve the measurement properties of the instrument?