Aim

Developing a measurement instrument

Description

When there is no instrument available that measures the construct of your interest, you may decide to develop a measurement instrument yourself. Therefore, the following steps need to be performed (see also Figure 1):

Step 1. Definition and elaboration of the construct intended to be measured.
The first step is to define your construct of interest. This could best be based on existing theory about the construct. When you are interested in a multi-dimensional construct, each dimension and its relation to the other dimensions should be described. It is also important to consider how your construct of interest differs from other closely related constructs. This can help you to further define what your want, but also what you do not want to measure. For example, if you want to measure participation, your need to think about whether participation is the same as social functioning or role functioning, or whether it is something else, and what makes it distinct from these other concepts. Models like the International Classification of Functioning (ICF) (WHO, 2011) or the model by Wilson and Cleary (1995) can be used as a conceptual model to relate the construct in which you are interested in to other related and unrelated constructs. In addition, you should decide whether the instrument will be based on a formative or reflective model (Fayer & Hand, 1997; De Vet et al. 2011). When all these issues are not described, it is very difficult, or impossible, to determine content validity, and to appropriately formulate hypotheses for assessing the construct validity of the instrument.

Step 2. Choice of measurement method (e.g. questionnaire, physical test)
The type of measurement instrument should correspond closely to the construct to be measured. Some constructs form an indissoluble alliance with a measurement instrument, e.g. body temperature is measured with a thermometer; and a sphygmomanometer is usually used to assess blood pressure in clinical practice. The options are therefore limited in these cases, but in other Physical functioning,       for example, can be measured with a performance tests, observations, or with an interview or self-report questionnaire. Often, the precise construct that is measured differs between measurement types. For example, with a performance test for physical functioning, information about what a person can do will be obtained, while by interview or self-report questionnaire information about what a person perceives he/she can do will be obtained.

Step 3. Selecting and formulating items
To get input for formulating items for a multi-item questionnaire you could examine similar existing instruments from the literature that measure a similar construct, e.g. for a different target population, and talk to experts (both clinicians and patients) using in-depth interview techniques. In addition, you should pay careful attention to the formulation of response options, instructions, and choosing an appropriate recall period (Van den Brink & Mellenbergh, 1998).  

 

Step 4 Scoring issues 
Many multi-item questionnaires contain e.g. 5-point item scales, and therefore are ordinal scales. Often a total score of the instrument is considered to be an interval scale. With this assumption, more statistical analysis are suitable to use. To calculate (sub)scores, however, you need to choose how you will calculate those. For example, are you going to add the items, use the mean score of each item, or calculate Z-scores. Also important to decide, is whether all items are equally important, or whether you are going to use (implicit) weights. Note, that when an instruments has 3 subscales, with 5, 7 and 10 items respectively, the total score calculated as the mean between the mean score of each subscale differs from the total score calculated as the mean of all items. Are all subscales unidimensional? And is there evidence that the subscores can be summed into a total score? What does this total score mean? These questions should be answered. Another important issue that you should describe is how to deal with missing values.

Step 5. Pilot study
Developing a measurement instrument is an iterative process. It is sensible to (regularly) test your instrument in small groups of people. A pilot test is intended to test the comprehensibility, relevance, and acceptability and feasibility of your measurement instrument.

Step 6. Field-testing
See guideline Evaluation of measurement properties.

 

 

For more information we refer to the following textbooks:

• De Vet et al. 2011. Measurement in Medicine.
• Streiner DL, Norman G. 2008. Health measurement scales: a practical guide to their development and use. 4th ed. Oxford University Press.
• Van den Brink WP, Mellenbergh GJ. 1998. Testleer en testconstructie. Boom , Amsterdam
  

Term	Explanation
Concept	Global definition and demarcation of the subject of measurement.
Construct	A well-defined and precisely demarcated subject of measurement. By psychologists used for unobservable characteristics, such as intelligence, depression, or health-related quality of life.
Conceptual model	Theoretical model of how different constructs within a concept are related, e.g. the Wilson and Cleary* model of health status.
Patient-reported outcomes (PRO)	A measurement of any aspect of a patient’s health status that comes directly from the patient, without interpretation of the patient’s responses by a physician or anyone else.
Non-PRO measurement instruments	All other types of measurement instruments, e.g. clinician-based reports, imaging techniques, biochemical analyses, or performance-based tests.
Health-related quality of life (HRQL)	An individual's perception of how an illness and its treatment affect the physical, mental, and social aspects of his or her life.

V2.0   27 May 2011 Guideline 1.1B-08 rewritten and divided into 3 guidelines: 1.1B-08 a,b and c.