Aim

- Recruiting well-qualified data collectors
- To standardise data collection as best as possible

Description

A good data collector for scientific research needs to have a number of specific qualities that can be used as selection criteria when recruiting. The qualities required may, of course, differ from study to study. It is good practice to create a list of qualifications in advance detailing the aspects the data collector(s) need(s) to be able to fulfil for your study (see examples in details).

The data collectors need to be given sufficient background knowledge about the study. They need to know how and why certain details are being collected (the ultimate aim or the specific information the test/questionnaire will provide). This knowledge can be acquired through a protocol and training (where required).

The necessary background information needs to be established in a comprehensive data collection protocol, including a description of all the measurement/interview components. The data collectors need to be aware of this protocol.

For many studies it is important that the data are collected in an as standardised way as possible. This means that the interviews or test schedules need to be standardised. The data collector needs to conduct the interview or present the test in exactly the same way each time (as little as possible intra-observer variation). Comprehensive instructions and interview or test training will be required for this. Where there are multiple data collectors, mutual differences will need to be measured and minimalised (as little as possible inter-observer variation). Comprehensive training will also be required for this.

To train the data collectors properly, it is important to create a training protocol before the start of the study. This protocol will include: the instructions to be provided, who is conducting the training and whether the training will be repeated during the data collection. It is also important to indicate at what stage the data collectors will have received sufficient training. Fixed guidelines can be drawn up for this by setting a specific threshold (for instance Cohen’s Kappa > 0.6 for inter-observer variation), or the requirement that the data collector meets the same level of measurements/observations as a gold standard or highly experienced data collector. If there are multiple data collectors active within the study, the inter-observer variation needs to be measured and minimalised through training.

New data collectors, starting during the data collection process, should be trained by the same individuals who provided the original training, and not by other data collectors.

N.b.: In some cases it may be necessary for measurements to be generalisable to daily practice (e.g. effects of usual care). In this event comprehensive training will be undesirable.

A number of examples of qualification requirements for the data collectors are summarised below. These can be used when recruiting staff:
•           Trainable for conducting interviews/tests;
•           Experience or affinity with participant;
•           Knowledge of, or experience with the research topic;
•           Experience working with GP’s, doctors, specialists, etc.;
•           Experience using data files;
•           Experience using specific computer programmes;
•           Able to undertake simple administrative tasks;
•           Able to work with high levels of accuracy;
•           Able to work independently;
•           Have good communication skills;
•           Presentable;
•           Flexible;
•           Available for the whole study period;
•           Own car.

Experience in data collection might sometimes be helpful, and sometimes be unhelpful. From methodological literature about surveys it is known that data collectors’ experience is negatively associated with data quality. An explanation for this is the fact that people develop their own style in order to obtain an answer quickly and efficiently (they may prompt a response, or make suggestions, or just jot something down as they know that item non-response is not preferred, etc., etc.). They also appear to be less trainable for new studies. In large American surveys interviewers are sometimes replaced after 50 interviews in order to randomise background noise.

Some of practical tips about recruiting research assistants and procedures can be found here.

V1.3: 1 Jan 2010: Translation into English.
V1.2: 8 Jul 2008: Minor textual modifications.
V1.1: 19 Oct 2006: Reference to the quality handbook.

1)         Has a qualification list been created for recruiting the data collectors?
Have the data collectors been informed of the aim of the study and the tests/questionnaires?
2)         Have all the measurement/interview components been established in a protocol?
3)         Has a training protocol been established for the study?
4)         Have the data collectors been trained? If so, how and by whom?
5)         Are multiple data collectors involved in the study? If so, which measures have been taken to measure and reduce mutual variation?
6)         Have new data collectors been recruited during the data collection process? If so, who trained these data collectors? Which measures have been taken to measure and reduce mutual variation between the new data collectors?