Supporting and monitoring data collectors

Aim

To secure accurate data collection data collectors, by supporting the data collectors and monitoring the data collected.

 

Requirements

  • Standardized data collection procedures, including how to cope with missing data;
  • Quality monitoring of first set of data collected for each measurement instrument;
  • Problems/issues/missing data to be documented in a logbook by the data collectors;
  • Immediate implementation of modifications in test protocol;
  • Regular meetings to discuss data collection issues between the executing researcher and data collector(s).

 

Documentation

  • Standardized protocol for data collection, including how to cope with missing data
  • Protocol for quality monitoring of data entered;
  • Logbook with problems/issues/missing data, kept by the data collectors;
  • Problems/issues that lead to modifications in the test protocol.

 

Responsibilities

Executing researcher:
  • To supervise and monitor the data collectors;
  • To carry out analyses on the first set of data to monitor the quality of the data.
Project leaders:
  • To ensure that the data collectors are being supervised and monitored by the executing researcher;
  • To ensure that the executing researcher carries out analyses on the first set of data to monitor the quality of the data.
Research assistant:
  • To check for completeness of measurement instruments;
  • Attempt to retrieve any omitted data as quickly as possible (i.e. by telephone/mail);
  • To note problems/missing data in a logbook;
  • To hand over the logbook to the data entry clerks along with the data.

 

How To

Once data collectors are recruited, it is important to keep track of their data collection, and to support them in any problem or issue. After all, they are collecting data for your project. So, it is important to intensively support and monitor the data collectors. A lot of information and guidance for data collectors is included in this PowerPoint presentation about quality principles.
The first data set collected by data collectors should be analyzed to monitor the quality of the data. This is important, because potential adjustments need to be included immediately in the protocol. Prior to data collection, it is advised to clarify to the data collectors which kind of quality monitoring for each measurement instrument will be conducted during the data collection/fieldwork. These agreements must be documented. Examples of quality monitoring include multiple testing (test-retest by the same and/or different data collectors), calibration, monitoring completed questionnaires. Audio or video recordings can also be made of the tests/interviews. Interviews/tests can be randomly checked by the researcher or another data collector. The quality of an interview can be discussed with the interviewer and potentially with other data collectors and researchers.
Analyses of the initial data set at variable level can provide insight into the quality of the data, the data collector(s) and the measurement instruments. For instance it is advised to review the distribution of the variables and to look out for outliers, impossible values and missing values. This analysis may lead to modifications of the test protocol. This analysis may also lead to modifications in number of participants to be included or tested on the basis of a new power calculation. The analysis of the initial data should definitely not be used to answer the research question.
The data collectors themselves should also check their collected data for quality. During data collection, it is important for data collectors to check questionnaires for completeness. The data collectors attempt to retrieve any omitted data as quickly as possible, i.e. by telephone or email. It is advised to create a list in advance for questionnaires, distinguishing between core variables for which missing data will always need to be retrieved immediately, i.e. primary outcome measures, and variables for which missing data are less important, e.g. 1 item from a multi-item scale in a questionnaire. The data collectors should be provided with clear instructions about retrieving missing data for various types of variables. Clear, written instructions should be given to data collectors about what they need to do if measurements have not succeeded or data are missing for other reasons. For these kind of issues data collectors should maintain a logbook and record what the most significant problems were during the data collection process. These logbooks need to be handed over to the data entry clerks along with the data. Problems have to be discussed regularly with the researchers. It is important to determine whether problems are incidental or structural.

 

 Audit questions

  1. Has the quality monitoring been carried out on the first data set collected by data collector(s) and have the results of the quality monitoring been documented?
  2. Which forms of quality monitoring have been carried out?
  3. Have problems during the data collection been documented in a logbook?
    1. If yes, how have these problems been solved?
    2. If no, why not?
  4. Have agreements been made about retrieving missing data?
  5. Have data been monitored at variable level during the data collection process?
  6. Have test protocols been modified during the data collection process?
    1. If so, on what bases and has this been documented?

 

V2.0: 12 May 2015: Revision format
V1.1:  1 Jan 2010: English translation

V1.0:  8 Jul 2008