Systematic reviews

Aimshutterstock_105425666

To describe the documentation that is necessary when undertaking a systematic review.

 

Requirements

Clear documentation of the various steps undertaken during the review process.

 

Documentation

  • Search and selection criteria;
  • The full search that has been performed, mentioning the database with suppliers, the search dates and all search strategies with all search terms (in a appendix);
  • Selection of the articles presented in a flow diagram;
  • Evaluation of the methodological quality (i.e. risk of bias assessment) of the articles;
  • Data analysis and evaluation of the quality of the evidence.

 

Responsibilities

 

Executing researcher
Project leader
Ensures the necessary background knowledge is present, gained through sources and experts, prior to writing the protocol.
Consult sources and content experts before writing the protocol, if necessary.
Write the protocol. This should include the following broad topics:  Criteria for including studies (i.e. inclusion/exclusion criteria); Search methods for identification of studies; Data collection and analysis (e.g. selection of studies, data extraction and management, assessment of risk of bias, measures of treatment effect).
Inspect the protocol by reading and adding comments.
Consult a search specialist in the process.
Consult a search specialist in the process, if necessary.
Include two independent reviewers for the selection of the abstracts and full-text articles, and risk of bias assessment.
Ensure that the selection of articles and risk of bias assessment has been carried out by two independent reviewers.
Document the selection and evaluation of the articles in the report.
Check the documentation of the executing researcher as proposed in the section ‘documentation’.
Select a report format and follow the guidelines described for a proper report. See the ‘Checklists for writing’ in the table below.
Review the data extraction and evaluation for the most important outcome measures in the review.
If the executing researcher is also the project leader then ensure that your documentation is reviewed by an independent researcher. This may include a co-author.

 

How To

A systematic review is a comprehensive overview of the available evidence based upon the literature within a particular topic. In this guideline, we focus on the quality aspects when conducting a systematic review. It is imperative that you consult other sources when preparing your review, such as the Cochrane handbook. It is important to note that a systematic review is written together with at least two persons, preferably with the help of a search specialist. At least one reviewer should be a content expert. A well-conducted review includes the following steps:

 

Preparation
Books:
  • Cochrane Handbook for Systematic Reviews of Interventions (editor Julian P.T. Higgins). http://handbook.cochrane.org/
  • ‘Meta-analyses in mental health research: A practical guide’ by Pim Cuijpers
It is advisable to check the Prospero database for ongoing reviews in order to see if your review is currently being conducted, and to register your topic.

 

Library websites:

 

Type of systematic review
Guidelines for executing
Checklists for writing
Interventions; (R)CT’s
–       CRDs Guidelines for Systematic Reviews
–       Cochrane Handbook for Systematic Reviews of RCTs
–       PRISMA
Diagnostic test accuracy
–       Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy
–       STARD
Measurement instruments
–       CRDs Guidelines for Systematic Reviews
–       COSMIN
Meta-analyses
–       Cochrane Handbook Chapter 9 & Chapter 19
–       PRISMA
Observational studies
–       Meta-analysis of Observational Studies
–       MOOSE

 

Course
It is advisable to follow a course in conducting systematic reviews, for example, the course offered by EpidM. The VU library offers an online search course.

 

Advice
Should you be interested in working on a systematic review which examines outcomes instruments, then we recommend contacting Dr. Caroline Terwee or Prof. Riekie de Vet (from the clinimetric group).

 

Protocol
Write a protocol for your systematic review before you start your search. Examples of protocols for a Cochrane review of intervention studies or for diagnostic reviews can be found in the Cochrane Library or CRD Handbook. The CRD Handbook also describes examples of other types of review, such as clinical tests, public health interventions, adverse events, or economic evaluations. For other types of systematic reviews, examples include:
Albers G, Echteld MA, de Vet HC, Onwuteaka-Philipsen BD, van der linden MH, Deliens L. Evaluation of quality-of-life measures for use in palliative care: A systematic review. Palliat Med. 2010;24(1):17-37.
van den Dungen P, van Kuijk L, van Marwijk H, van der Wouden J, Moll van Charante E, van der Horst H, van Hout H. Preferences regarding disclosure of a diagnosis of dementia: a systematic review. Int Psychogeriatr 2014;26:1603-18
Rietmeijer-Mentink M, Paulis WD, van Middelkoop M, Bindels PJ, van der Wouden JC. Difference between parental perception and actual weight status of children: a systematic review. Matern Child Nutr 2013;9:3-22.

 

A systematic review may age rapidly; therefore, you need to plan properly when you perform the literature search and when you evaluate the articles. It is, therefore, best if you have your search strategy, selection criteria, data extraction formula, and evaluation list ready prior to actually searching the databases.

 

Searching and Search specialist
It is advisable to consult a search specialist when undertaking a systematic review. The university library employs search specialists, who will be able to advise you.

 

Selection and evaluation of articles
The selection, assessment of risk of bias and data extraction should be undertaken by two independent reviewers. The initial selection should be based upon the title, abstract, and keywords, and the final selection should be based upon the full tekst and combined with the methodological evaluation of the article, although this is not advised. Make sure to document the selection, separately for both independent reviewers. Various standardized checklists are available for assessing the methodological quality of (randomized) controlled trials, diagnostic research (e.g. QUADAS) and observational studies. Consult the Cochrane website for the most appropriate tool.
Disagreements between the reviewers regarding the selection and methodological assessment can be discussed during consensus meeting. If necessary, a third independent reviewer should be consulted in order to reach consensus.

 

Documentation
Proper documentation of all decisions and choices made is essential for a systematic review. Maintain a separate logbook (see the ‘Logbook’ guideline) for systematic reviews. The documentation which is required is noted under ‘Documentation’ above.
In addition:
  1. Store all references retrieved in a database (e.g. Reference Manager, EndNote, Mendeley). Make a selection of the references included at a later stage and store this in a second database;
  2. Make a note of the reason for exclusion when excluding any references. Consult the PRISMA flow diagram and be familiar with what is required in the final report. Note down all the comments from consensus meetings. This prevents you from forgetting why a given article was or wasn’t included;
  3. Note down how many references in total have been retrieved, how many references you have excluded on the basis of the title, how many on the basis of the abstract and how many on the basis of full-text articles;
  4. Evaluation of the methodological quality of the articles: include, for example in a table, the reasons for allocating a particular score. This may appear to be quite a bit of work, but it will make your report more transparent for the reader.

     

Reporting
Use reporting guidelines specifically developed for different systematic reviews.
A number of tools are available for summarizing results. A quantitative analysis is commonly used to statistically pool the results of the included studies. The decision to conduct a quantitative analysis should be based upon the relative homogeneity of the data, which will determine whether statistical pooling of the results is possible. When heterogeneity exists (i.e. too much variation) between studies then a qualitative analysis will be necessary. This might be also necessary if too few data are available for pooling. Note: it is advisable to discuss the data analysis an presentation with an expert.
In a qualitative analysis the results are summarized in a systematic way, taking into consideration various aspects such as methodological quality, consistency of results, statistical significance and clinical relevance of the effects. The conclusion can be formulated by evaluating the strength of the available evidence. In order to summarize these effects and for formulating levels of evidence, it is advisable to use GRADE.

 

Audit questions

  1. Has the search strategy been properly documented (i.e. date of the search, databases which have been searched, search terms have been properly noted)?
  2. Has the selection of articles been properly documented (i.e. reasons for exclusion of articles; PRISMA flow-chart of inclusion of articles has been properly completed)?
  3. Has the risk of bias assessment been properly documented (i.e. assessments by reviewers has been included; conclusion about consensus)?
  4. Has statistical analysis been documented in the form of a meta-analysis, if appropriate?

 

Questions?
Contact Dr. Sidney Rubinstein <s.m.rubinstein@vu.nl>

 

V3.0: 28 Oct 2016: Revision guideline
V2.0: 19 June 2015: Revision format
V1.3: 1 Dec 2011: Reference to the Prospero database
V1.2:  22 Feb 2010: Addition search course.
V1.1:  1 Jan 2010: Translation into English and updated.
V1.0:  29 Sep 2006.