Palabras clave: evidence gap maps, evidence-based medicine, systematic reviews as topic, data visualization

Introduction

The volume of scientific information being generated is overwhelming and is currently on the order of one manuscript every 10 to 20 seconds[1]. The large number of primary and secondary studies added to the various databases (and other sources) generates the need to create a systematic, exhaustive, and graphic approach to facilitate the search for answers in decision-making[1],[2]. Among the many available options for synthesizing evidence, the following designs stand out:

1. Systematic reviews, which have a rigorous methodology for producing detailed information on specific questions from the synthesis of primary studies, but usually require a considerable investment of resources and time[1].
2. Rapid reviews, useful in scenarios with a narrow time-window to achieve the product, by simplifying the steps of a traditional systematic review[3].
3. Scoping reviews, which explore the definitions and concepts of a particular field rather than answering a specific question[4].
4. Overviews of systematic reviews, which synthesize and critically evaluate the findings of multiple systematic reviews, in order to answer broader questions[5].
5. And evidence gap maps, discussed in the study.

Evidence gap maps are tools focused on facilitating researchers, clinicians, and policymakers (among other stakeholders) to visualize and explore the available evidence, both from primary studies and systematic reviews, around a specific question, through a graphical and ergonomic product[2],[6].

This article corresponds to the sixth in a methodological series of narrative reviews on general biostatistics and clinical epidemiology topics that explore and summarize, in an accessible language, published articles available via the main databases and specialized consultation texts. The series is aimed at training students, clinicians, and researchers. The Evidence-Based Medicine team from the School of Medicine of the Universidad de Valparaiso, Chile, in collaboration with the Research Department of Instituto Universitario Hospital Italiano de Buenos Aires, Argentina, and the Evidence Center UC, of the Universidad Católica, Chile have worked on the series. This article reviews the general aspects of evidence gap maps, main objectives and utilities, most relevant graphical options of mapping, interpretation, relevant aspects of their methodology, and comparison with other synthesis approaches.

General concepts

Evidence gap maps emerged in 2003, and by 2010, had less than ten publications[2]; however, over the next decade, they increased in popularity, reaching more than 73 publications as of 2017[6],[7],[8],[9]. The organizations most involved in their generation and dissemination include the International Initiative for Impact Evaluation (3ie) (www.3ieimpact.org) and the Campbell Collaboration (www.campbellcollaboration.org).

Evidence gap maps can be defined as thematic collections of evidence structured around a framework that graphically and schematically represents the types of interventions and outcomes relevant to a particular problem[6]. Therefore, they are conceived as a systematic, illustrative, and broad organization, with the intention of characterizing the breadth, depth, and methodology of the relevant evidence in relation to a certain question, finally elucidating the gaps in knowledge[10].

In 2016, Miake-Lye et al. conducted a systematic review about the definition and methodology of evidence gap maps, whose findings show heterogeneity when describing and developing them. Their results highlight the following key components to define an evidence gap map:

1. The review of evidence to identify gaps or need for future research.
2. The involvement of the target audience in the design of a user-friendly product.
3. The coverage of a broad field of evidence.
4. The systematization of the process.
5. The visual or graphic representation of the findings[2].

In summary, evidence gap maps involve a systematic search in order to identify the available evidence on a broad topic, and the systematization of knowledge gaps and/or future research needs, presenting the results in a user-friendly format, often with a graphical representation or accessible grid[2].

As a tool, evidence gap maps are restricted to providing information on the available evidence, knowledge gaps on different interventions and their outcomes, although the methodology is expanding to the areas of prognosis, diagnosis, and possibly other areas of knowledge. However, they do not allow answering a specific research question because they do not seek to synthesize the findings of the included studies. Consequently, evidence gap maps are not intended to provide recommendations or inform policy and practice but rather to be one of the sources of information in the development of these studies[11],[12].

Objectives and main uses

The purposes of evidence gap maps can be simplified into two broad axes as follows:

1. To facilitate evidence-based decision-making by presenting the available evidence in an accessible format.
2. To identify gaps in the available evidence in a user-friendly way, which can serve as an input to determine where future research should focus using limited resources in a more strategic way[6],[11].

1. To facilitate evidence-based decision-making by presenting the available evidence in an accessible format

Systematic reviews are frequently used to answer clinical questions, but with their increasing production it is becoming more difficult to identify which one offers the best evidence. Despite the efforts made by the organizations focused on evidence synthesis, including Cochrane and Campbell, systematic reviews are being published in a variety of information sources, without a mandatory central repository for authors or publishers. Furthermore, systematic reviews may be available in inaccessible formats (other than the traditional peer-reviewed and published manuscript, e.g., preprints, proceedings, protocols, conference abstracts, etc.). Unfortunately, the methodological language sometimes makes it difficult for the untrained clinician to identify the main findings, and these findings fail to impact their work as they should[6].

In response, evidence gap maps provide an accessible summary, capable of providing information (e.g., existing systematic reviews on a given topic), allowing the reader to quickly and efficiently compare the availability and quality of evidence on different interventions. However, if trained research teams are available for this purpose, the evidence gap maps can be produced quickly (in less than six months), while identifying the most up-to-date and best quality systematic reviews and ensuring the availability of updated findings when required by stakeholders[6].

2. To facilitate the strategic use of research resources by identifying gaps in the available evidence where future research should be focused

Asking a necessary and appropriate research question remains a challenge. Much of the available evidence does not meet this feature, and a surprising number of studies lack the necessary quality, development, and analysis[10]. In this way, more than 85% of the global annual investment in research is allocated erratically due to correctable problems[13], such as choosing to answer a non-priority question, conducting poorly designed studies, and failing to properly and transparently report on the results of the investigations[10]. In addition to this is the lack of systematization in the evaluation of the body of evidence in the field to be investigated[10], which gives rise to selective citation, cherry-picking, or the fallacy of incomplete evidence. This consists of the usual practice of choosing those papers that are more available, more attractive, or those that confirm our beliefs, ignoring evidence that is less accessible or that contradicts us[14].

Thus, the evidence gap mapping approach offers solutions. The visual representation it provides allows us to establish areas where new primary studies or systematic reviews can add more value depending on the type of gap identified. The types of breaches are as follows:

• Absolute gaps: Where there is little or no evidence, new primary studies would contribute to generating the necessary knowledge and as a substrate for future systematic reviews. It should be noted that the initial explorations can be a starting point for the emergence of new questions and lines of research, broadening the field of study[6].
• Synthesis gaps: Where there are a large number of primary studies, the synthesis of which in a systematic review can add particular value. By mapping the availability and quality of existing evidence syntheses, evidence gap maps can identify areas with low-quality systematic reviews or areas where an update of existing systematic reviews is needed⁶. Therefore, the development of an evidence gap map may be part of the initial steps in developing a systematic review protocol[15].

In addition, evidence gap maps can highlight a variety of issues related to the quantity, quality, and characteristics of the existing body of evidence. Thus, they can inform a strategic approach to evidence development on a particular problem. In fact, by identifying the areas of high relevance for policy development with a lack of evidence, evidence gap maps can be a tool to guide researchers and funders to ensure that limited research resources support studies that address the critical evidence gaps[6].

General methodology

Understanding the methodology of evidence gap mapping is a key element in characterizing this approach and differentiating it from other study designs. The following is a general summary of their methods (For a detailed guide with recommendations for conducting an evidence gap map, it is suggested to review leading publications[6],[11],[12],[16]):

1. Development of the interventions and outcomes framework: Review key policy documents, including existing impact evaluations and systematic reviews[6], and also involve the defined target audience to ensure an adequate selection of critical interventions and outcomes if the purpose is to serve as a reference for policy-making[16].
2. Definition of the inclusion criteria for studies: Focus on systematic reviews to promote knowledge transfer from secondary evidence, in addition to including primary studies. Inclusion criteria may depend on the extent of the map and available resources.
3. Search and selection of relevant studies: Use systematic and exhaustive methods (depending on time and available resources) for the identification and selection of the reports of systematic reviews and/or primary studies.
4. Extraction of data and critical appraisal of the retrieved studies: Retrieve data according to a data extraction form agreed by the authors and summarize the included studies[6]. Critical appraisal of studies is desirable, but not mandatory[15].
5. Analyses and graphical or visual representation: Fill in the evidence gap map with the information retrieved in the corresponding cells. Perform a descriptive analysis of the available studies, noting implications for practice and policy.

As mentioned previously, there is no standardized method for conducting an evidence gap map to date[2]. A recent instrument that can provide guidance for reporting this type of synthesis is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews (PRISMA-ScR), which provides a list of key items to check in each section of the manuscript to be published[17]. For more information on this extension, it is recommended to consult the article on panoramic reviews belonging to this methodological series or consult the extension directly at http://www.prisma-statement.org/Extensions/ScopingReviews[17].

Mapping forms and interpretation

The presentation of the results of an evidence gap map is varied, with three main forms as follows:

1. To include a visual representation of the results within the publication.
2. To reference a database that virtually hosts the information to be consulted.
3. To employ a process that culminates in a synthesis of the existing body of evidence without a graphical representation[2].

Most evidence gap maps use variations of crisscrossed tables (or grids) for the main findings, aggregating the count or sum of publications, organized into various domains, most often categorized by the available interventions and/or study designs while considering outcomes. It is also possible to find flow and bar charts to summarize the main findings of an evidence gap map. It is common to find more than one graphical representation per article[2].

The most commonly used forms of mapping are described in the following sections.

Bubble plot

This format graphs the information in five dimensions as follows:

1. The x-axis represents effectiveness in categories (e.g., no effect, uncertain effect, with effect, among others).
2. The y-axis represents the size of the body of available evidence (number of studies).
3. Each bubble represents an intervention.
4. The size of these bubbles corresponds to the number of studies for that intervention.
5. The color usually represents the risk of bias of the items for that intervention, but sometimes another meaning is given, which should be explicit in the map[15].

A bubble plot used to report the findings in a published evidence gap map can be found in Figure 1[18].

Figure 1.Bubble plot used by an evidence gap map.

Intervention-outcome framework 

One type of format frequently used is a digital grid that includes in-terventions and outcomes, with links to summaries of the studies included in their evidence gap maps[15]. Usually, the rows and columns correspond to interventions and outcomes, respectively. When plac-ing the cursor over an intervention or outcome, the platform reveals a brief description. The bubbles positioned in the intersection cells of both axes represent the studies in a given area, and their size is related to the number of studies. The color of the bubbles indicates the type of evidence and its degree of confidence in relation to their effects, with three colors assigned to systematic reviews (green: high confi-dence; orange: medium confidence; red: low confidence), purple assigned to protocols of a future systematic review, and gray assigned to impact evaluations[19]. This format is frequently used by the 3ie organization, an example of which is shown in Figure 2[20].

Figure 2. Digital intervention–outcome framework.

Another framework is used by the Sightsavers organization (www.sightsavers.org) (Figure 3). Their evidence gap maps are based on a matrix designed to capture evidence on specific interventions or areas. The areas for which reviews were found are located in the col-umns, while the rows correspond to the strength of the evidence. Bubbles within the cells denote the existence of systematic or narra-tive reviews, whereas placing the cursor over one of them displays a summary page[15]. The color code represents the methodological quali-ty of the review (green: high; yellow: medium; red: low)[21].

Figure 3. Glaucoma evidence gap map from the Sightsavers organization.

Flow or bar diagrams

Some authors of evidence gap maps have reported their main findings in flowchart formats (Figure 4) or bar charts (Figure 5), being like the diagrams widely used to describe the systematic search process in scientific articles.

Figure 4. Evidence map on publications of sugar-sweetened concoctions by outcome and study type.

Figure 5. Bar chart of health topics explored within the references included in an evidence gap map.

Comparison with other types of evidence synthesis 

Evidence gap maps and systematic reviews have in common their secondary study design, which collects the available evidence through a comprehensive systematic search approach, with an identical screen-ing process of studies. Compared to systematic reviews, evidence gap maps have a broader scope, require less data extraction, and do not synthesize effect estimators. They do not require critical appraisal of the quality of the evidence (although it is recommended)[15] and can be conducted in less time[6]. The product is also different; evidence gap maps report the available evidence (not the results of the interven-tions) to be used in the prioritization of future research, unlike sys-tematic reviews that report the results of the included studies, con-tributing to knowledge translation and the generation of recommen-dations for clinical practice and health policy-making[15].

Scoping reviews, such as evidence gap maps, correspond to an evi-dence synthesis approach that aims to map the existing literature in an area of interest[4]. They are the most similar study design, with identical methodologies to the point of sharing the PRISMA-ScR[17]. The main differences are that evidence maps involve the target audi-ence earlier in the research process, the rigor of the search strategy, and offering an accessible visual product or grid in a user-friendly format[2]. 

Table 1 describes the main differences of evidence gap maps with other options of evidence synthesis. 

Table 1. Comparison of evidence gap maps with other forms of synthesis.

Conclusions

Evidence gap maps are a relatively new tool in literature review, offering a systematic search approach that identifies a field of available evidence, highlighting areas where knowledge gaps exist. They stand out for early involvement of the target audience, in order to design a user-friendly and easily accessible graphic product that summarizes the findings. The most commonly used formats are the bubble plot and the intervention–outcome framework.

Their production has been increasing in recent years, as they are of valuable use to decision-makers in defining the agenda and funding for future research, supporting evidence-based policy-making, and ensuring a faster and less resource-intensive final product than other study designs.

Notes

Contributor roles
BSM: conceptualization, investigation (original draft preparation), writing (review and editing), project administration. NMC, CL, JVAF: writing (review and editing). JPB: conceptualization, resources, writing (review and editing), supervision. EMA: writing (review and editing), supervision.

Acknowledgments
We are grateful to the Chair of Evidence-Based Medicine of the School of Medicine of the Universidad de Valparaíso, Chile, for promoting this series and the collaboration of the Research Department of the Instituto Universitario del Hospital Italiano de Buenos Aires, Argentina.

Competing interests
The authors declare that they have no conflicts of interest.

Funding
The authors declare that they have no external sources of funding associated with this article.

Language of submission
Spanish.

Figure 1.Bubble plot used by an evidence gap map.

Figure 2. Digital intervention–outcome framework.

Figure 3. Glaucoma evidence gap map from the Sightsavers organization.

Figure 4. Evidence map on publications of sugar-sweetened concoctions by outcome and study type.

Figure 5. Bar chart of health topics explored within the references included in an evidence gap map.

Table 1. Comparison of evidence gap maps with other forms of synthesis.

Esta obra de Medwave está bajo una licencia Creative Commons Atribución-NoComercial 3.0 Unported. Esta licencia permite el uso, distribución y reproducción del artículo en cualquier medio, siempre y cuando se otorgue el crédito correspondiente al autor del artículo y al medio en que se publica, en este caso, Medwave.