As discussed in the first article of this series (doi: 10.5867/medwave.2014.05.5966), there are instances where the evidence from scientific studies helps us make better decisions about the care we provide for our patients [1].

These instances are characterized by the fact that we are able to identify two or more care alternatives for a specific clinical setting, and which can be translated into a specific clinical question [2] that usually comprises the following elements: a Population (typically a disease, e.g. a population of patients with diabetes); a clinical course of Action to be assessed (a therapeutic Intervention or a diagnostic Test); a Comparison (the alternative to which the assessed action is compared, e.g. Is test A better than B?, or Is medication A superior to administering nothing at all?); and the Aim being pursued (in the case of therapy decisions this aim is the Outcome; e.g. to reduce mortality or improve quality of life).

What sort of evidence to look for

Systematic reviews were invented to deal with the difficulty in having several studies answering similar questions ‒often with discordant results‒ thus enabling what we call evidence based medicine [3]. A good systematic review constitutes the best evidence (a more reliable study design or with a higher probability of being true), at least at the time of publication, since it synthesizes all existing information for a specific question by going through a process that minimizes bias and error. Systematic reviews are essential in evidence based practice, and therefore should be the main aim of our search [4]. Unfortunately, with the increasing output of information these become outdated relatively quickly, and it is difficult for users to know if they are still valid unless compared to other similar reviews or by conducting an exhaustive review of the literature [5],[6].

How to select the “best evidence”

Contrary to what occurred years back, today we have scientific evidence (including systematic reviews) for the vast majority of decisions that clinicians make in their regular practice, and the volume of knowledge output is multiplying at great speed [7]. It could be said that we are in full “medical knowledge transition” from the lack of evidence to excessive evidence, especially in the topics of greater interest to clinicians and researchers. For this reason, today it is especially relevant to decide which information is the most reliable or, in practical terms, which systematic review is the most reliable among all those available.

If we are able to obtain all relevant systematic reviews, we will have the best possible approximation (without performing a new systematic review) to the “body of evidence” related to our question [4].

The three main elements proposed to determine which the best systematic review is, are:

• Date of publication (or, ideally, the date of the search conducted by the systematic review): for obvious reasons, more recent reviews have a greater probability of including the updated body of evidence for a question.
• Number of studies included: the greater the amount of studies (germane to the question asked), the greater the probability of it being representative of the body of evidence.
• Quality of the review: better quality reviews have a greater probability of finding the body of evidence and of avoiding errors and biases that lead to conclusions which are far from the truth.

Where to search for evidence

There are multiple search resources, such as general search engines (Google, Wikipedia), health research databases (PubMed), and databases specialized in health evidence. This is a rapidly changing area, where there is still no definitive answer to “which is the best place to search for?” In this article we propose to always begin with database specialized in health evidence Epistemonikos (8). For a more comprehensive review of the various alternatives and approximations to evidence, we recommend reading the respective chapter in “Users' Guides to the Medical Literature” [9].

Epistemonikos (“what is worth knowing”, in Greek) is a multilingual, collaborative database, which attempts to group together all relevant evidence for health decision-making, and it is currently the greatest collection of systematic reviews and other types of evidence worldwide. A unique characteristic of this database is that it connects different articles answering a similar question, and it offers visual tools to help determine which is possibly the best available evidence [10].

How to choose the terms for a search using Epistemonikos

A good search begins with the correct formulation of the question [2]. To transform the clinical question into a search strategy it is sufficient to follow these recommendations:

1. Not all components of the question should be used in the search. A search with few terms is generally safer. Typically, for a treatment question, one term is sufficient for the intervention and another for the population of interest.
2. Comparison is only used if it is an active comparison. It is unnecessary ‒and could even be counterproductive‒ to use terms such as ‘placebo’, ‘no treatment’, ‘standard therapy’, and so on.
3. The aim (or Outcome) is not usually part of the search strategy. There are multiple relevant outcomes for each question, and how they are denominated and measured varies greatly.
4. Avoid terms that are not specific or unrelated to the question components. For example, terms such as ‘treatment’, ‘prevention’, or ‘diagnosis’.
5. Rarely use Boolean operators: Epistemonikos, same as most sites, allows using the main logical or Boolean operators to combine terms cases (see Figure 1). They can be useful in selected. The advanced search interface allows traditional Boolean strategies.
   

Figure 1. Logical or Boolean operators

Systematic reviews in context

Not all systematic reviews are the same. Hence we should learn to recognize how reliable they are [11]. Looking to see in which journal they are published or who is the author is insufficient. The only exception are the systematic reviews conducted by members of the Cochrane Collaboration, which ‒on average‒ are of better quality than non-Cochrane reviews [12] (we will further elaborate on this in a future article).

Furthermore, systematic reviews are not designed for users unaccustomed to technical research language, and do not take into account all the elements necessary for decision-making. This is why it is often of much use to read what is known as a “structured abstract” of the review (abstract and independent critical analysis by another group).

On the other hand, various therapeutic alternatives or courses of action are ever more frequently found for one same clinical situation, and hence the answer to “which is the best management option for condition X?” will often require assessing multiple systematic reviews. Overviews of systematic reviews and clinical practice guidelines are a good alternative in these cases [13]. Nevertheless, given the breadth of their approach, producing them is expensive and they are outdated more quickly than individual reviews.

How to access the best evidence in 4 steps using Epistemonikos

Step 1: run a simple search
One characteristic of Epistemonikos is that it can perform a search in several languages, obtaining results that include not only articles in that language but all the articles. A simple search such as “chewing-gum abdominal surgery” or “abdominal postoperative chewing-gum” is sufficient to find relevant information. If the terms are known in English they can also be used, or a combination of terms in English and any other of the 9 languages supported by the database.

Step 2. identify a systematic review that answers your question
From the search results it is necessary to identify a systematic review that answers the question. For this purpose, the most efficient way is to use the connections among articles in the database. For example, if a study is found that answers the question, we can follow the connection toward the systematic reviews that include the study (see Figure 2).

The filter for systematic reviews to the left of the search engine can also be used.

One very important point is that systematic reviews have been selected by collaborators following strict criteria, so the user does not need to determine if it is really a systematic review or not. Traditional search engines such as PubMed provide “possible systematic reviews”, and hence the user should discriminate if the result is or not a systematic review.

Figure 2. Connection between a primary study and the systematic reviews that include it in Epistemonikos (www.epistemonikos.org)

Step 3: analyze the body of evidence for the question
Once a review has been selected we should use the tools offered by Epistemonikos to assess other systematic reviews for the same question.

When accessing the article abstract (by clicking on the title) a diagram appears to the right showing the systematic reviews that answer a similar question and a link to the “matrix of evidence” tool (see Figure 3).

Figure 3. Connection between a systematic review and related evidence

The matrix of evidence shows all systematic reviews sharing primary studies with the original review, and all of the studies identified among these reviews (see Figure 4). The percentage of primary studies in common is an efficient indicator of the degree of similarity of the question. In other words, if two reviews share a study, they probably answer a related question; if they share many studies they probably answer the same question.

Figure 4. Screen capture of a matrix of evidence

The user can edit the matrix of evidence to reflect the question asked. Typically, reviews and studies answering related questions but not exactly the same question should be removed.

Matrixes can be saved to the user account, and whenever new related evidence appears, an automatic warning is shown, making it very easy to keep the “body of evidence” updated for a specific question, situating the new evidence in this context (see Figure 5).

Figure 5. The matrix of evidence tool

Step 4. situate the best evidence in context
Not all systematic reviews are equally reliable (even if including all existing studies for the question asked) and are not specifically designed for a user unaccustomed to technical research language. Furthermore, these reviews do not consider all the elements necessary for decision-making. That is why it is often more useful for us to read a structured abstract of the review (abstract and independent critical analysis by another group), an overview of systematic reviews, or a clinical practice guideline. All these types of evidence are interconnected in Epistemonikos.

Notes

Interests
Epistemonikos foundation is an organization that intends to bring information closer to health care decision-makers through the use of technology. Its main output is the Epistemonikos database (www.epistemonikos.org).

The authors have completed the ICMJE. IN declares not to have potential conflicts of interest with the subject of this article and has received fees for methodological support in the preparation of clinical practice guidelines in the last 36 months by the American Gastroenterological Association and the Kingdom of Saudi Arabia. JC declares not having potential conflicts of interest with the subject of this article.

Figure 1. Logical or Boolean operators

Figure 2. Connection between a primary study and the systematic reviews that include it in Epistemonikos (www.epistemonikos.org)

Figure 3. Connection between a systematic review and related evidence

Figure 4. Screen capture of a matrix of evidence

Figure 5. The matrix of evidence tool

Esta obra de Medwave está bajo una licencia Creative Commons Atribución-NoComercial 3.0 Unported. Esta licencia permite el uso, distribución y reproducción del artículo en cualquier medio, siempre y cuando se otorgue el crédito correspondiente al autor del artículo y al medio en que se publica, en este caso, Medwave.