Research papers
Medwave 2019;19(11):e7748 doi: 10.5867/medwave.2019.11.7748
General concepts in biostatistics and clinical epidemiology: observational studies with cohort design
Gabriel Lazcano, Cristian Papuzinski, Eva Madrid, Marcelo Arancibia
References | Download PDF |
To Download PDF must login.
Print | A(+) A(-) | Easy read

Key Words: observational study, cohort studies, relative risk, incidence, bias, epidemiology, biostatistics

Abstract

Cohort studies evaluate the relationship between exposure to a specific event or phenomenon and the occurrence of an associated out-come of interest (or lack thereof). This methodological design has been widely used in certain areas of medicine, such as the study of cardiovascular risk factors and the effects of ionizing radiation in humans. It is a useful study design, especially for research involving low-occurrence exposures, because it can be easily adapted to various contexts. This article, which provides an overview of observational cohort studies, is part of a methodology series on general concepts in biostatistics and clinical epidemiology developed by the Chair of Scientific Research Methodology at the University of Valparaíso’s School of Medicine in Chile. It describes historical, practical, and theoretical concepts related to cohort studies; essential elements in cohort study design, and variations and derivations of it; potential types and sources of bias in these types of observational/longitudinal studies, and various methods researchers can use to address/minimize them.


 

Key ideas

  • Cohort studies analyze the relationship between exposure to a specific event or phenomenon and resulting outcomes and can, therefore, be used to estimate risk and establish causation.
  • This type of study design is especially useful for studying low-occurrence exposures.
  • Cohort studies can be expensive to carry out and difficult to reproduce. They are also prone to confounding and selection bias, but various methods can be used to address/minimize these effects
Introduction

The term “cohort” can be traced to the Roman cohors, an historical term for subunits of a legion whose origin is usually attributed to military reforms promoted by Roman general/statesman Gaius Marius in the 2nd century BC, which accelerated the transition from militias to professional armies[1]. In the epidemiological realm, we can find a primordial antecedent for cohort studies in life tables—instruments developed in the 17th century to document causes of death and estimate mortality; using the extensive documentation of recurrent epidemics, in 1662, English statistician John Graunt published the first life table, a format that would be refined by Edmond Halley in 1693[2]. At the end of the 19th century, the emergence of the modern insurance industry and the need to calculate projected risk of insured parties led to the creation of research formats designed to describe the natural history and effects of various interventions in American patients with tuberculosis in the early 20th century, and these studies resembled current prospective cohorts[3],[4],[5]. After the Second World War, flagship studies renowned for vast, rich data shaped both the contemporary cohort study and basic understanding of the concept of “risk”[6]; this included the Framingham Study on cardiovascular risk[6], which is still being implemented; the British Doctor’s Study, which generated important data on the risk of developing lung cancer from tobacco use[7]; and the study of Japanese survivors of both atomic bombs8, which, along with the cohort study of mortality in uranium miners in the Colorado Plateau[9], provided much of the current knowledge about the effects of radiation on humans[10].

Both its military origin—analysis of the experiences and functions of various groups in Roman legions—and its current, epidemiological meaning—grouping data by exposure to specific events or phenomena, such as risk factors or medical interventions—illustrate the prototypical concept of cohorts: classifying individuals with a common outcome by certain characteristics, a concept that is described in more detail below.

This is the fourth article of a six-part methodology series on general topics in biostatistics and clinical epidemiology, based on data from published articles available in the top medical/scientific databases and specialized reference texts. The series is designed for undergraduate and graduate students and is developed by the Chair of Scientific Research Methodology at the University of Valparaíso’s School of Medicine in Chile. The objective of this article is to provide an overview of the main theoretical and practical concepts of cohort studies.

Preliminary concepts

The cohort study has an observational, longitudinal, and analytical design: it analyzes the effects of exposure to a certain event or phenomenon that occurs “naturally” (without the intervention of the researchers), over time, and allows us to observe the resulting changes (or lack of them), using statistical hypothesis tests[11],[12].

Unlike case-control studies, which, as addressed in a previous article in this series, analyze the effects of exposure to different events or phenomena on an outcome of interest, cohort studies separate study population groups based on expected differences in outcomes following exposure to a common event or phenomenon. Based on the particular features of these two types of studies, another fundamental difference between them can be inferred—directionality—with case-control studies analyzing data from effect to cause, and cohort studies analyzing data prospectively (from cause to effect)[13].

Data collection for a cohort study can be prospective, retrospective, or ambidirectional/ambispective. Regardless of which of these three data collection methods is used, the analytical directionality in a cohort study will remain prospective (cause to effect). Using the first type of data, the study population’s baseline is exposure to a specific event or phenomenon, and the analysis identifies the occurrence of various expected outcomes (or lack thereof). With retrospective data collection, from the perspective of the researcher, both the exposure and the outcome have already occurred. In the case of ambidirectional or ambispective data collection, as described in Example 1, the exposure has already occurred, before the study, but an outcome has yet to materialize; this type of analysis is useful in assessing events/phenomena that take time to transpire and exposures that could trigger multiple outcomes of interest[14],[15].

Example 1. A study with an ambidirectional/ambispective cohort design, conducted in China[16], analyzed the adverse effects of therapy used in multi-drug-resistant tuberculosis. A retrospective study was carried out using the medical records of 751 patients who received treatment between May and July 2009, and a follow-up, a prospective evaluation, was conducted in December 2016.

Cohort studies analyze an element of causality or temporal relationship of cause and effect that is not present in cross-sectional or case-control studies[14],[17]. Therefore, no study participant—exposed or unexposed to the specified event or phenomenon—can have the outcome of interest at baseline. For example, in a cohort study of a specific pathology, every included study participant would, at baseline, be “healthy” for that condition.

This temporality (emergence of a specified outcome only after a certain lapse of time) is one of the nine criteria for cohort studies proposed by English epidemiologist and statistician Sir Austin Bradford Hill: 1) strength of association, 2) consistency, 3) specificity, 4) temporality, 5) biological gradient, 6) plausibility, 7) coherence, 8) experimental evidence, and 9) analogy[18]. Although this checklist is not meant to be a rigid one, it provides a valid framework for conducting cohort studies in epidemiological research and can be easily adapted to the current highly technical world in which molecular biology and genomics offer mechanistic explanations that complement and strengthen the findings found at the epidemiological level[18]. Of these nine criteria, temporality is the only one considered essential for describing causality between an exposure and an outcome[19].

For the reasons cited above, just as case-control studies have the best design for studying outcomes with very low occurrence, cohort studies have the best design for studying exposures of low occurrence, such as environmental disasters.

Designing a cohort study

There are five critical questions to answer when designing a cohort study[12],[20],[21],[22]: 1) “Who is at risk?”; 2) “Who should be considered exposed?”; 3) “Who constitutes an appropriate (unexposed) control?”; 4) “How will the occurrence of an event be objectively estimated?”; and 5) “How will the participants be monitored?”

The first question refers to the study participants’ susceptibility to developing the outcome of interest. Although both exposed and unexposed individuals can be included in the cohort, none of them should have experienced the outcome of interest at baseline, and everyone should theoretically be able to develop it. This goes hand in hand with the third question, which addresses the fact that unexposed individuals must have characteristics (age, sex, ethnicity, and socioeconomic status, among others) similar to those in the exposed group, to reduce potential biases. To be comparable, members of the unexposed group can have either an “internal” origin (e.g., admission at the same hospital, during the same period), relative to the exposed group, or an “external” origin (e.g., admission at a different hospital or records from a different set of national statistics); generally, the study of comparable groups with internal origin is more desirable[12].

The second and fourth questions refer to the essential characteristic of any robust study protocol regardless of its design: precise definitions of what is being studied. This implies, in the practical sense, the use of updated and clearly established clinical criteria, and clear explanations/descriptions of any instrument, scale, score, or discriminatory values ​​used in the study. In addition, researchers should always consider the characteristics of the population being monitored in case they could affect the measurement of the outcome of interest[11],[12],[13],[23].

The use of robust, standard methods for selecting the study population is also extremely important in cohort studies because losses to follow-up can unbalance cohort groups and affect the occurrence of the outcome(s) of interest, creating potential bias. As illustrated in Example 2, the choice of study design features can have unexpected outcomes and should be selected carefully[20],[21],[22],[24]. Various aspects of cohort study design that should be considered are listed below:

  • Survey method: face-to-face, telephone calls, or remote, such as electronic or paper surveys
  • Survey scope and frequency: using very long surveys or requiring numerous visits to health facilities can result in loss of interest or fatigue among study subjects
  • Logistics/availability of participating health facilities (if there is a need for population sampling or measurements)
  • Redundancy mechanisms: alternative ways of communicating with the patients in case they become unavailable or any changes in the usual means of contact arise.
  • Cost: ensure the sustainability of the study for the entire planned research period.

Example 2. The UK COSMOS study[22], conducted in the United Kingdom, seeks to assess the effects of mobile telephony on the health of its users. The authors, faced with a low response rate in the early stages of recruitment, decided to make more use of digital media, such as web questionnaires and text message reminders. Although they were able to improve recruitment levels using digital media, this change in the study design resulted in new challenges inherent to those chosen technologies, such as having to develop, and design attractive websites optimized for display on mobile phones and tablets.

Different types of cohort study designs: advantages, disadvantages, and potential uses

Given the very specific elements that are essential for carrying out a cohort study, it seems worthwhile to ask the following question before developing/designing one: “Is a cohort study the correct design for evaluating my research hypothesis?” Alternatively, more broadly: “What is the most appropriate design for answering my research question?” As long as the abovementioned criteria can be met, cohort studies are a good choice for analyzing a hypothesis, given the adaptability of the classic modalities of this type of study and their variations/derivations to different contexts and scenarios, according to the temporality of the data collection.

To summarize, the advantages of cohort studies include the directionality of their analyses (from exposure to outcome) and the temporality of their data collection, which allows for inference, with proper design, of a cause-effect relationship between the exposure and the outcome, when a significant statistical association is found. Therefore, adequately designed cohort studies allow for direct estimation of incidence and absolute and relative risks[12],[23],[25].

Prospective cohort study
A prospective cohort study design has innumerable advantages. First, once the required sample size for exposed and unexposed subjects has been attained, prospective assessments of an upcoming outcome can be carried out (the outcome must be a future event, however; it cannot have occurred prior to the study baseline). Another strength of this design is that it allows for direct assessment of the exposure, which enables the establishment of clear temporality regarding the occurrence of the outcome and the evolution of a condition, and ensures the quality of data collection, which is not registry-dependent. In addition, knowledge of the outcome will not affect the registration of exposures. Despite their relatively high cost, cohort studies are the best means of assessing exposure to events or phenomena of rare occurrence, especially those that cause damage (e.g., environmental disasters), given that the only other types of studies that can provide causality data are clinical trials, which, for ethical reasons, cannot be carried out to assess these types of events. For the same reasons, prospective cohort studies are also a good choice for describing and analyzing the natural history of a disease. 

One of the main disadvantages of the cohort design is its relatively high economic cost, which makes it inefficient for studying events of rare occurrence and challenging to reproduce in future studies. Furthermore, cohort studies have the same weakness as all observational designs, compared to their experimental peers—the potential for confounding variables that can distort results[11],[17],[23].

Retrospective cohort study
In retrospective cohort studies, both the exposure and outcome occurred before the analysis, and existing records are used to reconstruct the sequence of events (i.e., exposure and outcome are past events at the study baseline). In addition to having the abovementioned strengths of all cohort designs, retrospective cohort studies have a lower cost and higher performance versatility than prospective versions. However, these latter features are only advantageous if the required data are available and accurately recorded. In addition, researchers have limited control over the construction of the sample and the quality of the monitoring. Still, retrospective cohort studies are seen as the modality of choice for studying rare events and are an attractive alternative in hospital or institutional settings, where it is feasible to find good-quality records[23].

Ambispective or ambidirectional cohort study
The third variant of cohort studies, ambispective or ambidirectional studies, use preexisting data, document events, and phenomena that have already occurred, and allow for prospective/future monitoring. As a hybrid model, this type of cohort study has advantages and disadvantages compared to prospective and retrospective designs[15]. This type of cohort study is useful for analyzing exposures that may have both short- and long-term outcomes, or more than one outcome[12]. Another scenario in which this type of study can be useful is the analysis of health crises such as outbreaks, epidemics, or any other events that imply an unexpected exposure for which data must be collected retrospectively. Also, strictly speaking, lifestyle habit studies can be categorized as ambispective cohort studies since the exposure—which would in most cases, continue to develop over time—has already occurred before the study baseline[26].

Group cohort study
For group cohort studies, the essential elements are exposure-based sampling, follow-up, and calculation of absolute risk, and the existence of an unexposed control group is not a sine qua non condition[25]. Based on classical definitions, group cohort studies would be used for a case series[27]. However, this type of cohort design uses outcome-based sampling, and although it may consider a specific exposure, there is no follow-up or absolute risk calculation[25]; moreover, these types of case series can be “timeless”—that is, they can analyze cases that occurred at different times. The group cohort can also be useful for studying new interventions in small samples. The absence of an unexposed group prevents both hypothesis testing and the calculation of relative risks, but these types of studies can be used to record the incidence of outcomes of interest, such as adverse effects (Example 3).

Example 3. A group cohort study[28], conducted in Italy, documented the success rates and complications that occurred when using a flapless technique for installing immediate-loading implants in patients with edentulous jaw. In the study, 33 patients were recruited. The patients were monitored at the time of installation, and after 12 months of follow-up, to assess the stability of the implants.

Multiple-cohort study
Multiple-cohort studies can be used when different samples are subjected to different exposures or to different levels of the same exposure (when studying an unexposed group)[23],[29]. When monitoring more than one group, relative risk calculation and statistical association tests are possible (Example 4).

Example 4. A double cohort study[30] assessed the appropriate technique for minimally invasive surgical management of patients with rectal prolapse. Two cohorts were proposed, each with a different technique: laparoscopic ventral rectopexy, carried out in a health facility in the Netherlands, and rectopexy through laparoscopic resection, performed in the United States. The study designed according to the preferred technique in the respective countries of the two facilities participating in the study.

Use of external cohorts

External cohorts can be considered a particular type of multiple-cohort study in which preexisting external sources of information (e.g., censuses, population registries) are used. In this type of study, samples of individuals with similar characteristics but from another center or institution can also be included. One disadvantage of this study design is that basic characteristics differ across individuals from different groups. External cohorts can also be used when studying rare exposures or when it is impossible to recruit a control group[12],[23].

Case-control study nested in a cohort

A case-control study nested in a cohort study or nested case-control study is a case-control study of the individuals in the cohort who present the outcome. This research method was addressed in a previous article in this series (General concepts in biostatistics and clinical epidemiology: observational studies with case-control design, Doi 10.5867/medwave.2019.10.7716).

Case-cohort study

A case-cohort study design is derived from case-control study design but carried out using a cohort divided into two groups: a case group, which includes all those who developed the outcome of interest until a specific point in time, and a sub-cohort comprising randomly sampled individuals from the general cohort, regardless of whether they developed the outcome of interest31. This design is particularly useful for studies of large cohorts with multiple outcomes of interest where collecting data for each event would be inefficient. The advantage of this design compared to a nested case-control study is the ability to reuse the sub-cohort in the assessment of each different outcome (whereas a nested case-control study would require building a new control group for each different event). Because controls are representative of the general cohort, they provide a basis for estimating incidence and prevalence in the source population[31],[32],[33] (Example 5).

Example 5. The MORGAM (MONICA, Risk, Genetics, Archiving, and Monograph) Project[34] is an international collaborative study that analyzes the association between certain phenotypes and genotypes with the occurrence of cardiovascular events. The study compared cases identified from a randomly constructed sub-cohort. The efficiency of this type of cohort design is illustrated in this project: it was not cost-effective to perform genotyping for the entire cohort, so it was very useful to the researchers to have a sub-cohort that could be reused for several outcomes.

Frequency and association measures

Association measures can be expressed as probabilities (risks) or rates (incidence). The risk calculation determines the proportion of all individuals presenting the outcome of interest over the study period. The incidence calculation includes the unit of time, expressing the “speed” at which an outcome occurs[15].

The risk calculated for each group that makes up a cohort is called the absolute risk; it is possible to determine associations between various levels of absolute risk by calculating ratios (risk ratio or relative risk) or differences (reduction of absolute risk, risk difference, or attributable risk)[35]:

The calculation of incidence is mathematically similar, but it includes, in the denominator, the period when there is a risk of the outcome:


If the period in which there is a risk of occurrence of the outcome of interest is equal for all members of the cohort, the value obtained corresponds to the incidence rate; if the period of risk is different for different members of the cohort, the calculation includes the individualized period of risk for each member of the cohort, which is called, in that case, incidence density. In some cases, cumulative incidence, which is, mathematically, the same as absolute risk, and does not strictly correspond to a rate, is also included[35],[36].

In the literature, the operational definitions of these measures are often inconsistent, and/or confusing; researchers should be sure to provide clear/accurate descriptions of any measures they use[37],[38]. Because it is a unique and static value, the incidence rate may not be accurate or useful for reporting phenomena whose occurrence is variable over time. In these cases, survival analysis techniques can be used to analyze the outcome of interest in one or more populations. The use of life tables (actuarial analysis) or Kaplan-Meier curves will enrich the data that is generated[39]. Survival studies also allow for comparison of survival curves, which in turn allows for inferential analysis, to estimate differences between them, using parametric and non-parametric methods. Cohort studies are the most appropriate methodological design for survival analyses[39].

Biases and control strategies

Given their nature as observational and longitudinal studies, cohort studies can have multiple types and sources of bias. Among the different types of potential bias, selection bias resulting from loss to follow-up stands out. The first steps to address this type of bias should be taken in the design stage, as explained in the section above called “Designing a cohort study.” When losses to follow-up are unavoidable, artificial censoring to correct selection bias can be carried out using inverse probability-of-censoring weights; by estimating the probability of loss to follow-up, a correction coefficient is constructed that seeks to model the behavior of the sample without losses, allowing for correction of the risk estimate[40]. Using these simulations, it has been determined that losses to follow-up of 20% or more result in significant risk of bias, so significant effort should be made, including during the design stage, to avoid them[41]. So-called “healthy worker bias” can also occur when the risk of presenting a specific condition is compared between a cohort of workers that are exposed to a specific event or phenomenon and a cohort obtained from the general population that is unexposed to the event/phenomenon. In this case, the cohort of workers may appear to be a protective factor (relative risk less than 1), and their risk may even be less than the estimated risk for the general population, meaning that their specific type of work protects them from presenting the outcome of interest. However, this phenomenon could also be explained by optimal health conditions (young age, lack of chronic pathologies, i.e., "healthy" status) that these workers may have had before employment and/or that has been intensified through the labor selection process. This type of bias can be avoided by comparing different strata of the worker cohort, selected according to the number of weekly work hours, for example, or the type of work performed, among other factors[42].

The potential biases that require attention by researchers, especially during the design process, vary according to the type of cohort study that is used. In a retrospective cohort study or the retrospective part of an ambispective study, the quality and readability of the records used for data collection, and the participants themselves, are potential sources of bias, if questionnaires or interviews are used to gather data complementary to the exposure. In the first case (bias from medical records, for example), any missing data must be rigorously reported, but when the analysis is carried out, three different strategies are recommended[24]: 1) ignore the missing specific data and only include each individual in the analyses of data that he/she has; 2) censor/omit the entire record of any individual with missing data (regardless of what type of data is missing, omit the individual from all analyses); and 3) assign an average or baseline value. All three strategies skew the sample in unpredictable ways, so whichever one is used must be clearly described in the study protocol. In addition, any instruments used for questionnaires or in interviews must be clearly described, prior to their use, in language understandable to patients; in addition, whoever applies the instrument or conducts the interview should not know the exposure status of the participant[22],[43].

In prospective studies or the prospective part of an ambispective design, errors in measurement can lead to erroneous classifications of both the exposure status of the participant and the occurrence of the outcome of interest. This should be handled in the design stage, with all criteria clearly defined from both clinical and operational perspectives (e.g., threshold values, scales to be used, and laboratory methods, among others)[14],[26],[44].

The confusion phenomenon, addressed in previous articles in this series[45],[46], can also occur in cohort design. Therefore, the use of stratified analysis and multivariate statistical regression techniques could be useful in cohort studies to, through mathematical modeling, examine and control the effect of each of the recorded variables, identifying potential confounding variables[43].

Final considerations

While cohort studies have a strong historical background in the development of modern and contemporary medicine[10], they are far from being an outdated method, given the many tools available now for refining their results, and the new developments in biostatistics, and growing computer power, that allows for more agile implementation and application of them. Therefore, as some authors have pointed out, although the potential fit of cohort designs should be analyzed on a case-by-case basis, and these types of studies should not be applied randomly, or for mere novelty, the fact that they are not used as often as they could be is usually due to researcher habits (better familiarity with other study designs), or ignorance, rather than lack of feasibility or technological support[32],[40],[47].

This methodological design has allowed for the study of associations of great relevance for public health and medicine, especially in relation to exposures to harm. Cohort studies are the design of choice for studying incidence, survival of a condition, and protective and risk factors. Unlike other types of observational studies, cohort studies allow for the establishment of a causal relationship, because the type of data analysis they use meets the criterion of temporality—the one element essential for determining causal hypotheses—that is, moving from cause to effect (Figure 1).

Notes

Authorship contributions
CP, EM, MA are scholars in the Chair of Scientific Research Methodology at the University of Valparaíso School of Medicine (Chile), in which the development of this methodological series is described as a research activity of the teaching assistants of the course. All authors contributed to the planning and writing of the original manuscript. GL, CP, MA developed the Introduction and the sections on preliminary concepts, construction of a cohort study, and types of cohort designs. GL, CP, EM, MA contributed to the development of the section on frequency and association measures and bias and control strategies. GL provided the examples described in the article. GL, CP, MA developed Figure 1.

Funding statement
The authors declare that there were no external sources of funding.

Conflicts of interest
The authors have completed the ICMJE conflict of interest declaration form and declare that they have not received funding for the completion of this report; have had no financial relationships in the last three years with organizations that might have an interest in the published article; and have no other relationships or activities that could influence the published article. Forms can be requested by contacting the corresponding author or the editorial board of the Journal.

Ethics
This study did not require evaluation by an institutional review board because it is a research review.

Figure 1. Infographic of cohort studies. Source: designed by the authors.



Licencia Creative Commons Esta obra de Medwave está bajo una licencia Creative Commons Atribución-NoComercial 3.0 Unported. Esta licencia permite el uso, distribución y reproducción del artículo en cualquier medio, siempre y cuando se otorgue el crédito correspondiente al autor del artículo y al medio en que se publica, en este caso, Medwave.

 

Los estudios con diseño de cohorte evalúan la relación entre una exposición y la ocurrencia o no de un evento de interés, comenzando el análisis desde la exposición. Habiendo sido diseños muy utilizados en algunas áreas de la medicina, como la descripción de factores de riesgo cardiovascular o los efectos de la radiación ionizante en humanos, representan una herramienta con características atractivas debido a su adaptabilidad a numerosos contextos, sobre todo en el estudio de exposiciones de baja ocurrencia. Esta revisión es la cuarta entrega de una serie metodológica sobre conceptos generales en bioestadística y epidemiología clínica desarrollada por la Cátedra de Metodología de la Investigación Científica de la Escuela de Medicina de la Universidad de Valparaíso, Chile. En este artículo, se abordan conceptos teóricos generales sobre los estudios de cohorte, considerando aspectos históricos, generalidades sobre la construcción de un estudio utilizando este diseño, presentando distintas variantes y diseños derivados de interés, y potenciales sesgos a los que se puede ver enfrentado el investigador.

Authors: Gabriel Lazcano[1], Cristian Papuzinski[1,2], Eva Madrid[1,2], Marcelo Arancibia[1,2]

E-mail: marcelo.arancibiame@uv.cl

Citation: Lazcano G, Papuzinski C, Madrid E, Arancibia M. General concepts in biostatistics and clinical epidemiology: observational studies with cohort design. Medwave 2019;19(11):e7748 doi: 10.5867/medwave.2019.11.7748

Submission date: 19/10/2019

Acceptance date: 29/11/2019

Publication date: 16/12/2019

Origin: This article is one of several “Methodological notes” prepared by the course on Research Methodology of the School of Medicine of the University of Valparaíso

Type of review: reviewed by three external peer reviewers, double-blind

Comments (0)

We are pleased to have your comment on one of our articles. Your comment will be published as soon as it is posted. However, Medwave reserves the right to remove it later if the editors consider your comment to be: offensive in some sense, irrelevant, trivial, contains grammatical mistakes, contains political harangues, appears to be advertising, contains data from a particular person or suggests the need for changes in practice in terms of diagnostic, preventive or therapeutic interventions, if that evidence has not previously been published in a peer-reviewed journal.

No comments on this article.


To comment please log in

Medwave provides HTML and PDF download counts as well as other harvested interaction metrics.

There may be a 48-hour delay for most recent metrics to be posted.

  1. Goldsworthy AK. The complete Roman army. Thames & Hudson 2003. | Link |
  2. Bacaër N. Halley’s life table. En: A Short History of Mathematical Population Dynamics. London: Springer London; 2011: 5-10. | CrossRef |
  3. Brown L, Pope EG. The postdischarge mortality among the patients of the Adirondack Cottage Sanitarium. Ethical Publishing Company; 1904.
  4. Sartwell PE. The time factor in studies of the outcome of chronic disease. Am Rev Tuberc. 1951 May;63(5):608-12. | PubMed |
  5. Comstock GW. Early studies of tuberculosis. Natl Cancer Inst Monogr. 1985 May;67:23-7. | PubMed |
  6. Kannel WB, Dawber TR, Kagan A, Revotskie N, Stokes J 3rd. Factors of risk in the development of coronary heart disease--six year follow-up experience. The Framingham Study. Ann Intern Med. 1961 Jul;55:33-50. | PubMed |
  7. Doll R, Hill AB. Smoking and carcinoma of the lung; preliminary report. Br Med J. 1950 Sep 30;2(4682):739-48. | PubMed |
  8. Ozasa K, Grant EJ, Kodama K. Japanese Legacy Cohorts: The Life Span Study Atomic Bomb Survivor Cohort and Survivors' Offspring. J Epidemiol. 2018 Apr 5;28(4):162-169. | CrossRef | PubMed |
  9. Waxweiler RJ, Roscoe RJ, Archer VE, et al. Mortality follow-up through 1977 of the white underground uranium miners cohort examined by the United States Public Health Service. [on line]. | Link |
  10. Samet JM, Muñoz A. Evolution of the cohort study. Epidemiol Rev. 1998;20(1):1-14. | PubMed |
  11. Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet. 2002 Jan 5;359(9300):57-61. | PubMed |
  12. Grimes DA, Schulz KF. Cohort studies: marching towards outcomes. Lancet. 2002 Jan 26;359(9303):341-5. | PubMed |
  13. Carlson MD, Morrison RS. Study design, precision, and validity in observational studies. J Palliat Med. 2009 Jan;12(1):77-82. | CrossRef | PubMed |
  14. Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009 May;63(5):691-7. | CrossRef | PubMed |
  15. Madrid Aris E, Martínez Lomakin F. Moving towards a destination: considerations about cohort studies in less than 1000 words. Medwave. 2014 Jan 2;14(1):e5877. | CrossRef | PubMed |
  16. Zhang Y, Wu S, Xia Y, Wang N, Zhou L, Wang J, et al. Adverse Events Associated with Treatment of Multidrug-Resistant Tuberculosis in China: An Ambispective Cohort Study. Med Sci Monit. 2017 May 18;23:2348-2356. | PubMed |
  17. Silva LC. [Temporal sequence in observational studies to establish causality]. Medwave. 2014 May 2;14(4):e5944. | CrossRef | PubMed |
  18. Fedak KM, Bernal A, Capshaw ZA, Gross S. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015 Sep 30;12:14. | CrossRef | PubMed |
  19. Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005 Nov 3;2:11. | PubMed |
  20. Seubsman SA, Kelly M, Sleigh A, Peungson J, Chokkanapitak J, Vilainerun D. Methods used for successful follow-up in a large scale national cohort study in Thailand. BMC Res Notes. 2011 May 27;4:166. | CrossRef | PubMed |
  21. Fawcett J, Garrett N, Bates MN. Follow-up methods for retrospective cohort studies in New Zealand. Aust N Z J Public Health. 2002;26(3):256-61. | PubMed |
  22. Toledano MB, Smith RB, Brook JP, Douglass M, Elliott P. How to Establish and Follow up a Large Prospective Cohort Study in the 21st Century--Lessons from UK COSMOS. PLoS One. 2015 Jul 6;10(7):e0131521. | CrossRef | PubMed |
  23. Hulley SB, Cummings SR, Newman TB. Designing Cross-Sectional and Cohort Studies. In: Designing Clinical Research. Philadelphia: 2013:85-96.
  24. Kang H. The prevention and handling of the missing data. Korean J Anesthesiol. 2013 May;64(5):402-6. | CrossRef | PubMed |
  25. Dekkers OM, Egger M, Altman DG, Vandenbroucke JP. Distinguishing case series from cohort studies. Ann Intern Med. 2012 Jan 3;156(1 Pt 1):37-40. | CrossRef | PubMed |
  26. Sessler DI, Imrey PB. Clinical Research Methodology 2: Observational Clinical Research. Anesth Analg. 2015 Oct;121(4):1043-51. | CrossRef | PubMed |
  27. Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. 2010 Dec;126(6):2234-42. | CrossRef | PubMed |
  28. Cannizzaro G, Leone M, Esposito M. Immediate functional loading of implants placed with flapless surgery in the edentulous maxilla: 1-year follow-up of a single cohort study. Int J Oral Maxillofac Implants. 2007 Jan-Feb;22(1):87-95. | PubMed |
  29. Mathes T, Pieper D. Clarifying the distinction between case series and cohort studies in systematic reviews of comparative studies: potential impact on body of evidence and workload. | CrossRef | PubMed |
  30. Formijne Jonkers HA, Maya A, Draaisma WA, Bemelman WA, Broeders IA, Consten EC, et al. Laparoscopic resection rectopexy versus laparoscopic ventral rectopexy for complete rectal prolapse. Tech Coloproctol. 2014 Jul;18(7):641-6. | CrossRef | PubMed |
  31. Prentice RL. A Case-Cohort Design for Epidemiologic Cohort Studies and Disease Prevention Trials. Biometrika 1986;73:1. | CrossRef |
  32. Vojvodic M, Shafarenko M, McCabe SJ. Case-Cohort Studies: Design and Applicability to Hand Surgery. J Hand Surg Am. 2018 Aug;43(8):762-767. | CrossRef | PubMed |
  33. Cologne J, Preston DL, Imai K, Misumi M, Yoshida K, Hayashi T, et al. Conventional case-cohort design and analysis for studies of interaction. Int J Epidemiol. 2012 Aug;41(4):1174-86. | CrossRef | PubMed |
  34. Kulathinal S, Karvanen J, Saarela O, Kuulasmaa K. Case-cohort design in practice - experiences from the MORGAM Project. Epidemiol Perspect Innov. 2007 Dec 4;4:15. | PubMed |
  35. Porta M. A Dictionary of Epidemiology. Oxford University Press. | CrossRef |
  36. Tenny S, Boktor SW. Incidence. 2019 Jan 2. StatPearls. Treasure Island (FL): StatPearls Publishing; 2019 Jan-. | PubMed | Link |
  37. Kartsonaki C. Survival analysis. Diagnostic Histopathol 2016;22:263–70. | CrossRef |
  38. Spronk I, Korevaar JC, Poos R, Davids R, Hilderink H, Schellevis FG, et al. Calculating incidence rates and prevalence proportions: not as simple as it seems. BMC Public Health. 2019 May 6;19(1):512. | CrossRef | PubMed |
  39. Clark TG, Bradburn MJ, Love SB, Altman DG. Survival analysis part I: basic concepts and first analyses. Br J Cancer. 2003 Jul 21;89(2):232-8. | PubMed |
  40. Howe CJ, Cole SR, Lau B, Napravnik S, Eron JJ Jr. Selection Bias Due to Loss to Follow Up in Cohort Studies. Epidemiology. 2016 Jan;27(1):91-7. | CrossRef | PubMed |
  41. Kristman V, Manno M, Côté P. Loss to follow-up in cohort studies: how much is too much? Eur J Epidemiol. 2004;19(8):751-60. | PubMed |
  42. Chowdhury R, Shah D, Payal AR. Healthy Worker Effect Phenomenon: Revisited with Emphasis on Statistical Methods - A Review. Indian J Occup Environ Med. 2017 Jan-Apr;21(1):2-8. | CrossRef | PubMed |
  43. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002 Jan 19;359(9302):248-52. | CrossRef | PubMed |
  44. Tripepi G, Jager KJ, Dekker FW, Zoccali C. Selection bias and information bias in clinical research. Nephron Clin Pract. 2010;115(2):c94-9. | CrossRef | PubMed |
  45. Barraza F, Arancibia M, Madrid E, Papuzinski C. General concepts in biostatistics and clinical epidemiology: Random error and systematic error. Medwave. 2019 Aug 27;19(7):e7687. | CrossRef | PubMed |
  46. Cataldo R, Arancibia M, Stojanova J, Papuzinski C. General concepts in biostatistics and clinical epidemiology: Observational studies with cross-sectional and ecological designs. Medwave. 2019 Sep 25;19(8):e7698. | CrossRef | PubMed |
  47. Xue X, Agalliu I, Kim MY, Wang T, Lin J, Ghavamian R, et al. New methods for estimating follow-up rates in cohort studies. BMC Med Res Methodol. 2017 Dec 1;17(1):155. | CrossRef | PubMed |
Goldsworthy AK. The complete Roman army. Thames & Hudson 2003. | Link |

Bacaër N. Halley’s life table. En: A Short History of Mathematical Population Dynamics. London: Springer London; 2011: 5-10. | CrossRef |

Brown L, Pope EG. The postdischarge mortality among the patients of the Adirondack Cottage Sanitarium. Ethical Publishing Company; 1904.

Sartwell PE. The time factor in studies of the outcome of chronic disease. Am Rev Tuberc. 1951 May;63(5):608-12. | PubMed |

Comstock GW. Early studies of tuberculosis. Natl Cancer Inst Monogr. 1985 May;67:23-7. | PubMed |

Kannel WB, Dawber TR, Kagan A, Revotskie N, Stokes J 3rd. Factors of risk in the development of coronary heart disease--six year follow-up experience. The Framingham Study. Ann Intern Med. 1961 Jul;55:33-50. | PubMed |

Doll R, Hill AB. Smoking and carcinoma of the lung; preliminary report. Br Med J. 1950 Sep 30;2(4682):739-48. | PubMed |

Ozasa K, Grant EJ, Kodama K. Japanese Legacy Cohorts: The Life Span Study Atomic Bomb Survivor Cohort and Survivors' Offspring. J Epidemiol. 2018 Apr 5;28(4):162-169. | CrossRef | PubMed |

Waxweiler RJ, Roscoe RJ, Archer VE, et al. Mortality follow-up through 1977 of the white underground uranium miners cohort examined by the United States Public Health Service. [on line]. | Link |

Samet JM, Muñoz A. Evolution of the cohort study. Epidemiol Rev. 1998;20(1):1-14. | PubMed |

Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet. 2002 Jan 5;359(9300):57-61. | PubMed |

Grimes DA, Schulz KF. Cohort studies: marching towards outcomes. Lancet. 2002 Jan 26;359(9303):341-5. | PubMed |

Carlson MD, Morrison RS. Study design, precision, and validity in observational studies. J Palliat Med. 2009 Jan;12(1):77-82. | CrossRef | PubMed |

Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009 May;63(5):691-7. | CrossRef | PubMed |

Madrid Aris E, Martínez Lomakin F. Moving towards a destination: considerations about cohort studies in less than 1000 words. Medwave. 2014 Jan 2;14(1):e5877. | CrossRef | PubMed |

Zhang Y, Wu S, Xia Y, Wang N, Zhou L, Wang J, et al. Adverse Events Associated with Treatment of Multidrug-Resistant Tuberculosis in China: An Ambispective Cohort Study. Med Sci Monit. 2017 May 18;23:2348-2356. | PubMed |

Silva LC. [Temporal sequence in observational studies to establish causality]. Medwave. 2014 May 2;14(4):e5944. | CrossRef | PubMed |

Fedak KM, Bernal A, Capshaw ZA, Gross S. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015 Sep 30;12:14. | CrossRef | PubMed |

Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005 Nov 3;2:11. | PubMed |

Seubsman SA, Kelly M, Sleigh A, Peungson J, Chokkanapitak J, Vilainerun D. Methods used for successful follow-up in a large scale national cohort study in Thailand. BMC Res Notes. 2011 May 27;4:166. | CrossRef | PubMed |

Fawcett J, Garrett N, Bates MN. Follow-up methods for retrospective cohort studies in New Zealand. Aust N Z J Public Health. 2002;26(3):256-61. | PubMed |

Toledano MB, Smith RB, Brook JP, Douglass M, Elliott P. How to Establish and Follow up a Large Prospective Cohort Study in the 21st Century--Lessons from UK COSMOS. PLoS One. 2015 Jul 6;10(7):e0131521. | CrossRef | PubMed |

Hulley SB, Cummings SR, Newman TB. Designing Cross-Sectional and Cohort Studies. In: Designing Clinical Research. Philadelphia: 2013:85-96.

Kang H. The prevention and handling of the missing data. Korean J Anesthesiol. 2013 May;64(5):402-6. | CrossRef | PubMed |

Dekkers OM, Egger M, Altman DG, Vandenbroucke JP. Distinguishing case series from cohort studies. Ann Intern Med. 2012 Jan 3;156(1 Pt 1):37-40. | CrossRef | PubMed |

Sessler DI, Imrey PB. Clinical Research Methodology 2: Observational Clinical Research. Anesth Analg. 2015 Oct;121(4):1043-51. | CrossRef | PubMed |

Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. 2010 Dec;126(6):2234-42. | CrossRef | PubMed |

Cannizzaro G, Leone M, Esposito M. Immediate functional loading of implants placed with flapless surgery in the edentulous maxilla: 1-year follow-up of a single cohort study. Int J Oral Maxillofac Implants. 2007 Jan-Feb;22(1):87-95. | PubMed |

Mathes T, Pieper D. Clarifying the distinction between case series and cohort studies in systematic reviews of comparative studies: potential impact on body of evidence and workload. | CrossRef | PubMed |

Formijne Jonkers HA, Maya A, Draaisma WA, Bemelman WA, Broeders IA, Consten EC, et al. Laparoscopic resection rectopexy versus laparoscopic ventral rectopexy for complete rectal prolapse. Tech Coloproctol. 2014 Jul;18(7):641-6. | CrossRef | PubMed |

Prentice RL. A Case-Cohort Design for Epidemiologic Cohort Studies and Disease Prevention Trials. Biometrika 1986;73:1. | CrossRef |

Vojvodic M, Shafarenko M, McCabe SJ. Case-Cohort Studies: Design and Applicability to Hand Surgery. J Hand Surg Am. 2018 Aug;43(8):762-767. | CrossRef | PubMed |

Cologne J, Preston DL, Imai K, Misumi M, Yoshida K, Hayashi T, et al. Conventional case-cohort design and analysis for studies of interaction. Int J Epidemiol. 2012 Aug;41(4):1174-86. | CrossRef | PubMed |

Kulathinal S, Karvanen J, Saarela O, Kuulasmaa K. Case-cohort design in practice - experiences from the MORGAM Project. Epidemiol Perspect Innov. 2007 Dec 4;4:15. | PubMed |

Porta M. A Dictionary of Epidemiology. Oxford University Press. | CrossRef |

Tenny S, Boktor SW. Incidence. 2019 Jan 2. StatPearls. Treasure Island (FL): StatPearls Publishing; 2019 Jan-. | PubMed | Link |

Kartsonaki C. Survival analysis. Diagnostic Histopathol 2016;22:263–70. | CrossRef |

Spronk I, Korevaar JC, Poos R, Davids R, Hilderink H, Schellevis FG, et al. Calculating incidence rates and prevalence proportions: not as simple as it seems. BMC Public Health. 2019 May 6;19(1):512. | CrossRef | PubMed |

Clark TG, Bradburn MJ, Love SB, Altman DG. Survival analysis part I: basic concepts and first analyses. Br J Cancer. 2003 Jul 21;89(2):232-8. | PubMed |

Howe CJ, Cole SR, Lau B, Napravnik S, Eron JJ Jr. Selection Bias Due to Loss to Follow Up in Cohort Studies. Epidemiology. 2016 Jan;27(1):91-7. | CrossRef | PubMed |

Kristman V, Manno M, Côté P. Loss to follow-up in cohort studies: how much is too much? Eur J Epidemiol. 2004;19(8):751-60. | PubMed |

Chowdhury R, Shah D, Payal AR. Healthy Worker Effect Phenomenon: Revisited with Emphasis on Statistical Methods - A Review. Indian J Occup Environ Med. 2017 Jan-Apr;21(1):2-8. | CrossRef | PubMed |

Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002 Jan 19;359(9302):248-52. | CrossRef | PubMed |

Tripepi G, Jager KJ, Dekker FW, Zoccali C. Selection bias and information bias in clinical research. Nephron Clin Pract. 2010;115(2):c94-9. | CrossRef | PubMed |

Barraza F, Arancibia M, Madrid E, Papuzinski C. General concepts in biostatistics and clinical epidemiology: Random error and systematic error. Medwave. 2019 Aug 27;19(7):e7687. | CrossRef | PubMed |

Cataldo R, Arancibia M, Stojanova J, Papuzinski C. General concepts in biostatistics and clinical epidemiology: Observational studies with cross-sectional and ecological designs. Medwave. 2019 Sep 25;19(8):e7698. | CrossRef | PubMed |

Xue X, Agalliu I, Kim MY, Wang T, Lin J, Ghavamian R, et al. New methods for estimating follow-up rates in cohort studies. BMC Med Res Methodol. 2017 Dec 1;17(1):155. | CrossRef | PubMed |