COVID-19 Resources

  • Coordinating Center
  • NIH Collaboratory Trials
  • Core Working Groups
  • Steering Committee
  • Distributed Research Network
  • Table of Contents
  • How to Use This Site
  • Data and Resource Sharing
  • Training Resources
  • Publications
  • Knowledge Repository

Rethinking Clinical Trials

A living textbook of pragmatic clinical trials.

  • PRECIS-2 Case Study


What is a Pragmatic Clinical Trial?

Merrick Zwarenstein, MBBCh, MSc, MSc, PhD Ahmed Al-Jaishi, PhD Amit X. Garg, MD, PhD

Contributing Editor

Liz Wing, MA

Focusing on the trial’s intention is the first step in designing a trial that successfully answers its primary research question. For an introduction, read Promoting Both Internal and External Validity: Designing the Trial to Match Its Intention , which describes considerations for choosing a pragmatic or explanatory approach. While both approaches are valuable, their purposes are different and will lead to different design choices. As a result, designs that are more pragmatic will have conclusions and recommendations that are more useful for clinical or policy decision-making, and designs that are more explanatory will be more helpful in expanding scientific knowledge.

Case Study of Trial Design in a Renal Dialysis Setting

This case study illustrates the design of 2 randomized controlled trials that examined the temperature used in hemodialysis. Using the PRECIS-2 framework, we discuss insights about opportunities and constraints that a renal dialysis setting offers and show how the design of the trials aligned with their intention. Trial 1 was more explanatory in intention ( Odudu et al. 2015 ), and Trial 2 was more pragmatic in intention ( Al-Jaishi et al. 2020 ). We begin with some background on the 2 trials and the PRECIS-2 domains that will guide the step-by-step considerations for situating the trials on the pragmatic-explanatory continuum.

Trial 1 Characteristics (blue)

  • Personalized dialysis temperature compared with 37°C (98.6°F)
  • Unit of randomization: individual patient
  • Patients in Nottingham UK enrolled from September 2009 through January 2013
  • 1 year follow-up
  • 73 patients would have ~11,000 hemodialysis sessions during the trial
  • Individual-level consent
  • Trial-specific data collection
  • Primary outcome: change in the resting ejection fraction measured by cardiac magnetic resonance imaging (MRI) at 12 months compared with baseline
  • Cardiac structure, function, and aortic distensibility assessed by cardiac MRI

case study in clinical trials

Trial 2 Characteristics (red)

  • Personalized dialysis temperature compared with 36.5°C (97.7°F)
  • Unit of randomization: centers in Ontario followed from April 2017 to March 2021
  • Maximum 4 years follow-up
  • 84 centers, ~13,500 patients (~6000 entered at start of trial, ~7500 entered during trial)
  • ~4 million hemodialysis sessions during the trial
  • Consent by patient notification via poster and newsletter, allowing opt-out by patients or providers
  • Baseline and follow-up data from large databases
  • Primary outcome: a composite of cardiac death or hospital admission with myocardial infarction, stroke, or congestive heart failure

case study in clinical trials

Evaluating the Trials Using PRECIS-2

For study teams, the broad steps of evaluating a trial using the PRECIS-2 wheel include:

  • Defining the trial’s intention
  • Aligning the design to the intention
  • Identifying the trial’s location on the explanatory-pragmatic continuum (1-5) for each of the 9 domains (the spokes of the wheel)

In the case study, a very explanatory approach with many exclusions is represented closer to 1 . Such exclusions may included expected noncompliers, nonresponders, those at low risk of the primary outcome, children, elderly, or the use of or need for diagnostic tests not used in standard care to assess patient eligibility. On the other hand, a very pragmatic approach with criteria that are essentially identical to those in usual care is represented closer to 5 .

case study in clinical trials

Next, we evaluate the characteristics of both trials and describe the considerations behind each of the 9 PRECIS-2 domains.

(1) Eligibility, (2) Recruitment, (3) Setting

case study in clinical trials

In terms of the trial population, the individual-level trial (blue) was restrictive in its inclusion criteria, having individuals ≥16 years who started hemodialysis at the renal clinic setting within the last 180 days and who had the capacity to provide informed consent. In addition, individuals were excluded if they could not tolerate cardiac magnetic resonance imaging (MRI), were pregnant or lactating, or were classified with New York Heart Association grade IV heart failure.

In comparison, the MyTEMP Trial (red) had less restrictive inclusion criteria in order to increase generalizability. Inclusion was at the clinic (ie, cluster) level, where the medical director agreed for their clinic to be randomized to either trial arm, and whose clinic was expected to provide care per week to at least 15 different adult patients ≥18 years receiving conventional in-clinic hemodialysis.

Having a highly pragmatic approach for MyTEMP allowed for the rapid enrollment of clinics and patients (both patients receiving chronic dialysis at the time of randomization and future patients starting dialysis). However, because patients at low and high risk for experiencing the primary outcome were included, a large sample size was required, as nonselectivity reduces the effect size.

case study in clinical trials

(4) Organization

case study in clinical trials

The individual-level trial scored low (1) on the PRECIS-2 wheel because the intervention required a specific type of thermometer, trained research staff, cardiac MRI, and a dialysis machine with the ability to change the dialysis temperature by steps of 0.1 degrees. In comparison, the MyTEMP trial scored high (5) on the PRECIS-2 wheel because it required no specific expertise or equipment to implement the assigned treatment protocol.

However, because the assigned MyTEMP protocol became the new clinic policy, there was a need to clearly prescribe and set the dialysate temperature. Also, complex roles and social influences on the process of using a personalized dialysate temperature needed to be assessed.

The flowchart below shows the process of who needed to do what differently, the inter-relationships between different roles, and outcomes. In the MyTEMP trial, the leadership at each dialysis clinic changed the local policy to ensure alignment with the assigned temperature protocol. Physicians ordered the assigned temperature protocol for current patients at one time and as new patients received prescriptions for dialysate temperature. Nurses were then trained on the trial protocol and asked to follow physician orders to set the dialysate temperature. Nurses were also asked to report issues or adverse clinical symptoms related to dialysate temperature to the treating physician, who could change the dialysate temperature as they deemed appropriate.

case study in clinical trials

(5) Flexibility: delivery

case study in clinical trials

The individual-level trial required specific dialysis machines, research nurses, and specific thermometers to implement the study intervention. In comparison, the MyTEMP trial allowed the dialysate temperature to be set in the range of 0.5 to 0.9 degrees below the patient's body temperature. Also, the entire trial—at the clinic level—was implemented by clinicians and nurses rather than researchers. This meant that little local research infrastructure was needed in the recruiting clinics to implement the MyTEMP trial with high flexibility.

In the control arm of the individual-level trial, the dialysate temperature of patients was required to be set at 37°C. In the MyTEMP trial, clinics were asked to set the dialysate temperature to at least 36.5°C unless the patient or their nephrologist preferred another temperature be used.

(6) Flexibility: adherence, (7) Follow-up

case study in clinical trials

As discussed above, the individual-level trial employed research staff that closely followed patients for the trial duration. There were regular check-ups and feedback between research staff and participants. In comparison, the MyTEMP trial research staff had no contact with participants. However, each month the clinics were asked to randomly select 15 patients and record the dialysate temperature used for a single session. This allowed the study team to estimate the proportion of patients who were “adhering” to the clinic protocol. It should be noted that the level of adherence for the individual-level trial was measured at the participant level, whereas for the MyTEMP trial, adherence was measured at the clinic level.

case study in clinical trials

(8) Primary outcome, (9) Primary analysis

case study in clinical trials

The individual-level trial had a low PRECIS-2 score (1) for the primary outcome because it tested a change in the resting ejection fraction, which is a surrogate outcome. While important, a change in resting ejection fraction is less relevant to patients. In contrast, for the MyTEMP trial, the outcome was cardiovascular-related death or hospital admission for adverse cardiac events, which is of greater relevance to patients and their healthcare providers (5).

For the analysis domain, the individual-level trial used an intention-to-treat approach, which is a highly pragmatic approach. However, nearly 40% of the outcome data was missing due to dropouts or loss to follow-up. From the 73 patients randomized, 37 participants were assigned to control and 36 to the intervention. Of those participants, 28 and 26 were analyzed in the control and intervention arms, respectively. Among those analyzed, 5 participants in each arm required multiple imputations for missing outcome data at the 12-month follow-up period.

In contrast, the MyTEMP trial will also use an intention-to-treat approach, but loss to follow-up is expected to be minimal because that typically occurs when study participants emigrate from the (Canadian) province. For example, in Ontario less than 0.5% the population per year emigrates from the province.

case study in clinical trials

Pragmatic trials embedded in routine healthcare delivery are increasingly playing a vital role in filling the large gaps in knowledge about caring for patients on hemodialysis. Several fundamental questions in this healthcare setting appear to be particularly suited to pragmatic approaches to trial design. We used the MyTEMP trial to illustrate a more pragmatic intention, where the authors wanted to test whether use of a clinic-level protocol of personalized temperature-reduced dialysate results in a different rate of cardiovascular-related deaths or hospitalizations than a standard temperature dialysate. The MyTEMP trial was intentionally designed to be highly pragmatic and flexible because of:

  • Frequent and predictable patient encounters
  • Highly granular and uniform electronic data collection in routine care
  • Delivery of care by a small number of provider organizations

There is tremendous interest, both nationally and globally, in increasing the momentum for conducting pragmatic trials. Funders of research, including industry sponsors, are increasingly embracing this approach to reduce costs and generate findings that are rapidly translatable to practice.

Previous Section Next Section

  • Why Are We Talking About Pragmatic Trials?
  • The Embedded Pragmatic Clinical Trial Ecosystem
  • Differentiating Between RCTs, PCTs, and Quality Improvement Activities
  • Pragmatic Elements: An Introduction to PRECIS-2
  • Key Considerations for PCTs
  • Additional Resources

For an introduction, read Promoting Both Internal and External Validity: Designing the Trial to Match Its Intention .

Additional resources:

For other examples of PCTs conducted in dialysis settings, read about HiLo and TiME .

Review Pragmatic Elements: An Introduction to PRECIS-2 , and learn how to evaluate your trial using the PRECIS-2 tool .

Hear more from the authors in the November 13, 2020, Grand Rounds presentation .

back to top

Al-Jaishi AA, McIntyre CW, Sontrop JM, et al. 2020. Major Outcomes With Personalized Dialysate TEMPerature (MyTEMP): Rationale and Design of a Pragmatic, Registry-Based, Cluster Randomized Controlled Trial. Can J Kidney Health Dis. Feb 5. doi:10.1177/2054358119887988. PMID: 32076569.

Odudu A, Eldehni MT, McCann GP, McIntyre CW. 2015. Randomized Controlled Trial of Individualized Dialysate Cooling for Cardiac Protection in Hemodialysis Patients. Clin J Am Soc Nephrol. 10:1408-1417. doi:10.2215/cjn.00200115.

Version History

Published August 30, 2021

  • Skip to main content
  • Skip to FDA Search
  • Skip to in this section menu
  • Skip to footer links

U.S. flag

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

U.S. Food and Drug Administration

  •   Search
  •   Menu
  • For Patients
  • Clinical Trials: What Patients Need to Know

Basics About Clinical Trials

What are clinical trials.

Clinical trials are research studies in which people volunteer to help find answers to specific health questions. When carefully conducted, they are the safest and fastest way to find new treatments and ways to improve health.

Clinical trials are conducted according to a plan, called a protocol, which describes:

  • the types of patients who may enter the study
  • the schedules of tests and procedures
  • the drugs involved
  • the dosages, or amount of the drug
  • the length of the study
  • what the researchers hope to learn from the study.

Volunteers who participate in the study must agree to the rules and terms outlined in the protocol. Similarly, researchers, doctors, and other health professionals who manage the clinical trials must follow strict rules set by the FDA. These rules make sure that those who agree to participate are treated as safely as possible.

Learn more about the basics of clinical trial participation, read first hand experiences from actual clinical trial volunteers, and see explanations from researchers at the NIH Clinical Research Trials and You Web site.

Why are clinical trials done?

Clinical trials are conducted for many reasons:

  • to determine whether a new drug or device is safe and effective for people to use.
  • to study different ways to use standard treatments or current, approved treatments so that they will be more effective, easier to use, or decrease certain side effects.
  • to learn how to safely use a treatment in a population for which the treatment was not previously tested, such as children.

Who should consider clinical trials and why?

Some people participate in clinical trials because none of the standard (approved) treatment options have worked, or they are unable to tolerate certain side effects. Clinical trials provide another option when standard therapy has failed. Others participate in trials because they want to contribute to the advancement of medical knowledge.

Ensuring people from diverse backgrounds join clinical trials is key to advancing health equity. Participants in clinical trials should represent the patients that will use the medical products. This is often not the case—people from racial and ethnic minority and other diverse groups are underrepresented in clinical research. This is a concern because people of different ages, races, and ethnicities may react differently to certain medical products. Learn more about the clinical trial diversity initiative from the Office of Minority Health and Health Equity.

All clinical trials have guidelines, called eligibility criteria, about who can participate. The criteria are based on such factors as age, sex, type and stage of disease, previous treatment history, and other medical conditions. This helps to reduce the variation within the study and to ensure that the researchers will be able to answer the questions they plan to study. Therefore, not everyone who applies for a clinical trial will be accepted.

It is important to test drugs and medical products in the people they are meant to help. It is also important to conduct research in a variety of people, because different people may respond differently to treatments.  FDA seeks to ensure that people of different ages, races, ethnic groups, and genders are included in clinical trials. Learn more about FDA’s efforts to increase diversity in clinical trials .

Where are clinical trials conducted?

Clinical trials can be sponsored by organizations (such as a pharmaceutical company), Federal offices and agencies (such as the National Institutes of Health or the U.S. Department of Veterans Affairs), or individuals (such as doctors or health care providers). The sponsor determines the location(s) of the trials, which are usually conducted at universities, medical centers, clinics, hospitals, and other Federally or industry-funded research sites.

Are clinical trials safe?

FDA works to protect participants in clinical trials and to ensure that people have reliable information before deciding whether to join a clinical trial. The Federal government has regulations and guidelines for clinical research to protect participants from unreasonable risks. Although efforts are made to control the risks to participants, some may be unavoidable because we are still learning more about the medical treatments in the study.

The government requires researchers to give prospective participants complete and accurate information about what will happen during the trial. Before joining a particular study, you will be given an informed consent document that describes your rights as a participant, as well as details about the study, including potential risks. Signing it indicates that you understand that the trial is research and that you may leave at any time. The informed consent is part of the process that makes sure you understand the known risks associated with the study.

What should I think about before joining a clinical trial?

Before joining a clinical trial, it is important to learn as much as possible. Discuss your questions and concerns with members of the health care team conducting the trial. Also, discuss the trial with your health care provider to determine whether or not the trial is a good option based on your current treatment. Be sure you understand:

  • what happens during the trial
  • the type of health care you will receive
  • any related costs once you are enrolled in the trial
  • the benefits and risks associated with participating. 

What is FDA’s role in approving new drugs and medical treatments?

FDA makes sure medical treatments are safe and effective for people to use. We do not develop new therapies or conduct clinical trials. Rather, we oversee the people who do. FDA staff meet with researchers and perform inspections of clinical trial study sites to protect the rights of patients and to verify the quality and integrity of the data.

Learn more about the Drug Development Process .

Where can I find clinical trials?

One good way to find out if there are any clinical trials that might help you is to ask your doctor. Other sources of information include:

  • FDA Clinical Trials Search. Search a database of Federally and privately supported studies available through Learn about each trial’s purpose, who can participate, locations, and who to contact for more information.
  • Conduct more advanced searches
  • National Cancer Institute or call 1–800–4–CANCER (1–800–422–6237). Learn about clinical trials for people with cancer.
  • AIDS Clinical Trials and Information Services (ACTIS) or call 1–800–TRIALS–A (1–800–874–2572). Locate clinical trials for people with HIV.
  • AIDSinfo. Search a database of HIV/AIDS trials, sponsored by the National Institutes of Health’s National Library of Medicine.

What is a placebo and how is it related to clinical trials?

A placebo is a pill, liquid, or powder that has no treatment value. It is often called a sugar pill. In clinical trials, experimental drugs are often compared with placebos to evaluate the treatment’s effectiveness.

Is there a chance I might get a placebo?

In clinical trials that include placebos, quite often neither patients nor their doctors know who is receiving the placebo and how is being treated with the experimental drug. Many cancer clinical trials, as well as trials for other serious and life-threatening conditions, do not include placebo control groups. In these cases, all participants receive the experimental drug. Ask the trial coordinator whether there is a chance you may get a placebo rather than the experimental drug. Then, talk with your doctor about what is best for you.

How do I find out what Phase a drug is in as part of the clinical trial?

Talk to the clinical trial coordinator to find out which phase the clinical trial is in. Learn more about the different clinical trial phases and whether they are right for you.

What happens to drugs that don't make it out of clinical trials?

Most drugs that undergo preclinical (animal) research never even make it to human testing and review by the FDA. The drug developers go back to begin the development process using what they learned during with their preclinical research. Learn more about drug development .

This site is part of the Informa Connect Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 3099067.

  • Informa Connect

Real World Evidence in clinical trials: 3 case studies of successful implementation

case study in clinical trials

Share this article

Real World Evidence in clinical trials offers huge potential to change how studies are run. Today’s advanced technologies such as artificial intelligence platforms, commoditized data collection tools and large integrated healthcare IT infrastructures allow us to go one step further in collecting and utilizing data. However, there are clearly still huge hurdles to full implementation of RWE in clinical trials. Here we look at three case studies of real world evidence being successfully used in clinical trials as well as what data can be used in RWE studies.

Case Study 1: Salford Lung Studies

The innovative Salford Lung Studies are often showcased as great example of an RWE-based study. The GSK sponsored study examined the safety and effectiveness of a new treatment for chronic obstructive pulmonary disease. It involved over 2,802 patients treated by their own GPs in everyday clinical practice. The study had low exclusion criteria: 90% of screened patients were included in the study to replicate a representative sample of everyday practice. The goal of the study was to collect data that reflected medicine taking behaviors and collect valid data with minimal disruption to normal care, while enrolling a large generalizable proportion of eligible population. Deemed a success, 93% of participants remained for the duration of the study, and a number of conclusive clinical results were attained.

RELATED ARTICLE:  Real World Evidence: Revolutionizing the Clinical Trials Environment - WHITEPAPER

Case Study 2: Avelumab approval

Real-world evidence also played a key role in the recent approval of avelumab, a monoclonal antibody used to treat metastatic Merkle cell carcinoma (mMCC), a rare aggressive skin cancer. Since there are no standard of care for mMCC, investigators used data generated data from EHRs on observed clinical outcomes in a patient population that received chemotherapy to establish a benchmark for chemotherapy efficacy in a real-world setting. Researchers identified patients who responded to avelumab, and documented the benefits by contrasting it to the benchmarked data. Last year, the FDA granted accelerated approval to avelumab.

Case Study 3: PatientsLikeMe and amyotrophic lateral sclerosis (ALS)

Data from RWE can also be used to save costs and concentrate clinical studies on more promising candidates. The case of PatientsLikeMe and amyotrophic lateral sclerosis (ALS) is an excellent example. PatientsLike me as an important personalized healthcare network, with a large ALS community. Using its database, researchers found that 9% (348) of patients with ALS in the PatientsLikeMe community reported using lithium carbonate, a drug which had shown promise in a small study (16 treated patients, 28 controls), but which did not have regulatory approval. This created an opportunity to create a larger observational study, using data gathered from these patients, which was matched to multiple control groups. In the end, no difference in disease progression was observed after 12 months between the overall study group and those patients in the lithium carbonate treatment group (78 patients). Subsequent randomized studies reached the same conclusion that there was no clinical effect in the overall population.

Which data can be used in RWE studies?

Not all data is acceptable for RWE-studies. Hence, it is the submitter’s responsibility to show that the data is of sufficient quality. One way to do that is to follow the Hahn framework, which consists of three components:

  • Conformance – Does your data conform to specified regulatory standards?
  • Completeness – Relates to frequencies of data attributes present in a data set
  • Plausibility – Associated to the truthfulness of data

The issues around conformance are crucial, as data submitted to regulatory agencies need to follow the appropriate data standards. As an organization, to be able to work with RWD across multiple sources, data may need to be put into a common format, with common representation (terminologies, vocabularies, coding schemes). The FDA recognizes the importance of developing data standards to maximize the utility of RWD and is working on identifying relevant standards and methodologies for collection and analysis of RWD. Currently, the FDA and EMA both already have a number of guidance documents on the use, sharing and storing of electronic data. They provide recommendations on how to store and capture data and as well on the authenticity and reliability of your data storage. This can include implementing audit trails for electronic records, and how to archive records that are pertinent to clinical investigations.

Completeness has long been seen as a roadblock to REW adoption: critics have long believed that consumers by themselves are likely to submit incomplete data, or likely to forget and / or omit information from their records. This why the FDA require patient registries used in RWE studies have sufficient processes, such as those to gather follow-up information when needed, to ensure data quality, and to minimize missing or incomplete data.

Plausibility focuses on the believability of data. Where the first two criteria focus on the structure of data (conformance) and its presence (completeness), plausibility is uniquely concerned with the actual values being shared. This is another long-standing roadblock to the adoption of RWE, as critics fear that patients would fudge values rather than reporting truthfully due to their own self-interest. The automation of data collection (through wearable devices for example) alleviates this concern, as long as proper guidelines to validate the source and reliability of the data is supplied.

READ MORE:  Real World Evidence: Revolutionizing the Clinical Trials Environment - WHITEPAPER

About the Author: Jean-Francois Denault, MBA has been working with innovators and entrepreneurs in life sciences as a professional consultant for over fifteen years. He has worked with over 50 different clients in life sciences located all over the world. He has written a dozen articles for various publications and is the author of two life sciences marketing handbooks.

Related Articles

Andreas diacon, task applied science, announced as winner of christine pierre lifetime achievement award 2019, pharma’s pivotal role in social determinants of health, related event, clinical trials europe.

  • 29 Nov, 2023
  • Cologne, Spain
  • Open access
  • Published: 20 November 2007

Marketing and clinical trials: a case study

  • David Francis 1 ,
  • Ian Roberts 2 ,
  • Diana R Elbourne 3 ,
  • Haleema Shakur 2 ,
  • Rosemary C Knight 3 ,
  • Jo Garcia 3 ,
  • Claire Snowdon 3 , 4 ,
  • Vikki A Entwistle 5 ,
  • Alison M McDonald 5 ,
  • Adrian M Grant 5 &
  • Marion K Campbell 5  

Trials volume  8 , Article number:  37 ( 2007 ) Cite this article

21k Accesses

34 Citations

22 Altmetric

Metrics details

Publicly funded clinical trials require a substantial commitment of time and money. To ensure that sufficient numbers of patients are recruited it is essential that they address important questions in a rigorous manner and are managed well, adopting effective marketing strategies.

Using methods of analysis drawn from management studies, this paper presents a structured assessment framework or reference model, derived from a case analysis of the MRC's CRASH trial, of 12 factors that may affect the success of the marketing and sales activities associated with clinical trials.

The case study demonstrates that trials need various categories of people to buy in – hence, to be successful, trialists must embrace marketing strategies to some extent.

The performance of future clinical trials could be enhanced if trialists routinely considered these factors.

Peer Review reports

Results from randomised controlled trials (RCTs) make an important contribution to improving patient care. Some trials recruit a large number of patients and involve the collaboration of many doctors, nurses and other healthcare workers around the world. Because trials (especially large trials) can involve a substantial commitment of time and money, it is essential that they address important questions and use rigorous scientific methods. More recently, however, it has been recognised that good management and effective marketing are also essential to enable sufficient numbers of participating centres and patients to be recruited so that the study has enough statistical power [ 1 ]. This paper reports a case study of a novel application of a marketing approach from the world of business to a single clinical trial in order to develop a reference model for use in other trials.

Orientating concepts

Businesses strive to find customers and encourage them to buy what is on offer. Clinical trials strive to find doctors and patients and encourage them to sign up. Thus they face similar challenges and may need to adopt similar approaches to achieve their goals.

Clinical trials progress through distinctive stages, including study design, obtaining funding, finding participants, collecting and processing data, interpreting the results, and reporting. In some stages of a trial the key requirement is to do good science. However, in others the challenge is quite different – the key requirement is to install and operate a range of effective management techniques, similar to those required for marketing a product. Indeed, an experienced trialist observed that a trial is one fifth structure (science) and four-fifths process (i.e. management).

Marketing – the process of finding, winning over and retaining customers – is an important topic in management studies. Marketing has distinctive frameworks, methods and techniques – generally drawn from sociology and social psychology. Marketing became better understood in the 1960s [ 2 ] and now the discipline is ubiquitous within larger companies and many not-for-profit organisations. A definition of marketing by McDonald and Wilson [ 3 ] describes it as "a process for defining markets, quantifying the needs of the customer groups (segments) within these markets, determining the value propositions to meet these needs, communicating these value propositions to all those people in the organisation responsible for delivering them and getting their buy-in to their role, playing an appropriate part in delivering these value propositions to the chosen market segments (and) monitoring the value actually delivered" (pp11). (A value proposition can be defined as a clear statement of the tangible results a customer gets from using the products or services).

The marketing dimension is included only tangentially in the literature on clinical trials [ 1 ]. For example, trials are generally stated to need recruitment strategies, use of media and data tracking systems. However, the notion of a developing and working to achieve a formal marketing plan that covers all of the areas in the McDonald and Wilson definition is generally absent from descriptions of trial management.

This is not to suggest that trial managers consider the topic of marketing their trials to potential recruits lightly. Indeed, it is a dominant concern for many trialists. For example, the Diabetic Retinopathy Awareness Program study [ 4 ] undertook many initiatives to recruit volunteers and concluded, "these experiences substantiate the need for a comprehensive coordinated approach, using planned sources, to achieve recruitment success" (pp432). Farrell [ 5 ] has argued persuasively that it is lack of solutions to managerial issues that reduce the effectiveness of trials, and Rowe [ 6 ] suggested that, "to get patients into trials more efficiently pharma companies must begin to think like marketers".

It can be argued that marketing is especially important in clinical trials. Participation in a trial is a formal voluntary act, in that participants need to abide by a set of rules. Accordingly, not only is it necessary for people to volunteer, they also need to sign-up to behave in accordance with a set procedure [ 7 ]. In short, participants in a trial (be they clinicians or patients or their families) need to make a commitment, and undertake additional work, often without direct financial benefit to themselves.

From a marketing perspective, conducting a successful trial can be seen as a process with five main stages (Figure 1 ). The five stages follow McDonald and Wilson's definition but elaborate it significantly. The purposes and content of each stage is amplified in Table 1 .

figure 1

Five stages in marketing a trial.

Clinical trials require strategy, management, marketing and sales. Undoubtedly they undertake the activities listed in the table 1 in some way. However, what happens if those who define the strategy of a trial, establish its management processes, devise its marketing plan and attempt to sell the benefits of participation try to improve their practice by explicitly engaging with the discipline of management? The study described below provides some preliminary answers to this question.

This study is a component of a three-part project – STEPS (Strategies for Trials Enrolment and Participation) [ 8 ]. The first part included a quantitative analysis of the association between different patterns of recruitment in trials and factors thought likely to influence this pattern, based on an examination of Medical Research Council (MRC) and Health Technology Assessment (HTA) records [ 9 ]. This showed recruitment often fails to meet targets. The second part explored these issues further using qualitative analysis of transcripts from semi-structured interviews with key players in four trials considered by MRC/HTA as exemplars [ 10 ], with a particular focus on the complexity of financial negotiations. Here we report the third part based on an in-depth investigation of a single trial from a business perspective to assess its marketing strategy, in order to develop a reference model to aid future trials.

The Corticosteroid Randomisation after Significant Head injury (CRASH) trial [ 11 ] was a large scale RCT of the effect of corticosteroids compared to placebo in improving important health outcomes [ 12 ]. The trial aimed to recruit 20,000 head injured patients from hospitals world-wide. As the trial participants were unconscious the marketing strategy needed to focus on staff at participating hospitals (and not on the patients).

CRASH had a marketing challenge since it needed to engage the interest and collaboration of hundreds of people internationally, including members of ethics committees, surgeons, doctors, nurses and administrators. During the recruitment phase, one of us (DLF), a marketing and strategy specialist from the academic business sector, was invited to examine the trial as if it were a business, to comment on its marketing strategy and to help the trial team to understand and put in place a marketing plan over a two-year period. He was given access to all trial documents (apart from confidential investigators' personal details and patient data). He visited three participating hospitals in England, observed training sessions and interviewed or facilitated group discussions with doctors, nurses and ancillary staff. He also conducted 11 interviews, and held numerous meetings with members of the trial management team.

The methodological approach used techniques drawn from adaptive theory [ 13 ], case analysis [ 14 ] and action research [ 13 ]. The researcher's interview notes were analysed using a grounded theory framework [ 15 ] and the emerging model was compared with data from studies in commercial enterprises. The N-Vivo qualitative analysis software program was used to structure data initially but manual analytic methods were used later as often the purpose was to highlight what participating agents were not saying – rather than what they were saying. A professor of management (independent of the study team) checked the interpretative framework against the raw data. Emerging results were presented to a peer group (the STEPS research team) and to the team members of the CRASH trial. A one-day marketing workshop using an action research approach was held with the trial team to provide insights into the extent to which concepts and practices from the business world [ 15 ] might have relevance to management of clinical trials. Early in 2004 an additional five-hour workshop was held with representatives from three trials (one of which was the CRASH trial) to gain further insight into the practical implications of the findings, providing a further opportunity to validate the researcher's theory building process.

As the approach reported in this paper was part of a strategy to try to improve recruitment into the CRASH trial, the STEPS investigators decided that separate ethics committee approval would not be required for this process as the CRASH trial had already received approval from the North London MREC.

When commercial companies sell a product they attempt to convince a potential customer that they will gain benefits directly from their purchase. In the CRASH the trial managers were seeking to gain a commitment to engage from clinical professionals who would make no material gain for themselves. Accordingly, the CRASH trial was selling an opportunity for clinical professionals to participate in improving future clinical practice – an activity that can be seen as being akin to a charitable endeavour [ 7 ]. A challenge for the CRASH trial was to promote the idea that if a clinician signed up to the trial then medicine itself would progress and the clinician would be fulfilling a professional obligation.

A previously unexplored dimension of the marketing challenge was found to be the difficulty of gaining an evidence-based understanding the reasons why participants (in this case hospitals) signed up and what motivated them to fulfil a commitment that had no sanctions for non-performance. An analysis of feedback from participating hospitals concluded that they opted in for a variety of reasons, including the perceived merits of the study, the stature of the sponsors and advocates, the status provided to participants through participation and the affordability of participation (i.e how much time and effort would be required).

A tentative reference model was developed from the research date that facilitated an ongoing assessment of the sales and marketing capability of the trial.

The reference model

The reference model defines the capabilities required for successful marketing and selling of a medical trial that offered a holistic ideal type that the trial could use to define excellence [ 17 ]. It has four domains and 12 components and is illustrated as a wheel diagram (Figure 2 ). The twelve components are described below.

figure 2

Reference model.

Ia) Developing brand values

Brand values define what a brand is and what it is not – i.e. its personality. A clinical trial can be seen as a brand. Without explicit brand values it is impossible to communicate a coherent and persuasive perception of a trial's promise – i.e. what the trial intends to deliver to medicine, doctors, patients etc.

Ib) Gaining legitimacy and prestige

Trials need legitimacy – they need to be positively tagged by association with prestigious individuals and institutions (so a hospital doctor may say, "I know that this is an important trial because Professor X, who I know and respect, is supporting it). Legitimacy and prestige provide persuasive credibility key to gaining access to decision-makers who decide whether a trial should be supported and maintain engagement.

Ic) Signalling worthiness

It is vital to signal to likely participants that, "this trial will create greater value than the costs (time, effort or money) involved". Buy-in is more likely to occur when participants realise, and identify with, the potential benefits that will be delivered by the success of the trial. Methods for doing this include presentations at conferences, journal publications, advertising, public relations and training materials.

IIa) Providing simple, complete processes

Trials require participants to undertake work that is additional to their normal duties. Providing simple, complete processes reduces the costs of participation and increases the chances that involvement will be affordable.

IIb) Devising strategies for overcoming resistance

Potential participants frequently raise objections. Trials should have standard and persuasive answers to these. Having a persuasive answer for each objection increases the probability of making a sale.

IIc) Adopting an explicit marketing plan

The marketing of a trial is too important and too complicated to be done informally. A formal marketing plan is required that should include a definition of target market segments (groups that need to buy in to the trial) and the trial's unique selling points (USPs). It is to be expected that the marketing plan will need to be revised frequently – probably every quarter. It can be useful to have separate plans for dealing with (1) The Uninformed (Inform and persuade with targeted stories), (2) The Unconvinced (Address concerns point-by-point – "get to yes"), (3) The Laggards (Enrol, cajole, facilitate and target), (4) The Steady Performers (Reward, renew, upgrade and recognise) and (5) The Stars (Honour, learn from, and nourish).

IIIa) Engaging active sponsors, champions and change agents

Selling a trial to prospective participants requires persuasion. This requires enrolling sponsors (public advocates), champions (activists) and change agents (facilitators). Trial managers need a network of supporters to spread the message. Persuasion is more likely to occur if the advocate is respected and known personally to the prospective participant.

IIIb) Delivering a multi-audience, multi-level message

Trials need to convey sales messages through publicity, presentations, training materials, etc. These should be tuned to the distinctive needs of target groups – for example, surgeons are likely to be persuaded by different messages than administrators or nursing staff. Speaking in the language of the person being targeted and addressing their particular pattern of motivation is more likely to succeed than a one size fits all approach.

IIIc) Achieving buy-in (in public)

Public buy-in requires that intended participants announce their commitment to join the trial in a setting where others hear them. This is important because when someone states, in public, that they are willing to undertake an action, then they are more likely to abide by their commitment than if they take a silent decision – that can be forgotten easily.

IVa) Ensuring positive moments of truth

People evaluate organisations (including trial management teams) on the basis of their experiences at moments of truth. For example, if a doctor has a technical question about entering a patient into a trial she will gain a strong impression of the trial management team's competence by the way that the query is handled. If trialists behave well in a moment of truth then loyalty grows; if not, loyalty diminishes.

IVb) Providing frequent positive reinforcement

Positive reinforcement for existing participants should be an important part of a trial's participant retention strategy. It is more expensive to recruit new participants than to retain existing participants.

IVc) Facilitating incorporation into routines

Activities that become embedded as routines are more likely to be done than one-offs. Trial procedures should be incorporated into the routines of units undertaking the work.

We found that the CRASH trial faced challenges in marketing and selling that were mission critical – i.e. if goals were not achieved then the trial would fail. Farrell, amongst others, has been arguing for a greater recognition of the role of management in the conduct of clinical trials [ 5 ]. The key strength of the study reported here is that, for the first time in academic literature, it offers a reference model that provides a conceptual framework that can support and guide trial managers in assessing their marketing strengths and weaknesses [ 18 ]

The reference model described above should be seen as a tentative framework rather than a definitive template. It was developed from a theory-building process from a single trial and is best considered as a set of provocative hypotheses – later they may be developed as provisional audit tools. It may be that the reference model could be used as a diagnostic tool to identify if, and at what points, a trial is failing so that remedial interventions could be undertaken. An audit of the CRASH trial enabled components that were considered to be weaker than others to be identified and initiatives undertaken to improve in these areas. Further research is needed in other trials to explore whether the model is complete and correct and whether useful audit tools can be developed.

Clinical trials are not only research activities- they are also time-bound businesses that have two interdependent sets of processes – one clinical and the other managerial. In the main, since trials are seen as clinical endeavours, they are dominated by clinical issues and led by people with clinical skills. This is essential for certain policies and practices but this cultural bias can result in the managerial aspects of trials being, relatively, neglected. If this is true, even if only in part, it means that the radical improvement of clinical trials could require different ways of defining the challenges of running successful trials – in particular, to ensure that they are seen as management challenges that can benefit from the informed use of selected management processes and techniques.

These considerations suggest that looking within past trials for the answers to the problem of under-performing trials is necessary but will not be sufficient. In order to improve trials it will be necessary to look outside the world of clinical practice, into the worlds of business strategy, management, marketing and sales to gain a fuller understanding of what can be done to upgrade performance of clinical trials. This insight is not new. Donovan, Mills et al [ 19 ] state that the, "methodological literature (on trials) is almost exclusively statistical and epidemiological, and very little of it is concerned with the conduct or the particular demands that trials put on trialists and participants" (pp766).

This study could begin to change the ways that trial managers undertake their work. Also, it provides a different way to think about the skill sets and competencies needed by those who manage clinical trials. In essence, the message of this study is simple – even simplistic. It is that trials are both complex projects and businesses (they need to find customers). The key implication for clinicians is that insufficient attention to management issues and marketing or sales activities will degrade the performance of the trial.

There are significant implications for policy makers and funding bodies as well. If the tentative conclusions of this study are correct, then the funders will need to examine more than the scientific case before sponsoring a trial. They will need to see a marketing and sales plan, and be assured that all of the required elements of the business system will be developed. Since a successful trial requires both good science and good management, both need to be given their due weight.

But there are differences. Business is about profit. Medicine is driven by human values. It would be wrong to infer that publicly funded trials need to be more like businesses – rather, we suggest, that trials may benefit from using business concepts and business techniques.

This cross-disciplinary study was based on the premise that something new would be gained if a researcher from the world of business and management studied a clinical trial from his disciplinary perspective and worked with trialists to devise a useful framework. Since innovation is frequently facilitated by a clash of disciplines, it may be some of the insights needed to improve trial recruitment will come from fields other than medicine.

Lovato LC, Hill K, Hertert S, Hunninghake DB, Probstfield JL: Recruitment for Controlled Clinical Trials: Literature Summary and Annotated Bibliography. Controlled Clinical Trials. 1997, 18: 328-357. 10.1016/S0197-2456(96)00236-X.

Article   CAS   PubMed   Google Scholar  

Levitt T: Marketing Myopia. HBR. 1960, 45-56.

Google Scholar  

McDonald M, Wilson H: The New Marketing: Transforming the Corporate Future. 2002, Oxford , Butterworth-Heinemann

Schoenfeld ER, Greene JM, Wu SY, O'Leary E, Forte F, Leske C: Recruiting Participants for Community-Based Research: The Diabetic Retinopathy Awareness Program. AEP. 2000, 10 (7): 432-440.

CAS   PubMed   Google Scholar  

Farrell B: Efficient management of randomised controlled trials: nature or nurture. BMJ. 1998, 317: 1236-1239.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rowe JC, Elling ME, Hazlewood JG, Zakhary R: A Cure for Clinical Trials. McKinsey Quarterly. 2002

Wilson J, Musick M: Who Cares? Towards An Integrated Theory Of Volunteer Work. American Sociological Review. 1997, 62: 694-713. 10.2307/2657355.

Article   Google Scholar  

Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, Entwistle V, Garcia J, Roberts I, Grant A, The STEPS Group: Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study. Health Technol Assess. 2007, 11 (48): 1-126.

McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, Elbourne DR, Francis D, Garcia J, Roberts I, Snowdon C: What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006, 7 (9):

Snowdon C, Elbourne DR, Garcia J, Campbell MK, Entwistle VA, Francis D, Grant AM, Knight RC, McDonald AM, Roberts I: Financial considerations in the conduct of multi-centre randomised controlled trials: evidence from a qualitative study. Trials. 2006, 7 (34):

Edwards P, Arango M, Balica L, Cottingham R, El-Sayed H, Farrell B: Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months. Lancet. 2005, 365 (9475): 1957-1959. 10.1016/S0140-6736(05)66552-X.

Article   PubMed   Google Scholar  

Roberts IG, Yates D, Sandercock P, Farrell B, Wasserberg J, Lomas G: Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet. 2004, 364 (9442): 1321-1328. 10.1016/S0140-6736(04)17188-2.

Layder D: Sociological Practice: Linking Theory and Social Research. 1998, London , Sage

Chapter   Google Scholar  

Yin R: Case Study Research: Design and Methods. 1994, Thousand Oaks, Cal , Sage, 2nd

Eden C, Huxham C: Action Research. Essential Skills for Management Research. Edited by: Partington D. 2002, London , Sage Publications, 254-272.

Strauss A, Corbin J: Grounded Theory Methodology. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Edited by: Strauss A, Corbin J. 1990, Newbury Park, Cal , Sage, 273-285.

Martindale D: Sociological Theory and the Ideal Type. Symposium on Sociological Theory. Edited by: Gross LL. 1959, Evanston, Illinois , Row, Peterson and Company, 57-91.

Francis D, Woodcock M: Auditing Organizational Effectiveness. 2004, Farnborough , Gower

Donovan J, Mills N, Smith M, Brindle L, Jacoby A, Peters T, Frankel S, Neal D, Hamdy F: Improving design and conduct of randomised trials by embedding them in qualitative research: ProtecT (prostrate testing for cancer and treatment) study. BMJ. 2002, 325: 766-769. 10.1136/bmj.325.7367.766.

Article   PubMed   PubMed Central   Google Scholar  

Download references


We would like to thank the National Co-ordinating Centre for Research Methodology for commissioning this research; the MRC and the DH for providing funding; and the CRASH trial team. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Executive Health Department. The views expressed are not necessarily those of the funders and the funders had no involvement in the study.

Author information

Authors and affiliations.

Centre for Research and Innovation Management, Brighton, UK

David Francis

Nutrition and Public Health Interventions Research Unit, London School of Hygiene and Tropical Medicine, Keppel Street, London, UK

Ian Roberts & Haleema Shakur

Medical Statistics Unit, London School of Hygiene and Tropical Medicine, Keppel Street, London, UK

Diana R Elbourne, Rosemary C Knight, Jo Garcia & Claire Snowdon

Centre for Family Research, University of Cambridge, Free School Lane, Cambridge, CB2 3RF, UK

Claire Snowdon

Health Services Research Unit, University of Aberdeen, Health Sciences Building, Foresterhill, Aberdeen, UK

Vikki A Entwistle, Alison M McDonald, Adrian M Grant & Marion K Campbell

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to David Francis .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

The idea for STEPS was jointly conceived by the Principal Investigator, MKC, with AMG, VAE, DE, JG, CS, IR, DF, AMM and RCK. DF and IR proposed the case study. All authors contributed to the study design. DF, IR, DE and HS wrote the first draft of the manuscript, and all the authors read and approved the final manuscript. MKC is guarantor for STEPS, and DF is the guarantor for the case study

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Francis, D., Roberts, I., Elbourne, D.R. et al. Marketing and clinical trials: a case study. Trials 8 , 37 (2007).

Download citation

Received : 30 July 2007

Accepted : 20 November 2007

Published : 20 November 2007


Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Reference Model
  • Marketing Strategy
  • Trial Manager
  • Marketing Capability

ISSN: 1745-6215

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

case study in clinical trials

There’s a powerful story behind every headline at Ohio State Health & Discovery. As one of the largest academic health centers and health sciences campuses in the nation, we are uniquely positioned with renowned experts covering all aspects of health, wellness, science, research and education. Ohio State Health & Discovery brings this expertise together to deliver today’s most important health news and the deeper story behind the most powerful topics that affect the health of people, animals, society and the world.  Like the science and discovery news you find here? You can support more innovations fueling advances across medicine, science, health and wellness by giving today.


  • The Ohio State University
  • College of Dentistry
  • College of Medicine
  • College of Nursing
  • College of Optometry
  • College of Pharmacy
  • College of Public Health
  • College of Veterinary Medicine
  • Ohio State Wexner Medical Center
  • Ohio State's Comprehensive Cancer Center – James Cancer Hospital and Solove Research Institute

Subscribe. The latest from Ohio State Health & Discovery delivered right to your inbox.

How clinical trials work

Associate dean for Clinical and Translational Science College of Medicine

  • Share on Facebook
  • Share on Linkedin
  • Share via Email
  • Share this page

A woman receiving a shot in her arm from a female medical professional

You may have had sticker shock at some point when paying for a prescription.

Drugs can be expensive. The high price tag is largely because of the many years of research  a company has to do to prove their drug works, and that it’s safe for you to use.

The U.S. Food and Drug Administration (FDA) has nothing to do with setting prices for medications, but it is the agency that determines if a pharmaceutical drug should be allowed on the market.

Getting that approval can take 10 to 20 years of testing, first in a lab, then on animals and, finally, on people, in what’s called a clinical trial. People volunteer in clinical trials to have a chance to try the drug and for the drug’s producer to know if works and is safe.

At any given time, many clinical trials are going on at hospitals and universities across the country to test drugs or other treatments to cure a disease or make it easier to deal with symptoms.

Every clinical trial has at least three phases.

Phase 1 of a clinical trial

Number of volunteer participants: 20 to 100 people

Typically, the volunteers in Phase 1 don’t have the illness or condition the medication is supposed to treat. The exception to this is in cancer trials , in which all volunteers have the cancer that the medication is meant to treat.

The goal: Determine what, if any, side effects the drug causes, how well volunteers tolerate the drug and what dose gets best results.

Phase 2 of a clinical trial

Number of volunteer participants: Hundreds of people

In Phase 2 of a clinical trial, all volunteers have the condition that the medication treats (e.g., high blood pressure or high blood sugar).

The goal: Determine if the medication has the expected benefit, such as whether it lowered blood pressure or blood sugar as much as researchers had thought. In Phase 2, researchers also continue to focus on side effects and figuring out the best dose.

Phase 3 of a clinical trial

Number of volunteers: Thousands of people

The goal: Determine that the drug benefits people using it and doesn’t cause adverse side effects that outweigh its benefits. The benefits must exceed any side effects for it to be approved.

Sometimes, the FDA will require a fourth phase of trials. That’s when health care providers can prescribe the drug to participants and data is collected on the side effects and experiences of the hundreds of thousands of people who take it. Sometimes, a side effect won’t show up until the drug is given to a very large number of people.

Drug side effects

Whether someone is willing to tolerate difficult side effects of a medication depends on how life-threatening their disease is and how many other alternative medications are available to treat that disease. For example, a cancer patient may be willing to risk having side effects such as nausea, swelling and off-and-on numbing of their hands and feet if the chemotherapy medication shows promise in saving them from cancer. But those side effects wouldn’t be considered acceptable for a medication to treat high blood pressure.

Drug vs. placebo

Typically, clinical trials test a drug against not having any drug, to see if the medication being tested makes a difference in treating the illness. This means some participants will receive the drug being tested and some won’t. Typically, neither the medical staff nor the volunteer in the trial knows who gets what (called a “double-blind study”), so that participants are as unbiased as possible in reporting whether the drug helped with their symptoms.

The clinical trial volunteers who do not receive the medication being tested receive what’s called a “placebo.” That might be a syringe of saline solution or possibly a pill with no medicine in it.

If the medication being tested is for a serious illness or condition that can’t go untreated, all participants in the study will still receive standard medication already known to successfully treat that disease or condition. In this case, a group of trial participants might receive the new drug being tested along with the standard medication. At the same time, another group of study participants might get the standard medication plus a placebo. Then the researchers can compare the health outcome of each of the groups to see if the medication being tested is any better than the standard one.

Recruiting volunteers

One of the challenges with clinical trials is recruiting volunteers. It’s particularly challenging to find volunteers among:

  • People living in rural areas
  • People of color

Most clinical trials are done at major universities and large hospitals, which aren’t often found in rural areas. As a result, it can be difficult to recruit people who are able to drive to the study location for tests to see if the medication worked.

White people make up the most people who participate  in clinical trials, though there are a lot of efforts to attract more people of color. Efforts are made to ensure that drugs are being tested across a wide variety of people from different areas and ethnic groups. But more work needs to be done so that researchers have enough data about all groups of people who might eventually have the drug prescribed to them.

Clinical trials can only be done if enough people volunteer to be in them. It can be a time commitment, and not everyone can do that. For those who are able to participate, their contribution helps make more medications available to either stop illnesses or make it easier to live with them.

Ready to participate in a clinical study?

See what research studies need participants.

Julie Johnson

  • Clinical Research ,
  • Clinical Trial ,
  • Clinical Trial Volunteers ,
  • Medical Trials ,
  • Medication ,
  • Discovery & Innovation
  • Treatment Advances

Related websites

Articles on discovery & innovation.

Accelerating the pace of heart and lung research and discovery

Accelerating the pace of heart and lung research and discovery

By Beth Harvilla

Researcher Thomas Hund, PhD, accelerates the pace of heart and lung research and builds team science at The Ohio State University Dorothy M. Davis Heart and Lung Research Institute.

$20 million research project tackles mental health epidemic on massive scale

$20 million research project tackles mental health epidemic on massive scale

By Kelli Trinoskey

A $20 million grant is funding a new mental health research initiative on an unprecedented scale across all 88 Ohio counties.

Pickleball player plans return to sport after receiving high-tech shoulder replacement alternative

Health talks: national study to help inform regulation of tobacco products, stopping the deadly legacy of tobacco use and nicotine addiction.

By Steve Wartenberg

Get articles and stories about health, wellness, medicine, science and education delivered right to your inbox from the experts at Ohio State.

Required fields

Tell us more about yourself

By clicking "Subscribe" you agree to our Terms of Use . Learn more about how we use your information by reading our Privacy Policy .

Loading metrics

Open Access


Research Article

Institutional dashboards on clinical trial transparency for University Medical Centers: A case study

Roles Conceptualization, Data curation, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Berlin Institute of Health at Charité - Universitätsmedizin Berlin, QUEST Center for Responsible Research, Berlin, Germany

ORCID logo

Roles Conceptualization, Data curation, Investigation, Methodology, Software, Visualization, Writing – review & editing

Roles Software, Visualization, Writing – review & editing

Roles Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

  • Delwen L. Franzen, 
  • Benjamin Gregory Carlisle, 
  • Maia Salholz-Hillel, 
  • Nico Riedel, 
  • Daniel Strech


  • Published: March 21, 2023
  • Peer Review
  • Reader Comments

Fig 1

University Medical Centers (UMCs) must do their part for clinical trial transparency by fostering practices such as prospective registration, timely results reporting, and open access. However, research institutions are often unaware of their performance on these practices. Baseline assessments of these practices would highlight where there is room for change and empower UMCs to support improvement. We performed a status quo analysis of established clinical trial registration and reporting practices at German UMCs and developed a dashboard to communicate these baseline assessments with UMC leadership and the wider research community.

Methods and findings

We developed and applied a semiautomated approach to assess adherence to established transparency practices in a cohort of interventional trials and associated results publications. Trials were registered in or the German Clinical Trials Register (DRKS), led by a German UMC, and reported as complete between 2009 and 2017. To assess adherence to transparency practices, we identified results publications associated to trials and applied automated methods at the level of registry data (e.g., prospective registration) and publications (e.g., open access). We also obtained summary results reporting rates of due trials registered in the EU Clinical Trials Register (EUCTR) and conducted at German UMCs from the EU Trials Tracker. We developed an interactive dashboard to display these results across all UMCs and at the level of single UMCs. Our study included and assessed 2,895 interventional trials led by 35 German UMCs. Across all UMCs, prospective registration increased from 33% ( n = 58/178) to 75% ( n = 144/193) for trials registered in and from 0% ( n = 0/44) to 79% ( n = 19/24) for trials registered in DRKS over the period considered. Of trials with a results publication, 38% ( n = 714/1,895) reported the trial registration number in the publication abstract. In turn, 58% ( n = 861/1,493) of trials registered in and 23% ( n = 111/474) of trials registered in DRKS linked the publication in the registration. In contrast to recent increases in summary results reporting of drug trials in the EUCTR, 8% ( n = 191/2,253) and 3% ( n = 20/642) of due trials registered in and DRKS, respectively, had summary results in the registry. Across trial completion years, timely results reporting (within 2 years of trial completion) as a manuscript publication or as summary results was 41% ( n = 1,198/2,892). The proportion of openly accessible trial publications steadily increased from 42% ( n = 16/38) to 74% ( n = 72/97) over the period considered. A limitation of this study is that some of the methods used to assess the transparency practices in this dashboard rely on registry data being accurate and up-to-date.


In this study, we observed that it is feasible to assess and inform individual UMCs on their performance on clinical trial transparency in a reproducible and publicly accessible way. Beyond helping institutions assess how they perform in relation to mandates or their institutional policy, the dashboard may inform interventions to increase the uptake of clinical transparency practices and serve to evaluate the impact of these interventions.

Author summary

Why was this study done.

  • Clinical trials are the foundation of evidence-based medicine and should follow established guidelines for transparency: Their results should be available, findable, and accessible regardless of the outcome.
  • Previous studies have shown that many clinical trials fall short of transparency guidelines, which distorts the medical evidence base, creates research waste, and undermines medical decision-making.
  • University Medical Centers (UMCs) play an important role in increasing clinical trial transparency but are often unaware of their performance on these practices, making it difficult to drive improvement.

What did the researchers do and find?

  • We developed a pipeline to evaluate clinical trials across several established practices for clinical trial transparency and applied it in a cohort of 2,895 clinical trials led by German UMCs.
  • We found that while some practices are gaining adherence (e.g., prospective registration in increased from 33% to 75% over the period considered), there is much room for improvement (e.g., 41% of trials reported results within 2 years of trial completion).
  • We developed a dashboard to communicate these transparency assessments to UMCs and support their efforts to improve.

What do these findings mean?

  • Our study demonstrates the feasibility of developing a dashboard to communicate adherence to established practices for clinical trial transparency.
  • By highlighting areas for improvement, the dashboard provides actionable information to UMCs and empowers their efforts to improve.
  • The dashboard may inform interventions to increase clinical trial transparency and be scaled to other countries and stakeholders, such as funders or clinical trial registries.

Citation: Franzen DL, Carlisle BG, Salholz-Hillel M, Riedel N, Strech D (2023) Institutional dashboards on clinical trial transparency for University Medical Centers: A case study. PLoS Med 20(3): e1004175.

Academic Editor: Florian Naudet, University of Rennes 1, FRANCE

Received: April 28, 2022; Accepted: January 18, 2023; Published: March 21, 2023

Copyright: © 2023 Franzen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. The dashboard is openly available at: . Code to produce the dashboard is openly available in GitHub at: . Code to generate the dataset displayed in the dashboard is openly available in GitHub: . Data can be downloaded from the dashboard and are openly available on OSF at: . Raw data obtained from trial registries are openly available on Zenodo at: . Data for summary results reporting in the EUCTR are available via the EU Trials Tracker.

Funding: This work was funded by the Federal Ministry of Education and Research of Germany (BMBF 01PW18012, ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: The authors are affiliated to the Charité – Universitätsmedizin Berlin, one of the institutions included in this evaluation and in the dashboard.

Abbreviations: CONSORT, Consolidated Standards of Reporting Trials; CTIMP, Clinical Trial of an Investigational Medicinal Product; DOI, Digital Object Identifier; DORA, Declaration on Research Assessment; DRKS, Deutsches Register Klinischer Studien (German Clinical Trials Register); EUCTR, EU Clinical Trials Register; FDAAA, Food and Drug Administration Amendments Act; ICMJE, International Committee of Medical Journal Editors; OA, Open Access; OSF, Open Science Framework; STROBE, Strengthening the Reporting of Observational Studies in Epidemiology; TRN, trial registration number; UMC, University Medical Center; WHO, World Health Organization


Valid medical decision-making depends on an evidence base composed of clinical trials that were prospectively registered and reported in an unbiased and timely manner. The registration of clinical trials in publicly accessible registries informs clinicians, patients, and other relevant stakeholders about what trials are planned, in progress or completed, and aggregates key information relating to those trials. Trial registration thus reduces bias in our understanding of the existing medical evidence and disincentivizes outcome-switching and selective reporting [ 1 ]. For clinical trials to generate useful and generalizable medical knowledge gain, trial results should also be reported in a timely manner after trial completion per the World Health Organization (WHO) Joint Statement on Public Disclosure of Results from Clinical Trials [ 2 ]. Disclosure is a necessary but not sufficient component of transparency: Trial results should also be openly accessible and findable, in line with established guidelines [ 2 – 6 ]. However, several studies have shown that clinical trials are often not registered and reported according to these standards [ 7 – 11 ].

Audits of research practices can build understanding of the status quo, inform new policies, and evaluate the impact of interventions to support improvement. Examples include the European Commission’s Open Science monitor [ 12 ], the German Open Access monitor [ 13 ], the French Open Science Monitor in health [ 14 ], and institution-specific dashboards of select research practices [ 15 ]. Focusing on trial transparency, the EU Trials Tracker and the Food and Drug Administration Amendments Act 2007 (FDAAA) TrialsTracker [ 16 , 17 ] display up-to-date summary results reporting rates of public and private trial sponsors in a transparent and accessible way. The EU Trials Tracker served as a key resource for initiatives aiming to increase reporting rates of drug trials in the EU Clinical Trials Register (EUCTR) [ 18 , 19 ]. Based on the EU Trials Tracker, results reporting in the EUCTR has increased from 50% in 2018 to 84% (late 2022).

Research institutions such as University Medical Centers (UMCs) can incentivize practices for research transparency through their reward and promotion systems [ 20 , 21 ] and by providing education, infrastructure, and services [ 22 , 23 ]. However, internal and external assessments of research conducted at UMCs rarely acknowledge these practices [ 24 , 25 ]. Rather, traditional indicators of research performance such as the number of clinical trials, the extent of third-party funding, and the impact factor of published papers emphasize quantity over quality, which can entrench problematic research practices [ 26 ]. Initiatives such as the Declaration on Research Assessment (DORA) and the Hong Kong Principles have called for a change in the way researchers are assessed, and for more recognition of behaviors that strengthen research integrity [ 20 , 27 ]. The establishment of the Coalition on Advancing Research Assessment (CoARA) and the 2022 Agreement on Reforming Research Assessment emphasize this shift towards rewarding responsible research practices to maximize research quality and impact [ 28 ]. In turn, the UNESCO Recommendation on Open Science adopted in 2021 affirmed the need to establish monitoring and evaluation mechanisms relating to open science [ 29 ]. Audits of transparency practices could empower UMCs to support their uptake by highlighting where there is room for improvement and where to allocate resources. Comparative assessments between institutions could also provide examples of successes and stimulate knowledge transfer.

Audits that are based on open and scalable methods facilitate repeated evaluation and uptake at other organizations. Such an evaluation of transparency practices at the level of clinical trials led by UMCs requires reproducible and efficient procedures for (a) sampling all clinical trials and associated results publications affiliated to UMCs and (b) measuring select registration and reporting practices. We previously established procedures for identifying all clinical trials associated with a specific UMC and their earliest results publications [ 9 , 11 ]. In turn, an increasing number of open-source publication and registry screening tools have been developed in the context of meta-research projects aiming to increase research transparency and reproducibility [ 10 , 30 – 32 ].

The objective of this study was to perform a status quo analysis of a set of established practices for clinical trial transparency at the level of UMCs and present these assessments in the form of an interactive dashboard to support efforts to improve performance. While the general approach of our study is applicable for UMCs worldwide, this study focused on German UMCs.

Producing a dashboard for clinical trial transparency required the development of a pipeline consisting of 3 main steps: first, the identification of registered clinical trials led by German UMCs; second, the evaluation of select registration and reporting practices, including (a) the partly automated and partly manual identification of earliest results publications of these trials and (b) the application of automated tools at the registry and publication level; third, the presentation of these baseline assessments in the form of an interactive dashboard. An overview of the dependence of these steps on automated versus manual approaches is provided in S1 Supplement . The development of the dashboard was iterative and did not have a prospective protocol. The methods to develop the underlying dataset of clinical trials and associated results publications, however, were preregistered in Open Science Framework (OSF) for trials completed 2009 to 2013 [ 33 ] and 2014 to 2017 [ 34 ].

Data sources and inclusion and exclusion criteria

The data displayed in the dashboard relate exclusively to registered (either prospectively or retrospectively) clinical trials obtained from 3 data sources with the following inclusion and exclusion criteria:

  • The IntoValue cohort of registered clinical trials and associated results [ 35 ]. This dataset consists of interventional clinical trials registered in or DRKS, considered as complete between 2009 and 2017 per the registry, and led by a German UMC (i.e., led either as sponsor, responsible party, or as host of the principal investigator). Trials were searched for 38 German UMCs based on their inclusion as members on the website of the association of medical faculties of German universities [ 36 ] at the time of data collection. In line with WHO and International Committee of Medical Journal Editors (ICMJE) definitions [ 4 , 37 ], trials in this cohort include all interventional studies and are not limited to Clinical Trials of an Investigational Medicinal Product (CTIMP) regulated by the EU’s Clinical Trials Regulation or Germany’s drug or medical device laws. The dataset includes data from partly automated and partly manual searches to identify the earliest reported results associated with these trials (as summary results in the registry and as publication). The methods for sampling UMC-specific sets of registered clinical trials and tracking associated results are described in detail elsewhere [ 9 , 11 ]. Briefly, we used automated methods to search registries for clinical trials associated with German UMCs and manually validated the affiliations of all trials. We deduplicated trials in this cohort that were cross-registered in and DRKS (see more information in S2 Supplement ). Results publications associated with these trials were identified by means of a manual search across several search engines. This was complemented by automated methods to identify linked publications in the registry [ 10 ]. To reflect the most up-to-date status of trials, we downloaded updated registry data for the trials in this cohort on 1 November 2022 and reapplied the original IntoValue exclusion criteria: study completion date before 2009 or after 2017, not considered as complete based on study status, and not interventional. More detailed information on the inclusion and exclusion criteria can be found in S2 Supplement .
  • For assessing prospective registration in , we used a more recent cohort of interventional trials registered in , started between 2006 and 2018, led by a German UMC, and considered as complete per study status in the registry. We downloaded updated registry data for the trials in this cohort on 1 November 2022 and reapplied the same exclusion criteria as above except for completion date ( S2 Supplement ).
  • For assessing results reporting in the EUCTR, we retrieved data from the EU Trials Tracker on 4 November 2022 [ 16 ]. We found a sponsor name for 34 of the UMCs included in this study as of August 2021 (sponsor names in the EU Trials Tracker are subject to change). If more than one corresponding sponsor name was found for a given UMC (Bochum, Giessen, Heidelberg, Kiel, Marburg, and Tübingen), we selected the sponsor with the most trials. More detailed information can be found in S3 Supplement .

Analysis of registration and reporting practices

The dashboard displays the performance of UMCs on 7 recommended transparency practices for trial registration and reporting. In this study, we focused on adherence to ethical principles and reporting guidelines that apply to all trials. Compliance with a legal regulation was only assessed for summary results reporting in the EUCTR. For an overview of these practices, relevant guidelines and laws, the sample considered, and the measured outcome, see Fig 1 (sources in S4 Supplement ) and Table 1 . The data for these metrics were obtained through a combination of automated approaches and manual searches, several of which have been described previously [ 8 – 11 ]. In the following, we outline the methods used to generate the data for each metric. More detailed information can be found in the Methods page of the dashboard and in S5 Supplement .


  • PPT PowerPoint slide
  • PNG larger image
  • TIFF original image

Relevant guidelines and/or laws are provided for each practice (as of November 2022). A list of references can be found in S4 Supplement . An adaptation of this overview is included in the “Why these practices?” page of the dashboard. *DFG: According to the DFG guidelines at the time of writing, summary results should be posted in the registry at the latest 2 years after trial completion, or earlier if required by applicable legal regulations. BMBF, Bundesministerium für Bildung und Forschung; CIOMS, Council for International Organizations of Medical Sciences; CONSORT, Consolidated Standards of Reporting Trials; CTIMP, Clinical Trial of an Investigational Medicinal Product; DFG, Deutsche Forschungsgemeinschaft; ICMJE, International Committee of Medical Journal Editors; ICTRP, International Clinical Trials Registry Platform; WHO, World Health Organization; WMA, World Medical Association.


Prospective registration.

Raw registry data downloaded from and DRKS were further processed to determine the registration status of trials. We defined a trial to be prospectively registered if the trial was registered in the same or a previous month to the trial start date.

Bidirectional links between registry entries and associated results publications.

We extracted links to publications from the registry data and obtained the full text of publications. We then applied regular expressions to detect publication identifiers in registrations, and trial registration numbers (TRNs) in publications. The application of these methods on the IntoValue cohort was reported previously [ 10 ].

Summary results reporting in the registry.

For , we extracted the relevant information from the structured summary results field. For DRKS, we detected summary results based on the presence of keywords (e.g., Ergebnisbericht or Abschlussbericht) in the reference title. The summary results date in DRKS was extracted manually from the registry’s change history. We obtained summary results reporting rates in the EUCTR from the EU Trials Tracker. We retrieved historical data (percent reported, total number of due trials, and total number of trials that reported results) from the associated code repository [ 16 ].

Reporting as a manuscript publication.

The earliest publication found for each trial and its publication date was derived from the original IntoValue dataset [ 35 ]. Dissertations were excluded from publication-based metrics.

Open Access (OA) status.

To determine the OA status of trial results publications, we queried the Unpaywall database via its API on 1 November 2022 using UnpaywallR and assigned one of the following statuses: gold (openly available in an OA journal), hybrid (openly available in a subscription-based journal), green (openly available in a repository), bronze (openly available on the journal page but without a clear open license), or closed. As publications can have several OA versions, we applied a hierarchy such that only one OA status was assigned to each publication, in descending order: gold, hybrid, green, bronze, and closed.

Interactive dashboard

We developed an interactive dashboard to present the outcome of these assessments at the institutional level in an accessible way to the UMC leadership and the wider research community. The dashboard was developed with the Shiny R package (version 1.6.0) [ 38 ] based on an initial version developed by NR for the Charité –Universitätsmedizin Berlin [ 15 ]. The dashboard was shaped by interviews with UMC leadership, support staff, funders, and experts in responsible research who provided feedback on a prototype version [ 39 ]. This feedback led to the inclusion of several features to facilitate the interpretation of the data and contextualize the assessed transparency practices. The code underlying the dashboard developed in this study is openly available in GitHub under an AGPL license ( ) and may be adapted for further use.

We generated descriptive statistics on the characteristics of the trials and the transparency practices, all of which are displayed in the dashboard. We report proportions across UMCs (e.g., “Start” page) and per UMC broken down by start year (prospective registration only), completion year, publication year (open access), and registry (publication link in the registry, summary results reporting). We did not test specific hypotheses.

Software, code, and data

Data processing was performed in R (version 4.0.5) [ 40 ] and Python 3.9 (Python Software Foundation, Wilmington, Delaware, USA). With the exception of summary results reporting in the EUCTR (data available via the EU Trials Tracker ), all the data processing steps involved in generating the dataset displayed in this dashboard are openly available in GitHub: . The data displayed in the dashboard are available in OSF [ 41 ] and in the dashboard Datasets page. Raw data obtained from trial registries are openly available in Zenodo [ 42 ]. This study is reported as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for cross-sectional studies ( S6 Supplement ).

Characteristics of trials

The IntoValue dataset that this study is based on includes interventional trials registered in or DRKS, led by a German UMC, and reported as complete between 2009 and 2017 ( n = 3,113). Trials were found for 35 out of 38 UMCs searched. After downloading updated registry data for trials in this cohort, we excluded 91 trials based on our exclusion criteria (study completion date before 2009 or after 2017, n = 73; not considered as complete per study status, n = 16; not interventional, n = 2). After removal of duplicates, this led to 2,895 trials that served as the basis for most metrics ( Fig 2 ). For prospective registration in , we used a more recent cohort of interventional trials registered in , led by a German UMC, started between 2006 and 2018, and considered as complete per study status in the registry ( n = 4,058). After applying our inclusion criteria, this sample included 3,618 trials. S7 Supplement provides an overview of the characteristics of included trials stratified by registry. S8 Supplement provides flow diagrams of the trial and publication screening for each metric.


Flowchart of the trial screening (IntoValue). The box with the thicker contour highlights the starting point of the trial screening for other registry-based metrics (see Flowcharts 1–3 in S8 Supplement )., ; DRKS, German Clinical Trials Register; IV, IntoValue; UMC, University Medical Center.

Evaluation of trial registration and reporting practices

We developed an interactive dashboard ( /) to display the results of the evaluation of trial registration and reporting across UMCs. In the following, we highlight some of these results. More extensive evaluations of some of these practices are reported in separate publications, such as results reporting of trials [ 9 , 11 ] and links between trial registration and results publications [ 10 ].

Trial registration

Prospective registration : The proportion of trials led by German UMCs that were prospectively registered increased in both and DRKS over the period considered. Of 178 trials registered in and started in 2006, 58 (33%, 95% confidence interval 26% to 40%) were registered prospectively. A little more than a decade later, 144 of 193 (75%, 95% confidence interval 68% to 80%) trials started in 2018 were registered prospectively. Trials registered in DRKS followed a similar trend: While none of the 44 (0%, 95% confidence interval 0% to 10%) trials started between 2006 and 2008 were prospectively registered, this increased to 19 of 24 (79%, 95% confidence interval 57% to 92%) for trials started in 2017 ( S9 Supplement ). Among clinical trials registered in , the median per-UMC rate of prospective registration ranged from 30% ( n = 17/56) to 68% ( n = 127/186) with a median of 55% and a standard deviation of 8%. Per-UMC rates of prospective registration in DRKS ranged from 0% ( n = 0/1) to 75% ( n = 15/20) with a median of 44% and a standard deviation of 15%.

Reporting of a TRN in publications.

Of the 1,895 registered trials with a publication indexed in PubMed, 714 (38%, 95% confidence interval 35% to 40%) reported a TRN in the publication abstract. In turn, 1,136 of 1,893 registered trials for which the full text was available reported a TRN in the publication full text (60%, 95% confidence interval 58% to 62%) ( S9 Supplement ). Only 476 of 1,893 (25%, 95% confidence interval 23% to 27%) of trials reported a TRN in both the abstract and full text of the publication as per the ICMJE and Consolidated Standards of Reporting Trials (CONSORT) guidelines. The per-UMC rate at which clinical trial publications reported a TRN in the abstract ranged from 17% ( n = 13/75) to 56% ( n = 23/41) with a median of 38% and a standard deviation of 8%. The per-UMC rate at which clinical trial publications reported a TRN in the full text was higher, ranging from 43% ( n = 41/95) to 76% ( n = 32/42) with a median of 61% and a standard deviation of 7%.

Publication links in the registry.

Of 1,493 trials registered in with a publication, 861 (58%, 95% confidence interval 55% to 60%) had a link to the publication in the registration. In turn, only 111 of 474 trials registered in DRKS with a publication (23%, 95% confidence interval 20% to 28%) had a link to the publication in the registration. Among trials registered in with a publication, the per-UMC rate of publication links in the registration ranged from 32% ( n = 12/37) to 88% ( n = 28/32) with a median of 56% and a standard deviation of 12%. Among trials registered in DRKS with a publication, the per-UMC rate of publication links in the registration ranged from 0% ( n = 0/7) to 45% ( n = 5/11) with a median of 23% and a standard deviation of 13%.

Trial reporting

Summary results reporting..

We first assessed how many of the trials registered in or DRKS had summary results in the registry. The cumulative proportion of trials that reported summary results has stagnated at low levels between 2009 and 2017. Only 191 of all 2,253 (8%, 95% confidence interval 7% to 10%) trials registered in , and 20 of all 642 (3%, 95% confidence interval 2% to 5%) trials registered in DRKS had summary results in the registry ( S9 Supplement ). Per-UMC summary results reporting rates for all trials ranged between 0% ( n = 0/42) and 32% ( n = 8/25) (median of 7% and a standard deviation of 7%) for , and between 0% ( n = 0/23) and 50% ( n = 7/14) (median of 0% and a standard deviation of 9%) for DRKS. In contrast, reporting of summary results in the EUCTR was higher and increased over time: In almost 2 years, results reporting for due trials almost doubled from 41% ( n = 223/541, 95% confidence interval 37% to 46%) in December 2020 to 79% ( n = 647/813, 95% confidence interval 77% to 82%) in October 2022 (EU Trials Tracker) ( S9 Supplement ). At the time of data collection (November 2022), per-UMC summary results reporting rates in the EUCTR ranged between 0% ( n = 0/1) and 100% ( n = 14/14) across all included UMCs with a median of 82% and a standard deviation of 30%.

Timely reporting of results (2- and 5-year reporting rates).

Next, we assessed how many trials registered in or DRKS reported results in a timely manner. Reporting guidelines and German research funders have called on clinical trials to report (a) summary results in the registry within 12 and 24 months of trial completion and (b) results in a manuscript publication within 24 months of trial completion [ 2 , 43 – 45 ]. We therefore considered 2 years as timely reporting for both reporting routes. Of 2,892 trials registered in or DRKS with a 2-year follow-up period for reporting results as either summary results or a manuscript publication, 1,198 (41%, 95% confidence interval 40% to 43%) had done so within 2 years of trial completion.

While the 5-year reporting rate was unsurprisingly higher, 505 of 1,619 trials (31%, 95% confidence interval 29% to 34%) registered in or DRKS with 5-year follow-up between trial completion and the manual publication search had not reported results as a journal publication within 5 years of trial completion. Publication in a journal was the dominant route of reporting results, with summary results reporting rates below 10% across all completion years and follow-up periods. Per-UMC reporting rates as a manuscript publication ranged between 15% ( n = 7/46) and 58% ( n = 19/33) (2-year rate, median 39%, standard deviation 9%) and between 50% ( n = 24/48) and 87% ( n = 13/15) (5-year rate, median 70%, standard deviation 8%). Per-UMC reporting rates as summary results ranged between 0% ( n = 0/76) and 14% ( n = 6/43) (2-year rate, median 4%, standard deviation 4%) and between 0% ( n = 0/72) and 21% (9/42) (5-year rate, median 5%, standard deviation 5%).

The proportion of trial results publications that were openly accessible (gold, hybrid, green, or bronze) increased from 42% in 2010 ( n = 16/38, 95% confidence interval 27% to 59%) to 74% in 2020 ( n = 72/97, 95% confidence interval 64% to 82%) ( S9 Supplement ). Across all publication years, 891 of 1,920 (46%, 95% confidence interval 44% to 49%) trial publications were neither openly accessible via a journal nor an OA repository based on Unpaywall. Per-UMC rates of trial results publications that were OA ranged from 26% ( n = 10/38) to 72% ( n = 23/32) with a median of 55% and a standard deviation of 10%.

The key outcome of this paper is an interactive and openly accessible dashboard to visualize adherence to the aforementioned best practices for trial registration and reporting across German UMCs: . The dashboard displays the data in 3 ways: (a) assessment across all UMCs (national dashboard; see a screenshot in Fig 3 ); (b) comparative assessment between UMCs; and (c) UMC-specific assessment (see a screenshot for one UMC in S10 Supplement ).


Assessment of 7 registration and reporting practices across all included German UMCs (8 November 2022).

To allow for a better interpretation of the data displayed in the dashboard, absolute numbers are displayed in all plots as mouse-overs. A description of the methods and limitations of each metric is also provided next to each plot, with more detailed information in the Methods page. A FAQ page addresses general considerations raised in interviews with relevant stakeholders [ 39 ]. These interviews highlighted the importance of an overall narrative justifying the choice of metrics included. We therefore designed an infographic of relevant laws and guidelines to contextualize the clinical transparency metrics included in the dashboard (adapted from Fig 1 ).

Concerns about delayed and incomplete results reporting in clinical research and other sources of research waste have triggered debate on incentivizing individual researchers and UMCs to adopt more responsible research practices [ 20 , 22 , 23 ]. Here, we introduced the methods and results underlying a dashboard for clinical trial transparency, which provides actionable information on UMCs’ performance in relation to established registration and reporting practices and thereby empowers their efforts to support improvement. This dashboard approach for clinical trial transparency at the level of individual UMCs serves to (a) inform institutions about their performance and set this in relation to national and international transparency guidelines and funder mandates, (b) highlight where there is room for improvement, (c) trigger discussions across relevant stakeholder groups on responsible research practices and their role in assessing research performance, (d) point to success stories and facilitate knowledge sharing between UMCs, and (e) inform the development and evaluation of interventions that aim to increase trial transparency.

Trends in trial transparency

The dashboard displays progress over time and allows the data to be explored in different ways. While the upward trend for several practices (e.g., prospective registration, OA) is encouraging, there is much room for improvement with respect to established guidelines for clinical trial transparency. For example, less than half (45%) of trials registered in or DRKS and completed in 2017 reported results in a manuscript publication within 2 years of trial completion as per WHO and funder recommendations [ 2 , 43 , 44 ]. We observed a striking difference in the cumulative proportion of summary results reporting of drug trials registered in the EUCTR compared with trials registered in and DRKS. The uptake of summary results reporting in the EUCTR likely reflects the combined impact of the EU legal requirement for drug trials to report summary results within 12 months [ 45 ], the launch of the EU Trials Tracker and subsequent academic initiatives to increase reporting rates [ 8 , 18 ], as well as media attention [ 46 ]. This suggests that audits of compliance with respect to established guidelines and further awareness raising may also have the potential to increase results reporting rates of other types of trials.

Actionable areas for stakeholders

Some of the practices included in this dashboard can still be addressed retroactively, such as linking publications in the trial registration (realized for 49% of trials with a publication). These constitute actionable areas for improvement that UMCs can contribute to by providing education, support, and incentives. One important way to incentivize UMCs in this regard is to make responsible research practices part of internal and external quality assessment procedures. Other stakeholders such as funders, journals and publishers, registries, and bibliographic databases should complement these activities by reviewing compliance with their policies as well as applicable guidelines and/or laws. Salholz-Hillel and colleagues, for example, outlined specific recommendations for each stakeholder to improve links between trial registrations and publications [ 10 ]. UMCs and their core facilities for clinical research can, for example, use the data linked to the dashboard to inform principal investigators about the transparency of their specific trials. We are currently finalizing such a “report card” approach at the Charité - Universitätsmedizin Berlin [ 47 ].

Scalability beyond German UMCs

The datasets and methods used in this study can be scaled: This has been demonstrated in another European country (Poland) [ 48 ] and is currently underway in California, USA [ 49 ]. While the generation of the underlying dataset of clinical trials and associated results publications involves manual checks (approximately 10 person-hours per 100 trials), the assessment of transparency practices is largely automated. Institutions in possession of an in-house cohort of clinical trial registry numbers and persistent identifiers (e.g., Digital Object Identifier (DOI)) from matched journal publications, however, could achieve results more quickly. The code to create the dashboard is openly available and can be adapted to other cohorts.

Stakeholder and community engagement

The uptake of this dashboard approach by UMCs and other stakeholders depends on their respective attitudes and readiness. We previously solicited stakeholders’ views on an institutional dashboard with metrics for responsible research. While interviewees considered the dashboard helpful to see where an institution stands and to initiate change, some pointed to the challenge that making such a dashboard public might risk incorrect interpretation of the metrics and harm UMCs’ reputation [ 39 ]. While similar challenges with interpretation and reputation apply to current metrics for research assessment (e.g., impact factors and third-party funding), this stakeholder feedback demonstrates the need for community engagement when introducing novel strategies for research assessment. In this regard, a Delphi study was performed to reach consensus on a core outcome set of open science practices within biomedicine to support audits at the institutional level [ 50 ]. A detailed comparative assessment of existing monitoring initiatives and lessons learned could further support these efforts.

Updates and further development of the dashboard

We are planning regular updates of the registry data for trials already in the dashboard, as well as the inclusion of more recent cohorts of trials with at least 2 years follow-up (e.g., trials completed 2018 to 2021 assessed in 2023). Besides these updates, further transparency practices may be integrated into the dashboard in the future, e.g., dissemination of results as preprints, the use of self-archiving to broaden access to results [ 51 ], adherence to reporting guidelines [ 3 ], or data sharing [ 52 ]. Beyond transparency, other potential metrics could reflect the number of discontinued trials [ 53 ] or the proportion of trials that inform clinical practice [ 54 ]. The development of such metrics should acknowledge the availability of standards and infrastructure pertaining to the underlying practices [ 23 ] and differences between study types and disciplines [ 27 ]. Future versions of the dashboard may also display additional subpopulation comparisons, such as different clinical trial registries or UMC particularities [ 55 ].


A limitation of this study is that inaccurate or outdated registry data (e.g., incorrect completion dates or trial status) may have impacted the assessment of transparency practices described in this study. To mitigate this limitation, we updated the registry data with the most recent data we could obtain. The update-related changes suggest no systematic bias in the comparison across UMCs. Another limitation is that the trial dataset may contain more cross-registrations than we identified. For the aforementioned “report card” project, we manually verified 168 trials and found only 2 missed cross-registrations (1%). We therefore believe that missed cross-registrations represent only a small portion of our sample. Moreover, the assessment of each practice in the dashboard applies to a specific subset of trials or publications and comes with unique limitations, largely resulting from challenges associated with manual or automated methods (outlined in more detail in S5 Supplement ). More generally, the dashboard focuses on interventional trials registered in or DRKS and does not display how German UMC drug trials only registered in the EUCTR perform on established transparency practices (except for summary results reporting in the registry). We are considering including all drug trials in the EUCTR conducted by German UMCs in future developments of the dashboard.

UMCs play an important role in fostering clinical trial transparency but face challenges doing so in the absence of baseline assessments of current practice. We assessed adherence to established practices for clinical trial registration and reporting at German UMCs and communicated the results in the form of an interactive dashboard. We observed room for improvement across all assessed practices, some of which can still be addressed retroactively. The dashboard provides actionable information to drive improvement, facilitates knowledge sharing between UMCs, and informs the development of interventions to increase research transparency.

Supporting information

S1 supplement. use of automated vs. manual approaches across methods..

S2 Supplement. Inclusion and exclusion criteria.

S3 Supplement. Selected sponsor names in the EU Trials Tracker.

S4 Supplement. Sources for Fig 1 .

S5 Supplement. Detailed methods and limitations of registration and reporting metrics.

S6 Supplement. STROBE checklist for cross-sectional studies.

S7 Supplement. Characteristics of included trials.

S8 Supplement. Flow diagrams of the trial and publication screening.

S9 Supplement. Screenshots of the “Start” page of the dashboard.

S10 Supplement. Screenshot of the “One UMC” page of the dashboard.


We would like to acknowledge Tamarinde Haven and Martin Holst for their valuable input that shaped the dashboard. We acknowledge financial support from the Open Access Publication Fund of Charité – Universitätsmedizin Berlin and the German Research Foundation (DFG).

  • View Article
  • PubMed/NCBI
  • Google Scholar
  • 2. World Health Organization. Joint statement on public disclosure of results from clinical trials. World Health Organization [Internet]. 2017 [cited 2021 Jun 15]. Available from: .
  • 4. International Committee of Medical Journal Editors (ICMJE). Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. ICMJE [Internet]. 2021 [cited 2021 Dec 21]. Available from: .
  • 12. European Commission. Open science monitor. European Commission | Research and innovation [Internet]. [cited 2021 Nov 12]. Available from: .
  • 13. Open Access Monitor. Open access monitor [Internet]. [cited 2021 Mar 22]. Available from: .
  • 14. French Open Science Monitor in health. French Open Science Monitor [Internet]. [cited 2022 Feb 4]. Available from: .
  • 15. BIH-QUEST Center for Responsible Research at Charité - Universitätsmedizin Berlin. Charité Dashboard on Responsible Research. Charité Metrics Dashboard [Internet]. 2021. Available from: .
  • 16. EBM DataLab. EU Trials Tracker. 2018 [cited 2021 Jul 26]. Available from: .
  • 17. EBM DataLab. FDAAA TrialsTracker. 2018. Available from: .
  • 19. Cochrane Sweden. Cochrane Sweden collaborates on trial transparency report. Cochrane [Internet]. 2020 [cited 2021 Nov 2]. Available from: .
  • 27. San Francisco Declaration on Research Assessment. DORA [Internet]. 2012 [cited 2020 Dec 16]. Available from: .
  • 28. Coalition for Advancing Research Assessment. Agreement on reforming research assessment. CoARA [Internet]. 2022 Jul 20. Available from: .
  • 29. UNESCO. UNESCO Recommendation on Open Science. UNESCO | Open Science [Internet]. 2021 [cited 2022 Apr 29]. Available from: .
  • 33. Wieschowski S, Kahrass H, Schürmann C, Strech D, Riedel N, Siegerink B, et al. IntoValue. OSF. 2017[cited 2022 Nov 9].
  • 34. Wieschowski S, Strech D, Riedel N, Kahrass H, Bruckner T, Holst M, et al. In2Value 2. OSF. 2020 Jun. Available from: .
  • 36. Mitglieder | Medizinischer Fakultätentag. Medizinischer Fakultätentag [Internet]. [cited 2020 Aug 31]. Available from: .
  • 37. World Health Organization (WHO). Glossary. World Health Organization [Internet]. [cited 2021 Dec 21]. Available from: .
  • 38. Chang W, Cheng J, Allaire J, Sievert C, Schloerke B, Xie Y, et al. shiny: Web Application Framework for R. 2021. Available from: .
  • 40. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2021. Available from: .
  • 43. Deutsche Forschungsgemeinschaft. Klinische Studien: Stellungnahme der Arbeitsgruppe „Klinische Studien”der DFG-Senatskommission für Grundsatzfragen in der Klinischen Forschung. 2018. Available from: .
  • 44. BMBF. Grundsätze und Verantwortlichkeiten bei der Durchführung klinischer Studien. 2019. Available from: .
  • 46. Berndt C, Grill M. In Deutschland erforscht, im Nirwana versunken. Sü [Internet]. [cited 2022 Jul 14]. Available from: .
  • 47. Salholz-Hillel M, Franzen D, Müller-Ohlraun S, Strech D. Protocol: Clinical trial “report cards” to improve transparency at Charité and BIH: Survey and Intervention. OSF. [cited 2022 Jul 14]. Available from: .
  • 49. Wieschowski S, Strech D, Franzen D, Salholz-Hillel M, Carlisle BG, Malički M, et al. CONTRAST–CalifOrNia TRiAlS Transparency. OSF. 2022 May. Available from: .

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • My Account Login
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Open access
  • Published: 12 February 2024

Markers of imminent myocardial infarction

  • Stefan Gustafsson 1   na1 ,
  • Erik Lampa 1   na1 ,
  • Karin Jensevik Eriksson   ORCID: 2 ,
  • Adam S. Butterworth   ORCID: 3 , 4 , 5 , 6 ,
  • Sölve Elmståhl 7 ,
  • Gunnar Engström 7 ,
  • Kristian Hveem 8 , 9 ,
  • Mattias Johansson 10 ,
  • Arnulf Langhammer 8 , 11 ,
  • Lars Lind 1 ,
  • Kristi Läll 12 ,
  • Giovanna Masala 13 ,
  • Andres Metspalu 12 ,
  • Conchi Moreno-Iribas 14 , 15 ,
  • Peter M. Nilsson 7 ,
  • Markus Perola 16 ,
  • Birgit Simell 16 ,
  • Hemmo Sipsma 17 ,
  • Bjørn Olav Åsvold 8 , 9 , 18 ,
  • Erik Ingelsson 1 ,
  • Ulf Hammar 1 , 19 ,
  • Andrea Ganna 20 , 21 , 22 ,
  • Bodil Svennblad 2 , 23 ,
  • Tove Fall 1 , 19 &
  • Johan Sundström   ORCID: 1 , 24  

Nature Cardiovascular Research volume  3 ,  pages 130–139 ( 2024 ) Cite this article

6641 Accesses

1 Citations

424 Altmetric

Metrics details

  • Myocardial infarction
  • Prognostic markers

Myocardial infarction is a leading cause of death globally but is notoriously difficult to predict. We aimed to identify biomarkers of an imminent first myocardial infarction and design relevant prediction models. Here, we constructed a new case–cohort consortium of 2,018 persons without prior cardiovascular disease from six European cohorts, among whom 420 developed a first myocardial infarction within 6 months after the baseline blood draw. We analyzed 817 proteins and 1,025 metabolites in biobanked blood and 16 clinical variables. Forty-eight proteins, 43 metabolites, age, sex and systolic blood pressure were associated with the risk of an imminent first myocardial infarction. Brain natriuretic peptide was most consistently associated with the risk of imminent myocardial infarction. Using clinically readily available variables, we devised a prediction model for an imminent first myocardial infarction for clinical use in the general population, with good discriminatory performance and potential for motivating primary prevention efforts.

Despite declining age-standardized rates, myocardial infarction remains the leading and increasing cause of death globally 1 . Prevention of myocardial infarction is highly prioritized 2 , but the targeting of primary preventive efforts is hampered by inefficient means of identifying individuals at the highest risk for an imminent myocardial infarction (IMI). This could be partially explained by the inability of most risk prediction models to account for the highly dynamic nature of the period leading up to a myocardial infarction. For instance, traumatic events, such as a cancer diagnosis or loss of a spouse, markedly increase the risk of myocardial infarction 3 , 4 . In addition, the degree of stenosis in the culprit lesion in the coronary artery appears to increase in the months just before the myocardial infarction 5 . Nonetheless, to date, most biomarkers have been investigated over several years of follow-up because of a low number of individuals with a first myocardial infarction shortly after baseline in the general population. Hence, a large population-based study focusing on identifying biomarkers of an IMI is needed.

Primary prevention for asymptomatic risk factors over a long period is costly, and motivation among patients and providers is limited even for secondary prevention 6 . Risk prediction in the short term based on biomarkers of IMI might tilt the scales for prevention, as the knowledge of an increased risk of a first myocardial infarction within the ensuing few months might motivate patients and doctors to consider preventive strategies.

We hypothesized that circulating biomarkers of the dynamic biological processes that operate in the months preceding a myocardial infarction could be measured and used to assess risk. We tested this in a new nested case–cohort study and devised a prediction model for an imminent first myocardial infarction.

We assembled a new nested case–cohort study, the Markers of Imminent Myocardial Infarction (MIMI) study. The study includes initially cardiovascular disease-free individuals in six European general population-based cohorts who developed a myocardial infarction within the first 6 months after the baseline examination, with up to four cohort representatives per case (Fig. 1 and Supplementary Table 1 ). The case–cohort design allows for time-to-event analyses and derivation of accurate prediction models; it is also less prone to certain biases than the case–control design 7 . After exclusions, data of 2,018 individuals weighted to represent the full cohort of 169,053 persons were available for analysis (420 IMI cases and 1,598 subcohort representatives). Their characteristics at baseline are shown in Extended Data Table 1 .

figure 1

The distribution of MIMI participants across Europe is shown, with the participating countries and cohort centers indicated. Cases ( n  = 420) were initially sampled, and center-specific strata based on sex and median age were constructed. From each cohort center, up to four subcohort representatives were drawn for each case from the same stratum. A subcohort ( n  = 1,598) weighted to represent the total cohort ( N  = 169,053) based on the number of individuals in the age and sex strata in the total cohort was thus assembled. NA, not applicable.

Thereafter, we determined the levels of 817 proteins (some duplicates) and 1,025 metabolites in biobanked plasma samples from the cohort baseline examinations in a core laboratory and harmonized 16 clinical variables between the cohorts. We divided the study sample into a discovery sample (EpiHealth, Trøndelag Health Study (HUNT) and Lifelines; 70% of the sample) and an external validation sample (European Prospective Investigation into Cancer and Nutrition—Cardiovascular Disease (EPIC-CVD), Estonian Biobank study and Malmö Preventive Project (MFM); 30% of the sample). Considering the limited sample size of the study, we also performed an internal validation as an exploratory analysis by randomly splitting the study sample into a 70:30 discovery/validation sample, repeated in 100 random draws.

We investigated the associations of proteins, metabolites and clinical variables with the risk of a first myocardial infarction within 6 months after baseline using weighted, stratified Cox proportional-hazards regression models in the discovery sample. Biomarkers that passed multiple testing bounds (a Benjamini–Hochberg false discovery rate (FDR) of <0.05) were verified in the same models in the validation sample (this was done in the external and internal validation sets), with directionally consistent results at P  < 0.05 considered replicated.

In one-by-one models adjusting for technical covariates (season, storage time and plate; Fig. 2 ), 48 proteins, 43 metabolites and 3 clinical variables (age, sex and systolic blood pressure) were found to be associated with IMI after the discovery–validation process (Fig. 3 and Supplementary Table 2 ).

figure 2

The associations of 817 proteins, 1,025 metabolites and 16 clinical variables with the risk of a first myocardial infarction within 6 months in the full MIMI study, adjusted for technical covariates, are shown by biomarker category (clinical, metabolite or protein). HR relates to a doubling of the concentration of proteins and metabolites and a one-unit higher level of clinical biomarkers on their original scale (for example, years, mmol l −1 ). The top 25 biomarkers that passed external validation and ranked on how many internal validation splits the biomarker passed the replication criteria in the model adjusted for technical covariates in addition to the external validation are highlighted. a IL-6 and b KIM1 were measured on multiple Olink panels and tested in separate statistical tests. n  = 420 cases and 1,598 noncases.

figure 3

The top 25 biomarkers that passed external validation and ranked on how many internal validation splits the biomarker passed the replication criteria in the model adjusted for technical covariates in addition to the external validation are shown. Each predictor is represented by two rows, with the discovery result (blue) presented first and the validation result presented second (red). The results are sorted by predictor type (clinical, metabolite or protein) and effect size from the combined analysis of the discovery and validation samples. P value was calculated based on a 2 d.f. Wald test for metabolites analyzed using the missing indicator method (biomarker and missing indicator) and a 1 d.f. Wald test otherwise (biomarker only), two-sided in both cases. The 95% CI of the point estimate (log(HR)) was calculated for the biomarker only and might include 1 even if P  < 0.05 from the 2 d.f. (biomarker + indicator) Wald test. a IL-6 and b KIM1 values were determined from multiple Olink panels and tested in separate statistical tests. n  = 296 cases and 1,121 noncases in the discovery sample; n  = 124 cases and 477 noncases in the validation sample.

Thereafter, we investigated promising markers in models further adjusting for age and sex. Among them, brain natriuretic peptide (BNP) was the only biomarker with a borderline significant association with IMI (HR per doubling of BNP level (95% confidence interval (95% CI)) = 1.33 (1.15, 1.55), P  = 1.63 × 10 −4 , FDR = 0.11 in the discovery sample and 1.40 (1.00, 1.94), P  = 0.049 in the validation sample; Extended Data Fig. 2 ). BNP was the only biomarker with a suggestive association in the internal validation, passing the formal replication criteria in 22 of 100 random splits. By comparison, stem cell factor (SCF) and interleukin-6 (IL-6), biomarkers with a weaker support of an association, replicated in only 5 or 4 of 100 random splits. The cumulative hazard of IMI by fourths of BNP is shown in Extended Data Fig. 3 . The associations of BNP with IMI in sensitivity analyses excluding one cohort at a time and in a random-effects meta-analysis were similar, as shown in Extended Data Figs. 4 and 5 . For some of the 94 variables, we observed substantial between-cohort heterogeneity in the estimates when they were evaluated in a random-effects meta-analysis (Supplementary Table 3 ). The addition of interaction terms between sex and the biomarkers did not reveal any additional associations. Associations with IMI within 3 months (185 cases) were similar to those within 6 months (Extended Data Fig. 6 ).

In a model investigating the total effect of the BNP–IMI association (with a priori selected confounders, not mediators, according to Extended Data Fig. 1 ), adjusting for age, sex, weight, height, creatinine and systolic blood pressure, the association of BNP with IMI remained similar (HR (95% CI) = 1.34 (1.14, 1.57), P  = 3.12 × 10 −4 in the discovery sample and 1.51 (1.05, 2.18), P  = 0.028 in the validation sample; per doubling of BNP level).

We then investigated the association of the most promising marker, BNP, with the coronary artery calcium score (CACS) at a cardiac computer tomography examination in an external population-based cohort of 1,586 participants of the Swedish CArdioPulmonary bioImage Study (SCAPIS) who were free from self-reported cardiovascular disease. Here, a higher CACS was not notably associated with a higher BNP level (odds ratio (95% CI) = 1.14 (0.91, 1.42), P  = 0.25; per doubling of BNP level) in an ordinal regression model adjusting for the same covariates as in the total-effects model.

Finally, we investigated the possibility of developing a clinical risk prediction algorithm for a first IMI using clinically available variables and a weighted Cox ridge regression model. The prediction model achieved an internally validated C-index of 0.78, indicating a good ability to discriminate between IMI cases and noncases. When validating the model in the UK Biobank, a C-index of 0.82 was obtained, while a calibration plot showed some overestimation of 6-month IMI risks. As a comparison, the recalibrated SCORE2 achieved C-indexes of 0.77 (MIMI cohort) and 0.81 (UK Biobank) and overestimated the IMI risks in both samples (Extended Data Fig. 7 ). A nomogram based on the model is shown in Fig. 4 , with a worked example of its intended use displayed in Extended Data Fig. 8 and its cross-validated calibration presented in Extended Data Fig. 7 . An interactive web application is presented at . Coefficients for predicting IMI from the model are shown in Supplementary Table 4 .

figure 4

A nomogram for predicting IMI risk based on the final clinical model is shown. Each variable value contributes points (ruler at the top) that are summed up and translated to the predicted risk of a myocardial infarction within 6 months (bottom two rulers). Equation, β coefficients, 6-month survival and mean variable values are provided in Supplementary Table 4 . A worked example is shown in Extended Data Fig. 8 . The model is also presented as an interactive web application at

No biomarkers improved risk prediction in a LASSO (least absolute shrinkage and selection operator) Cox regression model; the variable selection by the LASSO was unstable, with the 95% bootstrap CI on the model size being 0–128 variables. No biomarkers improved risk prediction in a random forest model using 2,000 trees; it also ranked BNP and N-terminal pro-BNP (NT-proBNP) at the top but with very large CIs (Supplementary Table 5 ).

We here set out to identify and test biomarkers and the predictability of an imminent first myocardial infarction using a new case–cohort consortium of individuals without prior cardiovascular disease and with biobanked blood samples. From more than 1,800 biomarkers, we identified 48 proteins, 43 metabolites and 3 clinical variables associated with the risk of an imminent first myocardial infarction independent of technical covariates. Further analyses revealed BNP as the only biomarker consistently associated with IMI risk. We also derived a prediction model to discriminate between subsequent cases and noncases. The IMI phenotype has rarely been studied prospectively in the general population and with a broad panel of biomarkers. The findings may have implications for both clinical primary prevention studies and further etiological studies.

In the current study, higher BNP levels in individuals without a known cardiovascular disease were linked to a higher risk of a first myocardial infarction within 6 months in several models. Cardiomyocytes produce BNP in response to strain 8 , and NT-proBNP measurement is a pillar of the clinical management of heart failure 9 but is not used in diagnosing myocardial infarction 10 . Diastolic dysfunction is an early feature of myocardial ischemia, and a higher BNP level in this context is likely underpinned by diastolic dysfunction caused by subclinical ischemia 11 in individuals with some degree of coronary stenosis. This is supported by the weak association of BNP and CACS observed herein, although the association should be interpreted carefully. The noncausal explanation is further supported by the noncausality suggested by Mendelian randomization studies (acknowledging that associations of genetically determined lifelong BNP levels with coronary disease may have limited relevance to a temporally boxed-in series of events): a genetic variant affecting the expression of the BNP gene ( NPPB , rs198389) is not associated with cardiovascular endpoints 12 or coronary artery disease 13 . The influence of chance on the finding is low, as NT-proBNP was also significantly associated with IMI in the discovery sample, with a borderline association in the validation sample (Extended Data Fig. 5 ). While BNP may hence reflect an underlying coronary artery disease, it did not add materially to a risk prediction model for IMI composed of more readily available biomarkers.

Several known mechanisms implicated in atherosclerosis and ischemia were represented among the other 94 biomarkers associated with an IMI in both the discovery and validation samples after adjusting for technical covariates, including inflammation (IL-6) 14 , extracellular matrix metabolism (WAP four-disulfide core domain protein 2 (WFDC2)) 15 , hypertrophy (adhesion G-protein-coupled receptor G1 (AGRG1)) 16 , apoptosis (triggering receptor expressed on myeloid cells 1 (TREM1), tumor necrosis factor receptor superfamily member 10B (TRAIL-R2)) and cell adhesion (AGRG1). We also observed associations with markers representing mechanisms less often implicated in coronary diseases, such as markers of kidney injury (kidney injury molecule 1 (KIM1)) 17 , appetite regulation (growth differentiation factor 15 (GDF15)) 18 , and an α-amino acid found in dietary supplements and associated with paracetamol use (pyroglutamine) 19 . While some associations may be causal, others, such as associations with levels of chitinase-3-like protein 1 (CHI3L1) 20 , pleiotrophin (PTN) or KIT, may more likely be responses to myocardial ischemia. These findings may accelerate further etiological studies of acute coronary events.

We here developed a prediction model for IMI in the general population. An imminent infarction is difficult to predict; the signals are weak, and we faced power limitations. The model achieved good discriminative ability, with acceptable calibration in the lower risk range. It is possible to transpose to other settings by entering the base hazards and variable means of those settings, for example, interactively at . Given the increasing global burden of deaths from myocardial infarction, the importance of predicting them and increasing the individual motivation for preventing such deaths may be substantial; this can be tested in clinical trials.

The current study has several limitations. First, the use of multiple cohorts introduced heterogeneity. We addressed this at the sampling, biomarker analysis and statistical analysis stages, with the resulting limitation that the heterogeneity decreases statistical power. The strengths are the same as in other multicenter studies, including that only biomarkers with consistent importance in different settings are brought forward. Other study limitations are inherent to the uncertainty of ranking the top findings and the inability of one-by-one strategies to capture complex interrelationships. The instability of the variable importances from the random forest was unsurprising, as such methods are notoriously data hungry and require far larger datasets than classical modeling techniques 21 . While the studied markers are easily obtainable by a simple blood test or clinical assessment, a limitation is that a blood sample will not always capture tissue-specific processes. In addition, our study was limited to proteins and metabolites that remain stable in the freezer for many years. The biomarker analyses used herein are currently not available in clinical practice, and we lacked the clinically available and more precise immunoassay measurements of, for example, NT-proBNP and cardiac troponin; hence, imprecision in the proximity extension assay and ultra-high performance liquid chromatography–mass spectrometry (UPLC–MS) technologies may preclude definitive mechanistic insights and maximal clinical utility. Further, making causal assumptions is fundamentally challenging in a multimarker landscape where many causal pathways are unknown. Most markers could be potential mediators in pathways for known causes of myocardial infarction, including age and sex. Consequently, we provided models adjusted for technical covariates only and models with further biological covariate adjustment. Thus, some associations could be explained by confounding by, for example, age and sex. Notably, mediators of causal effects are also important to identify, with implications for prediction and use as treatment targets.

In conclusion, we identified biomarkers associated with the risk of an imminent first myocardial infarction, including BNP. Delineation of the distinct biological processes that operate in the months before the first myocardial infarction will be key to discovering prevention targets. We developed and validated a prediction model with a fair ability to discriminate between persons with and without risk of an imminent first myocardial infarction. Risk prediction in the short term may enhance the motivation of patients and doctors for primary prevention.

Study sample and outcome

The MIMI study sample draws biobanked blood and data from six European cohorts of the BBMRI-LPC (Biobanking and Biomolecular Research Infrastructure—Large Prospective Cohorts) collaboration 22 , as shown in Fig. 1 and Supplementary Table 1 . After sample size determination, we supplied each cohort with a standardized protocol (in which all definitions are described in detail) and an R script for selecting cohort representatives for the subcohort ( Supplementary Notes ).

Cohort participants with biobanked samples (at least 250 μl of plasma or serum; eventually, only plasma was included) and no previous clinical cardiovascular disease were eligible for inclusion in the present study. The exclusion criteria were previous clinical cardiovascular disease (defined as the presence at any time before baseline of any of the following: myocardial infarction, coronary procedure, heart failure, structural heart disease, tachyarrhythmias, stroke, thromboembolic disease and peripheral vascular disease) and renal failure.

Individuals with acute myocardial infarction (International Classification of Diseases, tenth revision (ICD-10), I21; ICD-9, 410.0–410.6 and 410.8) as the primary cause of hospitalization or death within 6 months after baseline were defined as IMI cases. We included both ST-elevation and non-ST-elevation myocardial infarctions; we encouraged efforts to include only type 1 myocardial infarctions by not counting cases with any of the following ICD codes in secondary positions: anemia (for example, ICD-10, D50–D64; ICD-9, 280–285), tachyarrhythmias (for example, ICD-10, I47–I49; ICD-9, 427), heart failure (for example, ICD-10, I50; ICD-9, 428), renal failure (for example, ICD-10, N17–N19; ICD-9, 584–586), chronic obstructive pulmonary disease (for example, ICD-10, J43–J44; ICD-9, 491, 492 and 496), sepsis and other severe infections (for example, ICD-10, A40–A41; ICD-9, 038), or hypertensive crises.

Up to four cohort representatives per available IMI case were randomly drawn from the full cohort to the subcohort in 50 strata based on sex, age (above/below median) and study center in a stratified case–cohort design 7 . All 420 IMI cases, and 1,598 subcohort representatives, were drawn from the full cohort of 169,053 participants, as summarized in Fig. 1 .

Clinical variables (age, sex, height, weight, waist circumference, systolic and diastolic blood pressure, triglycerides, high-density lipoprotein (HDL) cholesterol, non-HDL cholesterol, low-density lipoprotein (LDL) cholesterol, total cholesterol, glucose, diabetes status, highest education, smoking status, previous smoking exposure, alcohol intake and physical activity) were harmonized between the cohorts ( Supplementary Notes ). Non-HDL cholesterol was calculated as total cholesterol − HDL cholesterol. LDL levels were calculated using the extended Martin–Hopkins equation 23 .

All blood samples were randomized into appropriate measurement plates, stratified by cohort (with a similar number from each cohort on every plate), and aliquoted into the plates. Quality controls are summarized below and described in detail in the Supplementary Notes .

Protein measurements were done using the Olink proximity extension assay (Olink), a highly specific 92-plex immunoassay. Overall, 829 proteins across nine panels (cardiometabolic, cardiovascular II, cardiovascular III, development, immune response, inflammation, metabolism, oncology II and organ damage) were analyzed, including 804 unique proteins (considering overlap between panels). Relative protein values on a log 2 scale are reported, with each protein value normalized by plate by centering all plates at the same median, assuming random plate placement. Values below the assay’s lower limit of detection (LOD) were also included in the analyses.

Metabolites were analyzed using the UPLC–tandem MS (UPLC–MS/MS)-based Metabolon platform (Metabolon) by four different methods: reversed-phase UPLC–MS/MS with positive-mode electrospray ionization (early and late phase), reversed-phase UPLC–MS/MS with negative-mode electrospray ionization, and hydrophilic interaction LC/UPLC–MS/MS with negative-mode electrospray ionization. Overall, 1,135 metabolites were captured, including 925 with known identity and 210 with unknown identity. Relative metabolite levels were determined and normalized by analysis day. Metabolite levels were log 2 transformed, and nondetectable levels (<LOD or metabolite not present in the sample) were constant value imputed to a value below the minimum metabolite value (minimum/sqrt(2)).

Samples that did not satisfy the quality control criteria were initially excluded; exclusion filters were applied separately for the proteomics and metabolomics analyses, and only samples passing quality control for both analyses were included in the analysis set. For the proteomics analysis, samples with more than 50% of panels failing for technical reasons were excluded ( n excluded = 33). For the metabolomics analysis, samples were excluded because of low volume or detection of fewer metabolites than expected ( n excluded = 4). Consequently, samples for 420 cases and 1,598 subcohort representatives remained for analysis.

Next, biomarkers with an extremely high proportion of nondetectable or below-LOD measurements were excluded, with the same exclusion filters for proteins and metabolites. Biomarkers had to be detected in all six cohorts with at least 30 detectable values across all cohorts (~1.5% of the MIMI samples) or were otherwise excluded. Consequently, 817 proteins (some duplicates) and 1,025 metabolites were retained for analysis.

Statistical analysis

All analyses were done using R (version 4.1.1) 24 with the glmnet 25 , mice 26 , rms 27 , ranger 28 and survival 29 add-on packages.

One-by-one etiological analyses

In the discovery sample, the associations of all clinical variables (listed in Extended Data Table 1 ), proteins and metabolites with IMI were analyzed in separate weighted, stratified Cox proportional-hazards regression models adjusting for covariates, as described below. Inverse sampling probability weights (Borgan II) were applied to account for the case–cohort design in a stratified model, allowing for a different shape of the baseline hazard for each MIMI cohort (six levels) and using a robust variance estimator (Huber–White). Nonlinear relationships between continuous covariates (not including the biomarkers) and IMI were modeled using restricted cubic splines, and all factor variables were considered unordered.

Associations with an FDR (Benjamini–Hochberg) of <0.05 were taken forward to the validation sample, in which directionally consistent results with P  < 0.05 were considered replicated.

Cox proportional-hazards models adjusting for technical covariates (season, storage time and plate) were initially applied. Replicating biomarkers from the model adjusting for technical covariates were investigated in a model further adjusting for age and sex. A model allowing for an interaction between the biomarker and sex was further tested. Replicating biomarkers in the model adjusted for age and sex were then subjected to causal assumptions (Extended Data Fig. 1 ), and a bias-minimized model for each biomarker was investigated, estimating the total effects (including the effects of mediators).

Missingness and sensitivity analyses

Clinical variables with high missingness (previous smoking exposure, alcohol intake and physical activity) were not used in the analyses. Protein values below the LOD were included in the analyses; nondetectable metabolite levels were replaced with a constant value, and a missing indicator was added, as described below. The remaining missing values in the covariates were multiple imputed ( n imputations = 20) using chained equations including the outcome, clinical covariates and other variables correlated with the variable in the imputation model 30 . Regression results across imputed datasets were combined using Rubin’s rules 31 .

Interactions with sex were investigated by analyzing an interaction term for sex and each biomarker in models adjusting for technical covariates, age and sex. The interaction terms and all terms including the biomarker were tested using a multivariable chi-squared test with the same multiple-testing correction described above, requiring directionally consistent discovery and validation results.

The following secondary sensitivity analyses were included: random-effects inverse variance-weighted meta-analyses (DerSimonian–Laird) combining per-cohort results, leave-one-out analyses investigating the influence of single cohorts, complete-case analyses not imputing missing values in the clinical covariates, and analyses limiting the follow-up time to 3 months.

Simultaneous modeling and development of a prediction model

To attempt predicting this phenotype, we developed a prediction model for IMI using age, sex, anthropometric variables (height, weight and waist circumference), variables routinely collected in the laboratory (LDL cholesterol, HDL cholesterol, creatinine, glucose and triglycerides), systolic and diastolic blood pressure, smoking status (never, former or current) and education level. Regression coefficients were estimated using a weighted Cox ridge regression model, which shrinks coefficients toward zero using an L 2 penalty to accommodate overfitting. The strength of the penalty (lambda) was determined using tenfold cross-validation over a grid of 250 lambda values, repeated 100 times. The lambda selection was repeated in each imputed dataset, and the coefficients associated with the lambda giving the lowest cross-validated deviance were extracted. The final coefficient set was obtained by taking the median of the coefficients from each imputed dataset. A single-imputed dataset was used for validation and calibration. The C-index, which indicates a model’s ability to rank the risks, was determined using 100 repeats of tenfold cross-validation. A calibration curve was constructed using 100 repeats of tenfold cross-validation 32 . All modeling steps were repeated in each fold to assess the calibration accuracy objectively. The model containing only clinical variables was then reduced by approximating the linear predictor from the full model through stepwise regression. Predictions from the full model were used as the outcome in a linear model wherein variables were dropped sequentially until R 2  > 0.95. This yielded a highly parsimonious final model incorporating the main drivers of predictions. The prediction model was compared to SCORE2, a validated prediction model for the 10-year risk of cardiovascular disease developed using multiple European cohorts 33 . The 10-year survival probability and the covariate mean values used in the SCORE2 equations were replaced with the estimated 6-month survival probability and mean values from the current data to calculate the SCORE2-estimated 6-month cardiovascular disease risk 34 . Two additional external validations of the model were performed in the UK Biobank. First, all coefficients and covariate mean values in Supplementary Table 4 were used to validate the model. Second, the model was recalibrated using mean values and the estimated baseline risk from the UK Biobank cohort before validation.

To evaluate whether any biomarkers added to the clinical prediction model improve risk prediction, we used the linear predictor from the prediction model as an offset in a LASSO Cox regression model. Before model fitting, all proteins and metabolites were adjusted for technical variables. Briefly, each biomarker was used as the outcome variable in a regression model with all technical variables as covariates. The residuals from these models were used in place of the original biomarker values in the LASSO model. The LASSO model fitting was bootstrapped 250 times to investigate the stability of the variable selection.

As the biomarkers may have nonlinear associations with the outcome and interact with one another, and prior knowledge about nonlinearities and interactions among these variables is scarce, a random forest with 2,000 trees was fitted to the data as an exploratory analysis. Briefly, the random forest fits survival trees to bootstrap data samples using a random subset of the variables in each tree, handling interactions and nonlinearities naturally. A variable importance measure is associated with each variable and calculated based on the number of splits in which a variable is involved. The random forest was bootstrapped 250 times to obtain CIs for the variable importance measures.

Further analysis of relevant biomarkers

The associations of proteins detected using the Olink panels cardiovascular II and cardiovascular III with the CACS were available for testing in individuals free from cardiovascular disease (self-reported myocardial infarction, angina, coronary intervention, heart failure, atrial fibrillation, stroke and peripheral artery disease) for 1,586 participants at the Malmö or Uppsala centers of SCAPIS 35 . A higher CACS reflects a higher myocardial infarction risk. Proteins replicated in the primary MIMI analysis (BNP) were tested for an association with the CACS using an ordinal regression model adjusting for age, sex, body mass index, systolic blood pressure, creatinine, center, Olink plate, analysis date and season.

This study was approved by the Uppsala Ethics Authority (Dnr 2016/197). All Estonian Biobank participants signed a broad informed consent form. The study was carried out under ethical approval 258/M-21 from the research ethics committee of the University of Tartu and data release J08 from the Estonian Biobank. The Lifelines protocol was approved by the University Medical Center Groningen medical ethical committee under number 2007/152. The study was performed in accordance with the Declaration of Helsinki. The EpiHealth study was approved by the ethics committee of Uppsala University, and all participants provided informed written consent. The MFM was approved by the previous regional research committee in Lund, Sweden (2014/643), and all participants provided informed consent. Ethical review boards of the cohorts in EPIC-CVD approved the study protocol, and all participants provided written informed consent. Participation in the HUNT study was based on informed consent, and the Data Inspectorate and the Regional Ethics Committee for Medical Research in Norway approved the study.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

Data supporting the findings of this study are provided in the article and related files. Raw data are not publicly available, as they contain sensitive personal information, but may be obtained from the original cohorts upon request, with varying processes, requirements and response times. For example, researchers can apply to use the Lifelines data used in this study; information on how to request access to Lifelines data and the conditions of use can be found at . Data accession codes for this study are described below.

Code availability

The prediction model equation is available at . Code is available at . All code used to analyze UK Biobank data is deposited at the UK Biobank repository. Questions about the analyses and code can be sent to the corresponding author.

Lozano, R. et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380 , 2095–2128 (2012).

Article   PubMed   PubMed Central   Google Scholar  

European Society of Cardiology, European Heart Network, European Commission, World Health Organization. European heart health charter. Eur. J. Prev. Cardiol . (2007).

Carey, I. M. et al. Increased risk of acute cardiovascular events after partner bereavement: a matched cohort study. JAMA Intern. Med. 174 , 598–605 (2014).

Article   PubMed   Google Scholar  

Fang, F. et al. Suicide and cardiovascular death after a cancer diagnosis. N. Engl. J. Med. 366 , 1310–1318 (2012).

Zaman, T. et al. Angiographic lesion severity and subsequent myocardial infarction. Am. J. Cardiol. 110 , 167–172 (2012).

Kotseva, K. et al. Cardiovascular prevention guidelines in daily practice: a comparison of EUROASPIRE I, II, and III surveys in eight European countries. Lancet 373 , 929–940 (2009).

Ganna, A. et al. Risk prediction measures for case–cohort and nested case–control designs: an application to cardiovascular disease. Am. J. Epidemiol. 175 , 715–724 (2012).

Liang, F. & Gardner, D. G. Mechanical strain activates BNP gene transcription through a p38/NF-κB-dependent mechanism. J. Clin. Invest. 104 , 1603–1612 (1999).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Heidenreich, P. A. et al. 2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation 145 , e895–e1032 (2022).

PubMed   Google Scholar  

Gulati, M. et al. 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR guideline for the evaluation and diagnosis of chest pain: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation 144 , e368–e454 (2021).

Sabatine, M. S. et al. Acute changes in circulating natriuretic peptide levels in relation to myocardial ischemia. J. Am. Coll. Cardiol. 44 , 1988–1995 (2004).

Article   CAS   PubMed   Google Scholar  

Johansson, A. et al. Genome-wide association and Mendelian randomization study of NT-proBNP in patients with acute coronary syndrome. Hum. Mol. Genet. 25 , 1447–1456 (2016).

Folkersen, L. et al. Genomic and drug target evaluation of 90 cardiovascular proteins in 30,931 individuals. Nat. Metab. 2 , 1135–1148 (2020).

Rai, H. et al. Association of interleukin 6 −174 G/C polymorphism with coronary artery disease and circulating IL-6 levels: a systematic review and meta-analysis. Inflamm. Res. 70 , 1075–1087 (2021).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Yamamoto, M. et al. HE4 predicts progressive fibrosis and cardiovascular events in patients with dilated cardiomyopathy. J. Am. Heart Assoc. 10 , e021069 (2021).

Zhang, Y., Si, Y., Ma, N. & Mei, J. The RNA-binding protein PCBP2 inhibits Ang II-induced hypertrophy of cardiomyocytes though promoting GPR56 mRNA degeneration. Biochem. Biophys. Res. Commun. 464 , 679–684 (2015).

McCarthy, C. P. et al. Derivation and external validation of a high-sensitivity cardiac troponin-based proteomic model to predict the presence of obstructive coronary artery disease. J. Am. Heart Assoc. 9 , e017221 (2020).

Yang, L. et al. GFRAL is the receptor for GDF15 and is required for the anti-obesity effects of the ligand. Nat. Med. 23 , 1158–1166 (2017).

Liss, D. B., Paden, M. S., Schwarz, E. S. & Mullins, M. E. What is the clinical significance of 5-oxoproline (pyroglutamic acid) in high anion gap metabolic acidosis following paracetamol (acetaminophen) exposure? Clin. Toxicol. (Phila.) 51 , 817–827 (2013).

Deng, Y. et al. Upregulated microRNA-381-5p strengthens the effect of dexmedetomidine preconditioning to protect against myocardial ischemia–reperfusion injury in mouse models by inhibiting CHI3L1. Int. Immunopharmacol. 92 , 107326 (2021).

van der Ploeg, T., Austin, P. C. & Steyerberg, E. W. Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med. Res. Methodol. 14 , 137 (2014).

Simell, B. A. et al. Transnational access to large prospective cohorts in Europe: current trends and unmet needs. N. Biotechnol. 49 , 98–103 (2019).

Sajja, A. et al. Comparison of methods to estimate low-density lipoprotein cholesterol in patients with high triglyceride levels. JAMA Netw. Open 4 , e2128817 (2021).

R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2018).

Friedman, J., Hastie, T. & Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33 , 1–22 (2010).

van Buuren, S. & Groothuis-Oudshoorn, K. mice: multivariate imputation by chained equations in R. J. Stat. Softw. 45 , 1–67 (2011).

Article   Google Scholar  

Harrell, F. E. Jr. rms: regression modeling strategies. (2020).

Wright, M. N. & Ziegler, A. ranger: a fast implementation of random forests for high dimensional data in C++ and R. J. Stat. Softw. 77 , 1–17 (2017).

Therneau, T. A package for survival analysis in R. (2022).

White, I. R., Royston, P. & Wood, A. M. Multiple imputation using chained equations: issues and guidance for practice. Stat. Med. 30 , 377–399 (2011).

Article   MathSciNet   PubMed   Google Scholar  

Rubin, D. B. Inference and missing data. Biometrika 63 , 581–592 (1976).

Article   MathSciNet   Google Scholar  

Austin, P. C., Harrell, F. E. Jr. & van Klaveren, D. Graphical calibration curves and the integrated calibration index (ICI) for survival models. Stat. Med. 39 , 2714–2742 (2020).

Article   MathSciNet   PubMed   PubMed Central   Google Scholar  

SCORE2 Working Group and ESC Cardiovascular Risk Collaboration. SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur. Heart J . 42 , 2439–2454 (2021).

D’Agostino, R. B. Sr., Grundy, S., Sullivan, L. M. & Wilson, P. CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 286 , 180–187 (2001).

Bergström, G. et al. The Swedish CArdioPulmonary BioImage Study: objectives and design. J. Intern. Med. 278 , 645–659 (2015).

Download references


Transnational access to the large European prospective cohorts was provided by the BBMRI-LPC project funded by the European Commission’s Seventh Framework Programme (grant no. 313010, J.S.).

The Estonian Biobank study was supported by the European Union through the European Regional Development Fund (project no. 2014-2020.4.01.15-0012) and by institutional research funding IUT (IUT20-60) of the Estonian Ministry of Education and Research. We thank M. Alver (Estonian Biobank) for help with phenotype data.

The Trøndelag Health Study (HUNT) is a collaboration between the HUNT Research Centre (Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, NTNU), Trøndelag County Council, Central Norway Regional Health Authority, and the Norwegian Institute of Public Health.

The Lifelines initiative was made possible by a subsidy from the Dutch Ministry of Health, Welfare and Sport; the Dutch Ministry of Economic Affairs; the University Medical Center Groningen (UMCG), Groningen University; and the provinces in the north of the Netherlands (Drenthe, Friesland, Groningen). Lifelines is a multidisciplinary, prospective, population-based cohort study examining the health and health-related behaviors of 167,729 persons living in the north of the Netherlands in a unique three-generation design. The study used a broad range of investigative procedures in assessing the biomedical, sociodemographic, behavioral, physical and psychological factors that contribute to the health and disease of the general population, with a particular focus on multimorbidity and complex genetics.

EPIC-CVD was supported by funding from the European Commission Framework Programme 7 (HEALTH-F2-2012-279233), European Research Council (268834), Novartis, UK Medical Research Council (G0800270, MR/L003120/1), British Heart Foundation (SP/09/002, RG/13/13/30194, RG/18/13/33946), and National Institute for Health and Care Research (NIHR) Cambridge Biomedical Research Centre (BRC-1215-20014). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The coordination of EPIC was financially supported by the International Agency for Research on Cancer (IARC) and the Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, which has additional infrastructure support provided by the NIHR Imperial Biomedical Research Centre (BRC). The national cohorts were supported by the Danish Cancer Society (Denmark); Ligue Contre le Cancer, Institut Gustave Roussy, Mutuelle Générale de l’Education Nationale, Institut National de la Santé et de la Recherche Médicale (INSERM) (France); German Cancer Aid, German Cancer Research Center (DKFZ), German Institute of Human Nutrition Potsdam-Rehbruecke (DIfE), Federal Ministry of Education and Research (BMBF) (Germany); Associazione Italiana per la Ricerca sul Cancro (AIRC), Compagnia di SanPaolo and National Research Council (Italy); Dutch Ministry of Public Health, Welfare and Sports (VWS), Netherlands Cancer Registry (NKR), LK Research Funds, Dutch Prevention Funds, Dutch ZON (Zorg Onderzoek Nederland), World Cancer Research Fund (WCRF), Statistics Netherlands (Netherlands); Health Research Fund (FIS)–Instituto de Salud Carlos III (ISCIII), regional governments (Andalucía, Asturias, Basque Country, Murcia and Navarra), Catalan Institute of Oncology (ICO) (Spain); Swedish Cancer Society, Swedish Research Council and county councils of Skåne and Västerbotten (Sweden); Cancer Research UK (14136 to EPIC-Norfolk, C8221/A29017 to EPIC-Oxford), Medical Research Council (1000143 to EPIC-Norfolk, MR/M012190/1 to EPIC-Oxford) (United Kingdom). We thank all EPIC participants and staff for their contribution to the study, the laboratory teams at the Medical Research Council Epidemiology Unit for sample management and at Cambridge Genomic Services for genotyping, S. Spackman for data management, and the team at the EPIC-CVD Coordinating Centre for study coordination and administration.

SCAPIS was supported by Hjärt-Lungfonden, the Knut and Alice Wallenberg Foundation, the Swedish Research Council and VINNOVA.

This research was conducted using the UK Biobank resource under application no. 52678.

Grants were received from the European Research Council (no. 801965), Swedish Research Council, VR (2019-01471) and Hjärt-Lungfonden (20190505) (T.F.).

Grants were received from AFA Försäkring (160266), the Swedish Research Council (2016-01065), Hjärt-Lungfonden (20160734) and A. Wiklöf (J.S.).

The computations were enabled by resources in projects sens2019006 and sens2020005 provided by the Swedish National Infrastructure for Computing (SNIC) at UPPMAX, partially funded by the Swedish Research Council through grant agreement no. 2018-05973.

The funding sources for the different cohorts and the sponsors of the current work did not have any part in the collection, analysis and interpretation of data or in the decision to submit the paper for publication. Where authors are identified as personnel of the International Agency for Research on Cancer/World Health Organization, the authors alone are responsible for the views expressed in this article, and they do not necessarily represent the decisions, policies or views of the International Agency for Research on Cancer/World Health Organization.

Open access funding provided by Uppsala University.

Author information

These authors contributed equally: Stefan Gustafsson, Erik Lampa.

Authors and Affiliations

Department of Medical Sciences, Uppsala University, Uppsala, Sweden

Stefan Gustafsson, Erik Lampa, Lars Lind, Erik Ingelsson, Ulf Hammar, Tove Fall & Johan Sundström

Uppsala Clinical Research Center, Uppsala University, Uppsala, Sweden

Karin Jensevik Eriksson & Bodil Svennblad

BHF Cardiovascular Epidemiology Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK

Adam S. Butterworth

BHF Centre of Research Excellence, University of Cambridge, Cambridge, UK

NIHR Blood and Transplant Research Unit in Donor Health and Genomics, University of Cambridge, Cambridge, UK

HDR UK Cambridge, Wellcome Genome Campus and University of Cambridge, Cambridge, UK

Department of Clinical Sciences, Skåne University Hospital, Lund University, Malmö, Sweden

Sölve Elmståhl, Gunnar Engström & Peter M. Nilsson

K.G. Jebsen Center for Genetic Epidemiology, Department of Public Health and Nursing, NTNU, Norwegian University of Science and Technology, Trondheim, Norway

Kristian Hveem, Arnulf Langhammer & Bjørn Olav Åsvold

HUNT Research Center, Department of Public Health and Nursing, NTNU, Norwegian University of Science and Technology, Levanger, Norway

Kristian Hveem & Bjørn Olav Åsvold

Genomic Epidemiology Branch, International Agency for Research on Cancer (IARC/WHO), Lyon, France

Mattias Johansson

Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway

Arnulf Langhammer

Estonian Genome Centre, Institute of Genomics, University of Tartu, Tartu, Estonia

Kristi Läll & Andres Metspalu

Clinical Epidemiology Unit, Institute for Cancer Research, Prevention and Clinical Network (ISPRO), Florence, Italy

Giovanna Masala

Navarra Public Health Institute, Pamplona, Spain

Conchi Moreno-Iribas

IdiSNA, Navarra Institute for Health Research, Pamplona, Spain

Finnish Institute for Health and Welfare, Helsinki, Finland

Markus Perola & Birgit Simell

Lifelines Cohort Study, Groningen, Netherlands

Hemmo Sipsma

Department of Endocrinology, Clinic of Medicine, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway

Bjørn Olav Åsvold

Science for Life Laboratory, Uppsala University, Uppsala, Sweden

Ulf Hammar & Tove Fall

Institute for Molecular Medicine Finland, University of Helsinki, Helsinki, Finland

Andrea Ganna

Analytic and Translational Genetics Unit, Massachusetts General Hospital, Boston, MA, USA

Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard, Cambridge, MA, USA

Department of Surgical Sciences, Uppsala University, Uppsala, Sweden

Bodil Svennblad

The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia

Johan Sundström

You can also search for this author in PubMed   Google Scholar


J.S., S.G., E.L., E.I., U.H., A.G., B. Svennblad and T.F. were responsible for the study conception and design. S.G., E.L. and K.J.E. performed the statistical analyses. J.S. wrote the final draft together with S.G. and E.L. A.S.B., S.E., G.E., K.H., M.J., A.L., L.L., K.L., G.M., A.M., C.M.-I., P.M.N., M.P., B. Simell, H.S. and B.O.Å. were responsible for data acquisition for the different cohorts. All authors contributed to manuscript preparation and provided critical comments on the final manuscript. All authors had full access to all study data and had final responsibility for the decision to submit the paper for publication.

Corresponding author

Correspondence to Johan Sundström .

Ethics declarations

Competing interests.

The authors declare the following competing interests: A.S.B. reports grants outside this work (from AstraZeneca, Bayer, Biogen, BioMarin, Bioverativ, Novartis and Sanofi) and personal fees from Novartis. E.I. is now an employee at GlaxoSmithKline. S.G. is an employee of Sence Research AB. J.S. reports stock ownership in Anagram kommunikation AB and Symptoms Europe AB outside the submitted work. All other authors declare no competing interests.

Peer review

Peer review information.

Nature Cardiovascular Research thanks Ji Chen, Nicholas Mills and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 causal assumptions for developing bias- minimized models..

Directional acyclic graph (DAG) showing the best-guess relationship between the exposure (BNP) and the outcome (IMI) and other factors influencing this relationship together with the expected direction.

Extended Data Fig. 2 Variables associated with risk of an imminent myocardial infarction, further adjusted for age and sex.

Scatter plot comparing the point estimate (log[HR]) and corresponding 95% confidence interval from models adjusting for technical covariates (x-axis) and models additionally adjusting for age and sex (y-axis) in the full MIMI study. The 95% C.I. is calculated for the biomarker only whereas the p-value in the tables is based on the 2 d.f. Wald test (two-sided) of biomarker and missing indicator (when used), hence the 95% C.I. might overlap with null for a p-value < 0.05. N=420 cases and 1598 non-cases.

Extended Data Fig. 3 Association of BNP with Risk of an Imminent Myocardial Infarction.

Kaplan–Meier graph of unadjusted cumulative hazard of an imminent myocardial infarction (IMI) by fourths (Q) of brain natriuretic peptide (BNP), weighted by sampling weights.

Extended Data Fig. 4 Leave-one-out analyses of the association of BNP with imminent myocardial infarction.

Model with BNP and technical covariates in leave-one-out analyses where one cohort was omitted at time.

Extended Data Fig. 5 Associations of BNP and NT-proBNP with imminent myocardial infarction in the individual cohorts.

Forest plot of the regression results in the model adjusting for technical covariates performed per cohort for all available BNP measurements (BNP and N-terminal pro form). Hazard ratio and corresponding 95% confidence interval is presented. NT-proBNP is measured on both the CVD2 (*) and Metabolism panel (**). Individual cohort sample sizes are given in Supplementary Table 1 .

Extended Data Fig. 6 Comparison of associations of biomarkers with risk of IMI within 3 vs 6 months.

Regression estimates from models with time to IMI within 3 vs 6 months as outcomes. MI, myocardial infarction.

Extended Data Fig. 7 Calibration of the prediction model.

a Internal calibration. Cross-validated calibration curves for predicted probabilities of an imminent myocardial infarction from the MIMI model (solid black line) and the SCORE2 model (dashed black line). b External calibration. Calibration curves for predicted probabilities of an imminent myocardial infarction from the original MIMI model (solid black line), the MIMI model recalibrated to the UKBB data (dotted black line), and the original SCORE2 model (dashed black line). The diagonal gray line in both panels is the line of ideal calibration where predicted probabilities match the observed fraction experiencing the event.

Extended Data Fig. 8

Worked example of nomogram use. A 78-year-old (73 points) smoking (13 points) low-educated (10 points) man (23 points) with diabetes (8 points), height 1.71 m (28 points), waist circumference 110 cm (22 points), LDL cholesterol 4.5 mmol/L (21 points) and HDL cholesterol 1.2 mmol/L (39 points) will have a total score of 73 + 13 + 10 + 23 + 8 + 28 + 22 + 21 + 39 = 237 points, corresponding to a 6- month risk of a first myocardial infarction of circa 1.58%. The model is also presented as an interactive web application on

Supplementary information

Supplementary information.

Supplementary Notes and References.

Reporting Summary

Supplementary tables.

Supplementary Tables 1–5.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit .

Reprints and permissions

About this article

Cite this article.

Gustafsson, S., Lampa, E., Jensevik Eriksson, K. et al. Markers of imminent myocardial infarction. Nat Cardiovasc Res 3 , 130–139 (2024).

Download citation

Received : 28 April 2023

Accepted : 04 January 2024

Published : 12 February 2024

Issue Date : February 2024


Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Challenges and opportunities for biomarker discovery to predict imminent myocardial infarction.

  • Marie de Bakker
  • Dorien M. Kimenai
  • Nicholas L. Mills

Nature Cardiovascular Research (2024)

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

case study in clinical trials

ONS Voice Home

  • Latest Articles
  • Clinical Practice
  • ONS Leadership
  • Get Involved

Oncology Nurses Help Patient Finds Her Music Again

  • News and Views

Informed Consent Helps Patients Understand Their Rights and Responsibilities as a Clinical Trial Participant

Informed Consent Helps Patients Understand Their Rights and Responsibilities as a Clinical Trial Participant

  • Share on Twitter
  • Share on Facebook
  • Share on Pinterest
  • Share on LinkedIn
  • Email Article
  • Print Article

A comprehensive, ongoing process that involves providing potential and enrolled research participants with adequate information about a clinical trial’s purpose, procedures, risks, and benefits, informed consent helps individuals make sure they fully understand the implications when deciding whether they want to participate. 

Informed consent is more than just obtaining a patient’s signature on a document; it’s a continuous process that involves answering participants’ questions and providing information as the study progresses or as the subject or situation requires. Nurses have a responsibility to facilitate informed consent and ensure that participants fully comprehend the trial information and their role as clinical trial participants. 

The U.S. Food and Drug Administration has an overview of the basic elements of informed consent.  

Protecting Patients’ Rights 

Informed consent respects and upholds clinical trial participants’ rights to:  

  • Receive detailed information about the trial, including its purpose, procedures, risks, and benefits, in a language that they can understand  
  • Ask questions if they do not understand something 
  • Withdraw from the trial at any time without facing any consequences 

However, researchers conducting an extensive systematic review and meta-analysis spanning studies published over three decades found that only 52.1%–75.8% of participants in clinical trials understood different components of informed consent—and those numbers didn’t budge at all in the 30-year time span. Specifically , approximately: 

  • 75% understood the nature of study, potential benefits, and that they had the freedom to withdraw at any time. 
  • 65%–70% understood the study’s purpose, potential risks, and side effects; confidentiality; availability of alternative treatment if they withdraw; and treatments being compared. 
  • 50% understood the concepts of placebo and randomization. 

As oncology nurses, we have a duty to provide comprehensive education to people with cancer. When their cancer treatment is part of a clinical trial, our patient education duties include providing clinical trial information and decision-making support throughout their participation.  

According to the Health Insurance Portability and Accountability Act (HIPAA), participants also have the right to privacy and confidentiality and access to medical care. As nurses, it is our responsibility to maintain the highest standards of our patient’s privacy throughout the clinical trial and to advocate for their well-being to ensure that they receive the care and support that they need. 

Defining Participants’ Responsibilities 

After they agree to participate in a clinical trial, patients have responsibilities such as attending scheduled visits, adhering to the clinical trial protocol, complying with their medication schedule, and reporting adverse events. Nurses should inform patients about their responsibilities during the informed consent process and emphasize the importance of meeting the requirements during their participation. 

Informed consent is an essential component of clinical trials. As nurses, we play an important role in explaining the process, providing information, and advocating for the well-being of those who choose to participate. By keeping patients well-informed and supported throughout a clinical trial, we can help them make informed decisions about their participation. 

Implications for Oncology Nurses: Case Study 

Using Jane, a newly diagnosed patient with cancer, as an example, here’s one way oncology nurses can include informed consent as an ongoing process throughout a patient’s participation in a clinical trial.  

Jane is enrolled in a phase II oncology clinical trial during her clinic visit. At this visit, the oncology nurse and physician conduct a thorough informed consent by explaining the elements included in the document. At each of Jane’s future return visits during the trial, the oncology nurse assesses gaps in Jane’s understanding of the trial, addresses her questions, any emerging safety concerns, and updates her about modifications to the trial protocol and relevant protocol findings.   

During cycle 3, Jane’s creatinine clearance level is less than 50 ml/min. According to the clinical trial protocol, that level requires a dose reduction of the investigational product. The oncology nurse ensures Jane’s comprehension about the need for a dose reduction and her willingness to continue participating in the trial. 

Six months into the trial, the data and safety monitoring board identifies a new potential side effect based on interim results analysis which warrants an update to the informed consent. The nurse collaborates with the physician to discuss the updated side effects with Jane, answers any questions she has, and assists in reconsenting her with the most updated informed consent document. During the reconsent process, the nurse conducts intermittent comprehension assessments to ensure that Jane understands the updated information. 

By diligently continuing to ensure Jane fully understands the study and her rights and responsibilities at every visit throughout the trial, the oncology nurse is providing comprehensive informed consent for her as a participant. 

  • Oncology clinical trials
  • Patient Education
  • Patient advocacy

Folgen Sie uns:

case study in clinical trials

  • Umzug in Wien | Privat
  • Firmenumzug Wien / Büroumzug
  • Verpackungsmaterial
  • Küchenmontage Wien
  • Möbelmontage Wien
  • Räumung Wien
  • Entsorgung Wien
  • Klaviertransport Wien
  • Auslandsumzug
  • Umzug Wien nach Niederösterreich
  • Umzug Wien Burgenland
  • Umzug Wien Baden
  • Umzug Wien St. Pölten
  • Umzug Wiener Neustadt
  • Umzug Wien Vorarlberg
  • Umzug Wien Salzburg
  • Umzug Wien Innsbruck – Die besten Tipps 2023
  • Umzug Wien Deutschland
  • Umzug Wien München
  • Umzug Wien Berlin

Umzugsfirma Wien: Umzugsservice, schnell & günstig

Umzug Wien UmzugHelden mobile

Umzug Anfrage

Berechnung / besichtigung.

Umzug Wien und Österreich mit UmzugHelden

Umzugsfirma Wien - Die Vorteile:

Der allumfassende umzugsservice.

Privat Umzug in Wien Umzughelden

Planen Sie bald einen Umzug? Wir unterstützen Sie gerne dabei!

Unsere garantie bei jeder übersiedlung:.

Möbelpacker Wien UmzugHelden

Zufriedene Kunden sind unsere grösste Priorität.

Beste Umzugsfirma Wien Umzughelden

Die neuesten Bewertungen  von echten Kunden:

case study in clinical trials

Unsere Firmenkunden

Küche & Co Logo - UmzugHelden Partner

Oft gestellte Fragen - FAQ

Wie lange dauert ein umzug, was kostet umzug in wien oder in österreich, klaviertransport oder pianotransport in wien, was kostet ein  umzug innerhalb österreich, neueste artikel im umzug-blog, umzug mit kindern ohne stress – so geht’s, umzug mit tieren – die wichtigsten tipps, wieviele leihkartons wien benötigt man bei einem umzug.

UmzugHelden Wien Logo

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • Current issue
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Online First
  • What can Safety Cases offer for patient safety? A multisite case study
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • Elisa Giulia Liberati 1 ,
  • Graham P Martin 1 ,
  • Guillaume Lamé 1 , 2 ,
  • Justin Waring 3 ,
  • Carolyn Tarrant 4 ,
  • Janet Willars 4 ,
  • Mary Dixon-Woods 1
  • 1 THIS Institute (The Healthcare Improvement Studies Institute), Department of Public Health and Primary Care , University of Cambridge , Cambridge , UK
  • 2 Laboratoire Genie Industriel , CentraleSupélec, Paris Saclay University , Gif-sur-Yvette , France
  • 3 Health Services Management Centre , University of Birmingham , Birmingham , UK
  • 4 Department of Population Health Sciences , University of Leicester , Leicester , UK
  • Correspondence to Dr Elisa Giulia Liberati, THIS Institute (Public Health and Primary Care), University of Cambridge, Cambridge, UK; elisa.liberati{at}

Background The Safety Case is a regulatory technique that requires organisations to demonstrate to regulators that they have systematically identified hazards in their systems and reduced risks to being as low as reasonably practicable. It is used in several high-risk sectors, but only in a very limited way in healthcare. We examined the first documented attempt to apply the Safety Case methodology to clinical pathways.

Methods Data are drawn from a mixed-methods evaluation of the Safer Clinical Systems programme. The development of a Safety Case for a defined clinical pathway was a centrepiece of the programme. We base our analysis on 143 interviews covering all aspects of the programme and on analysis of 13 Safety Cases produced by clinical teams.

Results The principles behind a proactive, systematic approach to identifying and controlling risk that could be curated in a single document were broadly welcomed by participants, but was not straightforward to deliver. Compiling Safety Cases helped teams to identify safety hazards in clinical pathways, some of which had been previously occluded. However, the work of compiling Safety Cases was demanding of scarce skill and resource. Not all problems identified through proactive methods were tractable to the efforts of front-line staff. Some persistent hazards, originating from institutional and organisational vulnerabilities, appeared also to be out of the scope of control of even the board level of organisations. A particular dilemma for organisational senior leadership was whether to prioritise fixing the risks proactively identified in Safety Cases over other pressing issues, including those that had already resulted in harm.

Conclusions The Safety Case approach was recognised by those involved in the Safer Clinical Systems programme as having potential value. However, it is also fraught with challenge, highlighting the limitations of efforts to transfer safety management practices to healthcare from other sectors.

  • Patient safety
  • Qualitative research
  • Risk management

Data availability statement

No data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: .

Statistics from

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Safety Cases are a well-established regulatory technique in some areas, requiring organisations to make the case to the relevant regulator that they have put in place adequate measures to reduce risks in their systems to a level ‘as low as reasonably practicable’ (ALARP).

Importing of safety practices from other sectors has a long track record in healthcare, but little is known about the potential of the Safety Case approach when applied to clinical pathways.


It was difficult for clinical teams to use the Safety Case as intended (to show that risks had been reduced to ALARP), not least because they often identified issues that front-line staff could not address.

Safety Cases were sometimes used instead to attract senior leaders’ attention and to make the case for better support and resourcing, but some issues were beyond the control even of organisational leadership.


Safety Cases may have some potential in healthcare, but their optimal use in this sector may require modifications, particularly if they are considered for regulatory purposes.


Patient safety remains a major challenge for healthcare, despite more than two decades of sustained policy, practice and research attention. 1 2 The initial enthusiasm for borrowing practices and methods from other safety-critical industries (such as aviation) at the outset of the patient safety movement 3–5 has been tempered by experience. 6–12 It is now widely recognised that attempts to transfer approaches between contexts require care and caution, and should be supported by theory and empirical evaluation. 13–15 This paper seeks to contribute to addressing this need through examination of an attempt to introduce into healthcare a specific safety approach—the Safety Case—that is already used in other industries (including oil, transport and mining) both as a regulatory technique, 16 and, more rarely, as a quality management approach without regulatory mandate (eg, in the automotive industry). 17 18

The specifics of the Safety Case approach vary between sectors and regulators, 19 but the general principles are listed in box 1 . In brief, a claim to operational safety is justified through a series of linked arguments that explain how safety has been secured, with supporting evidence , including the processes in place to control risk. Where used as a regulatory technique, Safety Cases are produced by organisations to ‘make the case’ to the relevant regulator that they have put in place adequate measures to reduce risks in a product or system to a level ‘as low as reasonably practicable’ (often abbreviated as ALARP). The regulator then reviews the Safety Case and either grants the organisation licence to operate, or may require further risk assessments, justification of the measures proposed or additional risk mitigations. 20

Typical features of safety cases

Safety Cases are developed to ‘make the case’ that risk has been reduced to a level ‘as low as reasonably practicable’ (ALARP). To do so, Safety Cases integrate various forms of prospective risk management analysis, based on the idea that operators are better placed than external regulators to assess risks in their own systems. The core of the Safety Case is typically a risk-based argument and corresponding evidence to demonstrate that all risks associated with a particular system have been identified, that appropriate risk controls have been put in place, and that there are appropriate processes in place to monitor the effectiveness of the risk controls and the safety performance of the system on an ongoing basis. 23

Safety cases typically contain:

A description of the system and its operational context;

How safe the system is claimed to be and the criteria by which safety is assessed;

How hazards have been identified and how the risks they pose have been assessed;

What kind of risk control measures have been put into place and why they are effective; and

Why the residual level of risk is acceptable. 23

Safety Cases are typically reviewed and assessed by an external regulator, for example, in the nuclear or petrochemical industries in the UK. However, some industrial sectors have also deployed the approach outside of a regulatory requirement. For example, the automotive industry uses Safety Cases that are part of the ISO26262 standard, but this is not mandated by regulators. 17 18

As an approach requiring organisations to proactively describe what procedures and actions they are putting in place to control risk, Safety Cases can be contrasted with prescriptive, compliance-oriented approaches, where organisations are required to show that they have met externally imposed safety standards. 21 Because they are written for a specific system and its context of use, they are intended to be more adaptable to specific situations than generic safety standards, and also more responsive to rapid change in technologies or practices. 22

On the face of it, the Safety Case would appear to have value as an approach to safety management in healthcare, particularly in its potential for prospective identification and control of risk. However, the Safety Case approach has only rarely been used in healthcare, and only in a very limited number of applications (eg, development of information systems and medical devices). 23 24 In this article, we develop an analysis of the application the Safety Case approach within the UK National Health Service (NHS) using a case study of the first documented attempt to apply the principles of the methodology to clinical pathways. As the approach was deployed outside a regulatory context, our analysis focuses on the transferability of an approach to risk management that is proactive, structured, and tailored in nature and that presents evidence about the safety of specific clinical systems and existing mitigations in a single ‘case’ document.

Case study: the Safer Clinical Systems programme

Our analysis draws on an evaluation we conducted of a programme known as Safer Clinical Systems, which is designed to improve the safety and reliability of clinical pathways based on learning adapted from a range of hazardous industries. It seeks to enable organisations to make improvements to local clinical systems and pathways through a structured methodology for identifying risks and re-engineering systems to control risk and enhance resilience. 25 26 Use of the principles of the Safety Case approach is a centrepiece of the Safer Clinical Systems programme, although outside a regulatory context.

Funded by the Health Foundation, the Safer Clinical Systems programme was developed by a team at Warwick University and tested over a number of phases. Following initial development, a ‘testing phase’ involving eight NHS hospital sites (seven in England, one in Scotland) ran from 2011 to 2014. An ‘extension phase’ (2014 to 2016) involved further work by five of these sites and one new site.

Each participating hospital site ( table 1 ) was required to establish a multidisciplinary clinical team. Sites in the testing phase were advised by a support team of clinicians and experts, received inperson training, had access to other resources (such as a reference manual and telephone support) and were required to report their progress regularly. Sites in the extension phase had less bespoke support and were expected instead to build on their previous learning.

  • View inline

Sites involved in the programme

A requirement of participating teams was that they use the Safer Clinical Systems approach to proactively assess risks and hazards in their clinical pathways and that they produce Safety Cases at the end of their projects describing the risks and how they were being mitigated. The Safety Cases were expected to be similar in format to those used in other sectors, 27 comprising a description of the clinical pathway covered, the key hazards identified through structured analysis using prescribed tools, the risk controls implemented, and, critically, a ‘safety claim’ and associated ‘confidence argument’—a pronouncement on the current safety of the system concerned, and a statement explaining how risks had been made ALARP. Rather than being presented to an external regulator, as would be the case if the Safety Case were being used as a regulatory technique, the principal intended audience in this programme was the senior leadership (executive and board level) within organisations.

Evaluation methods

To study the testing and extension phases of the Safer Clinical Systems programme, we used a mixed-methods, longitudinal design, involving interviews, ethnographic observations, and documentary analysis across the nine participating sites. The analysis we report here is based primarily on interviews and documentary analysis. Ethnographic observations (over 850 hours) provided valuable data on how clinical teams carried out their Safer Clinical Systems projects in practice in the context of existing and competing demands, but are not reported in detail here.

Across the nine sites, we conducted 89 semistructured interviews in the testing phase and 39 in the extension phase with participating clinical team members and programme leaders. Sampling at the sites sought to purposefully include a range of different roles in the programme, including the clinical leaders of each project and others. We also conducted 5 semistructured interviews in the testing phase, followed by 10 in the extension phase, with organisational senior leadership, comprising executive team/board members. Interviews explored general experiences of the programme as well as specific exploration of using the Safety Case approach. Participants were informed of the aims and commissioners of the evaluation. All interviews were conducted by experienced social scientists using topic guides ( online supplemental material 1 ). Interviews were conducted either in person or by telephone, between November 2012 and June 2016, and were digitally audio recorded and then transcribed for analysis.

Supplemental material

Analysis, conducted by EL and guided by the wider team, was based on the constant comparative method 28 combining inductive and deductive approaches. We coded interviews and observations using an inductive approach, deriving codes directly from each interview and then progressively clustering codes in higher order categories and themes. To strengthen explanatory power, this inductive strategy was complemented by theoretical concepts drawn from the wider literature.

GL and EL conducted a documentary analysis of the Safety Cases prepared by the clinical teams ( table 2 ). We used recommendations and guidelines for writing and maintaining safety cases in other sectors, 29–31 to organise the Safety Cases’ content thematically, and identified their main strengths and weaknesses in terms of completeness, presence of appropriate evidence and analyses to support the claims, consistency with the site’s safety improvement objectives, readability, and presence of a safety claim and confidence argument.

Format and content of 13 Safety Cases reviewed

Finally, we organised our higher order themes and overall reflections using concepts and themes proposed by recent works on the topic. 19 32 Regular team meetings and correspondence provided oversight of the analytical approach, consistency and adequacy of codes, and reporting. Given the nature of the programme, we did not undertake a formal test for theoretical saturation for the interviews or the Safety Cases.

Across the testing and extension phases of Safer Clinical Systems, we undertook 143 interviews with participants across programme leadership, clinical teams and organisational leadership. We analysed 13 submitted Safety Cases; although 14 should have been developed, one site from the extension phase struggled to implement the programme in full and did not produce a Safety Case.

In presenting our analysis below, we consider, first, participants’ views on the Safety Case as a novel approach to understanding and managing safety risk in healthcare, and second, the work that went into developing Safety Cases. We then turn to the analysis of Safety Cases themselves.

Views on the value of safety cases

By the end of the programme, members of the project teams and senior leadership in the participating organisations had largely come to see the Safety Case as a valuable approach, with the potential to make hazards visible in an accountable, systematic and scientific way. The analytical steps required to compile a Safety Case, such as process mapping the patient pathway, were seen to be particularly useful in proactively identifying threats to safety, rather than reactively managing incidents once they had happened. The role of Safety Cases in enabling an overarching, system-wide view of the hazards, rather than focusing on what happens in particular segments of the pathway, was also welcomed. Broadly, teams valued the possibilities of new ways of thinking about risk.

I like the idea that you just have one document that you can hand to somebody and say how safe is your system. I like the concept that you can say ‘Well this is what our system is like just now’. (Project participant)

Some organisational senior leaders agreed, at least in principle, that Safety Cases could offer value, and recognised the importance of a prospective approach to safety.

We have immensely complex systems which could be simplified and therefore made a bit more reliable. […] So something which looks at that could certainly be a useful thing, because it’s saying ‘Well actually here is a little nest of complexity which you can reduce, but it’s also a significant risk to the patient, because you’re missing information or you’re hurrying things through.’ […] (Senior leader)

Other senior leaders, however, were not always clear on the practicalities of the approach, and some found it difficult to identify the added value of Safety Cases. They suggested, for example, that existing risk management tools performed very similar functions.

If you look at our risk register, mitigation is the last box, we spend a good amount of time on the other things, but if we were to spend any time on a particular risk it would be on mitigation […]. And so that sounds like a very similar process, and so I’m back to what the delineation is between Safety Case and risk register. (Senior leader)

Some project teams saw the Safety Case as useful for a secondary reason: that of securing the attention and interest of senior leaders in their organisations. Their hope was that, by providing new evidence and analysis of the riskiness of clinical systems, senior management attention, support, and resources might be solicited.

So they’ve [senior management] actually kind of bought into it, so I think they will feel pressure to deliver. (Project participant)

However, as we explain below, the exact fit of Safety Cases into the existing ecology of tools and documents in healthcare was not clear to all participants.

Preparing safety cases

Project teams were required to learn new techniques to prepare the Safety Cases, including use of systematic methods to identify and assess risks in their clinical pathways, to propose risk controls and to identify metrics that could be used to monitor systems. Production and communication of Safety Cases also required skills in making persuasive claims, structuring arguments and presenting evidence compellingly. The participating teams were, understandably, unfamiliar with many of these skills, and expressed uncertainties about the expected structure, content and style of the Safety Case itself, especially in terms of what issues to emphasise and how to evidence them. Participants described compiling and drafting the Safety Case as labour-intensive and difficult.

I think the other bit that we have been challenged by is the actual writing of the Safety Case and again it is because it is fairly new to healthcare in general. I think we are going to go through a few reiterations before we fully understand what it is and how to use it. (Project participant)

Notwithstanding the training and support received in the ‘testing’ phase, teams continued to report difficulties with preparing and drafting Safety Cases well into the extension phase. A recurrent source of ambiguity related to the size and scope of the clinical system that the Safety Cases should target. The first, diagnostic, step in the Safer Clinical Systems process involved defining the clinical pathway of focus. However, determining the boundaries of the pathway was far from straightforward. Furthermore, clinical pathways typically involved dozens of technological systems (eg, infusion pumps, IT systems) and sociotechnical processes (eg, guidelines, multidisciplinary meetings). Each might be amenable to risk assessment and management individually, but making sense of their connections, aggregate risks and potential interactions was a much more complex task.

It’s not a linear process and you do go back trying to understand another bit of the process that you thought you understood, but actually didn't as (…) you had hoped. (Project participant)

Once the pathways and their components had been determined (or at least approximated), project teams used a range of methods recommended by the Safer Clinical Systems programme, mostly derived from similar activities in other industries, to assess hazards and risks. The teams found the processes often challenging and time-consuming, with much discussion about the relative merits of different sources of data and evidence. Despite the challenges, teams generally concluded that conducting a systematic risk assessment using structured tools offered important new insights about clinical pathways.

What I’ve loved doing is, is talking to the staff and actually understanding what goes on, because it’s only when you understand what goes on that you can put it right… You’ve worked in the hospital for years and there’s still things you didn’t realise actually went on and things that people did that you didn't realise that they actually did. That was quite an eye-opener. (Project participant)

This new understanding through structured risk assessment enabled teams to identify multiple shortcomings that had potential to harm patients. The hazards they unearthed varied greatly in scale, level of risk posed and tractability to intervention. Some problems identified were amenable to resolution by the project teams, typically those with their roots in suboptimal service planning and pathway design, failures in communication among staff, or unclear distribution of responsibility or ownership of key processes. In response to these, most, but not all, sites designed or implemented some risk controls and documented them in their Safety Cases.

[Staff are] given the freedom and the autonomy to go ahead and do whatever things they think might be necessary to make things better. And that’s what people do, there is very much a culture of promoting change there, so they talked about small cycles of change, doing PDSA [Plan Dp Study Act] cycles, and there’s a number of different projects that are running (Observation notes)

The extent to which these risk control interventions were consistent with the principles of the Safer Clinical Systems programme varied by site. Some project teams were able to draw on extensive experience, while others foundered at this stage. Common to all sites, however, was the identification of issues that were well beyond the scope of control of the front-line teams themselves. These vulnerabilities tended to originate from deep-rooted institutional and organisational pathologies or constraints. The importance of these problems, including, for example, staffing levels, was beyond doubt. Exactly what to do about them was less clear. Some project teams made valiant attempts to at least mitigate the risks through local work, but others appeared to accept that standard quality improvement efforts would not solve the issues. Some teams described the ongoing failure to mitigate the risks in their Safety Cases, in part, as noted above, in the hope that action from senior level might be provoked.

There were other things that were discussed at the [meeting] that they thought would be good as a team to change… but with some of them, they just knew it would be impossible to do so, so actually they didn't even bother to write them down. (Observation notes) And the team very bravely went to the board and said, you know, our Safety Case is showing and we're telling you that our processes are unsafe, so it alerted people to the issues. […] So that was the strength of it. (Project participant)

However, as we now describe, for senior organisational leaders, both the imperative offered by the Safety Case and their own ability to act were less clear.

Content of, and responses to, safety cases

Our documentary review showed that submitted Safety Cases were highly variable in format and length ( table 2 ). Some were highly structured, clearly written and precise in the use of evidence; others were harder to follow, lacking in clarity and less well organised. Our review also found that the descriptive elements (analysis of risk and hazards) were much better achieved than the assurance components (the safety claim and the confidence argument). Indicative, perhaps, of the intractability to local-level intervention of some of the hazards uncovered, or the lack of expert safety science input in the project teams, most Safety Cases focused more on what had been done to determine the risk than on the level of safety that had been achieved in mitigating it. The documents also varied in the extent to which they reported the residual risks—those that remained despite the implementation of risk controls—in a clear and transparent way. For instance, one Safety Case noted that the diagnostic process had found 99 ways in which the pathway could fail, that the level of reliability in the microsystem remained lower than acceptable, and that radical re-design was needed. Others were more circumspect. Accordingly, while they documented sometimes-extensive mitigations, none of the Safety Cases could make an unambiguous safety claim supported by a powerful confidence argument. Some teams were not clear about how the evidence gathered and analyses conducted would contribute to the safety claim. Some sites listed project activities in lieu of offering an actual safety claim, reporting what they had done rather than the level of safety they had reached.

It was a useful, […] a really good repository for all the stuff we've done in the project, which I find really good. And has been good when people ask ‘What did you do?’ then you can say that this is what we did, so that’s useful. I'm not sure about whether people use it for what it is meant to be, which is to prove the pathway is now safe, I’m not sure whether it is used for that really. (Project participant)

Sometimes, safety claims were reported for each identified hazard (comparing levels of risk before and after the interventions they had implemented) rather than at the level of the clinical system. No site explicitly discussed whether risks had been reduced ‘as low as reasonably practical’. Some sites claimed improvements as a result of the interventions they had implemented, but these did not always stand up to statistical scrutiny. 33

The response of senior leadership to the Safety Cases submitted by teams varied. Some focused on the potential of the Safety Case for supporting organisational-level decision making in relation to risk reduction, resource allocation and strategic prioritisation.

I think it would be easier to respond to a Safety Case rather than more so the [other quality and safety] data I get. Because it’s back to first principles, what are we actually here to do… Then if we have an unsafe system everything else needs to fall in behind that, no matter cost pressures, no matter personal opinion, no matter all the other complexities in a big system. If an element is at risk, then that will always be made a priority. (Senior leader)

Not all senior leaders, however, were so confident that the insight offered by Safety Cases would or should inevitably lead to action. Some of the issues identified in the Safety Cases were beyond the ability not only of front-line teams to solve, but also of organisational leaders. Issues such as staffing levels, IT interoperability, and securing timely discharge required at least interorganisational coordination, resourcing, coordination, and support across the whole healthcare system. Additionally, the prevailing approach to risk management, and the perceived unavoidability of risks in the complex systems of healthcare, meant that the insights offered by a Safety Case might be unwelcome or not necessarily candidates for priority attention. In a system that relied primarily on retrospective risk management approaches, such as incident reporting and investigations, the need to tackle risks of recurrence (where problems had already manifested as serious incidents or ‘near misses’, and might do again) could easily take precedence over addressing seemingly ‘theoretical’ risks (problems identified through a detailed prospective analysis but yet to occur).

Because you’re saying actually ‘That was a potential harm on our risk management system, and we knew about it, and we were accepting that we don’t have enough money to address all of these issues at one time’. So there is, if you like, a prioritisation and rationing of where we put money according to the level of risk. […] It’s a bit like county councils putting crossings on roads, or a zebra crossing. You’re waiting for the fatality to occur before actually that will get the funding. (Senior leader)

Some feared that, given the legal obligation of boards to take action in response to safety risks that were revealed to them, an unintended consequence of the Safety Case approach might be to distract organisational focus from areas that were at least as worthy of attention but lacked the spotlight offered by the Safety Case. There was a perception that to have a Safety Case for every pathway or area of practice would likely be impossible, and that too many Safety Cases would be overwhelming.

The complexity of health care is such that there are hundreds of complex connected pathways that patients are on and so… You in theory could write hundreds [of Safety Cases] and that would then become meaningless because if you write hundreds no one would ever read them. So, I think it might be helpful in some specific examples… Rather than being something that could cover everything that we do to patients. (Senior leader)

Consequently, Safety Cases might serve not to assure about control of risks, but to unnerve—and unnerve leaders who were not always well placed to act, given the scope of their control and the other priorities they faced. In a system where Safety Cases were new, without an established function in safety management, and covering only a small proportion of safety-critical activity, the information they provided was not always readily actionable from a managerial perspective and, moreover, had potential to create uncontrolled reputational risk.

The danger is that what you have is a legal requirement to spend money on a Safety Case that actually is of low, relative risk to harms that are occurring in the absence of Safety Cases. So what you get is a spurious diversion of money to a wheel that has been made very squeaky, but actually isn’t causing harm… There’s the risk of diversion to get a perfect patch in one part of the system while everything else is actually terrible. (Senior leader) (A danger) is, you know, if it does get into the wrong hands, particularly with the media, because there’s not the openness and the ability to manage some of this data, which needs explanation. But we do pride ourselves on being a very open and transparent board. (Senior leader)

Our examination of an attempt to introduce the principles and methodologies of the Safety Case approach into healthcare suggests that the approach was broadly welcomed by participants in our study, but was fraught with challenge. In other sectors, the Safety Case rests on the ALARP principle. While the Safety Cases produced by participating teams in the Safer Clinical Systems programme did present proactive analyses of risks, they did not show that the risks in clinical pathways on which they focused had been reduced as far as reasonably possible. Instead, teams identified multiple residual risks that had resisted efforts at control and mitigation by the teams themselves. These findings emphasise the importance of careful consideration of context and implementation when transferring safety management approaches from one setting to another. 12 34–36 The evidence underlying other industrial risk management techniques (eg, Failure Modes and Effects Analysis, 37 ‘5 Whys’ 10 or Root Case Analysis 11 ) is also weak, but the regulatory function of Safety Cases warrants specific caution. Sujan et al ’s review of various sectors nonetheless concluded that even with the differences in regulatory context, healthcare organisations could benefit from using the Safety Case approach to develop understanding and exposition of their current levels of risk. 19 Our study does suggest that Safety Cases show some promise as a way of structuring more responsive, adaptable and specific proactive safety management practices in healthcare settings, but further careful development and evaluation are needed, particularly if consideration is given to using them for regulatory purposes. 19

An important feature of the programme we examined—essentially a feasibility study—was that the Safety Case approach was being used outside the regulatory frameworks and infrastructures characteristic of use of the technique in most other sectors. Without an external regulatory requirement to satisfy, participating organisations in the Safer Clinical Systems programme may not have felt a strong imperative to make the responses that might otherwise be expected; absent the spectre of regulatory action, senior leadership may not have felt compelled to reduce the risks ALARP. However, even when Safety Cases are part of a regulatory framework, they are not always rigorous or successful in controlling risk 38 or showing they have been reduced ALARP. 39 While our study does not allow conclusions to be drawn about what might happen if Safety Cases were included in a regulatory regime in healthcare, it does allow insights into the nature of the challenges that might be anticipated should regulators consider introducing the approach in healthcare settings.

Some of the challenges we identified arose from the mismatch between the complexity and interdependencies of clinical pathways, with their often unbounded character, and the more tightly defined (and often more mechanical or technical) applications of the approach in other industries. 22 40 Future research might usefully clarify whether and how the scope of a Safety Case could best be defined for healthcare settings, noting that the highly dynamic and interdependent nature of multiple subsystems of care may defy attempts to impose clear boundaries. These kinds of questions are becoming increasingly prominent in safety science as recognition grows that the development of networked complex systems (eg, unmanned aircraft systems) requires a shift from relatively static prelaunch assessment to a dynamic approach that can accommodate changes in the system’s properties and behaviour during its life-cycle. 41 42

Other challenges arose in the demanding nature of the expertise, skill and time commitment required to engage in the tasks of conducting safety analyses, identifying and testing risk controls, and compiling a Safety Case. The variable quality of the Safety Cases submitted by clinical teams in this programme is likely to be linked to variable competencies and available capacity. In contrast, in safety-critical industries where these risk assessment techniques originated, the design of effective risk controls is the responsibility of safety/reliability engineers with extensive training and expertise. For healthcare, use of the Safety Case approach will require additional resource and new dedicated roles with specific expertise, rather than relying on making further demands of existing clinical teams. 40 43 The resourcing implications of a wholesale effort to shift the regulatory system and culture of an entire sector could, however, be enormous, especially given the volume and complexity of activity in healthcare and the number of diverse clinical pathways.

An additional set of challenges was more cultural in character, and related to the revelatory potential of the Safety Case. On one hand, participants—especially clinical teams—appreciated the value of the Safety Case in offering a proactive, prospective and rigorous approach to identifying safety risks. Some also saw it as a means of attracting managerial attention and obtaining resources. 44 But leaders in organisations were not always convinced that the approach offered much that was new, suggesting that more evidence would be needed to demonstrate the added value of Safety Cases—especially in moving beyond description to solution, 45 and adding value over current approaches such as risk registers. A further concern at the leadership level was that it was unclear whether areas that did have a Safety Case should be considered to have a stronger warrant for action than those that did not. A framework for supporting prioritisation of risks is likely to be helpful in any future use of Safety Cases. However, current tools, such as risk matrices, may be flawed, 46 47 so better tools should be investigated.

Even less tractable was what to do about some of the problems reported in the Safety Cases. Clinical teams had done their best to implement risk controls where they could, but they did not have sufficient power and access to resources to address those that were institutional or structural in character. They therefore often fell back on weaker administrative measures, like training or procedures. 8 Yet organisational leaders were often similarly challenged, given their limited capacity and resources for radical systems re-design, improved staffing, IT infrastructure, or other major re-engineering or influencing of activities outside the organisation itself. These findings are indicative of broader problems with the selection of risk controls in health services 44 48 that may need to be addressed before Safety Cases could achieve their potential.

Our study has a number of strengths, including its in-depth, mixed-methods, longitudinal design with engagement both with the project teams and senior leaders in organisations. It was limited in its ability to assess the impact of the Safety Case approach in improving safety, not least because of issues with data on processes and outcomes. 33


The Safety Case approach offers promise in principle as a safety management approach in healthcare, but substantial challenges need to be addressed before further deployment, particularly in regulation. Further experimentation with the use of Safety Cases in healthcare might therefore more profitably focus on how to make the most of their assets—including the new insights offered by prospective, system-wide risk analysis—while managing their potential unintended consequences.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study involves human participants and was approved by the East Midlands – Leicester Research Ethics Committee (12/EM/0228). Participants gave informed consent to participate in the study before taking part.


We thank the people from the nine sites who participated in the Safer Clinical Systems programme and the support team. We also thank colleagues on the evaluation team, including Sarah Chew, Liz Shaw, Liz Sutton, and Lisa Hallam.

  • Illingworth J ,
  • Fernandez Crespo R , et al
  • Panagioti M ,
  • Keers RN , et al
  • Dixon-Woods M ,
  • Goeschel CA , et al
  • Catchpole K ,
  • Liberati EG ,
  • Peerally MF ,
  • Dixon-Woods M
  • Waring J , et al
  • Radnor ZJ ,
  • Marshall M ,
  • Pronovost P ,
  • Auerbach AD ,
  • Landefeld CS ,
  • Shojania KG
  • Portela MC ,
  • Pronovost PJ ,
  • Woodcock T , et al
  • Cleland G ,
  • Kelly TP , et al
  • Cleland GM ,
  • Sujan M , et al
  • Spurgeon P ,
  • Flanagan H ,
  • Cooke M , et al
  • Hawkins R , et al
  • UK Ministry of Defence
  • Haddon-Cave C
  • Woodcock T ,
  • Soukup T , et al
  • Lincoln J , et al
  • Franklin BD ,
  • Rivera AJ , et al
  • Borg J , et al
  • Fairbanks RJ
  • Braithwaite J
  • Naghavi M ,
  • Foreman K , et al
  • Clarkson PJ

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

  • Data supplement 1

Twitter @graham_p_martin, @carolynctarrant

Contributors EL and GL produced the first draft of the article, subsequently revised by GM, JWa, and MD-W. EL and JWi collected the data, analysed by EL and GL. All authors contributed to data interpretation, manuscript writing and reviewing, and approved the final version. MD-W was the study Chief Investigator and study guarantor.

Funding This study was funded by the Health Foundation, charity number 286967. The Healthcare Improvement Studies (THIS) Institute is supported by the Health Foundation – an independent charity committed to bringing about better health and health care for people in the UK. The views expressed in this publication are those of the authors and not necessarily those of the Health Foundation.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles

  • Editorial Changing the patient safety mindset: can safety cases help? Mark Sujan Ibrahim Habli BMJ Quality & Safety 2023; - Published Online First: 24 Nov 2023. doi: 10.1136/bmjqs-2023-016652
  • Editorial Changing the patient safety mindset: can safety cases help? Mark Sujan Ibrahim Habli BMJ Quality & Safety 2023; 33 145-148 Published Online First: 24 Nov 2023. doi: 10.1136/bmjqs-2023-016652

Read the full text or download the PDF:

College of Nursing

Driving change: a case study of a dnp leader in residence program in a gerontological center of excellence.

View as pdf A later version of this article appeared in Nurse Leader , Volume 21, Issue 6 , December 2023 . 

The American Association of Colleges of Nursing (AACN) published the Essentials of Doctoral Education for Advanced Practice Nursing in 2004 identifying the essential curriculum needed for preparing advanced practice nurse leaders to effectively assess organizations, identify systemic issues, and facilitate organizational changes. 1 In 2021, AACN updated the curriculum by issuing The Essentials: Core Competencies for Professional Nursing Education to guide the development of competency-based education for nursing students. 1 In addition to AACN’s competency-based approach to curriculum, in 2015 the American Organization of Nurse Leaders (AONL) released Nurse Leader Core Competencies (updated in 2023) to help provide a competency based model to follow in developing nurse leaders. 2

Despite AACN and AONL competency-based curriculum and model, it is still common for nurse leaders to be promoted to management positions based solely on their work experience or exceptional clinical skills, rather than demonstration of management and leadership competencies. 3 The importance of identifying, training, and assessing executive leaders through formal leadership development programs, within supportive organizational cultures has been discussed by national leaders. As well as the need for nurturing emerging leaders through fostering interprofessional collaboration, mentorship, and continuous development of leadership skills has been identified. 4 As Doctor of Nursing Practice (DNP) nurse leaders assume executive roles within healthcare organizations, they play a vital role within complex systems. Demonstration of leadership competence and participation in formal leadership development programs has become imperative for their success. However, models of competency-based executive leadership development programs can be hard to find, particularly programs outside of health care systems.

The implementation of a DNP Leader in Residence program, such as the one designed for The Barbara and Richard Csomay Center for Gerontological Excellence, addresses many of the challenges facing new DNP leaders and ensures mastery of executive leadership competencies and readiness to practice through exposure to varied experiences and close mentoring. The Csomay Center , based at The University of Iowa, was established in 2000 as one of the five original Hartford Centers of Geriatric Nursing Excellence in the country. Later funding by the Csomay family established an endowment that supports the Center's ongoing work. The current Csomay Center strategic plan and mission aims to develop future healthcare leaders while promoting optimal aging and quality of life for older adults. The Csomay Center Director created the innovative DNP Leader in Residence program to foster the growth of future nurse leaders in non-healthcare systems. The purpose of this paper is to present a case study of the development and implementation of the Leader in Residence program, followed by suggested evaluation strategies, and discussion of future innovation of leadership opportunities in non-traditional health care settings.

Development of the DNP Leader in Residence Program

The Plan-Do-Study-Act (PDSA) cycle has garnered substantial recognition as a valuable tool for fostering development and driving improvement initiatives. 5 The PDSA cycle can function as an independent methodology and as an integral component of broader quality enhancement approaches with notable efficacy in its ability to facilitate the rapid creation, testing, and evaluation of transformative interventions within healthcare. 6 Consequently, the PDSA cycle model was deemed fitting to guide the development and implementation of the DNP Leader in Residence Program at the Csomay Center.

PDSA Cycle: Plan

Existing resources. The DNP Health Systems: Administration/Executive Leadership Program offered by the University of Iowa is comprised of comprehensive nursing administration and leadership curriculum, led by distinguished faculty composed of national leaders in the realms of innovation, health policy, leadership, clinical education, and evidence-based practice. The curriculum is designed to cultivate the next generation of nursing executive leaders, with emphasis on personalized career planning and tailored practicum placements. The DNP Health Systems: Administration/Executive Leadership curriculum includes a range of courses focused on leadership and management with diverse topics such as policy an law, infrastructure and informatics, finance and economics, marketing and communication, quality and safety, evidence-based practice, and social determinants of health. The curriculum is complemented by an extensive practicum component and culminates in a DNP project with additional hours of practicum.

New program. The DNP Leader in Residence program at the Csomay Center is designed to encompass communication and relationship building, systems thinking, change management, transformation and innovation, knowledge of clinical principles in the community, professionalism, and business skills including financial, strategic, and human resource management. The program fully immerses students in the objectives of the DNP Health Systems: Administration/Executive Leadership curriculum and enables them to progressively demonstrate competencies outlined by AONL. The Leader in Residence program also includes career development coaching, reflective practice, and personal and professional accountability. The program is integrated throughout the entire duration of the Leader in Residence’s coursework, fulfilling the required practicum hours for both the DNP coursework and DNP project.

The DNP Leader in Residence program begins with the first semester of practicum being focused on completing an onboarding process to the Center including understanding the center's strategic plan, mission, vision, and history. Onboarding for the Leader in Residence provides access to all relevant Center information and resources and integration into the leadership team, community partnerships, and other University of Iowa College of Nursing Centers associated with the Csomay Center. During this first semester, observation and identification of the Csomay Center Director's various roles including being a leader, manager, innovator, socializer, and mentor is facilitated. In collaboration with the Center Director (a faculty position) and Center Coordinator (a staff position), specific competencies to be measured and mastered along with learning opportunities desired throughout the program are established to ensure a well-planned and thorough immersion experience.

Following the initial semester of practicum, the Leader in Residence has weekly check-ins with the Center Director and Center Coordinator to continue to identify learning opportunities and progression through executive leadership competencies to enrich the experience. The Leader in Residence also undertakes an administrative project for the Center this semester, while concurrently continuing observations of the Center Director's activities in local, regional, and national executive leadership settings. The student has ongoing participation and advancement in executive leadership roles and activities throughout the practicum, creating a well-prepared future nurse executive leader.

After completing practicum hours related to the Health Systems: Administration/Executive Leadership coursework, the Leader in Residence engages in dedicated residency hours to continue to experience domains within nursing leadership competencies like communication, professionalism, and relationship building. During residency hours, time is spent with the completion of a small quality improvement project for the Csomay Center, along with any other administrative projects identified by the Center Director and Center Coordinator. The Leader in Residence is fully integrated into the Csomay Center's Leadership Team during this phase, assisting the Center Coordinator in creating agendas and leading meetings. Additional participation includes active involvement in community engagement activities and presenting at or attending a national conference as a representative of the Csomay Center. The Leader in Residence must mentor a master’s in nursing student during the final year of the DNP Residency.

Implementation of the DNP Leader in Residence Program

PDSA Cycle: Do

Immersive experience. In this case study, the DNP Leader in Residence was fully immersed in a wide range of center activities, providing valuable opportunities to engage in administrative projects and observe executive leadership roles and skills during practicum hours spent at the Csomay Center. Throughout the program, the Leader in Residence observed and learned from multidisciplinary leaders at the national, regional, and university levels who engaged with the Center. By shadowing the Csomay Center Director, the Leader in Residence had the opportunity to observe executive leadership objectives such as fostering innovation, facilitating multidisciplinary collaboration, and nurturing meaningful relationships. The immersive experience within the center’s activities also allowed the Leader in Residence to gain a deep understanding of crucial facets such as philanthropy and community engagement. Active involvement in administrative processes such as strategic planning, budgeting, human resources management, and the development of standard operating procedures provided valuable exposure to strategies that are needed to be an effective nurse leader in the future.

Active participation. The DNP Leader in Residence also played a key role in advancing specific actions outlined in the center's strategic plan during the program including: 1) the creation of a membership structure for the Csomay Center and 2) successfully completing a state Board of Regents application for official recognition as a distinguished center. The Csomay Center sponsored membership for the Leader in Residence in the Midwest Nurse Research Society (MNRS), which opened doors to attend the annual MNRS conference and engage with regional nursing leadership, while fostering socialization, promotion of the Csomay Center and Leader in Residence program, and observation of current nursing research. Furthermore, the Leader in Residence participated in the strategic planning committee and engagement subcommittee for MNRS, collaborating directly with the MNRS president. Additional active participation by the Leader in Residence included attendance in planning sessions and completion of the annual report for , an initiative falling under the umbrella of the Csomay Center. Finally, the Leader in Residence was involved in archiving research and curriculum for distinguished nursing leader and researcher, Dr. Kitty Buckwalter, for the Benjamin Rose Institute on Aging, the University of Pennsylvania Barbara Bates Center for the Study of the History of Nursing, and the University of Iowa library archives.

Suggested Evaluation Strategies of the DNP Leader in Residence Program

PDSA Cycle: Study

Assessment and benchmarking. To effectively assess the outcomes and success of the DNP Leader in Residence Program, a comprehensive evaluation framework should be used throughout the program. Key measures should include the collection and review of executive leadership opportunities experienced, leadership roles observed, and competencies mastered. The Leader in Residence is responsible for maintaining detailed logs of their participation in center activities and initiatives on a semester basis. These logs serve to track the progression of mastery of AONL competencies by benchmarking activities and identifying areas for future growth for the Leader in Residence.

Evaluation. In addition to assessment and benchmarking, evaluations need to be completed by Csomay Center stakeholders (leadership, staff, and community partners involved) and the individual Leader in Residence both during and upon completion of the program. Feedback from stakeholders will identify the contributions made by the Leader in Residence and provide valuable insights into their growth. Self-reflection on experiences by the individual Leader in Residence throughout the program will serve as an important measure of personal successes and identify gaps in the program. Factors such as career advancement during the program, application of curriculum objectives in the workplace, and prospects for future career progression for the Leader in Residence should be considered as additional indicators of the success of the program.

The evaluation should also encompass a thorough review of the opportunities experienced during the residency, with the aim of identifying areas for potential expansion and enrichment of the DNP Leader in Residence program. By carefully examining the logs, reflecting on the acquired executive leadership competencies, and studying stakeholder evaluations, additional experiences and opportunities can be identified to further enhance the program's efficacy. The evaluation process should be utilized to identify specific executive leadership competencies that require further immersion and exploration throughout the program.

Future Innovation of DNP Leader in Residence Programs in Non-traditional Healthcare Settings

PDSA Cycle: Act

As subsequent residents complete the program and their experiences are thoroughly evaluated, it is essential to identify new opportunities for DNP Leader in Residence programs to be implemented in other non-health care system settings. When feasible, expansion into clinical healthcare settings, including long-term care and acute care environments, should be pursued. By leveraging the insights gained from previous Leaders in Residence and their respective experiences, the program can be refined to better align with desired outcomes and competencies. These expansions will broaden the scope and impact of the program and provide a wider array of experiences and challenges for future Leaders in Residency to navigate, enriching their development as dynamic nurse executive leaders within diverse healthcare landscapes.

This case study presented a comprehensive overview of the development and implementation of the DNP Leader in Residence program developed by the Barbara and Richard Csomay Center for Gerontological Excellence. The Leader in Residence program provided a transformative experience by integrating key curriculum objectives, competency-based learning, and mentorship by esteemed nursing leaders and researchers through successful integration into the Center. With ongoing innovation and application of the PDSA cycle, the DNP Leader in Residence program presented in this case study holds immense potential to help better prepare 21 st century nurse leaders capable of driving positive change within complex healthcare systems.


         The author would like to express gratitude to the Barbara and Richard Csomay Center for Gerontological Excellence for the fostering environment to provide an immersion experience and the ongoing support for development of the DNP Leader in Residence program. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

  • American Association of Colleges of Nursing. The essentials: core competencies for professional nursing education. . Accessed June 26, 2023.
  • American Organization for Nursing Leadership. Nurse leader core competencies. . Accessed July 10, 2023.
  • Warshawsky, N, Cramer, E. Describing nurse manager role preparation and competency: findings from a national study. J Nurs Adm . 2019;49(5):249-255. DOI:  10.1097/NNA.0000000000000746
  • Van Diggel, C, Burgess, A, Roberts, C, Mellis, C. Leadership in healthcare education. BMC Med. Educ . 2020;20(465). doi: 10.1186/s12909-020-02288-x
  • Institute for Healthcare Improvement. Plan-do-study-act (PDSA) worksheet. . Accessed July 4, 2023.
  • Taylor, M, McNicolas, C, Nicolay, C, Darzi, A, Bell, D, Reed, J. Systemic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Quality & Safety. 2014:23:290-298. doi: 10.1136/bmjqs-2013-002703

Return to College of Nursing Winter 23/24 Newsletter


  1. (PDF) Medical Journal of Clinical Trials & Case Studies Clinical Case

    case study in clinical trials

  2. Insight into the Different Clinical Trial Study Designs

    case study in clinical trials

  3. Case study: Clinical Trials

    case study in clinical trials

  4. Clinical Trials Case Study

    case study in clinical trials

  5. (PDF) Marketing and clinical trials: A case study

    case study in clinical trials

  6. Fundamentals of Clinical Trials

    case study in clinical trials


  1. Clinical Case Scenarios 5


  1. NIH Definition of Clinical Trial Case Studies

    NIH Definition of Clinical Trial Case Studies The case studies provided below are designed to help you identify whether your study would be considered by NIH to be a clinical trial. Expect the case studies and related guidance to evolve over the upcoming year.

  2. Clinical Trials Case Study

    This clinical trial case study discusses the process that led to the WATERFALL trial — from a question asked by a gastroenterology trainee more than two decades ago about fluid management...

  3. Case 7-2021: A 19-Year-Old Man with Shock, Multiple Organ Failure, and

    Presentation of Case Dr. Alexandra T. Lucas (Pediatrics): A 19-year-old man was admitted to the pediatric intensive care unit (PICU) of this hospital because of shock, multiple organ failure,...

  4. Case 22-2020: A 62-Year-Old Woman with Early Breast Cancer during the

    Case Records of the Massachusetts General Hospital Case 22-2020: A 62-Year-Old Woman with Early Breast Cancer during the Covid-19 Pandemic Laura M. Spring, M.D., Michelle C. Specht, M.D.,...

  5. Chasing WATERFALLs

    The WATERFALL randomized trial, published in the New England Journal of Medicine in 2022, showed that in patients with acute pancreatitis, early aggressive fluid resuscitation resulted in a...

  6. 22 Case Studies Where Phase 2 and Phase 3 Trials Had Divergent Results

    when the phase 2 study was relatively large and even when the phase 2 trials assessed clinical outcomes. In two cases, the phase 3 studies showed that the experimental product increased the ...

  7. PRECIS-2 Case Study

    For study teams, the broad steps of evaluating a trial using the PRECIS-2 wheel include: Defining the trial's intention. Aligning the design to the intention. Identifying the trial's location on the explanatory-pragmatic continuum (1-5) for each of the 9 domains (the spokes of the wheel) In the case study, a very explanatory approach with ...

  8. Basics About Clinical Trials

    Clinical trials are research studies in which people volunteer to help find answers to specific health questions. When carefully conducted, they are the safest and fastest way to find new...

  9. Real World Evidence in clinical trials: 3 case studies of successful

    Case Study 1: Salford Lung Studies. The innovative Salford Lung Studies are often showcased as great example of an RWE-based study. The GSK sponsored study examined the safety and effectiveness of a new treatment for chronic obstructive pulmonary disease. It involved over 2,802 patients treated by their own GPs in everyday clinical practice.

  10. Case Studies in Innovative Clinical Trials

    Each contribution is a practical case study that can speak to the benefits of an innovative approach but also balance that with the real-life challenges encountered A complete understanding of what is actually being done in modern clinical trials will broaden the reader's capabilities and provide examples to first mimic and then customize and ...

  11. Why Diverse Clinical Trial Participation Matters

    Goals of Increasing Diversity in Clinical Trials. Broadly, the goals of increasing diversity in clinical trial participation include earning and building trust, promoting fairness, and generating ...

  12. Marketing and clinical trials: a case study

    Using methods of analysis drawn from management studies, this paper presents a structured assessment framework or reference model, derived from a case analysis of the MRC's CRASH trial, of 12 factors that may affect the success of the marketing and sales activities associated with clinical trials. Results. The case study demonstrates that ...

  13. PDF NIH Definition of Clinical Trial Case Studies

    NIH Definition of Clinical Trial Case Studies . In 2014, NIH began a multi-faceted effort to enhance the quality, relevance, feasibility, and transparency of NIH-funded clinical trials. A key element of these stewardship reforms was the development of a clearer, more comprehensive definition of clinical trial. The NIH clinical trial definition is:

  14. How clinical trials work

    People volunteer in clinical trials to have a chance to try the drug and for the drug's producer to know if works and is safe. At any given time, many clinical trials are going on at hospitals and universities across the country to test drugs or other treatments to cure a disease or make it easier to deal with symptoms. Every clinical trial ...

  15. Institutional dashboards on clinical trial transparency for ...

    Author summary Why was this study done? Clinical trials are the foundation of evidence-based medicine and should follow established guidelines for transparency: Their results should be available, findable, and accessible regardless of the outcome. Previous studies have shown that many clinical trials fall short of transparency guidelines, which distorts the medical evidence base, creates ...

  16. Markers of imminent myocardial infarction

    Gustafsson, Lampa et al. used proteomics, metabolomics and clinical factors in a case-cohort study to identify biomarkers of imminent myocardial infarction and devise a prediction model for ...

  17. Informed Consent Helps Patients Understand Their Rights and

    Implications for Oncology Nurses: Case Study . Using Jane, a newly diagnosed patient with cancer, as an example, here's one way oncology nurses can include informed consent as an ongoing process throughout a patient's participation in a clinical trial. Jane is enrolled in a phase II oncology clinical trial during her clinic visit.

  18. Clinical Trials and Case Studies

    International Journal of Clinical Trials & Case Studies (IJCTCS) is a comprehensive, peer reviewed journal devoted to Clinical Trials & Case Studies. IJCTCS, published by SciDoc is an Open Access journal that includes high quality papers, which covers all major areas of Clinical Trials & Case Studies.

  19. PDF NIH Definition of Clinical Trial Case Studies

    This study is a clinical trial. Case #8a: The study involves the recruitment of research participants with disease X. It is designed to compare the diagnostic performance of approved devices A and B, both of which are used in clinical practice, to measure disease markers. Device A will be used in half of the patients; device B will be used

  20. What can Safety Cases offer for patient safety? A multisite case study

    Methods Data are drawn from a mixed-methods evaluation of the Safer Clinical Systems programme. The development of a Safety Case for a defined clinical pathway was a centrepiece of the programme. We base our analysis on 143 interviews covering all aspects of the programme and on analysis of 13 Safety Cases produced by clinical teams.

  21. Driving change: a case study of a DNP leader in residence program in a

    In this case study, the DNP Leader in Residence was fully immersed in a wide range of center activities, providing valuable opportunities to engage in administrative projects and observe executive leadership roles and skills during practicum hours spent at the Csomay Center. ... When feasible, expansion into clinical healthcare settings ...

  22. Clinical trials that are designed to fail

    Even worse are trials which are aliased with a political agenda. It's easy-peasy to design a trial to fail (the parachute trial was jumping from a height of 2 feet). That makes me a lot more critical when you see trials where the failure of the trial would be seen as a upside, because it is just so easy to botch a trial.