Grand Rounds December 16, 2022: The Use of EHR-Agnostic Clinical Decision Support to Prevent Thromboembolism in Hospitalized Medically Ill Patients (Alex C. Spyropoulos, MD, FACP, FCCP, FRCPC; Jeffrey Solomon, BFA)

Speakers

Alex C. Spyropoulos, MD, FACP, FCCP, FRCPC
Professor of Medicine – The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell
Professor – Institute of Health System Science – The Feinstein Institutes for Medical Research
System Director – Anticoagulation and Clinical Thrombosis Services
Northwell Health at Lenox Hill Hospital

Jeffrey Solomon, BFA
Senior Director, Usability Lab
Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY

 

 

Keywords

Pragmatic Clinical Trials, Cluster Randomized Trial, EHR solutions

 

Key Points

  • The majority of hospital-acquired Venous Thromboembolism (VTE) occurs in non-surgical medical inpatients, but Thromboprophylaxis for medically ill patients during and after hospital stays is underutilized. Electronic alerts incorporating VTE risk models could increase appropriate use of Thromboprophylaxis and reduce symptoms of VTE.
  • The study team’s health informatics group developed a novel clinical decision support (CDS) tool, called IMPROVE-DD VTE CDS, which can be integrated into different electronic health record (EHR) systems. The effectiveness of the intervention was then evaluated through a cluster randomized trial at four academic tertiary hospitals from December 21, 2020 to January 21, 2022.
  • This trial tested the hypothesis that the use of a platform-agnostic, EHR-embedded VTE risk model with integrated CDS would 1) increase rates of appropriate Thromboprophylaxis, and subsequently 2) reduce thromboembolism, compared to usual medical care in hospitalized, medically-ill patients.
  • The study team developed the tool using high-quality evidence from randomized trials related to risk factors for subpopulations and guideline recommendations. The solution’s interface was developed using workflow analysis, multiple iterations of the tool and usability testing.
  • The trial is the first to show that a universal EHR-integrated CDS tool using a validated VTE risk model (IMPROVE-DD) had a high adoption rate (77%), significantly increased rates of in-hospital appropriate Thromboprophylaxis and significantly reduced major Thromboembolic events without an increase in major bleeding at 30 days post-discharge compared to usual medical care. Thirty-day mortality was higher in the intervention hospital group.
  • The study team spoke about the tension that often exists between solutions that are tightly integrated into a specific EHR and the ability to disseminate them widely. Developers worked to find the “sweet spot” in a solution that balanced the two priorities. The tool’s success is attributed to workflow analysis, rapid prototyping, usability testing and its integration with the EHR.
  • The high baseline rate of appropriate in-hospital and at-discharge Thromboprophylaxis in the academic control hospitals suggests that the solution has an even greater potential for positive impact at non-academic or rural hospital settings.

 

Discussion Themes

How was usability tested and how does the tool fit into a clinician’s daily routine? With every iteration of testing, the team worked to identify barriers in provider workflow. The number of testing rounds, as well as the quality of usability testing, was critical in understanding workflows and the best moment to launch the tool. The team also engaged with a diverse array of providers from multiple hospitals and specialties. It was important to observe behaviors in real time and be open to making changes at multiple points during the design and testing process.

How did you derive and evaluate the evidence that was used in the IMPROVE-DD VTE CDS tool? The team derived the evidence based on a multivariate analysis from an international prospective registry. The team also conducted multiple external validation projects using the same weights and scores. In doing this, they were able to replicate the data among different populations.

– How can tools like this contribute to clinical decisions based on data for subgroups instead of broader guidelines? The study team is hopeful that this effort and others like it will promote clinical decisions that are based on evidence for patient subgroups. Clinical recommendations should be based on more specific data, as there are public health implications for guideline statements that are too broad.

-What is the explanation for the excess deaths at 30 days in the intervention group? The trial was designed before the COVID-19 pandemic, but it was conducted during the height of the pandemic in New York. Because of rebalancing of physicians, there were more COVID-19 patients in the intervention group. Researchers considered this an epiphenomenon that likely reflects the imbalance of the COVID-19 patients in the intervention group. More analysis is needed to confirm that hypothesis

Tags

#pctGR, @Collaboratory1

Toward Causal Inference in Cluster Randomized Trials: Estimands and Reflection on Current Practice

Methods: Minds the Gap Webinar Series
“Toward Causal Inference in Cluster Randomized Trials: Estimands and Reflection on Current Practice”
Fan Li, PhD; Yale School of Public Health
National Institutes of Health, Office of Disease Prevention

Cluster randomized trials (CRTs) involve randomizing groups of individuals to different interventions. While model-based methods are extensively studied for analyzing CRTs, there has been little reflection around the treatment effect estimands at the outset. In the first part of this presentation, we describe two relevant estimands that can be addressed through CRTs and point out that they can differ when the treatment effects vary according to cluster sizes. As a cautionary note, we demonstrate how choices between different analytic approaches can impact the interpretation of results by fundamentally changing the question being asked. In the second part, we revisit the linear mixed model as the most commonly used method for analyzing CRTs. The linear mixed model makes stringent assumptions, including normality, linearity, and typically a compound symmetric correlation structure, all of which may be challenging to verify. However, under certain conditions, we show that the linear mixed model consistently estimates the average causal effect under arbitrary misspecification of its working model. Under equal randomization, its model-based variance estimator, surprisingly, remains consistent under model misspecification, justifying the use of confidence intervals output by standard software. These results hold under both simple and stratified randomization, and serve as an important causal inference justification for linear mixed models. Caveats and extensions of our findings will also be mentioned.

For more information, visit https://prevention.nih.gov/education-training/methods-mind-gap/toward-causal-inference-cluster-randomized-trials-estimands-and-reflection-current-practice.

November 9, 2021: PPACT Study Finds Benefits of Cognitive Behavioral Therapy in Reducing Chronic Pain and Pain-Related Disability

Photo of Dr. Lynn DeBar
Dr. Lynn DeBar, principal investigator of PPACT

Patients who participated in a cognitive behavioral therapy (CBT) intervention as part of their regular care for chronic pain showed improved function and reduced pain compared to standard treatment, according to the results of the Pain Program for Active Coping and Training (PPACT) study. Although CBT did not reduce opioid use, patients who participated in a 12-week course on pain self-management led by primary care providers showed modest but sustained benefits that persisted for 12 months after the intervention.

Study results were published this month in the Annals of Internal Medicine.

The PPACT study, an NIH Collaboratory Trial, was a pragmatic, cluster randomized trial that enrolled 850 patients receiving long-term opioid therapy for chronic pain. Patients in the intervention group participated in 12 weekly, 90-minute group sessions that taught skills of muscle relaxation, guided imagery, cognitive restructuring, and yoga-based adapted movement. Patients in the usual care group continued to receive pharmacologic and nonpharmacologic treatment.

Figure from PPACT main outcomes paper
Source: Annals of Internal Medicine 2021 Nov 2. doi: 10.7326/M21-1436

Patients were followed for 12 months with primary outcome measures of pain impact on enjoyment of life, activity levels, and sleep. Researchers also assessed secondary outcome measures of pain-related disability and opioid use.

Compared to usual care, the CBT intervention reduced self-reported pain and pain-related disability and increased satisfaction with primary healthcare providers. Opioid use and dose remained the same in both the intervention group and the usual care group.

PPACT was supported within the NIH Collaboratory by the NIH Common Fund, the National Center for Complementary and Integrative Health (NCCIH), and the National Institute of Neurological Disorders and Stroke (NINDS). Learn more about the NIH Collaboratory Trials.

October 29, 2021: Embedding Pragmatic Trials into Emergency and Critical Care (Matthew W. Semler, MD, MSc; Jonathan D. Casey, MD, MSc)

Speakers

Matthew W. Semler, MD, MSc
Assistant Professor
Vanderbilt University Medical Center

Jonathan D. Casey, MD, MSc
Assistant Professor
Vanderbilt University Medical Center

Topic

Embedding Pragmatic Trials into Emergency and Critical Care

Keywords

Pragmatic clinical trials; Study design; Comparative effectiveness trials; Treatment effect; SMART trial; PreVent trial; Exception from Informed Consent (EFIC)

Key Points

  • Emergency medical clinicians are faced with common decisions in everyday practice with little to no data from randomized clinical trials to help inform their decisions.
  • Four barriers to comparative effectiveness trials in a critical care setting are a brief therapeutic window, patients with multiple co-morbidities, the inability of the patient to consent to research, and analyzing average treatment effect rather than individual treatment effect.
  • The PreVent Trial studied the use of bag-mask ventilation to prevent hypoxemia for patients who had been administered anesthesia in preparation for intubation.
  • Efficient, pragmatic trial procedures that don’t delay treatment enable comparative effectiveness randomized clinical trials to be conducted effectively.
  • After 50 years of debate about bag-mask ventilation during this interval period, the PreVent Trial found that bag-mask ventilation cut the rate of hypoxemia by 50% without affecting aspiration.
  • The SMART Trial was a cluster-randomized, multiple-crossover trial of fluid management that studied patient outcomes when Balanced Crystalloids were used versus Saline solution.
  • The large sample size of over 15,000 patients provided the SMART trial with the power to detect that a balanced crystalloid fluid prevented Major Adverse Kidney Events in 1% of patients compared to Saline solution. /li>
  • Exception from Informed Consent (EFIC), implemented in 1996 allows trials in emergency situations of the condition is life-threatening, existing treatments are unproven or unsatisfactory, and research involves no more than minimal risk.
  • Analyzing Individual Treatment Effects will allow clinical providers to tailor their decisions to their individual patient.

Discussion Themes

Clinical equipoise poses a challenge for comparative effectiveness trials.

Key to getting buy-in from clinician stakeholders is explaining the importance of the research to the patient.

 

Read more about PreVent trial and the SMART trial.

Tags

#pctGR, @Collaboratory1

October 22, 2021: The STAMP Trial: Increasing Engagement in Advance Care Planning and Lessons Learned from Partnering with Community Ambulatory Practices (Terri R. Fried, MD)

Speaker

Terri R. Fried, MD
Section Chief, Geriatrics
Professor of Medicine
Yale School of Medicine
Attending Physician
VA Connecticut Healthcare System

Topic

The STAMP Trial: Increasing Engagement in Advance Care Planning and Lessons Learned from Partnering with Community Ambulatory Practices

Keywords

STAMP Trial; Advance Care Planning; ACP; Patient engagement; Cluster randomized trial

Key Points

  • The STAMP (Sharing and Talking about My Preferences) Trial is a cluster randomized controlled trial aimed at increasing engagement in Advanced Care Planning (ACP).
  • The STAMP Trial first aimed to re-conceptualize advance care planning (ACP) to achieve the ultimate goal of enabling the patient or surrogate to make decisions in a future moment rather than making decisions in advance. In this way, ACP is a flexible act of communication that allows for in-the-moment advice of a patients care providers about the nuances of a particular clinical situation.
  • ACP is a Health Behavior that involves stages of change. The STAMP Trial uses a 10 minute survey to assess how ready a patient is to start the ACP communication process.
  • Patients are given ACP materials based on their stage of readiness as assessed by the survey.
  • Results showed a small increase in ACP planning for groups randomized to the study interventions, but that small increase applied over large numbers of patients could have a significant impact on the number of people participating in ACP.

Discussion Themes

Cluster randomized trial design is complex unless you are working with an intervention that is implemented in a whole health care system rather than individual patients.

Determining a denominator in a cluster randomized trial at the patient level was very difficult.

 

Read more about Dr. Fried’s work on the STAMP Trial.

Tags

#pctGR, @Collaboratory1

October 15, 2021: The Impact of Community Masking on COVID-19: A Cluster-Randomized Trial in Bangladesh (Jason Abaluck, PhD)

Speaker

Jason Abaluck, PhD
Professor of Economics
Yale School of Management

Topic

The Impact of Community Masking on COVID-19: A Cluster-Randomized Trial in Bangladesh

Keywords

COVID-19; Cluster-randomized trial; Masking; Community spread; NORM model

Key Points

  • Dr. Abaluck’s cluster-randomized trial on the impact of community masking in ~350,000 adults in 600 villages of Bangledesh aimed to answer 2 questions: What strategies or interventions will increase mask-wearing? and What is the impact of mask wearing intervention on COVID-10?
  • There are two major differences between the existing randomized trials of mask wearing for flu and the impact of community masking in Bangladesh trial. The first issue is the existing trials were randomized individually not by communities so you cannot evaluate weather masks act as source control. The second issue with the existing trials is the discrepancy between the number of people who attest to wearing a mask and the number who actually wore a mask.
  • The impact of community masking in Bangladesh trial used the NORM model to increase mask wearing. The NORM model distributes masks at No-cost, Offers information about why mask wearing is important, Reinforces mask wearing by having mask promoters give free masks and information in public places, and Modeling by trusted community leaders wearing masks and endorsing mask wearing.
  • The NORM model more than tripled mask usage(13% usage increased to 42%), increased physical distancing, and had a sustained impact that lasted at least 10 weeks.
  • Communities where the NORM model was employed saw a 9% reduction in symptomatic COVID-19 infections.
  • Mask wearing showed a greater reduction in COVID-19 among the elderly.

Discussion Themes

The NORM model would have different efficacy rates in different cultures and communities.

You can get some people to wear a mask by just distributing the masks in public places and asking them to wear them.

Masks are not an all-or-nothing protection. Masks have a dose-reduction effect and that dose-reduction translates into a transmission effect.

 

Read more about the Dr. Abaluck’s work on masking and COVID-19 in the recently published Discussion Paper.

Tags

#pctGR, @Collaboratory1, @Jabaluck

September 10, 2021: Effect of Salt Substitution on Cardiovascular Events and Death (Bruce Neal, MB ChB, PhD, FRCP, FAHA, FAHMS)

Speaker

Bruce Neal, MB ChB, PhD, FRCP, FAHA, FAHMS
Executive Director, George Institute Australia
Professor of Medicine, UNSW Sydney
Honorary Professor, Sydney Medical School, University of Sydney
Professor of Clinical Epidemiology, Imperial College London

Topic

Effect of Salt Substitution on Cardiovascular Events and Death

Keywords

Cluster randomized trial; Salt substitute; Stroke; Cardiovascular disease; SSaSS

Key Points

  • The SSaSS study is a pragmatic, cluster randomized trial on the effects of salt substitutes versus regular salt on stroke, major adverse cardiovascular events, and mortality.
  • The SSaSS study followed almost 21,000 people in 600 village clusters over 5 years. At the end of the 5 year study, 92% of the intervention group was still using salt substitute and 6% of control started using salt substitute.
  • Data from the study show evidence of lower blood pressure, lower risk of stroke, lower risk of major adverse cardiovascular events, and protection against premature death with no evidence of harm.
  • There was no evidence of any increased risk of hyperkalemia.

Discussion Themes

Successful recruitment approaches in this large, long-term trial required extensive engagement and networking with local health workers and community leaders.

Salt substitute as a method to lower stroke and cardiovascular risk is attractive because it looks and tastes the same as salt, and doesn’t ask people to change their behavior.

The use of salt substitutes to decrease the intake of discretionary sodium, the salt used in home cooking or sprinkled on food after cooking, may have more effect in developing countries.  Developed countries consume processed and packaged foods more often and have less discretionary sodium intake.  In developed countries, it may be necessary to encourage food manufacturers to decrease sodium and increase potassium in the processed and packaged foods they produce.

Many millions of lives could benefit from the effects of salt substitute if this could be implemented in developing countries around the world.

Read more about Dr. Neal’s work with the SSaSS study.

Tags

#pctGR, @Collaboratory1

Methods: Mind the Gap Webinar July 14: Overview of Statistical Models for the Design and Analysis of Stepped Wedge Cluster Randomized Trials

Speaker: 

Fan Li, PhD
Yale University School of Public Health

Description:

The stepped-wedge cluster randomized design has received increasing attention in pragmatic clinical trials (PCTs) and implementation science research. Since the design’s introduction, a variety of mixed-effects model extensions have been proposed for the design and analysis of PCTs. In this talk, Dr. Fan Li of Yale University will provide a general model representation and regard various model extensions as alternative ways to characterize secular trends, intervention effects, and sources of heterogeneity. He will also review key model ingredients and clarify their implications for the design and analysis of stepped-wedge trials.

Registration required: 

https://www.prevention.nih.gov/education-training/methods-mind-gap/overview-statistical-models-design-and-analysis-stepped-wedge-cluster-randomized-trials

April 23, 2020: New Workshop Summary on the Design and Analysis of Pragmatic Clinical Trials

In 2019, NIH Health Care Systems Research Collaboratory held a comprehensive workshop to explore and discuss statistical issues encountered with embedded pragmatic clinical trials (ePCTs). The new Workshop Summary describes panel discussions with the principal investigators and statisticians of NIH Collaboratory Trials and the challenges and solutions encountered during the design and analysis of their trials.

The 4 panel discussions covered the following topics:

  • Measurement and Data: Outcomes, Exposures, and Subgroups Based on EHR Data
  • To Cluster or Not to Cluster?
  • Choosing a Parallel Group or Stepped-Wedge Design
  • Unique Complications

This Workshop Summary also provides lessons learned and recommends tools to help others design and analyze future ePCTs. For more on the design and analysis of pragmatic clinical trials, see the tools provided by the Biostatistics and Study Design Core and Living Textbook chapters on Experimental Designs and Randomization Schemes and Analysis Plans.

March 17, 2020: Cheat Sheet on the Intraclass Correlation Coefficient

The NIH Collaboratory Biostatistics and Study Design Core has created an Intraclass Correlation Coefficient (ICC) Cheat Sheet to provide an introductory description of the ICC, which is important for the design and analysis of cluster-randomized trials.

“The intraclass correlation coefficient (ICC) is a descriptive statistic that describes the extent to which outcomes 1) within each cluster are likely to be similar or 2) between different clusters are likely to be different from each other, relative to outcomes from other clusters. The ICC is an important tool for cluster-randomized pragmatic trials because this value helps determine the sample size needed to detect a treatment effect.” —from the ICC Cheat Sheet

The tool is a 2-page handout that can be used in trainings or classes regarding pragmatic clinical trials involving cluster randomization.

For more on the ICC, see the Intraclass Correlation section in the Living Textbook or this in-depth working document on the ICC from the Biostatistics and Study Design Core. If you have questions, feedback or suggestions regarding this tool, please contact us at nih-collaboratory@dm.duke.edu.