Why Are We Talking About Pragmatic Trials?

What is a Pragmatic Clinical Trial

Section 1

Why Are We Talking About Pragmatic Trials?


Kevin Weinfurt, PhD


Contributing Editors

Karen Staman, MS

Jonathan McCall, MS

Liz Wing, MA

Healthcare in the United States is increasingly complex and expensive, and there is a great need for more evidence to inform decisions that lead to improved, efficient, and affordable care (Alper and Grossmann 2015). Care providers, researchers, administrators, payers, regulators, and the public agree that the provision of medical care should be informed by high-quality scientific evidence regarding the risks and benefits of treatments. Yet, this high-quality evidence—generated by conducting randomized controlled trials (RCTs) and disseminated through clinical practice guidelines—is severely lacking across a multitude of specialties (Tricoci et al. 2009; Roos et al. 2011; Wright et al. 2011; Koh et al. 2013; Feuerstein et al. 2014; Neuman et al. 2014). In the absence of this evidence, clinicians must make educated guesses to determine treatment based on personal judgment and knowledge of the patient, rather than on the consensus of a group of clinical experts (Tricoci et al. 2009). Clinicians and patients simply do not have enough evidence to effectively inform clinical decisions. For example, in the field of cardiology, which arguably has one of the most robust evidence bases among specialties, the majority of treatment recommendations are founded upon lower-quality trials, observational studies, or expert opinion (Tricoci et al. 2009).

In addition, when we survey the US clinical trials enterprise from a broad perspective, the kind of trials needed to provide medical evidence to support treatment decisions are, for the most part, not being done. Analyses of trials contained in the ClinicalTrials.gov database continually tell the same story—the vast majority of clinical trials are too small to provide sufficient statistical power to definitively answer clinical questions, they fail to address critical treatment priorities, and/or they suffer from shortcomings in design and execution that limit their usefulness (Califf et al. 2012; Pasquali et al. 2012; Alexander et al. 2013; Goswami et al. 2013; Hirsch et al. 2013; Lakey et al. 2013; Todd et al. 2013; Witsell et al. 2013; Inrig et al. 2014; Subherwal et al. 2014). In addition, the data from many of these trials are not being reported in timely and transparent ways (Anderson et al. 2015). Adding to these complications, there has been a steady drumbeat of revelations indicating that many findings published in the peer-reviewed literature are fundamentally unreliable (Ioannidis 2005; Open Science Collaboration 2015; Le Noury et al. 2015).

For much of modern medical history, many have believed that clinical research should be kept apart from routine patient care. As a result, we now have a system in which research data are collected using stand-alone systems. These systems are designed to ensure that the information gathered during research activities is valid and complete. However, having separate systems for research and care comes at a significant cost. There is growing concern that the results obtained from clinical research may not apply to “real-world” situations (Ioannidis 2005), because the research is often done under artificial conditions with volunteers who may not reflect the patients who actually live with a given disease or condition. What’s more, this “standalone” system requires enormous amounts of money and effort to sustain.

To counter these problems, many are now advocating a move to a learning health system in which tools such as computing power, connectivity, team-based care, and systems engineering techniques will produce a culture of continuous learning at lower cost (Institute of Medicine 2013). Ideally, clinical trials would be embedded within a system of healthcare where evidence is rapidly and continually fed back into clinical care, and clinical care itself would inform the further development of medical evidence. At the same time, the widespread use of electronic health records and advances in information technology and informatics are creating opportunities to combine very large, complex sets of data (“big data”) in ways that until now were almost unimaginable. As systems for managing data continue to improve within US health systems, the availability of electronic data is likewise improving rapidly.

Pragmatic clinical trials (PCTs) represent one approach that could support a goal of a learning health system by informing real-world practice with digital health data collected at the point of care.




back to top

Alexander KP, Kong DF, Starr AZ, et al. 2013. Portfolio of clinical research in adult cardiovascular disease as reflected in ClinicalTrials.gov. J Am Heart Assoc. 2:e000009. doi:10.1161/JAHA.113.000009. PMID: 24072529.

Alper J, Grossmann C. 2015. Integrating Research and Practice: Health System Leaders Working toward High-Value Care: Workshop Summary. Washington, DC: National Academies Press.

Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. 2015. Compliance with Results Reporting at ClinicalTrials.gov. New Engl J Med. 372:1031–1039. doi:10.1056/NEJMsa1409364. PMID: 25760355.

Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. 2012. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010. JAMA. 307:1838. doi:10.1001/jama.2012.3424. PMID: 22550198.

Institute of Medicine. 2013. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Smith M, Saunders R, Stuckhardt L, McGinnis JM, eds. Washington, DC: National Academies Press. http://www.ncbi.nlm.nih.gov/books/NBK207225/. Accessed January 28, 2015.

Feuerstein JD, Akbari M, Gifford AE, et al. 2014. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 89:16–24. doi:10.1016/j.mayocp.2013.09.013. PMID: 24388018.

Goswami ND, Pfeiffer CD, Horton JR, Chiswell K, Tasneem A, Tsalik EL. 2013. The state of infectious diseases clinical trials: a systematic review of ClinicalTrials.gov. PLoS ONE. 8:e77086. doi:10.1371/journal.pone.0077086. PMID: 24146958.

Hirsch BR, Califf RM, Cheng SK, et al. 2013. Characteristics of oncology clinical trials: insights from a systematic analysis of ClinicalTrials.gov. JAMA Intern Med. 173:972. doi:10.1001/jamainternmed.2013.627. PMID: 23699837.

Inrig JK, Califf RM, Tasneem A, et al. 2014. The landscape of clinical trials in nephrology: a systematic review of Clinicaltrials.gov. Am J Kidney Dis. 63:771–780. doi:10.1053/j.ajkd.2013.10.043. PMID: 24315119.

Ioannidis JPA. 2005. Why most published research findings are false. PLoS Med. 2:e124. doi:10.1371/journal.pmed.0020124. PMID: 16060722.

Koh C, Zhao X, Samala N, Sakiani S, Liang TJ, Talwalkar JA. 2013. AASLD clinical practice guidelines: a critical review of scientific evidence and evolving recommendations. Hepatology. 58:2142–2152. doi:10.1002/hep.26578. PMID: 23775835.

Lakey WC, Barnard K, Batch BC, Chiswell K, Tasneem A, Green JB. 2013. Are current clinical trials in diabetes addressing important issues in diabetes care? Diabetologia. 56:1226–1235. doi:10.1007/s00125-013-2890-4. PMID: 23564296.

Le Noury J, Nardo JM, Healy D, et al. 2015. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. h4320. doi:10.1136/bmj.h4320. PMID:26376805.

Neuman MD, Goldstein JN, Cirullo MA, Schwartz JS. 2014. Durability of class I American College of Cardiology/American Heart Association clinical practice guideline recommendations. JAMA. 311:2092–2100. doi:10.1001/jama.2014.4949. PMID: 24867012.

Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science. 349:aac4716–aac4716. doi:10.1126/science.aac4716. PMID:26315443.

Pasquali SK, Lam WK, Chiswell K, Kemper AR, Li JS. 2012. Status of the pediatric clinical trials enterprise: an analysis of the US ClinicalTrials.gov registry. Pediatrics. 130:e1269–e1277. doi:10.1542/peds.2011-3565. PMID:23027172.

Roos M, Brodbeck J, Sarkozy A, Chierchia GB, De Asmundis C, Brugada P. 2011. A critical analysis of the scientific evidence behind international guidelines related to cardiac arrhythmias. Circ Arrhythm Electrophysiol. 4:202–210. doi:10.1161/CIRCEP.110.958181. PMID: 21372270.

Subherwal S, Patel MR, Chiswell K, et al. 2014. Clinical trials in peripheral vascular disease: pipeline and trial designs: an evaluation of the ClinicalTrials.gov database. Circulation. 130:1812–1819. doi:10.1161/CIRCULATIONAHA.114.011021. PMID: 25239436.

Todd JL, White KR, Chiswell K, Tasneem A, Palmer SM. 2013. Using ClinicalTrials.gov to understand the state of clinical research in pulmonary, critical care, and sleep medicine. Ann Am Thorac Soc. 10:411–417. doi:10.1513/AnnalsATS.201305-111OC. PMID: 23987571.

Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC. 2009. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 301:831–841. doi:10.1001/jama.2009.205. PMID: 19244190.

Witsell DL, Schulz KA, Lee WT, Chiswell K. 2013. An analysis of registered clinical trials in otolaryngology from 2007 to 2010: ClinicalTrials.gov. Otolaryngol Head Neck Surg. 149:692–699. doi:10.1177/0194599813506545. PMID: 24107478PMCID: PMC4121965.

Wright JD, Pawar N, Gonzalez JSR, et al. 2011. Scientific evidence underlying the American College of Obstetricians and Gynecologists’ practice bulletins. Obstet Gynecol. 118:505–512. doi:10.1097/AOG.0b013e3182267f43. PMID: 21826038.


Weinfurt K. What is a Pragmatic Clinical Trial: Why Are We Talking About Pragmatic Trials?. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: http://www.rethinkingclinicaltrials.org/pragmatic-clinical-trial/what-is-a-pragmatic-clinical-trial/. Updated September 11, 2017.