The source of trial data for a PCT should be considered when determining the monitoring plan. Two main issues for PCTs are data quality and timeliness.
When using the EHR as the source of trial data, monitoring for data quality early in the trial can help ensure that valid results will be obtained (Ellenberg et al. 2015). Data reported by clinicians in the everyday care setting may have more variability than DMCs are used to seeing with traditional trials, which have highly controlled settings, strict reporting protocols, and separate systems for documentation (e.g., case report forms). In traditional trials, every effort is made to ensure consistency of measurement across sites. More heterogeneity may be observed in PCTs due to real-world variations in populations and approaches by healthcare delivery organizations and their practicing clinicians.
“Thus, DMCs may need to pay close attention to site-specific data to determine whether an emerging result may be attributable to one or two sites and is perhaps not widely generalizable” (Ellenberg et al. 2015).
As an element of data quality, completeness of data can also be a challenge in PCTs. Pragmatic trial protocols may allow flexibility in follow-up according to standard clinical practices at participating sites. The DMC and sponsor will need to agree on necessary data for follow-up in accordance with the risks of the trial intervention. Variations in follow-up practices across sites may also need to be accounted for in the randomization process to ensure that sites with more frequent follow-up practices are balanced across study arms.
Data obtained from sources such as claims data, state mortality data, or even the EHR will often have a delay and not be available real-time. In some trials, sites may perform data analysis locally due to privacy concerns, then submit results for centralized analysis and aggregation. This process, along with necessary data quality checks, can result in additional time before data are available for review (Ellenberg et al. 2015). Some determination will need to be made regarding the amount of delay that is acceptable to ensure adequate participant protection. DMCs should be involved in these discussions.
In a traditional study of a mental health intervention, a suicide attempt or suicide death would be considered a serious adverse event, calling for immediate reporting to the DSMB and a DSMB determination regarding relatedness of the event to study treatment. However, in SPOT, a pragmatic trial, information on deaths is obtained from state mortality data, which are delayed up to 16 months. Death was also an expected occurrence in this population at high risk for suicide. In this situation, it would not be sensible to stop the trial for investigation of a death 16 months after the death occurred. These special circumstances required negotiation with the DSMB to work out a more practical monitoring plan.
- Which PCTs Should Have a DMC?
- Monitoring Protocol Adherence
- Data Issues With Monitoring PCTs
- Monitoring for Serious Adverse Events
- Futility Assessment
- Case Study: Planning for Monitoring PCTs
- Role of Stakeholder Perspectives
- Special Training and Resources for DMCs of Pragmatic Trials
- Additional Resources
An NIH Collaboratory guidance document with best practices for assessing data quality in PCTs
Explores requirements for data quality assessment in comparative effectiveness research using distributed data networks