Big data, real-world data (RWD), and observational data have become ubiquitous and essential components of the drug development and commercialization process. Researchers, statisticians, and data scientists must generate evidence and gain insights from these massive data. Various types of RWD—such as claims, electronic health records, surveys, and digital data—provide opportunities for successful partnerships between academia and business, industry, and government organizations.

This is why RWD and its applications in the pharmaceutical industry was the topic of the fourth virtual meet-up sponsored by the National Institute of Statistical Sciences (NISS) and Merck and co-sponsored by the ASA’s Health Policy Statistics Section (HPSS) for the first time this year.

RCT vs. RWD

Randomized controlled trials (RCTs) have long been the gold standard for providing evidence to support the safety and efficacy of a new drug, vaccine, or biologic. Most pre-marketing product development programs are divided into three phases (phases 1, 2, and 3), with phase 3 consisting of large RCTs to confirm the safety and efficacy of a new molecular entity. However, complete reliance on RCTs comes at a price, as they are often not only costly but limited to specific clinical circumstances that generate biases in RCT results with respect to population averages.

On the other hand, RWD are collected from a more heterogeneous population, often in a less rigorous and observational manner, and do not provide the same assurances against biases due to confounding. Nonetheless, RWD have the potential to provide rich, diverse, and important information on compound performance in more realistic clinical settings.

This meet-up focused on the roles RWD can play in compound development, registration, and post-approval, which is an area of highly active research with ongoing debates.

Disruptive Innovation

The presenters highlighted the opportunities to embrace and meet the challenge arising from real-world evidence (RWE) and their related disciplines and applications. Anirban Basu’s presentation suggested disruptive innovation is needed in not only how evidence is used, but also in how it is produced. He explained the merits of replacing the evidence production infrastructure based on the current drug development regulatory paradigm with an alternative structure termed only-in research (OIR). In the OIR framework, sponsors would produce evidence of safety through phase II in the conventional way. However, much of the safety and efficacy information currently obtained from phase 3 RCTs would be produced by RWE generated by OIR randomized market access to the compound. Final regulatory approval would be based on the evidence generated from the OIR phase.

Douglas Faries discussed best practices and innovations using RWE within the existing drug development paradigm. He argued we should endeavor to improve the operating characteristics of RWE to the point where it can support reliable and valid decision-making that is acceptable to regulatory decision-makers. He elucidated some of the most important issues—including repeatability, bias, and unmeasured confounding—and gave insight into how they could be addressed to solidify RWD’s evidentiary foundation.

Commenting on the presentations, Kelly Zou noted many of the issues discussed were also topics at the recent Duke-Margolis Conference on RWE. She explained how infrastructure disruption is the key to Basu’s notion of an alternative development pathway, while Faries focused on innovative ideas to address the statistical challenges of RWD and how well these results agree with RCTs. Her final take-away was that although there could be infrastructure disruptions, quantitative skills and statistical thinking remain important.

Skill Sets to Leverage RWE for Decision-Making

Several years ago, Forbes published a short history of data science (DS), starting from John Tukey back in the 60s. According to the ASA’s statement on DS, it is an emerging and broad discipline with the following three areas as its foundation:

• Database management to enable transformation, conglomeration, and organization of data resources
• Statistics and machine learning to convert data into knowledge
• Distributed and parallel systems for providing the computational infrastructure to carry out data analysis

Thus, skills involving data access, integration, and interoperability are of growing importance.

For researchers to leverage RWD, innovative ideas; strong methodological training; and hands-on applications using database and analytic software tools such as Python, SQL, Hadoop, Spark, Hive, and Tableau (in addition to more traditional software coding choices such as SAS, R, SPSS, Stata, Matlab) are useful. Technical skills must be supplemented with interpersonal / communication skills—often using visualization methods to aid in interpretation of the complex data and analytics. Perhaps even more salient in this new world of data exuberance is some grounding in decision theory, to understand the applicability of a piece of evidence to real-world decision-making. Furthermore, analyses of RWD are relying more on sophisticated methods such as complex bias adjustments, advanced modeling, and machine learning techniques. Despite the new sources of data and new skills needed to handle big data, the foundational statistical skills—including research design, pre-specification, multiplicity, missing data, understanding bias, and uncertainty—still remain a critical skill set for RWD researchers.

Virtual Meet-Ups

This NISS-Merck-HPSS joint virtual meet-up was part of a roughly quarterly series hosted by NISS and Merck on emerging issues of interest to the pharma/biostatistics community. Previous meet-up topics include multiple endpoints in clinical trials, estimands and sensitivity analysis in clinical trials, and applications of machine learning in the pharmaceutical industry.

The format is usually two short talks by two invited speakers, followed by a panel discussion initiated by comments from a moderator and questions submitted from the audience. The meet-up lasts one hour and 15 minutes and a recorded version with presentation slides are shared on the NISS website. Connection information is made available on the NISS website (Upcoming Events section) and through advertisements on the ASA Connect and Biopharmaceutical Section bulletin boards.

The next meet-up, which will focus on statistical challenges in immuno-oncology, is scheduled for January 22, from 11:00 a.m. to 12:15 p.m. ET.

Editor’s Note: The participants are employees of their respective organizations. Views and opinions expressed in the meet-up and this article are the participants’ and do not necessarily reflect those of their employers.