Ruixiao Lu

From left: Peng Yang, Ron Yu, Ron Wasserstein, Steven Goodman, Imola Fodor, Jing Huang, Ruixiao Lu, Chris Barker, Whedy Wang, Jing Du, and Li Zhang

A leadership forum titled “Real-World Significance Beyond the P-Value” took place May 21 in the San Francisco Bay area, bringing together experts on the topic. The event was hosted by DahShu a 501(c)(3) non-profit organization promoting research and education in data science, and co-sponsored by the San Francisco Bay Area Chapter (SFASA). More than 100 people attended in person, while more than 200 participated online.

Before the panel started, Jing Huang of Veracyte and the founding president of DahShu, introduced the organization and future events. Ron Yu of Gilead Sciences and president-elect of SFASA, reviewed the history and activities of the chapter. Both called for more volunteers to build the community of statistics and data science.

Ruixiao Lu of Genomic Health—a cofounder of DahShu, past president of SFASA, and vice chair of District 6—made opening remarks and moderated the panel. A quick poll from the in-person attendees showed a majority of the audience didn’t think p-values or statistical significance should be abandoned.

Four prominent panelists shared their perspectives. Ron Wasserstein, ASA executive director, was the first presenter. He gave an overview of the 2016 ASA statement on p-values and the latest TAS special issue on statistical inference. He discussed the rationale of the proposal to abandon statistical significance and restated the importance of good statistical practice, proper use of statistical methods, and transparent reporting to promoting reproducibility and replicability. He also announced that Regina Nuzzo will be joining the ASA staff as senior adviser for statistics communication and media innovation. In this role, she will help the ASA improve communication of statistics policy issues, bring more statistics to journalists and journalism, and enhance public engagement and communication skills for statisticians and statistics students.

The second presenter was Steven Goodman of Stanford University. He gave a talk titled “P-Values and Epistemology of Biomedical Research,” which highlighted the role of statistical rules as social technologies, not just technical tools, to aid collective data interpretation and decision-making, as well as serve as guideposts for non-experts. He referred to the JAMA publication in April by his colleague John Ioannidis about the importance of predefined rules and prespecified analyses and argued that banning statistical significance could have unknown and unintended adverse social effects and foster confusion. He proposed using Bayes factors, particularly minimum Bayes factors, and illustrated why they are improvements over p-values. He concluded by saying there are no quick fixes and changing statistical practices must be accepted and driven by scientists reforming their own fields. He thinks the ASA and others need to develop a multifaceted strategy partnering and learning from sociologists, historians, and philosophers of science.

Whedy Wang and Imola Fodor spoke from their industry experiences as leaders in their institutions. Wang is vice president of biometrics at Theravance Biopharma. She discussed the use of p-value in confirmatory clinical trials, which are typically well-controlled phase III studies designed with an understanding of the clinical evidence—including scientific assumptions and effect sizes—generated from previous phases. The review and regulatory approval pathway the FDA established includes the consideration of statistical significance to control the type I errors in such settings. Wang discussed the impact of lowering the p-value threshold, as suggested by Daniel J. Benjamin and James O. Berger in the TAS special issue, on the sample size and development cost for a new product. She also illustrated the kind of clinical evidence beyond p-values reported and considered in regulatory submissions. She suggested that rather than abandoning the statistical significance ship, researchers should focus on proper interpretation of p-values, promoting good statistical practice with the goal of upholding high scientific rigor and integrity—communicating not just the signal of treatment effect but also the uncertainty.

Fodor, global head of oncology biostatistics for early development and the hematology franchise at Genentech/Roche, provided a cross-functional perspective, summarized the discussions in her work community, and shared her recommendations for what a practicing statistician should do: Do not abandon significance; understand its proper use; and communicate clearly and transparently. She also encouraged critical statistical thinking with scientific curiosity, proactive and collaborative communications, and education of the broader community to increase statistical and scientific literacy.

An active 30-minute floor discussion followed the panelist presentations. Attendees, mostly practicing statisticians in research institutes and health care industries, shared their stories and sought advice from the panelists. The event wrapped up after more than two hours of discussion.