JSM 2014 Commentary by Tyson Lee

Boston (nicknamed the Hub) welcomed more than 6,600 registered JSM (Joint Statistical Meetings) participants from throughout the world and recorded the second-largest JSM in history (to one previously held in Washington, DC).

Tyson Lee describes his experience at JSM 2014 en route UA flight 1699 from Boston to San Francisco on August 7, 2014.

Waltzing through the statistically minded, one can sense the heat and excitement with jargon such as “OLS,” “lasso,” “GPS,” “MAP,” “bootstrapping,” and “jackknifing” dancing in the air. This year celebrated the ASA’s 175th anniversary, which brought more festivity to an otherwise (stereotyped) uptight crowd. Bayesians joyfully declared that they don’t P anymore while brave frequentists sang love songs to their statistically significant others using such romantic phrases as “highly correlated,” “zero degree of freedom,” and “confidence interval.” Statistical ideas sparkled in leaps and bounces.

What is statistics? The question was first asked in 1839. Statistics presented different faces to different sciences. Can it be a science of itself? Steve Stigler from The University of Chicago, a longtime ASA Fellow, presented a keynote speech titled “7 Pillars of Statistical Thinking.” In his 30 years of research on the history of statistics, he summarized key statistical thinking into seven pillars:

1. Aggregation (means – first appeared in year 1635; least square mean – year 1805)
2. Information (Central Limit Theorem – year 1930)
3. Likelihood (MLEs, testing, p-value, Bayes, etc. – dated back to year 1763)
4. Inter-Comparison (%, t-test)
5. Regression
6. Design (ANOVA, design-based inference)
7. Residuals – year 1831

‘Misinformation’ Age

Presenters reported that with the huge amount of data available today, fallacies and misuse of statistics also run rampant. Kellogg lost a $4M lawsuit based on a biased study claiming “breakfast cereal boosts boys’ learning ability,” in which the control group did not receive any breakfast at all! Another study claimed that “wide-faced baseball players yielded higher batting averages,” failing to acknowledge the possible common causality from steroids use!!! Conflict of interest and lack of real control were the root causes of many problems. Has the information age become the misinformation age?! Statistical education is critical for today’s work force.

Missing Data Mitigation and Handling

Carol Robertson-Plouch from Eli Lilly presented “Mitigating Missing Data Risks,” quoting several NDA submissions rejected by the FDA due to substantial missing data. Is it

“M ss Data” or
“Messy Data” or
“Moss Data”?

You get the idea!

Missing data make it difficult to interpret the study results. Absence of data that were intended to be collected led to possibility of incorrect conclusions and potential non-approvability. Should the attitude be “let the horses out of the barn” and “let cowboys/cowgirls (statisticians) fix it”? In many cases, it is too late to fix it. The right approach is to “build a better barn door” by proactively minimizing occurrences of missing data. We all know that “an ounce of prevention is worth a pound of cure,” don’t we?

Other presenters discussed different types of missing data—MCAR (missing completely at random), MAR, and MNAR (missing not at random). Completer and single imputation analyses are okay for MCAR. Pattern mixture method can be used for MAR and MNAR. Thomas Permutt also advocated prevention and urged to try very hard. He suggested only using elegant statistical methods when all prevention efforts fail. He presented an approach of composite endpoint using discontinuation as an outcome.

Ralph D’Agostino presented eight rules of missing data handling. A best method is “simple, robust and assumption free,” he argued. It needs to be explainable to nonstatisticians. When mechanism of missingness is unknown, LOCF is notoriously bad. LOCF may be okay if patients improve over time, but it needs strong justification. FDA seldom granted such justifications, as Boguang Zhen from the FDA admitted. He championed that the best way to handle missing data is “not to have any.” He emphasized two key words on missing data handling for sponsors in their submission packages: minimization and justification. More reading is available.

Data Standards: ADaM (Analysis Data Model)

Steve Wilson, from FDA, presented challenges and progress on using ADaM. He saw two major trends following the recent approval of the PDUFA V by Congress:

• Binding guidance by year 2017
• Development of Therapeutic Area (TA) standards

FDA encourages “socializing deliverables” (in a social network paradigm) and “transcelerating” drug development by providing more standardized data. He proposed a “data standardization plan” analogous to the well-known SAP (Statistical Analysis Plan) to emphasize data traceability. He mentioned that CSS (Computer Science Symposium) published an Analysis Data Reviewer’s Guide and got on board with tools. Several industry practitioners also presented their perspectives on data standards as a tool to close the project management triangle gap.

All in all, JSM 2014 was a statistical banquet. I was immensely humbled by the vast knowledge base at the conference.

As my return flight took off from the Cradle of Liberty, my vision became clearer (despite hypoxia) at the high altitude over the clouds. In this world of omnipresence of variability, one thing is for certain: The Big Data age has dawned upon us with its infinite possibilities. This is an exciting age, and it is up to us to break free of the old dogma, raise the level of statistical literacy, take actions, and innovate.

Editor’s Note: Drafted en route UA flight 1699 from Boston to San Francisco on August 7, 2014. Tyson Lee is an executive director at Fibrogen, Inc. Comments can be sent to tysontlee@gmail.com. All rights reserved.