Below is a response to the article “Odds Are, It’s Wrong,” published in the March 27 issue of Science News. ASA members wrote the letter to the editor and cited the writer’s failure to clearly distinguish between the misuses of statistics and its methodological limitations. The letter, as edited by the magazine, was printed in the May 8 issue of Science News (Page 32) and is now online. Click here to view that version.
Dear Editor,
Odds are it’s wrong, but the chances that statistics is to blame are slim and fat. Tom Siegfried accurately portrays the importance of statistics in the conduct of science. However, his failure to clearly distinguish between the misuses of statistics and its methodological limitations leads to misleading conclusions about the role of statistics in the proliferation of erroneous scientific results. Furthermore, his characterization of statistics as a mutant form of mathematics rooted in the same principles that guarantee profits for Las Vegas casinos is unscientific at best. It only hinders the realization that statistics is the discipline that is best positioned to contribute to the solution of the problems that he so entertainingly describes.
Statisticians have long recognized the challenges presented by multiple testing, the interpretation of observational data, and, more recently, the analysis of high-dimensional data. Siegfried rightfully acknowledges the many statisticians and biostatisticians who have persistently and repeatedly written eloquently on these issues. He also notes that appropriate methods, such as those for false discovery control, are available to ameliorate the problems. Yet, he curiously persists with the theme that statistics is defective when it is the misuse of statistical methods that is the main culprit in the situations he describes.
Siegfried has fired a shot across the bow of science that, although not perfectly on target, serves as a call for further discussion among statistical scientists and researchers in the relevant disciplines, such as the medical, social, and behavioral sciences. There is a need to educate statistical practitioners at all levels, as gross misuse of statistical methods borders on scientific misconduct. However, it is also important to realize that while statistics usually plays the role of the fall guy in these matters, there are other, more fundamental, factors involved, such as the pressure to publish and obtain funding (positive results sell); the public’s, and hence the media’s, appetite for palatable findings (chocolate is good for your health); and data hoarding (no one can see how thoroughly I searched my data to find a p = .049).
A chisel in a skillful artist’s hand can produce a beautiful sculpture, and a scalpel in an experienced surgeon’s hand can save a person’s life. Similarly, statistical techniques used properly by an honest and knowledgeable scientist can be equally impressive at illuminating complex phenomena, thus promoting scientific understanding and shortening the time between scientific discovery and its impact on societal problems. If misused, they can produce the counterproductive results that Siegfried describes. Such erroneous results, however, should not be viewed as a failing of statistics.
Sastry G. Pantula
President, American Statistical Association
Jef Teugels
President, International Statistical Institute
Len Stefanski
Editor, Journal of the American Statistical Association (Theory and Methods)
Leave a Reply