SPAIG Finds Statistical Community Slightly Shaken, Not Greatly Stirred Regarding Big Data

Barry Nussbaum, SPAIG Committee Chair

In last October’s issue of Amstat News, the SPAIG Committee announced its intention to look at two pervasive concerns of the statistical community: Big Data and declining response.

SPAIG (Statistical Partnership among Academia, Industry, and Government), by its very name, is concerned with the collaborative efforts of academe, industry, and government. As Big Data concerns seem to encroach upon the statistical field, SPAIG is concerned with statisticians’ involvement with the computer experts who are increasingly collecting and analyzing Big Data.

So, our committee, with the help of additional ASA members who responded to our October notice, distributed a pilot set of questions. As with most pilots, we started with a small, nonscientific set of two questions for a handful of statisticians, balanced across academe, industry, and government. The questions (forming probably the smallest questionnaire you’ll ever see) were:

1. I think big data are (answer all that apply) –

a. A real threat to the statistics community (y/n)
b. A major innovation requiring advanced computer skills (y/n)
c. Just another passing fad (y/n) 

COMMENTS: 

2. In response to big data (answer all that apply) –

a. My office has done nothing yet (y/n)
b. My office has assigned big data projects to our computer whiz kids (y/n)
c. I am screaming that they better be good statisticians too (y/n) 

COMMENTS:

And what did we learn? The questions were certainly on people’s minds and appeared salient. The statistics community is concerned about Big Data, convinced it is not just a passing fad, but sees no cause to get head over heels excited either. The folks who offered comments were somewhat more diverse, with remarks ranging from “no threat to statistics” to “disciplines will move on without our help.” Some comments noted Big Data provides “more opportunities (for statisticians)” and suggested that proactively “we should jump on the bandwagon in collaborating with other disciplines.”

Well, collaboration is what the SPAIG committee, and indeed statistics, are all about. Our next step will be to formalize our little survey and get a more representative pulse of our profession. It’s time to get into the sandbox with the Big Data gurus. Many of you already have!

Contact Barry Nussbaum, SPAIG Committee chair, at Nussbaum.barry@epa.gov or (202) 566-1493 with questions, comments, or concerns.