Rob Santos

I once said to a journalist that if I was on a beach and suddenly observed an approaching tsunami, I wouldn’t need to know the wave height was 23.9 feet, approaching at 30.7 miles per hour—plus or minus 0.5—at a 95 percent level of confidence. All I’d need to know was a huge wave was approaching fast, so I better get the heck out of there with great haste (paraphrasing and embellishing). In fact, anyone on the beach who took the time to collect precise measurements would likely achieve results too late or not at all for them to make an important decision. Fate would intercede.

Similarly, if I wanted to try a different restaurant in Austin, I would likely consult one of the many online crowd-sourced eatery ratings, knowing full well such ratings are nowhere near a true representation of patron satisfaction and some businesses submit their own ratings to bolster their business or disparage the competition. You just keep that in mind when you see the ratings and take your chances.

At the other extreme, a large-scale clinical trial of a COVID vaccine featuring tens of thousands of participants is needed to demonstrate vaccine efficacy and gauge negative side effects. Here, high levels of scientific rigor and statistical precision are necessary to ensure and advance public health.

Turning to the climate, I rely on well-developed—albeit far-from-certain—hurricane projection models to decide how to prepare for an approaching super-storm. I often marvel at the National Hurricane Center’s assorted model projections plotted onto a map to see how congruous or convergent (or squiggly) their trajectories appear. Even when model trajectories vary considerably, sufficient information can be conveyed to allow an effective safety plan to be prepared.

Both sets of examples illustrate uncertainty of various types: systematic error from biases and perturbations from random error. Both also illustrate how we and greater society live with varying levels of uncertainty and quality of information to make decisions affecting our lives. And both exemplify the importance of aligning uncertainty and data quality with our ultimate objectives.

The quality of the information we need is directly tied to the specific task at hand. Experiencing a lousy dinner at a restaurant you chose via potentially unreliable online ratings is admittedly annoying, but it’s not the end of the world. Fleeing a beach because you see a large wave approaching can save your life, even though it is based purely on a single, inexpensive, and potentially imprecise personal observation (i.e., n=1). And the result of a large, high-cost, time-consuming, rigorous clinical trial is well worth the investment of resources and time when it comes to advancing public health. This brings us to one of my favorite topics: the concept of “fitness for use.”

“Fit for use,” aka “fit for purpose,” is critical to the statistical and broader research communities. You don’t need a multi-million-dollar rigorous statistical survey to understand what population segments prefer tea over coffee and why. Well-designed focus groups may suffice. But rigorous, high-cost statistical surveys are needed to estimate the US unemployment rate or the Consumer Price Index. These statistics can literally affect the global economy and thus need to be of the highest quality.

That is why researchers—including statisticians—must align project timelines, available resources, and methodologies to achieve explicit research objectives. Doing so makes a study “fit” for its intended “use” (i.e., that of achieving the research objective). It takes creativity and technical skill to achieve fitness for use; it is an exquisite art!

As an aside, this reminds me of the age-old iron triangle: “Fast; cheap; good: Pick two.” Presumably, it’s impossible to achieve all three, although I suspect there are some rare examples. Note the COVID vaccines were fast and good, albeit very expensive. But getting back to the concept of fit for use, there are inevitably occasions when scientific rigor, time, and resources/methods simply cannot be aligned to achieve the research objective. In such circumstances, it is always good practice to revisit each element to see if an acceptable revision can be made to accommodate the research objective. But when that is not possible, the research objective will need to be modified. Or the study may need to be terminated at the planning stage.

I long ago stopped counting the instances in which potential clients contacted me to conduct a high-quality survey of an important topic, only to be gobsmacked by the requisite cost and timeline. Fortunately, an in-depth conversation often revealed they did not need a high-quality survey to address their research question. The insights they sought could be gained in a timely fashion with a different, inexpensive research approach.

A great example of this involved the collection of member feedback on personal experiences of sexual assault and harassment at ASA events by the Task Force on Sexual Harassment and Assault. I served as a member of this group. I and others quickly advised against conducting a scientific sample survey because of the almost certain low participation rate and high cost associated with a sensitive-topic study. And, in fact, this was not necessary. It was only necessary to document that harassment/assault experiences befell some of our members, as well as the range of scenarios and manifestations in which these appalling interactions occurred. Those could be collected by simply providing all association members an opportunity for confidential feedback. Many members chose to participate using a feedback form developed by the task force members. Thus, the task force gathered the needed data to garner findings, reach conclusions, and make recommendations.

Being outside of academia for decades, I am unaware of the extent to which the concept of fit for use has found a place in the statistical ­curriculum. I hope it has, because it is foundational to the design of effective research and knowledge gain. In fact, I’d say invoking the concept of fit for use is one of my secrets to success. Try it out and see for yourself!

In closing, I will note with sadness that this is my final President’s Corner column. I hope you enjoyed reading these as much as I enjoyed writing them. I worried I would need to stop earlier in the year due to my nomination for US Census Bureau Director. But I will begin this new chapter in January 2022, at which time I will regrettably resign from the ASA Board.

The opportunity afforded by these columns allowed me to promote what I thought was most important this year—sharing ourselves to build community. We are all part of an amazing statistical community and deserve to help and support each other. Please remember that.

It’s been a privilege and honor to serve you as president. I leave you in the superb hands of the board, each member of whom is an exceptional leader. I especially thank Executive Director Ron Wasserstein, Donna LaLonde, and the entire ASA staff for their untiring support. And special thanks to incoming president, Kathy Ensor, who took on additional board duties once my nomination became public. Finally, I thank you, our beloved ASA membership, for your volunteerism (i.e., sections, chapters, outreach groups, caucuses, etc.) and your support of each other during these tumultuous times of COVID. I will always hold fond memories of my time serving you.