Mark C. Otto and MinJae Lee
If you’re racing at 90 miles per hour perpendicular to your goal, how much progress are you making? Before anything else—before the analysis, the model, or the answer—you must be sure you are solving the right problem.
Take the Fish and Wildlife Service, which issues construction permits designed to protect and minimize impact on nesting bald eagles. A half mile from a nest is considered “safe.” Closer than 330 feet is a problem. And between 330 feet and a half mile is the uncertain zone, where most permitting decisions are made.
Our statistician’s instinct is to dive in and determine what happens in that uncertain zone. But a study would take years, and in the meantime, permits can only be issued beyond the safe distance mark, cutting off construction entirely in dense nesting areas. But what if we framed the problem differently? What if we issued permits within the uncertain zone, on the condition that the builder is responsible for any reported nest failures? As distances and covariates are recorded and permit thresholds are refined, both the eagles and builders get the security they need. The model can come later. The decision moves now.
Framing the problem is the first step in the PrOACT-structured decision-makxing process, as described in Smart Choices, the 1999 book by John Hammond, Ralph Keeney, and Howard Raiffa. It’s also described in a short video from the Alliance for Decision Education. Many decisions that arrive on our desks come as problems imposed by a boss, client, or deadline. The PrOACT move: Reframe them as opportunities.
In pharmaceuticals, we are often given a clinical trial to design, but if we work closer with physicians as a project takes shape, we can help develop the actual drug rather than just conduct analysis. This happens routinely in collaborative research. Investigators often begin by asking, “What statistical method should I use?” or “How many participants do I need?” Those are important questions, but they’re rarely the first questions. The conversation becomes more productive when we step back and ask, “What scientific question are we trying to answer?” or “What decision will this study ultimately inform?” Once that objective is clear, the study design, analysis plan, and sample size often become much easier to define.
… Framing is one of the most valuable and often overlooked contributions statisticians make to collaborative research. Frame first, then refine.”
Statistical methods should follow the scientific question, not define it. In that sense, framing is one of the most valuable and often overlooked contributions statisticians make to collaborative research. Frame first, then refine.
The FWS was once asked to develop a permitting process for wind power facilities that would prevent harm to eagle populations. The team responded with population models on the impact of wind projects on eagles, and they asked wind companies to provide additional information. With this data, they adapted a complex model to the size of the project, local eagle density, and expected mortality. They also evaluated constraints. Concern was greater for golden eagles, whose populations were stable under current pressures, whereas bald eagle populations were growing exponentially. Two years of surveys were required for each permit, and the mortality model proved fairly accurate. But wind power companies were also responsible for mortality of bats and other birds, so the two-year survey requirement was costly.
What have others done? Rhode Island’s Ocean Special Area Management Plan was the first to develop offshore wind in the US. They proactively mapped where power was needed, wind patterns, shipping lanes, and bird population locations. From that rich GIS and statistical analysis, they defined where offshore projects could be permitted. The major economic and environmental work was already done, so companies could apply directly for permits without having to conduct their own studies. Rhode Island treated the problem as an opportunity and turned it into one.
This is what statisticians are good at: deciding what the population is, how to measure it, what is being compared, and which assumptions are doing the work. These are framing skills disguised as study-design skills, and they belong in the room from the beginning—not just at the analysis stage. Many of the statistical consultations we participate in are really conversations about framing the decision before choosing the analysis. The decision-maker can help shape the frame, too. The US government is often a decision-maker, and it can bring together opposing sides of a problem to work through the SDM process and determine the best solution for affected parties. As kids intuitively know, parents make the decisions, so they propose problems that work for their parents, too. “You can shop while we meet our friends at the mall.” For statisticians, framing is not a new skill to learn—it’s something we do every day. The opportunity is to recognize it earlier, before the discussion turns to methods and models.
Next time, we’ll work through the rest of the PrOACT process—objectives, alternatives, consequences, and trade-offs—and see how each, like framing, is a skill that statisticians already use.
Mark Otto
MinJae Lee
Deciding Factors, a column by Mark Otto, explores decision analysis and how statistical scientists can apply their expertise to make better choices in their careers, communities, and personal lives.
Leave a Reply