Mark C. Otto and MinJae Lee
Alice, lost in Wonderland, asks the Cheshire Cat which way she ought to go. The Cat replies that it depends a good deal on where she wants to get to. Alice doesn’t much care. “Then,” says the Cat, “it doesn’t matter which way you go.” Values tell you where you want to get to. Without them, we are just spinning the bottle. Objectives turn those values into directions, and the rest of this article focuses on them.
Consider a rough decomposition offered by Spencer Greenberg and Jeremy Stevenson in The 12 Levers: Productivity is the product of time, efficiency, and importance. Time is bounded—24 hours a day—with less available for productive work and for avoiding burnout. Efficiency is bounded—you can raise it, but not above one. Importance has no ceiling. Working twice as long or twice as focused on the wrong problem doesn’t bring you any closer to a solution.
Importance is where the leverage lives, and importance is set by objectives. That framing should register with any statistician: Two of the three factors are capped, and the third is not. Objectives set the third factor. In research, this is a reminder that statistical efficiency cannot compensate for asking the wrong scientific question.
Ralph Keeney’s central argument in Value-Focused Thinking is that most people start a decision by considering the alternatives they can see and then figure out which is best. He suggests going the other way: Start with objectives and let them suggest alternatives you would not have generated on your own. This is not a rhetorical flourish. In Keeney’s work, groups asked to list objectives before alternatives generated roughly 25% more alternatives than those that started with alternatives. The car story from last month makes the mechanism visible: Once safety, cargo space, and neighborhood walkability are added to the objective list, alternatives that had not been on the table (the Subaru, the minivan, the no-car option) come into view. Alternatives are limited by what you can imagine. Objectives more easily expand what you can imagine.
In Smart Choices, Keeney and his coauthors distinguish between means objectives and fundamental objectives. The way to separate them is to ask why—up to five times, or until no other reason comes to mind. In the eagle permitting work we discussed two articles ago, “maintain the eagle population” was a means objective. Ask why: because we want the species to survive and nothing else comes to mind. That is a fundamental objective. Only fundamentals should be used to evaluate alternatives; means are the objectives you use to get there. Write each in verb-object form—minimize cost, maximize coverage, reduce risk. That form will feel familiar to statisticians because it is the same form we already use to write objective functions: a verb (minimize) and an object (residual sum of squares). Value-focused thinking is objective-function thinking, moved from the model to the decision. The analogy is not exact, but the mindset is familiar: First define what we are trying to optimize, then determine how best to get there.
Using a partial list of objectives can fail in two ways. The first is missing an objective—a value nobody said out loud, yet it decides the outcome anyway. The classic case is the family that says it wants a comfortable house. They live happily in their comfortable home until they sell. The neighborhood did not become more appealing over time, and they could not afford a nicer home next. Hidden objectives, such as resale appreciation, are often not discovered until well after the fact. The second failure is the systematic omission—forgetting cost, for example, so every alternative looks affordable, and the family goes for pie-in-the-sky. The correction for both is to ask other people. Keeney consistently found that groups that consulted outsiders identified more objectives than those that did not, and the objectives identified later were often the ones that determined the choice. Cost is almost always an important limiting objective.
Another set of objectives comes from outside the problem; a company has strategic objectives that underlie all its decisions. In one of Keeney’s examples, Southern California Edison values low-cost, reliable power service, greater use of renewable energy, high productivity, rewarding good employees, and improving social and economic conditions. You have values in your own life, such as integrity and trust-building, that you apply to all your personal decisions. We will learn more about these values when we discuss rationality and habits.
The same holds across research—every study, every model comparison, and every research question is an implicit choice about what to measure and what to declare as an outcome. These objectives determine the trial’s direction and success. Structured Decision Making asks us to make them explicit, in the room, with the stakeholders whose decision will be evaluated against the objectives we settle on. In a clinical trial, the choice of primary endpoint reflects the study objectives, and once that choice is on paper, the sample size, the analysis plan, and the interpretation all flow from it. In collaborative research, investigators often come to statisticians with an endpoint already in mind. But before asking how to analyze that endpoint, it is worth asking why it matters and whether it truly represents the study’s fundamental objective. A convenient or familiar endpoint may be a means objective rather than the study’s fundamental objective. Get that right first.
Objectives, once we have them, are usually not equally important—we will address how to weight them after we generate alternatives, which we will do next time. You will see, as with objectives, that the first alternatives that come to mind are often not the best.
Mark Otto
MinJae Lee
Deciding Factors, a column by Mark Otto, explores decision analysis and how statistical scientists can apply their expertise to make better choices in their careers, communities, and personal lives.
Leave a Reply