Missile Reliability
During engineering design, you expect to find failure modes. For example, in a missile design, you might fire five missiles and see two failures—say an engine ignition or battery failure. The engineers then try to fix the failures. After the fixes are incorporated, how do you calculate the reliability?

To simplify, the original approach (using the Lloyd method) turns failures into successes if those failures don’t occur again in the next few shots. The method is simple, direct, and easily calculated. It led to a reliability estimate of about 0.93 in our case. The raw score would be the result if the failures were not turned into success, but retained as failures. Both methods are, in some sense, wrong. Using the raw score fails to account for the effort (and possible success) to fix failure modes. Yet, the Lloyd method gives too much credit to the fix, because it does not recognize that observing one failure may prevent, or shadow, the possibility of seeing another.

What we have here is a system that has many failure modes, each with its own failure rate, and the failure rate of the whole system is a combination of all failure rates. When an engineer removes a failure mode, the combined failure rate will change. Common experience is that each failure mode follows a Poisson distribution. So, the growth in reliability is nonhomogeneous Poisson process (more complicated because the events are discrete). The method is not simple, not direct, and not easily calculated.

In this case, the absence of a statistician in the beginning allowed both the government and contractor to be fooled about the reliability of the system, illustrating our point that statisticians have an important role in helping the government determine how best to use the available data.

We also have reached out to the Defense Science Board (DSB), partly through ASA leadership, and have been told that statisticians will be recommended to DSB study panels that require statistical expertise. We continue to push for a statistician to be appointed to DSB.

Outreach to Fellow Statisticians

Recognizing the importance of having more statisticians involved in national security projects to broaden our influence, we have undertaken many activities to educate the community about the many interesting problems and their potential for statisticians. In addition to inviting statisticians to this summer’s meeting, we’ve been organizing JSM sessions that show the breadth of activity in defense and security—from cyber security to biosurveillance to more traditional areas such as reliability. We also are organizing the third Quantitative Methods in Defense and National Security (QMDNS) conference, which was established to link defense problems to statistical solutions. Participants include both defense industry workers and statisticians.

SDNS also established a speakers program, again with funding from an ASA member initiative, to provide speakers and travel funding to universities and colleges interested in hearing a presentation about statistical aspects of national security challenges. We have a dozen people on the speakers list and encourage you to both invite a speaker to your institution and add your name to the list.

Success Stories

While there is much more that statistics can contribute to this field, statisticians have made an impact on national security problems. One example is a major defense acquisition program that ran into reliability problems because of faulty methodology for tracking reliability improvement during development. It was a statistician who laid out the rules for estimating reliability, allowing the program to get back on track (see “Missile Reliability”). Other examples can be found by reading Statistics in Defense and National Security:
The Present–You Need More Than Statistics—You Need the Right Statistics (PDF).

Next Steps and Lessons

Our next steps for outreach to policymakers are to continue and broaden our current activities by reaching more officials and statisticians and revisiting those we’ve already met. For outreach to statisticians, we hope our enthusiasm for the many interesting problems statistics can address in this field was manifest. We welcome your involvement and advice.

As our history indicates, this has been a long road, but we can point to real progress for statistics in national security issues. Clearly, much remains to be done. One lesson we’ve learned is that we need to do a better job of educating decisionmakers about what statisticians do. For those who realize the value of statistics, our experience is that sometimes the statistics is not always done by statisticians. So, we need to communicate more convincingly that statisticians can bring state-of-the-art techniques to a problem, which may yield better solutions faster.

For other sections exploring how to better reach policymakers, we highly recommend using the resources available from the ASA, especially funding through member initiatives.

ASA Science Policy Actions

ASA president sends letter to Secretary of Education Arne Duncan regarding stature and autonomy of the National Center for Education Statistics

ASA president joins leaders of 17 other science organizations in signing letter to all U.S. senators on scientific consensus of climate change science

ASA president sends letter of support for Bureau of Justice Statistics director nominee, James P. Lynch

ASA hosts meeting on election auditing attended by statisticians, election officials, political scientists, computer scientists, and voting advocates