Helping Congress and the administration do what is possible

This column is written to inform ASA members about what the ASA is doing to promote the inclusion of statistics in policymaking and the funding of statistics research. To suggest science policy topics for the ASA to address, contact ASA Director of Science Policy Steve Pierson at pierson@amstat.org.

Contributing Editors
Clifford Spiegelman is a distinguished professor of statistics at Texas A&M, where he has been on the faculty for 24 years. He is also a senior research scientist at the Texas Transportation Institute. His applied research interests include chemometrics, transportation statistics, environmetrics, and statistical forensics.






Adina Schwartz is a professor in the department of law, police science, and criminal justice administration at John Jay College of Criminal Justice in the City University of New York. Her research interests include the scientific underpinnings of the forensic identification sciences, standards for the admission of expert testimony, and cyber-surveillance law.




Kate Philpott holds a BS in molecular biology from The College of William and Mary, an MS in forensic science from Virginia Commonwealth University, and a JD from the University of Virginia. She has worked for the state and federal governments and the defense community in legal and scientific capacities. She is currently an independent consultant.

In 2009, a report from a committee of the National Research Council of the National Academy of Sciences found that “no forensic method other than nuclear DNA analysis has been rigorously shown to have the capacity to consistently and with a high degree of certainty support conclusions about ‘individualization’ (more commonly known as ‘matching’ of an unknown item of evidence to a specific known source).” Also in the report Strengthening Forensic Science in the United States: A Path Forward (the “NRC report”) (PDF download), the committee found that many of the hallmarks of good science—strong statistical underpinnings, the identification of sources and rates of error, rigorous and detailed protocols, and well-documented analyses—were lacking in forensic science. These criticisms of forensic science and calls for reform—presaged by Donald Kennedy’s 2003 Science editorial, “Forensic Science: Oxymoron?”—are not merely theoretical, as evidenced by the ever-escalating discovery of cases in which faulty forensics played a role in wrongful convictions and of scandals in forensic laboratories across the country.

The NRC report’s leading recommendation for forensic science reform, which the ASA has endorsed, was for congressional creation of a national institute of forensic science (NIFS), to be independent of all existing federal agencies and, in particular, of the Department of Justice (DOJ). Unfortunately, budgetary difficulties and political realities appear to have stymied the creation of NIFS to spearhead forensic science reform—at least temporarily. Obstacles to creating a NIFS-like entity should not be a barrier, however, to undertaking forensic science reform now. Here, we propose five practical reforms that can and should be implemented.

I. Transparency

The absence of publicly available information has severely hindered assessment of the strengths and weaknesses of the practices of particular laboratories and the scientific underpinnings of forensic science disciplines. Therefore, we propose that for each forensic science discipline (e.g., fingerprint identification, firearms and toolmark identification, postmortem interval determination (time of death estimates)), all federal laboratories and all other laboratories that receive federal funds or grants or use federal databases make the following available on the Internet, free of charge:

• All studies relied on as evidence of the validity of the discipline and/or of the particular procedures that the laboratory uses to apply the discipline
• All laboratory protocols (e.g., standard operating procedures, quality assurance manuals, methods for estimating uncertainty, standards for documentation)
• Records of all proficiency testing of the laboratory’s analysts over the past five years, to include completion rates on tests and reported results and scores on all test questions, as well as the texts of questions, descriptions of test designs, and procedures for administering tests
• All audit findings with regard to the laboratory’s performance over the past five years (e.g., the procedures and standards employed and the decisions reached by any accrediting agency in regard to the initial or continued accreditation of the laboratory, to withdrawing or denying accreditation, or to conditioning accreditation on remedial measures)

These requirements can be met for minimal cost, as most laboratories already have an Internet presence.

II. Evaluation Research

Although the NRC report’s recommendations have been endorsed by distinguished scientists, jurists, academicians, and the ASA, they have encountered substantial resistance from forensic scientists, some of whom have claimed the report did not take proper account of their research. Therefore, we propose that Congress request the National Academy of Sciences or, alternatively, the Office of Science and Technology Policy (OSTP), through the National Science and Technology Committee, to appoint a standing committee(s) to assess the strengths and weaknesses of the validation studies, protocols, proficiency testing, and procedures and standards of accrediting agencies that will be posted on the Internet, in accord with proposal I. Because National Academies panelists volunteer their time and energies, expenses would not be large.

Further, in light of the significant design flaws that the NRC committee found in forensic studies and methodologies, we propose that the majority of the committee membership consists of scientists or experts drawn from the particular scientific and engineering disciplines in which the applied forensic science disciplines are grounded. A minority of the membership should be comprised of representatives of the forensic science disciplines and clinical and/or industrial quality control experts.

III. Identifying Research Needs

The NRC report found that the conclusions reached in most forensic disciplines (e.g., the identification of one particular person as the source of a fragmentary fingerprint or one particular firearm as the one that fired a bullet) are not scientifically justified based on the extant literature. Therefore, we propose that Congress request the National Academy of Sciences or, alternatively, the OSTP to appoint a standing committee(s) composed of a minority of forensic scientists and a majority of research scientists, with no ties to law enforcement and inclusive of a quality assurance expert and a statistician, to identify research needed to test the claims made in each forensic science discipline. The evaluation research to be undertaken in accord with proposal II should assist in the identification of research needs. Since committee members would be volunteers, expenses would not be large.

IV. Long-Term Research

As the NRC report recognized, many of the problems with forensic science are due to the long-standing subordination of scientific needs to those of law enforcement. The funds currently within the Department of Homeland Security (DHS) and DOJ budgets for forensic science research should be tapped to the extent possible to fund such research at independent scientific agencies that are not connected to either DOJ or DHS, with sole responsibility for all decisions in regard to the use of the tapped funds to be vested in such independent scientific agencies.

Additionally, to track the requirement for federal funding of clinical trials that has led to constructive input from the statistical community and better patient outcomes for decades, forensic science research grant applications to any of the federal agencies must be required to include a statistics section that explicitly states a scientifically valid experimental design and specifies a data analysis plan. An inadequate experimental design section should automatically disqualify the proposal for funding.

V. Education

For forensic science to be adequately reformed, judges and attorneys must be equipped to understand the relevant science. Therefore, Congress will encourage all relevant federal agencies (e.g., National Institute of Justice, National Science Foundation, National Institute of Standards and Technology, National Institutes of Health, the Administrative Office of the U.S. Courts) to sponsor seminars and courses to educate judges, defense lawyers, and prosecutors to understand and critically assess the scientific underpinnings and reliability of forensic science disciplines. The majority of the faculty for each program should be researchers drawn from the particular scientific and engineering discipline(s) in which applied forensic science disciplines are grounded.

Conclusion

In recognition of budgetary difficulties and political realities, these proposals keep costs to taxpayers to a minimum. Although their enactment would not solve all problems, we believe the proposed reforms are a doable first step toward ensuring that only first-rate science—beholden only to truth and not to law enforcement or any other partisan interest—is used in the courtroom. The integrity of the American judicial system demands that Congress and the administration speedily enact these reforms.

Science Policy Actions

ASA signs letters in support of budgets for Bureau of Economic Analysis, Energy Information Administration, and the National Science Foundation

ASA signs statement opposing cuts to U.S. Census Bureau, Bureau of Justice Statistics, Economic Research Service, and National Agricultural Statistics Service

ASA supports Office of Government Ethics Proposed Rule to better facilitate the service of federal scientists in professional society leadership

ASA signs letters to senators opposing removal of Senate confirmation of Bureau of Justice Statistics and National Center for Education Statistics heads