Home » Cover Story

ASA Forensic Science Committee Celebrates 10 Years

2 January 2023 966 views No Comment

Last fall marked 10 years since the ASA Board of Directors created an ad hoc committee on forensic science. The Advisory Committee on Forensic Science’s creation recognized and formalized the ongoing critical work of ASA members to ensure a strong statistical perspective in the forensic science reform effort energized by the 2009 National Academy of Sciences report, Strengthening Forensic Science in the United States: A Path Forward.

With a charge to “advise the ASA on forensic science reform issues and to ensure that statistics and statisticians maintain a visible profile in the forensic sciences and forensic science reform,” the committee and broader statistical community have been integrally involved in the broad efforts to strengthen the forensic science disciplines.

The committee asked current and former members to reflect on the progress made, contributions by the statistical community, and priorities for future work. What follows is a compilation of responses from Maria Cuellar, current committee chair and an assistant professor at the University of Pennsylvania; Danica Ommen, current committee vice chair and an assistant professor at Iowa State University; Alicia Carriquiry, distinguished professor and president’s chair at Iowa State University and director of the Center for Statistics and Applications in Forensic Evidence; Karen Kafadar, commonwealth professor and former chair of statistics and co-director of the Center for Statistics and Applications in Forensic Evidence at the University of Virginia; Julia Mortera, a professor at Università Roma Tre; Chris Saunders, a professor at South Dakota State University; and Hal Stern, chancellor’s professor and co-director of the Center for Statistics and Applications in Forensic Evidence at the University of California, Irvine.

Statistical Advances in Strengthening Forensic Science

Strengthening Forensic Science in the United States: A Path Forward explicitly identified the need for statisticians to be involved in developing the statistical foundations of forensic science techniques, leading to greater involvement of the statistics community in ongoing efforts to create standards for forensic science and forensic science research.

Statisticians are conducting research they believe is missing from the current forensic science literature, including constructing algorithms that use statistical analyses to evaluate evidence. When research is conducted in collaboration with forensic practitioners, the chances that the new methods are useful and used in real casework are much improved.

In the specific case of pattern comparison disciplines (e.g., fingerprint analysis, firearms analysis, footwear analysis), the use of statistical learning algorithms to extract information from images and quantify similarities and dissimilarities between two items has led to the development of objective data-based methods that have the potential to address some of the shortcomings of subjective approaches that rely on visual comparisons and experience.

The gradual acceptance of a likelihood ratio framework for evaluating the weight of evidence in favor of one of two propositions under consideration also represents a step forward. Even though wide adoption of likelihood ratios in the United States is still in the future, forensic scientists and legal professionals have begun to realize the importance of thinking about the chances of observing the evidence under different scenarios.

Several universities have taken the initiative to train the next generation of statisticians and scientists on how statistical concepts apply to forensic science disciplines. New courses and programs have been developed, along with research experiences that prepare students for careers in forensic science, criminal justice, or related fields.

Contributions to the Forensic Science Community

The committee has already had a significant effect on the forensic science community by focusing on two areas: providing a mechanism for communicating the role of statistics and statisticians in ongoing attempts to improve forensic science and publicizing the needs of this important science policy domain within the statistics field.

The committee has played a large role ‘behind the scenes’ to ensure funding for high-quality forensic statistics research is available and has advocated for the appropriate use of statistical language and methods. The committee has strived to develop meaningful relationships with many federal agencies, including the FBI, National Institute of Standards and Technology, and National Institute of Justice, as well as with local forensic laboratories. The committee has written public statements such as the ASA Position on Statistical Statements for Forensic Evidence and letters and statements to Congress and the Department of Justice. Members of the committee have also interacted with congressional staff members as they consider legislation and funding bills to support the forensic science enterprise.

Additionally, the committee has made efforts such as writing articles for Amstat News and sponsoring sessions at the Joint Statistical Meetings to draw statisticians’ attention to the need for research in this area and the impact their research can have.

Overall, the statistics community has contributed significantly to forensic science improvement efforts, especially in research collaborations. One noteworthy effort is the National Institute of Standards and Technology–funded Center for Statistical Applications in Forensic Evidence. Headquartered at Iowa State University, the center is a multi-university consortium that has provided statisticians with the means and a supportive, collaborative environment to improve the accuracy of the analysis and interpretation of forensic evidence. CSAFE has reached thousands of forensic practitioners worldwide through its training and education program.

The statistics community has played an important role in communicating to many in the forensic science and legal communities the importance of probabilistic thinking when analyzing and interpreting evidence. Notions such as uncertainty, reproducibility, repeatability, and coincidental matches were largely absent in forensic practice, but statisticians have successfully communicated the central role they must play in any forensic analysis.

Challenges for Statisticians and Researchers

Even though significant research and infrastructure progress has been made in the last decade, much work remains. We’ve identified some of the major challenges statisticians and researchers continue to face.

Committee members recruited new members to their ranks, resulting in a growing number of statisticians working on forensic science issues. However, there are more problems to deal with than there are statisticians to work on them. A challenge moving forward is finding statisticians and researchers to continue working on these issues. In addition, funding levels to conduct research at the interface of statistics and forensic science remain low, limiting the progress that can be made. Additional research investments by the federal government would have direct benefits to forensic science and the indirect benefit of attracting more researchers into the field.

What is the source of the evidence found at the crime scene? The “question of source,” determining whether a crime scene sample and suspect sample come from the same source, is one of the fundamental forensic science problems that distinguishes forensic science from other scientific disciplines, and the lack of statistical infrastructure to support this problem is problematic. While statisticians may try to place this problem within the context of statistical learning or model selection problems, it is important—as with any applied problem—for statisticians to understand the forensic practitioner’s needs and where statistics and probability will help. This kind of understanding can take time to develop, but until the forensic statistics community truly listens to forensic scientists, we risk solving the wrong problem.

Even now, there is a significant gap between the views of many in the forensic practitioner community and the views of statisticians and other scientists engaging in forensic science research. Understandably, practitioners do not want to hear their way of doing analyses can be improved or, worse, that they may be wrong. Practitioners may fear losing some degree of reputation for their discipline. Yet, effective collaboration requires forensic practitioners be open to new approaches to analyzing and interpreting evidence and willing to introduce more quantitative methods into their practice. Statisticians need to communicate clearly that the target of research is an improvement of methods and processes and not the performance of individual forensic analysts.

Statisticians also have much to learn about technology transfer. What is the best way to introduce new technologies into forensic laboratories? Challenges are numerous. Practitioners may be resistant to changing their current practice and trying new methods. They may be reluctant to trust an algorithm they need help understanding, has not been extensively tested, or may rely on hidden and untestable assumptions. Some research in medicine has suggested that even when practitioners see for themselves the high performance of an algorithm, they still do not want to use it. In this light, it is essential that algorithms proposed for the criminal justice system be designed with fairness, accountability, and transparency in mind.

A common concern of practitioners is that quantitative approaches are being designed to replace human decision-making with algorithmic decision-making. While we know this is not true, it once more reflects that we statisticians need to be more effective at communicating what we are trying to achieve. It is critically important to understand barriers to adopting new quantitative technologies in forensic practice and determine the best way to use algorithmic results to guide human decision-making.

Communication challenges also arise in the courtroom when statisticians report the findings of statistical analyses to law professionals and lay members of the jury. Models are often presented in both the investigative phase and in court. It is difficult to describe statistical evidence and complex models to nonexperts in clear and understandable terms. Likelihood ratios are easily understood by statisticians but rarely by others. Statisticians must improve communication skills while delivering essential basic quantitative training in convenient formats to all those involved in criminal trials.

Recommendations for Moving Forward

The fair administration of justice, both to prevent the incarceration of innocent people and ensure the guilty are caught and punished, requires contributions from lawyers, judges, forensic practitioners, and scientists of all types. Statisticians’ expertise in study design, data analysis, data visualization, and data interpretation and reporting has much to offer in improving forensic science practice. We have provided a few recommendations that, if implemented, could go a long way to positively affect the criminal justice system.

As statisticians, we must continue to engage and partner with the forensic science community to identify pressing problems and work toward practical solutions that can be implemented. Increased research funding and activity are needed, including fundamental statistical research that can provide the foundation for most forensic disciplines. Researchers should also design realistic studies that honestly replicate the conditions under which forensic practitioners work, and any forensic method being used or could be used in the future should be tested using sound evaluation procedures.

We must also look at ways to train top-tier statistics students to work with forensic practitioners and legal professionals to solve the issues in analyzing and interpreting forensic evidence. Developing university courses and encouraging them to get involved in professional organizations would help interested students have opportunities to research important forensic matters.

Along with increased research activities comes the need for statisticians to find better ways to communicate our findings to not only forensic practitioners but also to judges, lawyers, and the general public. We must create publicly available resources the forensic and legal communities can use and share, including databases, code, and training opportunities.

If you have ideas or would like to get involved, contact ASA Director of Science Policy Steve Pierson.

Learn about activities informed by the Advisory Committee on Forensic Science.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Comments are closed.