A while back, one of us interviewed a junior candidate for a faculty position. This person showed a lot of promise, had a novel research agenda, and obviously was going to impact the profession in nontraditional ways. However, given current faculty evaluation guidelines, we could not make an offer in good conscience because their contributions would not be fairly rewarded. We don’t think this is an isolated problem.
Academia seems to have an increasing emphasis on a particular productivity—a focus on publishing many papers, with high prestige accorded to a narrow set of journals. This has several consequences:
- It places stress on faculty, particularly junior faculty who don’t yet have tenure.
- Researchers who think and write rapidly and work on many projects at once are admired, while researchers who focus on relatively few ideas may be pressured into becoming more like the first category.
- Counting the number of publications in prestigious journals makes things more difficult for faculty who don’t work on the research prioritized by those journals.
- Many lack the connections or the resources, usually funding and personnel, that help with having many publications in prestigious journals.
Giving faculty the ability to work in ways that suit them the best may help make academia more appealing, especially considering increased competition from industry. It is important to recognize there are many new ways to make contributions. Thinking carefully through these issues will impact promotion and tenure, graduate students’ theses, and junior faculty hiring.
We have several suggestions. For evaluations, have faculty submit, at most, some fixed (small) number of research manuscripts that best represent their research.
In addition, we should better understand our community’s strengths and weaknesses in evaluating work. For example, as a community, we’re good at deciding whether theorems are interesting, whether a proof is standard or requires ingenuity, whether a result has a large scope or small, and so on. We’re also good at recognizing whether a decision has been made to write a collection of papers that differ modestly as opposed to a smaller number of ‘bigger’ papers.
We are not as good at evaluating contributions that are distant from our communal training, so we should develop better procedures for evaluating an individual’s contribution to joint work and evaluating work that touches other fields.
We should ask letter writers to address only the scholarly contributions each paper makes. It’s good to ask whether the small set of publications to be evaluated represents a substantial intellectual contribution since this is the productivity we want to evaluate. It is fair to assess whether the contribution compares favorably to expectations in the discipline and perhaps to other individuals. Letter writers should be encouraged to not focus on extraneous proxies for quality such as the prestige of the journal in which the work appeared, grants related to the work, collaborators involved, and institutions involved. Department committees can assess these proxies, and external letters that rehash a vita add little to evaluation.
The scholarly contributions should allow for important technical contributions, impactful algorithm and software development, and contributions to other scientific domains (research driven by questions from other disciplines). Ideally, researchers should also be allowed to study esoteric subjects, provided they are doing something thoughtful and intellectually serious. While citation counts can be seen as a plus, the value of the research should not be equated to citation counts.
We regard the improvement of evaluation of faculty research as crucial given that modern statistics is a much more diverse discipline than it was just a decade ago, and it is important for us to respond to new challenges. We hope our suggestions will catalyze further discussion.
Bertrand Clarke has been head of the department of statistics at the University of Nebraska for 11 years. Before that, he worked at the University of Miami and University of British Columbia as a full professor. He earned his PhD in statistics at the University of Illinois in 1989.
Murali Haran has been head of the department of statistics at The Pennsylvania State University since 2018. He earned his BS in computer science at Carnegie Mellon University in 1997 and his PhD in statistics at the University of Minnesota in 2003. He has been on the faculty at Penn State Statistics since 2004.
Galin L. Jones is the Dr. Lynn Y. S. Lin Professor of Statistics, director of the school of statistics at the University of Minnesota, and co-chair of the University of Minnesota Data Science Initiative. Jones is a Fellow of the American Statistical Association and the Institute for Mathematical Statistics and co-editor of the Journal of Computational and Graphical Statistics.
Steve MacEachern is distinguished arts and sciences professor of statistics at The Ohio State University. He holds a courtesy appointment as professor of psychology. MacEachern earned his BA in mathematics from Carleton College and his PhD in statistics from the University of Minnesota. He served as chair of statistics at Ohio State from 2015 to 2023.
Leave a Reply