Obituary: Arthur Pentland Dempster

Submitted by Ruobin Gong, Nan Laird, and the Dempster Family

Arthur Pentland Dempster, professor emeritus of theoretical statistics at Harvard University and a founding member of its department of statistics, died on January 30, 2026, at the age of 96. Over nearly seven decades of scholarship, Dempster shaped the foundations of statistical inference through a series of path-breaking research programs, including a theory of upper and lower probabilities (now widely known as Dempster-Shafer theory), the EM algorithm, and a geometric reformulation of multivariate analysis. Dempster’s contributions have left a permanent mark on the discipline and adjacent fields ranging from machine learning to artificial intelligence.

Dempster was born in Toronto in 1929 to Nelson J. Dempster, a World War I veteran, and Jessie MacAlpine, a pianist. They had three sons: John; Arthur; and Nelson. Arthur’s early interest in science and mathematics was influenced by his uncle and namesake, Arthur Jeffrey Dempster, The University of Chicago physicist whose pioneering work in mass spectrometry led to the discovery of uranium-235.

Dempster attended the University of Toronto, where he was a Putnam Fellow in 1951 and earned a BA in mathematics and physics in 1952. He entered Princeton University in 1954 and completed a PhD in mathematical statistics in 1956. His first sustained contact with the statistical community came at the summer statistical seminar held by the University of Connecticut, organized by Geoffrey Beall and John W. Tukey and attended by Paul Meier, Jerome Cornfield, David Blackwell, and other figures who would shape the discipline. At Princeton, Dempster studied under Sam Wilks’ statistics program and participated regularly in Tukey’s afternoon seminars, where applied problems brought by visitors were systematically analyzed before an audience of graduate students. His dissertation treated multivariate problems in which the number of variables exceeds the sample size, a regime now central to high-dimensional statistics.

It was at Princeton that Dempster met Elizabeth O’Neill; they married in January 1957.

Dempster spent 1956–1957 as a lecturer at the University of Toronto. In 1957–1958 he held an appointment in the mathematics research group at Bell Telephone Laboratories. At Bell Labs, Dempster was exposed to scientific computing and participated in an early large-scale data project to computerize personnel records for the entire Bell System. It was also at Bell Labs that Dempster developed his theoretical work on random allocation designs, published as a pair of papers in the Annals of Mathematical Statistics.

In early 1957, Frederick Mosteller recruited Dempster to the newly established department of statistics at Harvard. Dempster joined as assistant professor in 1958, shortly after the other founding faculty (Mosteller, William G. Cochran, John W. Pratt, and Howard Raiffa) took up their appointments. He was promoted to associate professor in 1961 and to full professor in 1964. He served as chair of the department for a total of 13 years between 1969 and 1985 and remained on the active faculty until his transition to emeritus status in 2005.

Dempster’s first major research program continued the multivariate themes of his thesis. Throughout the 1960s, he taught a graduate course on multivariate analysis with a distinctive geometric and computational orientation. Dempster developed this material into the book Elements of Continuous Multivariate Analysis. His brief note in the Annals on a paradox concerning inference about a covariance matrix was privately regarded by Dempster as among his most important contributions. The work anticipated by decades the difficulties of inferring high-dimensional covariance structures from small samples, a problem to which he would return in his later work on belief functions.

In the early 1960s, while immersed in R. A. Fisher’s Statistical Methods and Scientific Inference, Dempster began to appreciate the conceptual allure of Fisher’s fiducial inference and became convinced it was technically deficient. His response was a sequence of papers published between 1966 and 1968, in which he introduced a new framework for probabilistic reasoning under partial information. “New Methods for Reasoning Towards Posterior Distributions Based on Sample Data” and “Upper and Lower Probabilities Induced by a Multivalued Mapping” established the mathematical apparatus of random sets and multivalued mappings as a generalization of ordinary probability. A companion paper extended the framework to inference from finite populations. The synthesis appeared as “A Generalization of Bayesian Inference” in the Journal of the Royal Statistical Society, Series B, which made explicit the relationship between the new framework and Bayesian posteriors. A further paper that same year treated upper and lower probabilities generated by random closed intervals.

The mathematical apparatus of these papers was motivated initially by Fisher’s inability to extend fiducial reasoning to discrete data, beginning with the binomial distribution. The conceptual core was an explicit recognition of a third category of probability mass beyond “for” and “against” any assertion: the probability of “don’t know.” Dempster delivered an early account of the broader subjectivist program at the 1965 International Statistical Institute meetings in Warsaw under the title “A Subjectivist Look at Robustness,” a presentation that drew an appreciative letter from Bruno de Finetti. Glenn Shafer’s 1976 monograph, A Mathematical Theory of Evidence, reformulated Dempster’s theory of upper and lower probabilities. Dempster praised Shafer’s monograph for “freeing [Dempster’s theory] from the narrow statistical confines of random sampling and re-expressing it in terms of more general relevance as belief, support, and evidence.” Their contributions combined, now widely known as the Dempster-Shafer theory, became one of the principal alternative formalisms for reasoning under uncertainty widely employed in artificial intelligence and engineering contexts.

In the early 1970s Dempster began collaborating with Donald B. Rubin (then at the Educational Testing Service) on missing data problems and with Nan M. Laird (then at the Harvard Department of Statistics) on random-effects models. Their collaboration culminated in a landmark paper, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” published in the Journal of the Royal Statistical Society, Series B, with discussion. The paper formulated a general computational repertoire, namely an “expectation” step followed by a “maximization” step iterated until convergence, and demonstrated how it may be applied broadly to obtain maximum likelihood estimation in the presence of a broad class of incomplete data problems arising in exponential families, including random-effects models, censored and grouped data, and generally in models involving latent structures.

The 1977 paper accumulated more than 70,000 citations and was listed among Nature’s most-cited papers across disciplines. Its impact has been deepest in fields where latent-variable models and incomplete data are pervasive: machine learning; mixture modeling; tomographic image reconstruction; hidden Markov modeling; and quantitative genetics; to name a few.

Through the 1980s and 1990s, Dempster supervised graduate work that addressed the principal computational obstacle to applying Dempster-Shafer reasoning: the combinatorial cost of the random-set representation in high dimensions. Augustine Kong’s dissertation developed a “join tree” representation for combining belief functions locally. Kong’s representation is one of several closely parallel contributions to belief function local computation that emerged during that era. This program was in turn part of a broader convergence on local computation in graphical models that took place across statistics and machine learning research communities. Russell Almond’s dissertation extended this line of work and later appeared in book form.

Over the final two decades of his research life, Dempster returned to Dempster-Shafer theory as his principal scholarly focus. In “The Dempster-Shafer Calculus for Statisticians,” Dempster reinvigorated the research program by introducing the characteristic triple (p, q, r), corresponding to the probabilities “for,” “against,” and “don’t know” attachable to any formal assertion. Dempster argued for the necessity of an explicit “don’t-know” probability for the honest representation of inferential uncertainty. He called the resulting framework “DS-21,” a 21st-century reformulation organized around the careful construction of state spaces, the recognition that the “don’t-know” probability (r) is sensitive to the analyst’s choice of variables, and the deliberate use of “don’t-know” probability to address multiplicity and post-hoc subgroup inference. A retrospective synthesis appeared as “Statistical Inference from a Dempster-Shafer Perspective” in the COPSS volume Past, Present, and Future of Statistical Science.

Methodological and computational work proceeded in parallel. With Paul T. Edlefsen and Chuanhai Liu, Dempster developed a Dempster-Shafer treatment of Poisson counting data motivated by problems in high-energy physics, specifically the Banff upper-limits challenge associated with Higgs boson searches. The prohibitive computational cost to obtain Monte Carlo samples of the random-set objects at the heart of the theory, which Dempster had studied as a class of random convex polytopes in a 1972 Annals paper, stood for nearly half a century as the greatest obstacle to applied Dempster-Shafer inference. In 2021, this obstacle was overcome in collaboration with Pierre E. Jacob, Ruobin Gong, and Edlefsen in “A Gibbs Sampler for a Class of Random Convex Polytopes,” published in the Journal of the American Statistical Association, accompanied by discussions from Glenn Shafer, Persi Diaconis, and others.

Dempster believed, as he would often state plainly to colleagues, that the research program on Dempster-Shafer theory of belief functions is the most important of his principal scientific legacies.

Over nearly five decades of active service, Dempster taught generations of Harvard undergraduates and supervised more than 30 doctoral students, many of whom went on to leading positions in academic statistics and industry. His service and dedication to the department were a sustained force in shaping Harvard’s statistical research and teaching mission. He and Elizabeth opened their home, first on Moon Hill Road in Lexington, later on Walker Street in Cambridge, to colleagues, students, and visitors throughout his career. With funding support from former doctoral student Stephen Blyth, Harvard’s Department of Statistics established the Arthur P. Dempster Award in 2012 to recognize promising graduate student research in theoretical and foundational statistics.

Dempster was a Fellow of the Institute of Mathematical Statistics, American Statistical Association, and American Academy of Arts and Sciences. He held a Guggenheim Fellowship in 1967–1968.

Elizabeth (née O’Neill), Arthur’s wife of nearly 69 years, died on March 29, 2026, two months after his own passing. A classicist by training, Elizabeth read classics with first-class honors from Queen’s University in Kingston, Ontario, and held a master’s degree in Greek at Bryn Mawr College. Elizabeth and Arthur are survived by their children, Rebecca, Ben, and Sara; their spouses, Matthew, Yuri, and Eloy; and grandchildren, Rowan, Aidan, and Gavin.

The 2026 Joint Statistical Meetings in Boston, Massachusetts, will host a memorial session to celebrate Dempster’s life and work on Sunday August 2 at 2 p.m. Harvard Statistics will also host an alumni reception in honor of Dempster on the same day at 5 p.m. at the Maxwell-Dworkin building (33 Oxford St., Cambridge, MA 02138).

For details, download the references.