Contributing Editors
Arthur Kennickell is the chief of microeconomic surveys at the Board of Governors of the Federal Reserve System. He is an ASA Fellow and past winner of the Julius Shiskin Award.

Julia Lane is the program director of the Science of Science and Innovation Policy program at the National Science Foundation, a program that addresses gaps in science policy. Lane previously served as senior vice president and director of the economics department at NORC.

Among the obvious consequences of the financial crisis is the exposure of limitations in the available systems of information needed to detect, correct, and prevent such problems. Decades of neglect and underfunding have resulted in a U.S. statistical data infrastructure that is insufficient to respond to policy and research imperatives. At the same time, the failure to develop a flourishing culture of measurement in the economic profession means there has been little organized effort to mobilize the necessary support for change. Improvement will require not just money for new data initiatives and reform of existing programs, but also cultural change within the profession. Failure of the economics profession to respond risks a loss of capacity and credibility.

The current political environment is an important backdrop to bear in mind when addressing this issue. President Barack Obama is the first U.S. president to mention statistics and data in his inaugural address, and Office of Management and Budget Director Peter Orszag has called for evidence-based policymaking. As a result, federal agencies are being asked to manage their portfolios by using sound science, developing data sets, measuring outcomes, and evaluating performance.

Economics, Measurement, and Data

In the early history of the economics field, serious attention was devoted to the congruence of theory and data, as befits a scientific field. Particularly during the 1950s, such discussions appeared frequently in journals such as Econometrica, and the Cowles Foundation and the National Bureau of Economic Research showed serious interest. Later, Simon Kuznets and Wassily Leotief each won the Nobel Prize for contributions to economic measurement. Much of the early discussions focused on measurement questions related to national accounting, while information technology has recently made it feasible to make broad use of microdata to better address the behavior aspects of the field. Indeed, empirical researchers in labor economics and public finance, in particular, could not operate without microdata. Increasingly, policy work also requires microdata, where only aggregates were needed before.

Despite the increasing reliance on data, the broad engagement of economists with measurement has decayed. Increasingly, the profession has moved to a more passive approach to measurement, where the collection of information is implicitly treated as a second-class activity. Almost everywhere (there are brilliant exceptions, of course), graduate training relevant for measurement in economics has degenerated to perhaps the moment in the first econometrics course in which sampling distributions are introduced to give some foundation to t-statistics.

Aside from purely scientific motivations, this should be seen as an exciting time to be thinking about economic measurement. Advances in technology for the collection, storage, and sharing of data have been seized by such fields as biology (most notably in gene sequencing and inventorying of proteins), geoscience, and astronomy to make rapid advances. For economics, there is potentially a similar explosion of information—some of which has been filtered through new surveys or other classical measurement systems and much more that is created constantly by the functioning of the economy.

The relative passivity of economists may be seen as a rational response to the professional rewards for investing in data infrastructure development. The collection and dissemination of data has many of the features of a public good, and generally the profession fails miserably to assign academic credit for such work. Similarly, foundation funding is typically available for analytical research, and data collection is secondary. Even when data are collected, there is usually no incentive (at best) to document and disseminate those data.

Conceptual Framework: Economists and the System of Measurement

Economists can play at least four important roles in contributing to a system of measurement for economic research—from the small to the large scale. At the most granular level, economists can help drive the quality of the information measured. They can do this by direct engagement in the design of the data collection. The engagement of economists in the creation of surveys, such as the Survey of Consumer Finances or the National Longitudinal Surveys of Youth, led to the creation and transformation of entire fields of research. The input of economists at the many points at which decisions need to be made can have a large effect on the analytical outcomes based on a given information collection.

For example, at the simplest level, economists can help ensure that the complex chain of language used to describe the objects measured has the closest possible alignment with conceptual objectives from the beginning to the end. They can ensure that critical elements are retained on administrative and survey records to facilitate linkages across records, as in the Longitudinal Employer-Household Dynamics program. Economists also can apply economic principles to improve the quality of measurement systems, themselves. In surveys, for example, incentives can be designed to improve the quality of information collected by interviewers. The current incentive structure, in which interviewers are under serious pressure to complete interviews at minimal cost —rather than on the quality of the information they collect—seriously jeopardizes data quality.

Regardless of the quality of the data, they have little utility if there is no access. Economists can be proactive in promoting access. The agitation by German social scientists to “set the data free” resulted in the establishment of a new German data infrastructure for empirical research in the social and economic sciences and the transformation of German research. Economists should be much more vocal about the degradation of data quality associated with statistical disclosure limitation techniques and the burdens imposed by requiring researchers to physically go to research data centers to access data.

At a system-wide level, economists can improve the evaluation of measurement—a crucial component of a healthy measurement system—by identifying gaps in the data infrastructure that limit analysis. Systems of measurement should be driven as much as possible by research needs. Economists represent an important and heterogeneous group of data users, who possess understanding of the regularities and irregularities of a large number of particular data structures and the information gaps constraining research. Thus, for example, economists can provide the broad perspective necessary to evaluate the quality of microdata underlying a survey originally constructed for other purposes. The development of such a broad perspective would not only provoke improvement in the quality of particular measurement systems, but substantially advance the collective understanding and practice of economics in a more classically scientific way.