Home » Featured

Establishing and Maintaining Trust in a Federal Statistical Agency: A Discussion from the National Academies

1 October 2022 804 views No Comment
Robert Groves, with ASA Director of Science Policy Steve Pierson
    height= Robert Groves is the chair of the Committee on National Statistics and served as director of the US Census Bureau from 2009–2012.

    height= Hermann Habermann served as chief statistician of the United States from 1988–1992. He was also deputy associate director for budget at the Office of Management and Budget, deputy director and chief operating officer for the US Census Bureau, and director of the United Nations Statistics Division.

    height= Charles Rothwell was director of the National Center for Health Statistics from 2013–2018, capping a career in federal service that started in 1987.

    height= James Lynch was director of the Bureau of Justice Statistics from 2010–2012.

    The Office of Management and Budget, Congress, and National Academies deem professional independence necessary for an agency to establish trust in its products. Professional independence includes an agency’s autonomy over its budget, information technology, hiring, publications, contracting, and data collection and analysis.

    For their spring 2021 release of the seventh edition of Principles and Practices for a Federal Statistical Agency, the National Academies Committee on National Statistics hosted a public webinar that included a discussion between committee chair Robert Groves and Hermann Habermann, James Lynch, and Charles Rothwell. To further the understanding of the importance of statistical agency autonomy, the editors at Amstat News include the autonomy portions of this wide-ranging discussion below.

    To emphasize the importance of an agency’s autonomy, the ASA is also sharing stories collected from former statistical agency leaders and staff describing instances in which the professional autonomy of their agency was challenged or they had to stand up for the professional independence of their agency. If you would like to share a story, please contact ASA Director of Science Policy Steve Pierson.

    Robert Groves: It is not an overstatement that official government statistics are one of the building blocks of a democracy. They are indeed how an informed citizenry is nourished in a democracy. What makes the whole process work is the statistics being viewed as credible. And how is credibility earned by a statistical agency? Because of the technical nature of the work, we believe credibility derives largely from trust. So how does one build trust? Trust is built, we believe, by the agencies following the wisdom compiled in Principles and Practices.

    Principles and Practices do not review laws or regulations. They don’t have statutory authority. In some sense, however, they’re even more powerful than laws, if they are adopted as a code of ethical standards of behavior inside the agency. If the staff of statistical agencies adopt these as a way of living their professional lives, the benefits of official statistics can be derived.

    We have lived through a period in which these codes of ethics have been the life rafts of US federal statistics. There are heroes among us right now, the civil servants and the technical staff who have lived these ethical principles under some duress.

    For those of you who are new to the statistical agencies, I encourage you to read Principles and Practices. They offer guidance on how to behave in real-life, practical, day-to-day situations. Ideally, they describe the values that underlie the work cultures of all the US federal statistical agencies.

    Let me now turn to our panel of deeply experienced former members of the federal statistical agencies who, collectively, have decades of experience in living the Principles and Practices.

    Thinking over your time in federal statistical agencies, were there occasions when you found yourself drawing on Principles and Practices?

    James Lynch: Just prior to my time at the Bureau of Justice Statistics, there was a bit of a scandal, which was the firing of Bureau of Justice Statistics Director Larry Greenfeld. Greenfeld stood up to those at the US Department of Justice who wanted to push a certain set of adjectives in a press release concerning interactions between the police and African Americans. Greenfeld was overseeing the release of findings from the National Crime Victimization Survey, specifically the Police Public Contact Supplement, which focuses on the interaction between citizens and the police.

    The Police Public Contact Supplement showed that African Americans were not being stopped disproportionately but, once stopped, the kind of force they were subjected to was disproportionate. The Bureau of Justice Statistics laid this out in a report and a press release, resulting in the pushback from various people within the associate attorney general’s offices and Greenfeld’s firing.

    And so, when I came to the Bureau of Justice Statistics, this incident was in people’s minds, which was a benefit to me. People were sensitive to the independence of the Bureau of Justice Statistics as a statistical agency because the Greenfeld episode was considered in many ways a scandal. Principles and Practices also changed as a result of it.

    In the wake of Greenfeld’s firing, language was added to Principles and Practices to the effect that directors of statistical agencies should have the authority to release statistical information, including press releases and documentation without prior clearance. This was a real advantage to me when the press office, which wanted to control the message coming from the US Department of Justice, pushed back on Bureau of Justice Statistics releases. Having Principles and Practices and the idea that the Office of Management and Budget was supportive of it was very influential. The press folks backed off.

    Later, the Obama administration science adviser, John Holdren, issued a scientific integrity memo that went around to all the departments in the executive branch asking for policies for the treatment of science in their agency. Like every science agency, the Bureau of Justice Statistics had to respond to it and, in doing so, we relied heavily on Principles and Practices. As a result, the US Department of Justice lawyers simply accepted our position, which was not the case for other agencies throughout the department. Having Principles and Practices to rely on was helpful to me in a number of ways.

    Charles Rothwell: In the 1970s, before Principles and Practices, Peter Henle, a member of my extended family, was the chief economist at the Bureau of Labor Statistics and an associate commissioner. With Harold Goldstein, he was responsible for the publication of unemployment data—among other things—during the Nixon administration. While unemployment seemed to be declining, it was not in a significant way and so the Bureau of Labor Statistics did not report it as a decrease. This displeased the president, as documented in a variety of memos and the Nixon tapes, which in turn led to Goldstein retiring and Henle leaving the Bureau of Labor Statistics. The incident is what brought about many of the protections the Bureau of Labor Statistics now enjoys. In the next administration, Henle came back in a senior position, which included responsibility for the Bureau of Labor Statistics.

    Years later, I came to the National Center for Health Statistics as an associate director responsible for IT and data dissemination from a statistical unit within a state health department. I had the occasion to share with Henle all the great systems we were going to be developing.

    Keeping in mind this was before Principles and Practices, I explained we were having the secretary of the department help us release our data and make a statement in that release. There was silence. And then Henle stated, “I don’t think you’re doing the secretary a service or your agency a service. You need to be separate from a policy perspective, for both the department’s strength and your strength.” I really hadn’t appreciated that point.

    Not much later, the director of the National Center for Health Statistics handed me a copy of the brand-new publication, Principles and Practices, and there it was in black and white for me to understand. I learned from that, and I’m very much a supporter of Principles and Practices.

    Hermann Habermann: I would like to suggest three examples of how the Principles and Practices were relevant in my own experience. When I was deputy director at the Census Bureau, I was informed it had been decided to move the publication date of the report on the pervasiveness of poverty in the United States until after the election. I replied that while those above me had the authority to do that, I would have to resign to protest what I saw as the manipulation of official statistics for partisan political purposes. Official statistics should be independent of partisan politics. In the end, the date remained unchanged and I would like to think the argument about professional independence and the importance of credibility by the public in that belief had an impact.

    The second one illustrates the importance of leadership. When Janet Norwood was commissioner of the Bureau of Labor Statistics, the agency was caretaker of a database on workplace accidents for the Occupational Safety and Health Administration. The Senate Labor Committee wanted to examine the microdata. When the data was collected, the Bureau of Labor Statistics told the individual companies the data they supplied would be confidential. The Bureau of Labor Statistics informed the committee that, because of this, they could not release the data.

    It is not easy to deny a committee of the US Senate, and some tense negotiations followed. At that time, the Bureau of Labor Statistics did not have the confidentiality laws protecting its data it now enjoys. It did have the leadership of the head of the Bureau of Labor Statistics, who believed the credibility of the agency was at stake. In the end, the Bureau of Labor Statistics prevailed in its arguments about preserving the credibility of the agency.

    Finally, when I was at the Office of Management and Budget, the oil embargo was underway and the Justice Department suspected collusion among the oil companies, driving up gasoline prices. Knowing the Energy Information Administration had data from the oil companies on their pricing structure, the Justice Department requested it. The Energy Information Administration pointed out that the data was collected under a pledge of confidentiality and therefore it could not provide the data. The Justice Department replied that it interpreted the law differently.

    The Energy Information Administration could have referred to Principles and Practices and argued on the principles of independence and credibility and the public impact of not protecting their confidentiality pledge. Instead, they argued on a point of law and lost. As a result of that decision, the Energy Information Administration had to change their data collection program to indicate that, in certain cases, they could not protect the confidentiality of oil company data.

    Robert Groves: Let me pick up on being separate from policy. Principles one and four speak to independence from political influence but also the need to produce statistics that are policy relevant.

    How do you navigate the boundary between being policy relevant and not being a policymaker or promoter?

    Charles Rothwell: We need to be policy relevant. Our job is to inform the debate and not enter into the debate. There’s a great difference. During the Obama administration, the major piece of legislation enacted almost immediately was the Affordable Care Act, or Obamacare. It took several years for it to take effect, at which point everyone was very interested in what its impact would be.

    The National Center on Health Statistics had been collecting data on health insurance since about 1959 through the National Health Interview Survey. Because of that interest, our proposal to expand the National Health Interview Survey to be able to cover some of the major states on a quarterly basis was approved and we had the revised survey in the field on schedule.

    Several weeks before we were going to release the first quarterly report, I received a phone call from the White House, wanting to know details. I explained why it was not in their best interest to have their fingerprints anywhere on what the release stated. I offered to include them in the briefing to the secretary’s office 24 hours beforehand but also recommended against it. They agreed.

    Robert Groves: It seems clear that if a statistical agency enters the debate and is viewed by the public as engaged in policymaking, the agency’s credibility is threatened. If there’s a change of power or if the ideology of a new administration goes in another direction, the agency is tainted by memories of a policy position, and that’s why staying policy neutral is so important.

    Let me turn to the notion of leadership that has been brought up. Of course, it is not just the head of the agency who can exert leadership on these ethical codes. Sometimes, it’s someone fairly low level in the organization who evokes these codes of ethics and says this is not right.

    Hermann Habermann: That’s absolutely right. A critical responsibility of the head of the agency is setting the culture. Principles and Practices have to be more than some ideas in a book. Those ideas have to be embedded in the way an agency operates every day. It echoes what you just said before, Bob, and that’s part of the responsibility of an agency head. The person in charge of the agency has to foster that culture.

    I would also echo your point that there is an ethos for statistics agencies, a set of guiding principles and beliefs. I think most people work in a federal statistical agency because they believe reliable statistical information is fundamental to a democracy. And the Principles and Practices support that ethos. And as you pointed out, it has to be throughout the agency, not just the leadership.

    The evidence suggests that, increasingly, we find many of those in power do not share this ethos. They believe in making up the data to fit their narrative. We can waive the Principles and Practices at them, but will this be enough? We learned in the last administration that an entire statistical agency can be physically moved and its effectiveness damaged fairly easily. I am not certain we have paid sufficient attention to how we will maximize the likelihood of ensuring official statistics will retain its credibility and quality. Perhaps that is something the Academy can think about.

    In considering the future of the Principles and Practices, there is another issue we might think about. As we have discussed, the private sector is an increasingly important player in providing data to federal statistical agencies and alternative sources of statistical information to the public. Moreover, the introduction of new technologies is both an opportunity and a challenge. The Principles and Practices were developed in a world in which federal agencies were the dominant players and providers of statistical information. That world no longer exists. A major question for us now is how to adapt the Principles and Practices to this new world.

    The difference between the public and private sector has been characterized in the following way: In the public sector, institutions may only do what they have been permitted to do, while in the private sector, institutions may do whatever they wish that has not been explicitly proscribed. If the Principles and Practices are to remain a useful guide in building our ethos, we must understand how to build a new Principles and Practices that understands this characterization.

    Robert Groves: Let me take that up, slightly shaped: Do you think Principles and Practices and the existing laws provide enough to protect the agency’s integrity going forward or do we need more? Is new legislation required to undergird the independence of the statistical agencies?

    Charles Rothwell: Let me start with Principles and Practices. I think from a statistical perspective, internally for our agencies, they are both necessary and sufficient. However, as we’re dealing with our departments, the executive branch, Congress, and the public, I think Principles and Practices are still necessary, but not sufficient. Yes, the OMB directives are helpful, as are the existing specific laws in place to protect certain statistical agencies, as well as the Confidential Information Protection and Statistical Efficiency Act. But we need more, and I have several suggestions.

    If the statistical agencies are going to be able to seamlessly work together, which the pandemic clearly demonstrated we need to do, the visibility of the Office of Management and Budget office of the chief statistician needs to be raised and there should be legislation to better facilitate statistical agencies working together. I believe the federal statistical agency heads should report directly to the chief statistician—they can also report directly to the head of their host agency—they should be appointed in a similar way, and they should all have the proper protections. I also believe there should be an innovation fund, perhaps out of the office of the chief statistician, to fund the research for advancing government statistics.

    In short, my argument is not creating laws to protect us. I’m arguing for legislation that allows us to work better together; to modernize how we do business—both decentralized and centrally—and to have a senior person in OMB who guides us, holds us accountable, and moves us into the future.

    Hermann Habermann: Unfortunately, when we look around the world, and in our own country, the production of reliable, objective official statistics has been under assault. One example in our country has been the proposed addition of a citizenship question on the decennial census. The manipulation of the Census Bureau in this case for political purposes is well documented. While the question was not added to the census, the attempt caused damage to the Census Bureau and may have affected the response rate. While striking the question, the Supreme Court did not base its decision with any reference to what is contained in the Principles and Practices, but because of technical violations by the administration.

    Unfortunately, the willingness by political leaders to ignore fundamental principles of official statistics as codified in Principles and Practices (and other relevant codes) is not unique. My concern is what kinds of mechanisms are we going to have to push back? The fight against the citizenship question took an enormous effort on the part of many parties. In the end, the decision to eliminate the question was only five to four. The Principles and Practices are necessary but they do not seem sufficient.

    As I mentioned before, there are increasing cases in which official statistics are being manipulated for partisan political purposes. Too often, there’s a disregard for these principles by those who are in political power. That’s the difficulty. We believe in the principles, but not everyone else does.

    You also raised the question about legislation undergirding agency independence. Federal statistical agencies derive their authority and resources from the US government and are part of that government. I don’t see these agencies becoming independent from the authorizing, budget, and oversight roles of Congress. What I think many of us would like to see is professional independence. Legislation that would undergird the professional independence of statistical agencies is certainly needed.

    James Lynch: Considering I became director of the Bureau of Justice Statistics with no previous experience in a statistical agency, the most important thing about Principles and Practices and the statistical community is that they socialize you. A person can come in fairly ignorant of the idea and importance of a statistical agency’s independence and the delicate arrangements supporting independence. You get indoctrinated into the statistical system through your new colleagues in the agency and the system and you begin to think the way Principles and Practices want an agency head to think.

    We’ve seen the weakness of laws in recent years without the courage of people standing behind them. That courage is based on custom, so I don’t know laws will help. I think putting forward a plan for how a statistical agency or a statistical system can undergird and preserve a democracy, along with an understanding of it, will give people the courage to quit.

    The courage to quit is what we’ve always had. One of the pillars of this system is that people understand how you’re supposed to behave and why you’re supposed to behave that way. And when people do not allow you to behave that way, you quit. You quit visibly and in a noisy fashion. That kind of courage is what undergirds all of these things. So, I think the question is not to get a law but to ask, how do you inculcate that culture into the people who take these positions?

    Robert Groves: Should a statistical agency have the authority to collect information that is in its purview?

    Charles Rothwell: The Centers for Disease Control and Prevention, for a long time, was not able to collect information on firearm-related morbidity or mortality. The National Center for Health Statistics has no such restriction so, when I was asked why I was collecting such information, I pointed out the absurdity of publishing all mortality with cause of death except for those related to guns and that not providing any mortality data for the United States wasn’t an option.

    The National Center for Health Statistics collects a lot of very personal information in our health and examination survey, called NHANES, which could be politically sensitive. The National Center for Health Statistics also has the Family Growth Survey, which is very important for informing us about changes in sexual practices and current intent on families having more children. Seldom have we heard you can’t collect this information. Why? Because, as a federal statistical agency, folks know their personal information will be protected and the reports generated from their data will help form evidence-based policy decisions to help improve lives.

    James Lynch: The decision to collect or not collect information (when it is yours to make) is a bit more complicated than the decision to release information once collected. In considering what information to collect or not collect, it’s important to consider the end result and the cost and benefit for the statistical system writ large.

    When I was at the Bureau of Justice Statistics, many gun violence researchers pushed hard to get information on gun ownership in the National Crime Victimization Survey. After considerable thought, I came down on the negative side because I was convinced of three things. First, if we put that on the National Crime Victimization Survey, we would no longer have a National Crime Victimization Survey, or maybe even a Bureau of Justice Statistics, because political winds were blowing that way. Congress, at that time, strongly discouraged data collection on gun use and ownership whenever it was attempted. Second, I worried how the addition of such questions would affect the National Crime Victimization Survey response rates. Third, I thought they could get the information they needed elsewhere.

    Hermann Habermann: As I said, I do not think Congress is going to let the agencies have that kind of autonomy. Nevertheless, statistics agencies must continue to review and address the relevance of the information they collect—and now in a much quicker timeframe. The delegated authorities agencies now have do not allow them to respond adequately to these changing needs of society. The agencies need to be allowed more leeway from the Hill to better track societal changes and needs.

    It is nonetheless important to remember the other side of the coin of increased autonomy of statistical agencies. No one elected anyone in the statistical agencies. Currently, there is a political process to, hopefully, reflect the public good in determining if, for example, poverty statistics are more important than crime statistics. If statistics agencies were truly independent, where would the creation of that public determination come from?

    Robert Groves: Thank you for this rich discussion and to the National Academies Committee on National Statistics for a seventh edition of Principles and Practices.

    For the readers, especially those of you who may be new to the statistical community, I invite you to read Principles and Practices for a Federal Statistical Agency. They define how we should behave professionally. They define a set of aspirations that, if we all achieved them, would produce greater trust in statistical agencies, more ubiquitous credibility given their statistical products, and a support for democracy in unparalleled ways. So, I hope you got a flavor of how you, day to day, can use Principles and Practices to guide your own decisions.

    Five Principles

    Principle 1
    Relevance to Policy Issues and Society: Federal statistical agencies must provide objective, accurate, and timely information that is relevant to important public policy issues.

    Principle 2
    Credibility Among Data Users and Stakeholders: Federal statistical agencies must have credibility with those who use their data and information.

    Principle 3
    Trust Among the Public and Data Providers: Federal statistical agencies must have the trust of those whose information they obtain.

    Principle 4
    Independence from Political and Other Undue External Influence: Federal statistical agencies must be independent from political and other undue external influence in developing, producing, and disseminating statistics.

    Principle 5
    Continual Improvement and Innovation: Federal statistical agencies must continually seek to improve and innovate their processes, methods, and statistical products to better measure an ever-changing world.

    Ten Practices

    To fulfill these five principles, the following 10 practices are essential for statistical agencies to adopt:

    1. A clearly defined and well-accepted mission
    2. Necessary authority and procedures to protect independence
    3. Commitment to quality and professional standards of practice
    4. Professional advancement of staff
    5. An active research program
    6. Strong internal and external evaluation processes for an agency’s statistical programs
    7. Coordination and collaboration with other statistical agencies
    8. Respect for data providers and protection of their data
    9. Dissemination of statistical products that meet users’ needs
    10. Openness about sources and limitations of the data provided
    1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
    Loading...

    Comments are closed.