Home » Featured

State of the Science and Engineering Data Infrastructure: National Center for Science and Engineering Statistics

1 July 2021 1,591 views No Comment
Steve Pierson, ASA Director of Science Policy

Image shows Robert Groves, Jeri Mulrow, and Lynda Carlson in a set of headshots

Continuing our state of the data infrastructure series, the ASA Count on Stats team spoke with three experts on the National Center for Science and Engineering Statistics (NCSES). Housed in the National Science Foundation (NSF), NCSES is the principal federal statistical agency providing data on the state of science and engineering (S&E) enterprise; collecting essential data on science, technology, engineering, and mathematics (STEM) education; providing research and development activities, funding, and infrastructure; and supporting data innovations and solutions for the future.

The NCSES has expanded over the last two decades and is currently in the spotlight for its prominent role in evidence-based policymaking efforts. Lynda Carlson, a former NCSES director, was especially influential in elevating NCSES in the first half of the 2000s. Known as the Division of Science Resources Statistics until 2010, Carlson championed the renaming of the agency and an expanded mission, which were both realized in the America COMPETES Reauthorization Act of 2010.

Currently, NCSES Director Emilda Rivers chairs the government-wide Advisory Committee on Data for Evidence-Building, and NCSES is leading the Interagency Council on Statistical Policy efforts to establish a standardized application process for restricted-use data from federal statistical agencies. In addition, the US House of Representatives approved a provision in May for NCSES to lead a pilot project for the National Secure Data Service.

Figure shows an outline of a person shaded to represent that out of the NCSES' workforce,  there are 52 federal employees  (7% of the NCSES staff) and 2500+ federal contractors, 93% of the agency's staff

Figure 1: Breakdown of NCSES workforce by employee and contractor. Source: Presentation by NCSES Director Emilda Rivers, June 4, 2021.

Carlson, Robert Groves, and Jeri Mulrow speak highly of the many advantages NCSES enjoys as part of the NSF. They speak almost as passionately about how NSF staffing constraints hamstring NCSES, NSF, and the federal statistical system. The ASA checked the staffing numbers for the 13 federal statistical agencies and found NCSES has a budget-to-staff ratio of approximately $1.15 million per full-time employee, more than three times the median ratio for the 13 principal federal statistical agencies.

A consequence of the staffing constraints is a heavy reliance on contractors. According to a June 4 NCSES presentation, NCSES supports almost 50 times as many contractors as full-time employees. (See Figure 1.) In contrast, the Energy Information Administration (EIA), which has an annual budget for $126 million (for fiscal year 2021-FY21), reported at the same meeting 359 employees and 300 contractors. (The FY21 budget for NCSES is $66.7 million.)

The bottom line is, while hampered by staffing caps, NCSES overall benefits enormously from being part of NSF. Thanks to strong leadership and strategic hiring over the last 15 years, the NCSES profile continues to elevate for reasons that go well beyond its current vital role in evidence-based policymaking. NCSES, NSF, and the S&E enterprise would all be well served by allowing NCSES more staff.

NCSES seems to be the primary federal statistical agency that has grown in visibility and stature the most over the last 20 years. Would this be an accurate assessment, and what are the drivers?

Lynda Carlson: A unique part of NCSES is its place within the National Science Foundation. It’s housed in a “boutique” agency that, in some ways, acts as a quasi-government organization. Unlike some of the other agencies, they are in an agency that is into testing and trying and pushing the envelope. NSF is continually being refreshed because they have many Intergovernmental Personnel Acts (IPA) and have rotating assistant directors.

Being in NSF helps make the agency nimble. For example, when I was director, I decided we ought to change the name of the agency and have strengthened data confidentiality protections. So, I went to the NSF general counsel and they said to check with the director, who said, “Sure.” And so, in the Competes Act of 2010, our name was changed and NCSES obtained confidentiality protections for its data collected prior to the establishment of the Confidential Information Protection and Statistical Efficiency Act (CIPSEA) agency. Further, NCSES became a “named” entity in NSF, which meant it might be moved within NSF, but could not be eliminated or moved to another entity of government without congressional approval.

I was at the EIA for 23 years and there was no way anything like that would ever happen at the Department of Energy or any other agency. That’s what I mean by a unique place for change.

Since it has become NCSES, as opposed to the Division of Science Resources Statistics (SRS), it’s actually loved by the rest of NSF. They aren’t engaging in the kinds of budget battles I had when I was with the EIA. There’s more of an understanding or appreciation that they need a budget. However, they are really short staffed, which I’ll discuss more later.

Jeri Mulrow: I think being part of NSF is why they can really be innovators and lead the federal statistical agencies’ work on evidence-based policymaking. They can draw on the energy and innovations from the rest of NSF and get more collaborations and partnerships flowing to push this work ahead not only on the infrastructure side of things, but on the statistical and methodological techniques, as well as the use of other/ alternative data sources. Thinking further ahead, they are in a really good position to put all the pieces together.

Another thing about being part of NSF is they understand that research means you don’t always succeed on the first attempt. They don’t judge failure as a total waste and understand there’s this incremental building: You adjust and move forward as you try things.

In terms of visibility, it helps that NCSES is nimble enough to take on demonstration projects and show that important work can be done and ramped up. They’ve done a really nice job over the last decade or more to show you can make significant improvements in survey methodology and data collection activities. Because NCSES has the resources and control over how they publish and produce their data, they’re right at the forefront of where the federal statistical system ought to be. They also have the ability to contract out for expertise and the cachet to bring the experts who are really forward-looking. And they listen.

I think the recent Women, Minorities, and Persons with Disabilities in Science and Engineering report is another example of how NCSES is bringing forward these historical data collections that they’ve done and highlighting their current relevance.

We have these trends of data over time, and NCSES has been able to really highlight those in a way the audiences can actually understand. A lot of the agencies have been struggling with the value of statistics and presenting data clearly. NCSES and NSF are showing how this can actually be done.

Stepping back, the Science and Engineering Indicators (SEI) are likely NCSES’s best-known product. Tell us about NCSES products and data users.

Robert Groves: Let me first concentrate on the impact of this relatively small agency. Through the SEI, it has outsized influence on the discussion about science funding—way more than one would ever imagine.

SEI is the only assembly of data on public attitudes toward science investment, which is also important on the impact side. There are also two issues that have become increasingly important in the last few years. One is the demographic composition of the scientific workforce, where women and people of color are disproportionately underrepresented, which NCSES has helped to spotlight. The other is how the agency can inform the National Science Board and the foundation with regard to investments, not in scientific inquiry, but in the promotion, promulgation, and support of the skilled technical workforce, who generally are not bachelor’s degree recipients.

I also want to comment on what an incredibly cost-efficient agency NCSES is. It does its own data collection, but it also assembles data from other agencies.

I also want to praise NCSES for moving to a digital form for SEI, moving from an every-two-year book that not enough people read to a really quite accessible, nearly continuous release. That’s a huge advance and, in a way, NCSES is way ahead of other larger agencies that are still putting out books. This was a huge culture change.

In terms of data users, the White House Office of Science and Technology policy relies heavily on NCSES data. Congress also use NCSES data for various purposes, including funding deliberations. Further, with more than half of graduating PhDs going into the private sector, private industry is using the data to assess the incoming workforce, addressing such questions as what degrees they are getting, what skills they can bring, or what we need them to get degrees in.

Lynda Carlson: The Survey of Earned Doctorates (SED) and Survey of Doctorate Recipients (SDR) are well known and used extensively. On the R&D side, some of the work being done on innovation by Carol Robbins is sharp and interesting. I think the insights from that data show there are issues related to innovation in this country. For data users, I would add universities. They are always watching their competition, and not just in terms of how many research dollars they get but also what their graduates are studying. Also, industry is paying attention as the data relates to the Endless Frontier efforts.

How would you characterize the current state of NCSES and its data infrastructure?

Lynda Carlson: I’ve already mentioned NCSES’s short staffing. Recently, there were some releases and it pains me to see the analytic work is done by contractors. NCSES staff doesn’t have the chance to use and learn from their data as do staff at some other agencies. Rather, they are contract managers. And what that means is that the true knowledge of your own data is owned by a contractor, who will probably be turned over in another four years or five years. That’s a big problem. They are really behind in staff, and not just in terms of analytic capability.

Jeri Mulrow: NCSES is a leader in showing how things can be done. Its digital approach is more modern than most of the other agencies that produce data and release it to the public. The idea behind the data hub is modern, but other organizations are still missing some of the techniques and methods. Also going for them, as we noted above, they have control over their IT and how they publish and produce reports. And they listen and can fund research if they want.

As with most small agencies, NCSES has huge potential it is not able to tap into because it doesn’t have the staff. The statistical agencies have a great deal of knowledge and a long history working with data, not just survey data, but all those concepts about quality and interoperability. I think NSF doesn’t really understand how NCSES could contribute, beyond science and engineering indicators, to its mission, let alone the country’s understanding of this arena.

If they had more staff, they could more effectively manage their strengths and be thinking even more strategically ahead.

Bob Groves: All of the federal statistics agencies, including NCSES, are in the middle of a gigantic paradigm shift where new data resources have to be added in clear ways. These are technical matters that require analytic expertise; you can’t just hire off the street because the blending of data requires a deep understanding of the measurement steps of the data you’re blending, in addition to all the statistical issues. When small agencies fall below the minimum size of their technical core, they are threatened. I do believe there’s a minimum threshold and that an indirect indicator of this is the ratio of contract staff to full staff.

Where do you see NCSES’s biggest potential impact over the next several years?

Lynda Carlson: There is definitely a lot of focus on evidence-based policymaking, but something much bigger is about to happen to NSF and NCSES. The NSF reauthorization bill is going to bring a massive infusion of money and responsibility to NSF. It’s going to require a whole new series of data collections or a rethinking of the existing ones, because it’s not just on the R&D side. There will be an emphasis on innovation and technology and the future competitiveness of our country. This expansion will be much broader than NSF, so they’re going to have to broaden their interactions with other agencies.

Jeri Mulrow: The concepts around innovation and technology and the measures around those concepts are not simple. There’s been some advances on that and on the demographic side but not as much on the R&D or innovation and technology side.

Data linkages are another area in which there is more potential, and they could be even more powerful—not just within the data they collect themselves—but with other outside data sources—elementary education in math, science, and technology, for example—to really think broadly about how to foster a strong and robust STEM workforce and continued innovation. We are seeing that there are workforce leakages at all levels but don’t adequately know the drivers. Also, how do we get people in, melding the arts, science, and the humanities?

Regarding the National Secure Data Service proposed by the Commission on Evidence-Based Policymaking, NCSES is initiating a demonstration project called America’s Data Hub. It’s a small step to help answer what the pitfalls are, what the challenges are, and where the successes are. That allows NCSES and the other statistical agencies to see what challenges they run into, what’s viable, and how to scale it up from there.

NSF would benefit from further NCSES work on how to measure the impact of NSF funding. For example, on health and the economy. There are complexities with trying to figure out how to talk about that impact because the impact can potentially take years, if not decades, to be realized. That work would help raise NSF’s profile and stature.

We’ve mostly been talking about the potential national impact of NCSES but its potential influence stretches to the state level and internationally. The states are very interested in the economic impact of having an academic institution with a ranking. They want to know whether their graduates stay in the area and what they contribute. NSF’s profile could also be elevated more internationally with more NCSES work to understand the international flows and connections of science, technology, innovation, and people.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Comments are closed.