Dear Editor,

Keith Crank’s review of The Path Forward in the July issue of Amstat News asserts that the recommendations set forth in the report are not supported by the data presented. We disagree with Dr. Crank’s assertion and believe the data presented accomplish the report’s goal of providing reasoned arguments, supported by existing empirical evidence, on the future of graduate education.

The report addresses a multitude of complex issues by synthesizing and interpreting the available data from well over 100 studies and sources. While, as researchers, we agree that it is preferable to utilize data from one, single data set or time period to support any set of claims, no single source of available information could have accomplished this. Unlike with a traditional research study, we did not collect or analyze data specifically for this report, nor did we have a predetermined hypothesis to support or reject. Consequently, a major criterion in selecting data for the report was that it come from the most credible and current sources. When relying on pre-existing data, it is not always possible to perfectly address the issue being discussed, let alone have related data always align. Indeed, what is remarkable is that, despite the issues inherent in using data from multiple sources, these data tell essentially the same story.

Dr. Crank’s review claims that the report sometimes does not distinguish “between counts and percentages” and performs what he calls a “mixed metaphor of data use.” However, the examples he uses as supporting evidence focus on minor technical points that do not undermine the report’s findings or its conclusions.

For one of his examples, he suggests that the data presented are inconsistent because the discussion first cites data related to one age cohort and time period for graduate students and later cites data using a different age cohort and a slightly different time period for undergraduates, arguing that “It’s not clear why they couldn’t use the same time period and age cohort, and they give no explanation for the difference.” This is merely an example of the lack of exactly parallel data available when utilizing multiple sources. However, we feel that the similar time period makes the comparison valid, especially in the context of a policy report such as The Path Forward.

In another example, Dr. Crank cites a sentence containing two separate data points on the rate of college graduates earning advanced degrees and on increases in graduate enrollment. He correctly notes that the two data points refer to “totally different objects,” but the text never asks the reader to directly compare them in a statistical sense. The purpose was to juxtapose the good news on increasing graduate enrollments with the continuing problem of the small percentage of undergraduates who eventually complete a graduate degree. Since the mission of the report was to generate discussion among a wide range of audiences, the role of the data in the report was to provide supporting information around issues, not to focus on statistical nuance, significance, or modeling.

Dr. Crank also questions the report’s statement concerning the number of international students enrolled in doctoral programs needing to stay in the United States in order to keep the number of doctorates in the work force constant. He argues that since PhD production is increasing, “the stay rates would need to decrease to keep the total number of doctoral graduates living in the United States constant.” While Dr. Crank’s calculation is accurate as he frames it, there is an unstated assumption in the report that the total number of PhDs awarded will remain relatively constant in the short run. Perhaps this assumption should have been explicitly addressed, but, more importantly, it is an entirely reasonable one, as the year-to-year change in the total number of degrees is very small, and, as a proportion of the population, it has been flat for the last 30 years (see Figure 3 in the report). A change in immigration policies or enrollment decisions could have a dramatic effect on stay rates in the short run (and hence on the number of potential employees with PhDs available to U.S. employers), but changes in the overall number of PhDs produced are likely to be much more gradual. Therefore, the key concern for policymakers is indeed the number much more than the percent.

In a later commentary, Dr. Crank focuses on the employment projections cited in the report, indicating that the data do not support the need for more advanced degrees. He argues that since the projected increase in jobs requiring master’s degrees is about 2% per year over the next eight years, while the increase in master’s degree production had been 4% from 1995 to 2005, why are more resources needed if production is outpacing demand? If the report claimed that master’s degree production would continue to increase at 4% per year, this would be a valid criticism; however, such projections are neither cited nor suggested. Indeed, the report addresses many of the factors (including changing demographics, international participation, and completion rates for students) that make this rate of increase questionable.

Dr. Crank stated he was “excited to hear about this report, as someone who believes graduate education is important,” but, unfortunately, he has not used his forum to discuss the recommendations of the report or provide examples of data or statistics he feels might be better suited to make the case of why graduate education is important. We believe that the recommendations in the report are supported by the data provided and the types of analyses used and that The Path Forward report is already achieving its intended purpose of stimulating vigorous discussions about graduate education and its role in America’s future.

Cathy Wendler
Principal Director, Foundational and Validity Research, Educational Testing Service

Nathan Bell
Director, Research and Policy Analysis, Council of Graduate Schools