The Great Pressure We Are Under: Deliver Quantity And Quality
The research needed is not just about determining the quality of the existing AP program, but more critically to identify, insofar as possible, the mechanisms that have led to local “good, bad and ugly” implementations of an obviously very well intended program. Such research can help our profession make informed plans as how to improve or even reform it in order to achieve our dual goals to build and sustain a strong workforce for our profession’s future and to continuously raise statistical literacy in general. We are no longer in a stage where our central goal is merely to convince the general public, at least the scientific ones, of the importance of statistics. We are now very much desired, or even feared, as I argued in Meng (2009). We now need to deliver, not just in terms of quantity but more importantly quality. I put more emphasis on quality, because as we are all aware, without good quality, a product will not last for long even if it is highly demanded at some point.
A local painful experience which occurred when I became department chair reminds me well of the importance of not forgetting quality when quantity is being demanded. In 2004-2005, we had an unexpected increase of enrollment by about 85 students in our most basic introductory course, Stat 100. Therefore, literally at the last minute, we needed to find additional teaching fellows (TF) to staff 5 more sections, as Harvard’s policies require one TF section for every 17-20 students. We delivered the quantity, by hiring anyone who “could breathe and count” – a sarcastic phrase that sadly was not too far from the reality. The result amounted to “a mini crisis”: students complained, faculty complained, and even those TFs themselves complained because they were under pressure to do a job they clearly were not qualified for, nor were they given any training. Given the grand challenges we are facing, as listed and discussed in detail in a number of very recent articles (e.g., Brown and Kass, 2009; Meng, 2009), it would not be pure speculation to suppose that our profession would be facing similar “mini crises” or even big ones if we only focus on quantity. Of course, this is a well recognized issue by many. For example, much effort by ASA’s education department, such as the “Meeting within a Meeting” at JSMs and the STEW site (all available here ) is about improving the statistical education quality at all levels. My emphasis here is that in an assessment of a national program such as AP statistics, assessing its quality is as important, if not more so, than assessing its quantitative aspects.
The real pressure here, as in many other situations when the demand exceeds the supply, is maintaining both high quality and high quantity, or even just adequate quality and adequate quantity. And for educational endeavors, there is also often a cascade effect. As my colleague, Joe Blitzstein (the “Youtube” sensation mentioned in my op-ed), commented on an early version of this article:
Indeed, without enough qualified teachers, even the best designed curriculum and most well intended program can do more harm than help.
Joe is exactly right that in order to deal with this pressure, our most urgent task in terms of assessment is to learn as much as possible about the mechanisms that have led to “turn on” and “turn off” and, hence, we can be strategic with the limited resources we have. This key point is also emphasized by Kari Lock, a member of my “happy team,” in her comments to me:
We perhaps all have our educated guesses about, or even direct experiences of, what may or may not work. Surely the curriculums and teacher’s quality matter greatly, as does competition from other fields – we are not the only profession on the expanding horizon. (As discussed in Meng (2009), ironically, the rapid rising and evolution of other disciplines, such as life sciences, creates both demand of and competition for us.) And we may even know, at our local levels, which of those mechanisms are likely to be the dominating force.
However at the national level, I’d venture that currently we do not have a good understanding of the interplay of various factors. Many deep questions can and should be asked even if we yet need to estimate PON and POFF, at the national level. Here are a few that immediately come to mind.
- What kind of students are more likely being turned on or turned off? What kind of students are unlikely to be affected by the quality of the AP program?
- Are students turned on or off more because of course materials or because of their delivery? Or must it be the interaction of the two to have a strong effect, in either direction?
- For those “turn-off” courses, is the problem more due to teachers’ lack of teaching skill, lack of statistical knowledge, lack of experiences of statistics practice, or lack of enthusiasm they themselves have because of their own bad learning experiences? Or must it be a combination of two or more before driving students
away? - How widely spread is the notion that AP stat is a “softer alternative” to AP math? Is this largely due to the common perception that statistics is an “easy” branch of mathematics, or due to the effectiveness of the AP math program to attract top students, or due to students’ or even school administrations’ perceptions of who is getting to teach math courses and who is teaching stat courses?
- To what extent has the AP exam itself encouraged teaching for test-taking rather than for understanding and inspiring interest? In view of the rapid re-shaping of statistics by the scientific evolution, in what ways should we revise or even revamp the AP exam and curriculum in order to inspire and reflect students’ interests?
I am sure that none of these questions are new, and that the authors of RPFHS and readers have many more questions. As an example, here is an excellent and tough question from Joe Blitzstein, again in his comments to me:
Indeed, the ASA/CB study reported that having taken the AP stat course is a relatively important reason for the respondents not to take any college-level course in statistics. Therefore, in order to address Joe’s question and more generally the overall impact of AP statistics, we need to also examine the quality of statistical education at the college level, where the great shortage of qualified teachers is also a well-known problem, as discussed above. As a matter of fact, one may well question to what extent the “turn-off’ phenomenon exists at the college level, and whether it is in any sense better than at the AP level.
Evidently, questions like those listed above are often very hard to answer, and some perhaps are never answerable completely. Minimally, it will take significant human and other resources to conduct studies to address these questions. But I’d argue that the stakes are simply too high, especially considering how many other fields are competing intensely for future talent, for us not to give our “first gateway” to future statisticians the highest priority. I therefore urge ASA to assemble the best teams our profession can
offer to conduct studies of the AP program regarding its current impact and future directions and improvement, building upon the ASA/BC study. The plural “studies” is intended to point out that what is needed here is not just a single assessment study, but rather an on-going process to keep ourselves on our toes, to identify feasible ways for improvements, both short term and long term, and to ultimately offer a continuously renovated platform for “gateway” statistical education at a national level. The overall success of the AP program would be undoubtedly an effective arsenal in releasing the great pressure our profession is under, that is, to provide both high quantity and high quality future statisticians. We certainly face a great challenge here, but with challenges often come advances. My colleagues and I are well reminded of this by the fact that our current success as a model department at Harvard started from the aforementioned “mini crisis.”
Leave a Reply