An Even Harder Question: What Kind of Students Are We Turning Off?

An astute reader may have been wondering why I have not addressed an obvious question, that is, could it be that the “turn-off” experience is unique to Harvard undergraduates? At the time of writing my op-ed, Harvard undergraduates were my targeted population, so this question was of little interest. However, now that the “Harvard observation” is brought to the national level, I am indeed inspired by RPFHS to consider its general applicability. Coming with the inspiration, however, is perspiration.

I perspired not because of the realization that I committed an elementary error of extrapolation, but rather because of the realization that if the “turn-off” phenomenon is indeed more likely for students at Harvard or Harvard-like institutions, then our profession has an even deeper and perhaps more disturbing problem to worry about. Harvard undergraduates are undoubtedly a highly selected group. But they are not so different from undergraduates at many of Harvard’s peer institutions, nor did they all come from one school district or one state. It is, however, quite possible that academically strong students have a higher likelihood to walk away from a poorly taught subject, statistics or not, than academically less able ones. Other than the fact that the former will have more choices at their disposal, it is not hard to imagine that the former are also more likely to be turned off by mechanical teaching emphasizing memorization for testing, as alluded to in the aforementioned student’s email. Indeed, the latter may even prefer such mechanical teaching because it is less challenging than inspirational teaching which requires high-level creativity and independent thinking, the very traits we all look for when we recruit students (and faculty). Therefore, if Harvard undergraduates can be viewed as a sample of high achieving high school students and if such students report more “turned-off” experiences, then we must ask ourselves not only what percentage of students are turned off by poorly taught AP classes, but also what kind of students are more likely to be turned away.

I perspired more when this was connected with the following anecdote, first heard from a Harvard undergraduate and then independently from a high school student in California (the son of a friend). In both cases, the student reported that his high school guidance counselor advised students that they should consider taking AP stat courses only if they cannot survive AP calculus courses. This suggests that at least in some high schools AP statistics is perceived as a “softer alternative” for students who cannot yet handle calculus.

Before we all get enraged by such a condescending perception, let us collectively keep a cool head and ask a deeper question. Should we then consider the impact of the AP program on the overall quality of students it helps to attract in addition to how many it attracts? Consider two versions of Table 1, one for strong students and one for weak students (the dichotomy, of course, is for simplicity of illustration), labeled correspondingly with a subscript S for strong and W for weak. Then arithmetically it is possible that we have

P_OFF,S + P_OFF, W < P_ON,S + P_ON,W, but P_OFF,S > P_ON,S

That is, even if the program succeeds in attracting more students in total, we can still end up with a population with lower quality compared to the population we would have attracted if the program were not in place.

I have no data on this, and sincerely hope this is NOT true! What I have is a hunch that something could go wrong and we cannot detect it if we only look at the “In” column. But historically, hunches (or more formally “case studies”) have led to both happy and miserable discoveries, when followed up by well-designed studies; two cases I used in my Happy/Misery Course are the (Happy) Viagra Trial and the (Misery) Fen-Phen Study. It is in this spirit that I suggest that we consider the issue of quality when we design further studies on assessing the effectiveness of the AP program. Such studies are not easy at all, but if there is any profession which has the most, and best, experts for designing and conducting them, it must be us. Indeed, other professions are paying high consulting fees to us for conducting these types of assessment studies for them. It would be ironic if we could not engage our own profession, with the same rigor, to address a problem that is directly about our own future!

Mis-take of Passion?

My initial explanation for the lack of acknowledging the “turn-off” issue in RPFHS is that they were misled by the non-trivially edited version (without my knowledge) as printed in Amstat News (September, 2009), where the quotation marks were removed from the aforementioned literal quotation from a student on AP courses being boring. It therefore could be perceived, particularly as a subtitle, as a depiction of the entire AP statistics program. Consequently, RPFHS serves, justifiably, as a defense of the AP program by providing positive examples. But after I raised such a possibility, the editor assured me that RPFHS was written as a reaction to my original op-ed (which contains five puzzles; click here to view).

That leaves me wondering if the authors of RPFHS and I have shared a similar type of “mis-take of passion”, that is, when driven by our passion for wanting to see a particular outcome, we forego careful consideration or critical thinking that otherwise are quite customary to us. The phrase “mis-take” here is less a criticism of “mistake” but more an admonishment to ourselves that the human tendency of “mis-taking” passion in our effort to persuade is almost innate. Indeed, in a sense, it is my mis-take (or even mistake) of passion that has led to the current discussion, though I am hoping for forgiveness if the discussion ultimately leads to positive outcomes.

When THURJ invited me to write an op-ed, I said yes immediately even though I was completely overwhelmed by four courses and many other tasks. My passion obviously took over – where else could I find such an opportunity to potentially reach all current Harvard undergraduates, especially those who have already decided to stay away from statistics?

The article was initially circulated within Harvard, but the enthusiasm from readers soon encouraged me to make it available more generally. I was well aware of the selection bias in what one tends to hear, but when I read reactions such as the following, I told myself, well, why not let more students see it?

Retrospectively, my passion for attracting as many students as possible blinded me to a completely obvious fact, that once an article is printed and especially circulated in general, it will not be read by students only! It simply escaped me that my intended dialogue with some Harvard students, that is, “I understand you had a bad AP stat class, but please give statistics another try” could be taken as an assertion about the entire AP program. I of course could have said “Sorry you had a miserable experience, but AP stat classes worked well for many others”—one can imagine how persuasive that would be. But I should have added a footnote when circulating it to emphasize the intended purpose of mentioning AP statistics in my op-ed. I therefore thank RPFHS for providing this opportunity for me to prevent any future inappropriate quotations of my two sentences on AP stat courses.