Home » A Statistician's View, Departments

The Assessment of Teachers: Notes from a Conference

1 May 2014 404 views No Comment

Jan_amst_cover

We were interested to read the January Amstat News article “The Assessment of Teachers: Notes from a Conference” in Statistician’s View. This paints an optimistic picture of how value added models (VAMs) can be used to measure teacher effectiveness and provide a valuable statistics teaching tool.

Unfortunately, the reality is rather different. The use of these models is highly contested, and there is a large statistical and educational literature on their use. One key issue is that the confidence intervals associated with the value added estimate for any one teacher tend to be large, and sensitive to the assumptions of the model. This makes their use for purposes such as teacher promotion or salary determination extremely limited. Yet the article fails to mention this issue.

It also fails to mention the problem of unintended perverse side effects, which have been extensively documented, including teaching to a narrowed curriculum and heightened stress levels among teachers and students.

The illustrative models used in the paper are incorrect. For example, the first one has undefined terms and subscripts that don’t balance. It also omits the all-important terms for lagged effects of previous teachers. The concepts underlying “value added” models are not simple, and in addition to their strong parametric assumptions, they can become quite complex (see the 2014 Annual Review of Statistics and Its Application article, “Using League Table Rankings in Public Policy Formation: Statistical Issues,” by Harvey Goldstein for a discussion). Using them as didactic devices in teaching statistics to teachers seems highly questionable.

While statisticians have responded with complex models and analyses to the complex partially crossed and partially nested structure of children proceeding through school classes, the modeling and assessment of the effect of teachers in the development of their students is controversial, and not at all a routine procedure.

A final irony in this article is the suggestion that statisticians should take the lead in providing professional development to teachers! A willingness to be seen as partners alongside educational specialists would be welcome.

Harvey Goldstein, Fellow, British Academy
Professor of Social Statistics, Centre for Multilevel Modelling,University of Bristol, United Kingdom

Murray Aitkin, Fellow, ASA
Emeritus Professor of Statistics
University of Newcastle, United Kingdom

Author’s Response:

Thank you for taking the time to write a response to the Amstat News article about the assessment of teachers.

There is no doubt that the use of VAMs and growth models is highly contested. In fact, the conference the article discusses focused greatly on the pros and cons of the various models. In addition, the conference brought together both researchers and state and district policymakers to discuss the difficulties of implementation for these models. For example, an important issue discussed is that of nonrandom assignment of teachers to schools and students to teachers.

As noted in the article, the purpose of the article was not to discuss the nuances of the models, but to introduce the ideas of the models to a statistical audience and make the connection to teacher preparation. The policy brief from the conference discusses specific limitations and issues with the models discussed at the conference.

It is absolutely correct that the study of VAMs and growth models is complex. The idea of using this as a backdrop for teacher training does not mean teachers would be taught how to estimate VAMs and growth models and study their complexities. Instead, as stated in the article, “we would not expect teachers to take a course on and understand all the nuances and difficulties of estimating teacher performance measures; however, for example, understanding the complexities and interpretations of multivariate regression would be quite appropriate.” Furthermore, the study of regression could be set up, while working with teachers, using education data sets that would offer teachers an opportunity to understand some of the terminology surrounding their profession in policy discussions.

As to the suggestion that the illustrative models are “incorrect,” there are a variety of ways that these models may be specified, and those portrayed exemplify fairly common representations.

I completely concur with the response letter’s suggestion that teaching teachers in a professional development context should be done with the collaboration of discipline-specific individuals alongside education specialists. In fact, many statistics educators working directly with teachers are themselves education specialists or work directly with education specialists.

Anna E. Bargagliotti

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Comments are closed.