Anna E. Bargagliotti, Loyola Marymount University Department of Mathematics

On October 10 and 11 at Michigan State University, the conference titled “Using Student Test Scores to Measure Teacher Performance—the State of the Art in Research and Practice” took place. The conference was sponsored by the Institute for Education Sciences (IES) and led by PIs Cassandra Guarino, Mark Reckase, and Jeff Wooldridge. It brought together about 80 researchers, policymakers, educators, and other interested parties to discuss issues related to the policy implications of teacher performance models such as value-added models (VAMs) and growth models (GMs) (e.g., Colorado Growth Model).

The topic of the conference is relevant to statisticians and statistics educators because as discussion about how teachers should be prepared in statistics take place, it is important to consider how teachers should use and understand teacher performance measures. Since these measures are computed using statistical models, it is natural to discuss their use and computation while preparing teachers in statistics. This short piece is meant to summarize some of the ideas presented at the conference and connect these ideas to teacher preparation in statistics.

VAMs and GMs are statistical models estimating teacher effectiveness based on student performance. VAMs are intended to capture the contribution, or “value added,” of an input to student achievement. In general, VAMs, written in panel data form, look like:

Yit= αt+βj+Yi(t-1)γ+Xitζ+εit

where t indexes the grade, i the student, and j the teacher. Yit denotes the achievement of student i at time t while controlling for prior achievement in time t-1 on the right hand side with Yi(t-1). βj is a dummy variable for teacher j and thus provides the teacher effectiveness score. The vector Xit denotes a set of child, classroom, and school characteristics. As can be seen by the equation, VAMs rely on student standardized test scores and control for achievement on the prior test.

The GMs refer to models that estimate median growth in a teacher’s classroom. First, a growth percentile is computed for each student, and, second, a teacher rank is computed by finding the median growth percentile of the students in his/her class. To estimate the student percentile rank, quantile regression is employed. In particular, 100 percentile regressions are estimated (one for each percentile) while conditioning on prior test score. If QYit(τ|x) denotes the τth percentile of the current test score Yit, conditioned on prior test score Yi(t-1) then the model to estimate can be expressed as:

QYit(τ|Yi(t-1))= α+Yi(t-1)γ+Xitζ+εit

This equation represents the percentile τ of a student’s current test score conditional on the prior test score. A student’s percentile score is then computed by finding the predicted percentile closest to the score. When estimating the GM equations in practice, the test score on the right-hand side may be modeled nonlinearly, and typically the models do not control for child, classroom, and school characteristics.

Researchers working on VAMs and GMs are primarily concerned with estimating βj in the VAM equation and computing the median growth in a teacher’s classroom based on the quantile estimates for the students in the GM set-up. In both cases, the goal is to estimate teacher effectiveness in such a way that it accurately reflects the underlying effectiveness of a teacher’s ability, thus producing reliable measures.

To estimate these models, methodological choices must be made. Choosing an estimation method, thinking about measurement error present in the standardized tests, choosing between random teacher effects versus fixed teacher effects, and deciding on the inclusion of covariates to include in the model are just some of the statistical choices one must make to estimate the models. Related to these choices are the underlying assumptions each methodology relies upon. In general, estimating these models is complex due to issues of nonrandom sorting of students to teachers and teachers to students. There are several papers by the conference organizers and presenters, as well as others, that offer good resources for technical presentations of these models.

Teacher performance measures and pay for performance have been hot topics pushed to center stage due to NCLB requirements and Race to the Top federal funding opportunities. These national policies have required states and districts to incorporate teacher performance measures in teacher evaluation. States and districts may compute the performance measures and use them for several purposes (e.g., teacher assessment, summative information to inform teaching practices). Ideally, these forms of evaluation, among other uses, would be used to inform teachers of how to improve their teaching. Teachers who are able to use summative assessment in their classrooms to make decisions about teaching strategies could be an important “side-effect” gained from having teacher evaluation performance measures available.

As statisticians and statistics educators think about the statistical preparation of teachers, a key topic to discuss is how to train teachers to understand and make use of summative assessment in their classroom. Assessment and modeling concepts are by nature statistical and should be addressed by statisticians during the teacher preparation process. For in-service teachers, statisticians should be at the forefront of providing professional development to teachers on how to carry out assessment in the classroom, how to understand summative assessment prescribed by federal and state laws, and what the underlying modeling concepts are that are the backbone of teacher performance measures such as VAMs and GMs.

Through this, teachers may gain the ability to understand the language around the teacher performance debates. Of course, we would not expect teachers to take a course on and understand all the nuances and difficulties of estimating teacher performance measures; however, for example, understanding the complexities and interpretations of multivariable regression would be quite appropriate. Moreover, setting up the topic of regression from a modeling perspective is an important viewpoint for teachers to adapt and understand. Instead, teachers focus on single variable regression and discuss it in mathematical lessons while learning the equation of a line. Fitting a line of best fit becomes a calculator problem and the fundamental idea of modeling relationships is quickly lost.

To illustrate, suppose we have a theoretical relationship between several variables we would like to model. Teachers should be able to describe how to write a model (i.e., equation) that represents the relationship we believe may exist among the variables. Furthermore, teachers should be able to use common methods (e.g., OLS methods for regression) to estimate the parameters in the model.

Overall, teaching teachers about modeling in the context of teacher performance also can empower teachers to participate in the conversation revolving around their profession. Statistics is the natural home for this topic, and statistics educators and statisticians should capitalize on the opportunity to teach teachers about assessment and modeling in the education context.

Details and specific examples of how to teach teachers about VAMs and GMs can be found at Project-Set’s website. Also, more information about the conference discussed above and links to the papers and presentations can be found at the Michigan State University website.