NUT Report: "We have to do something."

When asked reasonable questions and explanations regarding education reform initiatives, the replies are predictable. When it comes to merit pay based on value-added measurement (VAM), the rationale is described as:
1) We need to know the truth.
2) We have to do something.

In addition to the two recent studies reported here and here, another group of assessment experts from the Economic Policy Institute sent a letter to the NY Board of Regents showing evidence that VAM is unstable and prone to error. Among the concerns expressed, this team of experts noted the following:

Teachers’ ratings are affected by differences in the students who are assigned to them. Students are not randomly assigned to teachers – and statistical models cannot fully adjust for the fact that some teachers will have a disproportionate number of students who may be exceptionally difficult to teach (students with poor attendance, who are homeless, who have severe problems at home, etc.) and whose scores on traditional tests have unacceptably low validity (e.g. those who have special education needs or who are English language learners). All of these factors can create both misestimates of teachers’ effectiveness and disincentives for teachers to want to teach the neediest students, creating incentives for teachers to seek to teach those students those expected to make the most rapid gains and to avoid schools and classrooms serving struggling students.

These experts also cited a RAND Corporation study that concludes:

....the research base is currently insufficient to support the use of VAM for high-stakes decisions about individual teachers or schools. It is important that policymakers, practitioners, and VAM researchers work together, so that research is informed by the practical needs and constraints facing users of VAM and that implementation of the models is informed by an understanding of what inferences and decisions the research currently supports.

Nevertheless, the NY Board of Regents ignored the evidence and joined the wave of national experimentation with no reasonable rationale nor justification.

How many expert studies are required before education reformers respond to the legitimate concerns over this massive implementation effort? Unstable, unreliable data will not reveal the "truth" and "We have to do something" are not adequate explanations. Decades of doing "something" that does not work is irresponsible policy making.

The following experts signed the letter:
Eva Baker, Distinguished Professor, UCLA Graduate School of Education
Director, National Center for Research on Evaluation, Standards and Student Testing (CRESST)
President, World Educational Research Association, 2010-2012
Past President, American Educational Research Association

Linda Darling-Hammond, Charles E. Ducommun Professor of Education, Stanford University
Past President, American Educational Research Association
Executive Board Member, National Academy of Education

Edward Haertel, Vida Jacks Professor of Education, Stanford University
Chair, Board on Testing and Assessment, National Research Council
Vice-President, National Academy of Education
Past President, National Council on Measurement in Education

Helen F. Ladd, Edgar Thompson Professor of Public Policy and Professor of Economics, Sanford School of Public Policy, Duke University
President, Association of Public Policy and Management

Henry M. Levin, William Heard Kilpatrick Professor of Economics and Education, Teachers College, Columbia University
Past President, Evaluation Research Society
Past President, Comparative and International Education Society

Robert E. Linn, Professor Emeritus, University of Colorado at Boulder
Past President, American Educational Research Association
Past President, National Council on Measurement in Education

Aaron Pallas, Professor of Sociology and Education, Teachers College, Columbia University
Fellow, American Educational Research Association

Richard Shavelson, Dean Emeritus and Margaret Jacks Professor Emeritus, Stanford University
Past President, American Educational Research Association

Lorrie A. Shepard, Dean & Distinguished Professor, University of Colorado at Boulder
Past President, American Educational Research Association
Past President National Academy of Education
Past President National Council on Measurement in Education

Lee S. Shulman, Charles E. Ducommun Professor Emeritus, Stanford University
President Emeritus, Carnegie Foundation for the Advancement of Teaching
Past President, American Educational Research Association