New Study Strikes Latest Blow Against ‘Value-Added’ Teacher Evaluation

In the first large-scale analysis of new systems that evaluate teachers based partly on student test scores, education researchers have found a weak to nonexistent relationship between value-added models (VAM) of teacher performance and the content or quality of classroom instruction. This at best tenuous correlation calls into question the appropriateness of using the data in evaluating teachers or improving classroom instruction, the report says.

This study, conducted by Morgan Polikoff and Andrew Porter and published by the American Education Research Association, is just the latest addition to the mountain of evidence that value-added teacher evaluation – a collection of statistical techniques used for analyzing student test scores – is an unreliable, cookie-cutter method that leads to unfair and inaccurate performance evaluations.

Still, as more states design and implement new evaluation systems, the pressure to attach an undue importance on value-added measures is proving hard to resist. Thirty-five states and the District of Columbia require “student achievement,” or test scores,  to be a “significant” or the “most significant” factor in teacher evaluations. Only ten states do not require test scores to be used in teacher evaluations.

Polikoff and Porter analyzed and evaluated data from 327 fourth and eighth grade math and English teachers in six school districts – New York City, Dallas, Denver, Charlotte-Mecklenburg, Memphis, and Hillsborough County, Florida.  The data was collected from a larger project funded by the Bill and Melinda Gates Foundation known as the Measures of Effective Teaching (oddly enough. the Gates Foundation also funded the AERA study)

The two researchers found that some teachers who were well-regarded based on measures such as classroom observations, student surveys, and other indicators nonetheless had students who tests scores that were below average. At the same time, some teachers whose students had higher test scores didn’t do so well on those other measures.

Polikoff said the results were surprising.

“What we expected to find was that there were strong positive relationships between instructional alignment with these measures of quality, that it would predict student learning on state tests,” Polikoff explained. “But what we actually found was that there were very weak to zero relationships between pedagogical quality with the value-added measures.”

Polikoff and Porter believe that value-added measures do provide some useful information, they nonetheless are not picking up the qualities most people think of as being associated with good teaching.

“Our results suggest that it’s going to be difficult to use these systems to improve teacher performance,” said Polikoff.  “Given the growing extent to which states are using these measures for a wide array of decisions, our findings are troubling.”

But with the rollout of Common Core State Standards proceeding, Polikoff says it is imperative that all stakeholders develop a deeper understanding of the ways effective teachers implement the standards in the classroom.

The findings in the study, the two researchers said, leads to a disconcerting question that should be addressed by policymakers who are pushing these unreliable models: “If VAMs are not meaningfully associated with either the content or quality of instruction, what are they measuring?”

Despite its unreliability, VAM is being called into question across the country. In 2013, teachers in Florida, supported by the National Education Association and the Florida Education Association, filed a lawsuit charging that the state’s new evaluation system, which used a bizarre formula that incorporates test data from students who some teachers have never taught, violated the equal protection and due process clauses of the 14th Amendment of the U.S. Constitution. Although the judge agreed that the system was unreliable and unfair, he dismissed the suit because the problems did not meet the standard to be declared unconstitutional. In April, the Tennessee Education Association took similar legal action against the state’s value-added assessment system.

  • Liz Moran Capone

    Why isn’t the NEA doing more on the national level? Where is the NEA effort to counteract the “pressure” that is “hard to resist”? Why aren’t more of my dues dollars going to resisting? How many teachers will be unfairly rated and then terminated, or will resign before that happens, before the NEA wakes up?

  • Test results seem to be one of the fair ways to evaluate teachers. Classroom visits give only a snapshot of what may be going on in the classroom. Student surveys mostly show how popular the teacher is, that is, how the students relate to the teacher emotionally. It’s not surprising to me that test results and student surveys don’t correlate. — taught for 47 years at public school and college levels

  • Christopher

    Most standardized tests are “snapshots” themselves. Objectives should be taught and tested periodically during the year rather than at the end of the year. As a principal I know which teachers are doing a good job teaching.

    Testing should inform instruction. Good teaching should produce growth in students. Every student may not pass but they should grow.

    Since the inception of NCLB testing companies have gotten a huge amount of state education dollars. There seems to be a need on the part of some entities to “prove” public education is a waste of tax dollars.

    I will leave it at that.

    • Bilgewater

      Spot on comment, Christopher.

  • Fran Dennis

    It seems so logical that students’ test scores should correlate with the quality of teaching. I wouldn’t have believed otherwise until I saw it. I taught chemistry at a community college. Two sections of the same course, during the same term, using the same materials and the same teacher (me) frequently had significantly different averages on very similar tests. The differences held up for the entire term, with one section consistently outperforming the other.

  • Mireille Ellsworth

    The TEACHERS ARE The NEA! To anyone who is asking, “What is our union doing about fighting these reforms with no research base?” I ask you, “What have YOU done to resist or get involved in the fight?” If nothing else, talk to as many people as possible about the insanity going on in schools nowadays. I talk to whomever will listen, people waiting in doctor’s offices, cashiers at stores, store managers and owners, etc. Talk to your colleagues at the school level to discuss what you (collectively) are going to do about it! Better yet, run for union office, join a grassroots organization (like the Badass Techers Association), or at least read Diane Ravitch’s book Reign of Error so you know the WHOLE story! We need EVERYONE in this fight or the dedicated teachers will either lose sight of what authentic teaching & learning is or will lose their jobs, which is just as bad as losing access to helping the upcoming generations learn to think critically in order to deal with the horrible legacy we are leaving them!

  • cheap hermes handbags for women
    I every time spent my half an hour to read this webpage’s articles every day along with a cup of coffee.

  • gluten intolerance information

  • Pingback: 2014’s Best and Worst Players in Public Education()