Bogus Ratings: Another Tool in the School Wars

New York City’s teachers union lost a two-year battle with the city’s Department of Education February 24 when the city released controversial teacher “data reports.” The reports rate more than 18,000 elementary and middle school teachers by their students’ performance on state standardized tests.

The publication of the reports has been widely criticized as yet another “blame the teacher” attack in the intensifying war over public education.

The reports used a statistical measure known as “value-added” to calculate the portion of a student’s progress from year to year that can be attributed to an individual teacher. The measure was adapted from similar formulas used to judge changes in the value of agricultural commodities caused by processing, such as a pig turned into bacon.

Value-added ratings can be erratic, showing a teacher’s effectiveness as high one year and low the next. (Read more about one teacher's experience here.)

Teachers were given scores from 0 to 100 based on tests taken between 2007 and 2010. A computer program predicted student performance based on the past year’s test results, with some adjustments made for factors like income level and race. Teachers were rated at, above, or below average based on how well their students’ actual performance matched predicted scores. Those that showed the most “progress” got the highest ratings.

But the scores have margins of error as high as 53 out of 100 points for English and 35 points for math. This means a teacher rated “below average” could just as easily be “average” and an “above average” teacher might really be somewhere in the middle. Some teacher’s scores are based on as few as 10 students.

Furthermore, many question whether it’s possible to isolate an individual teacher’s input from all other factors affecting student performance, such as income level, family situation, health, or even a child’s particular state of mind on test day.

“Teachers all over the city are incredulous,” said Kelly Wolcott, an eighth-grade teacher in Brooklyn. “There’s a lot of confusion, mostly due to the fact that there are a lot of errors.”

She says one of her colleagues received two different reports for the same set of kids in the same subject in the same year. Others report receiving scores for grades, subjects, and years they never taught. (The tests of course don’t measure learning in arts, history, or any other area, leading many to charge the emphasis on testing numbs the curriculum.)

Even the city says the scores alone are not an accurate measure of teacher performance. “The purpose of these reports is not to look at any individual score in isolation ever,” said Shael Polakow-Suransky, the chief academic officer for city schools.

Yet New York tabloids wasted no time, plastering pages with headlines screaming about the “city’s worst teachers.” One Los Angeles teacher committed suicide in 2010, after the Los Angeles Times published a front-page series rating individual teachers using a similar “teacher effectiveness” scale that scored each teacher based solely on student progress on a state test.

Bill Gates, whose foundation prominently backs the use of students’ test scores to evaluate teachers, called the publication of individual teacher scores “a big mistake” and “a capricious exercise in public shaming” in a New York Times editorial.

So why were such flawed data made public?

“They only put these out to place attention on the problem of ‘bad teachers,’ whether those teachers are really bad or not,” said Kit Wainer, a high school social studies teacher in Brooklyn. “The theory is that if the key problem is teacher quality, then you can solve it by changing the teaching staff at a given school.”

Indeed, replacing teachers at schools deemed to be “failing” has been central to the education overhaul sweeping cities across the country.

UP-ENDING SCHOOLS

New York City’s appointed education overseers ignored a series of protests by teachers, students, and community members when they confirmed “turnaround” plans for 33 schools March 5. The schools will close at the end of the school year and immediately reopen with new principals, new curricula, and new teachers. Turnaround requires that at least half the teaching staff at a school be replaced.

SUPPORT LABOR NOTES

BECOME A MONTHLY DONOR

Give $10 a month or more and get our "Fight the Boss, Build the Union" T-shirt.

In addition, evaluation scores were being used in tenure decisions even before the data were released to the public. Last year, under new evaluation protocols incorporating the data, only 58 percent of eligible teachers were granted tenure. Five years ago, that number was 99 percent.

“They’re trying to malign the work of public school teachers and undermine tenure to clear veteran teachers out of the system to replace them with ‘teaching temps’ like Teach for America,” Wolcott said. “They want a workforce that replenishes itself every three to four years.”

Replacing career teachers with young teachers who don’t stay in education means lower salary costs for a district. It also means weaker unions, as longstanding activists are replaced by an increasingly transitory membership with high turnover and little investment in teaching as a profession.

COURTHOUSE STRATEGY

The data reports were initially developed in 2008 under former schools Chancellor Joel Klein as part of a push to evaluate teachers on how well their students perform on standardized tests.

Under pressure from city leaders and their corporate-education allies, the city’s teachers union (UFT) supported the collection of data for the reports in 2008—on the condition that the data not be made public. Klein and his administration promised the data would not be released.

While the education department didn’t hand the scores over to the media outright, reporters were told, with a wink and a nod, to file requests under New York’s freedom of information law. The UFT fought release of the reports in court for nearly two years but on February 14 a judge cleared the way.

Some teachers are critical of the UFT’s reliance on press conferences and court cases to block the reports and convince the public.

“They should have immediately organized protests around the release of the reports,” said Megan Behrent, a high school teacher in Brooklyn. “The attack is immense, but the response from the UFT has been minimal at best.”

After the data were cleared for release, the normally sluggish Department of Education produced the data “very quickly” in response to reporters’ requests, said Gloria Brandman, a special ed teacher in Brooklyn.

“It’s part of their ongoing attack to discredit teachers and get rid of as many as possible,” she said, adding, “of course, our union agreed to these reports in the first place.”

RACE TO SOMEWHERE

Ironically, the city’s Department of Education announced last September that it would stop producing teacher data reports, not because of the unreliability of the data but because the state is now developing similar teacher evaluations, based on the same standardized tests.

In order to maintain federal Race to the Top funding, states were ordered to develop test-based teacher evaluation systems. And now they are placing great faith in them, despite the wild error ranges reported in New York City.

The New York State Education Department reached a deal February 16 with the New York State United Teachers union for a new system that would base a minimum of 20 percent of teachers’ evaluation scores on their students’ standardized tests. An additional 20 percent will be based on test-based metrics bargained over by the union and the school district.

Some teacher unionists blasted the test-heavy approach.

“You have done nothing to protect teachers or advance our agenda as professionals to be respected,” wrote the president of a local teachers union outside Buffalo. “NYSUT caved under pressure and ran from the good fight.”