Erasures are detected by the same electronic scanners that CTB/McGraw-Hill, D.C.'s testing company, uses to score the tests. When test-takers change answers, they erase penciled-in bubble marks that leave behind a smudge; the machines tally the erasures as well as the new answers for each student.
In 2007-08, six classrooms out of the eight taking tests at Noyes were flagged by McGraw-Hill because of high wrong-to-right erasure rates. The pattern was repeated in the 2008-09 and 2009-10 school years, when 80% of Noyes classrooms were flagged by McGraw-Hill.
On the 2009 reading test, for example, seventh-graders in one Noyes classroom averaged 12.7 wrong-to-right erasures per student on answer sheets; the average for seventh-graders in all D.C. schools on that test was less than 1. The odds are better for winning the Powerball grand prize than having that many erasures by chance, according to statisticians consulted by USA TODAY.
"This is an abnormal pattern," says Thomas Haladyna, a professor emeritus at Arizona State University who has studied testing for 20 years.
But when did it become criminal for a student to erase a wrong answer and mark a correct answer?
There can be innocent reasons for multiple erasures. A student can lose his place on the answer sheet, fill in answers on the wrong rows, then change them when he realizes his mistake. And, as McGraw-Hill said in a March 2009 report to D.C. officials, studies also show that test-takers change answers more often when they are encouraged to review their work. The same report emphasizes that educators "should not draw conclusions about cheating behavior" from the data alone.
Haladyna notes, however, that when entire classrooms at schools with statistically rare erasures show fast-rising test scores, that suggests someone might have "tampered with the answer sheets," perhaps after the tests were collected from students. Although not proof of cheating, such a case underscores the need for an investigation, he says.
Hmm. Why would anyone have cause to tamper with test sheets? Could bonuses for principals in the DC schools have anything to do with it? Bonuses worth thousands of dollars, offered by Rhee and her lieutenants for rapid gains on standardized tests?
Sorry; those who know aren't talking. Most of them evacuated after Rhee lost her job last November.
In April, state superintendent [Deborah] Gist left Washington to take a job as head of Rhode Island's state school system. Her successor, Kerri Briggs, then dropped the request for D.C. public schools to investigate its schools. Both Gist and Briggs, now director for education reform at the George W. Bush Institute in Texas, declined to comment.
But others are singing like canaries.
From the start, Rhee emphasized a need to raise scores, restore calm to chaotic schools and close those with lagging scores and small enrollments. She paid bonuses to principals and teachers who produced big gains on scores. She let go dozens of principals and fired at least 600 teachers. Others retired or quit.
Turnover was brisk. Richard Whitmire, author of The Bee Eater, a biography of Rhee, reported that Rhee hired 1,918 teachers during her three years in office –– about 45% of those on the payroll last October. Only 2,318 current teachers had been hired before Rhee took charge.
The pressure on principals was unrelenting, says Aona Jefferson, a former D.C. principal who is now president of the Council of School Officers, representing principals and other administrators. Every year, Jefferson says, Rhee met with each principal and asked what kind of test score gains he would post in the coming school year. Jefferson says principals told her that Rhee expected them to increase scores by 10 percentile points or more every year. "What do you do when your chancellor asks, 'How many points can you guarantee this year?' " Jefferson says. "How is a principal supposed to do that?"
Rhee churned through principals. The Washington Post reported that Rhee appointed 91 principals in her three years as chancellor, 39 of whom no longer held those jobs in August 2010. Some left on their own, either resigning or retiring; other principals, on one-year contracts, were let go for not producing quickly enough.
Union officials say the pressure for high test scores may have tempted educators to cheat.
"This is like an education Ponzi scam," says Nathan Saunders, head of the Washington Teachers' Union. "If your test scores improve, you make more money. If not, you get fired. That's incredibly dangerous."
Dana Goldstein, a columnist for the Daily Beast, took up the issue yesterday and explained why Saunders is right, that incentives for high test scores are dangerous.
In the social sciences, there is an oft-repeated maxim called Campbell’s Law, named after Donald Campbell, a psychologist who studied human creativity. Campbell’s Law states that incentives corrupt. In other words, the more punishments and rewards—such as merit pay—are associated with the results of any given test, the more likely it is that the test’s results will be rendered meaningless, either through outright cheating or through teaching to the test in a way that narrows the curriculum and renders real learning obsolete.
In the era of No Child Left Behind, Campbell’s Law has proved true again and again. When the federal government began threatening to restructure or shut-down schools that did not achieve across-the-board student “proficiency” on state reading and math exams, states responded by creating standardized tests that were easier and easier to pass. Alabama, for example, reported that 85 percent of its fourth-graders were proficient in reading in 2005, even though only 22 percent of the state’s students demonstrated proficiency on the National Assessment of Educational Progress, the gold standard, no-stakes exam administered by the federal government.
Simultaneously, instances of outright cheating were rising nationwide. The USA Today investigation on the probable cheating in Washington, D.C. is just one article in a must-read series based on student achievement data culled from 24,000 public schools across the country. The paper found 1,610 instances in which test score gains from year to year exceeded three standard deviations—a jump greater than that of 99.7 percent of all test-takers annually in any given state, the threshold at which statisticians agree that test results may be suspect.
The good news is that Campbell’s Law does not mean we should give up on assessing students and holding school systems accountable for their academic success. Research shows that certain kinds of exams—those that require essay writing on broad themes, for example—enhance student learning of key concepts. We can also assess students by requiring them to give oral presentations, or by looking for growth in portfolios of their work over the course of a year. Effective teachers produce students who excel when held to these more sophisticated standards, which are difficult to fudge or cheat.
Of course, creating better testing systems will be expensive, and implementing them will demand significant expertise on the part of school administrators. But as the Obama administration and national education reformers—Michelle Rhee chief among them—ask states and school districts nationwide to tie teacher evaluation scores and pay to student performance, it is crucial that we measure student academic growth in nuanced ways that encourage deep learning, not in over-simplified ways that create perverse incentives to dumb-down the curriculum and cheat.
No comments:
Post a Comment