What Do School Tests Measure?

The New York Times, August 4, 2009

According to a New York Times analysis, New York City students have steadily improved their performance on statewide tests since Mayor Michael Bloomberg took control of the public schools seven years ago. While statewide passing rates on the tests have risen in every grade on English and math tests, New York City’s scores have gone up even more, and across all neighborhoods. The racial achievement gap has been cut in half on some tests.

This is good news for Mayor Bloomberg, who has made standardized testing a linchpin of his administration’s stewardship of the schools. Critics say the results are proof only that it is possible to “teach to the test.” What do the results mean? Are tests a good way to prepare students for future success?

Sandra Stotsky, professor of education reform
James Comer, professor of child psychiatry
Bruce Fuller, professor education and public policy
Richard D. Kahlenberg, Century Foundation
Veda Jairrels, professor and author
Lance T. Izumi, Pacific Research Institute
Marcelo and Carola Suárez-Orozco, N.Y.U.’s immigration studies program
Marcus Winters, Manhattan Institute

Are These Tests Any Good?

Sandra Stotsky is professor of education reform at the University of Arkansas.

I know of no country that doesn’t test its K-12 students at some point before they graduate from high school, with some countries giving more tests than others and at different grade levels depending on the number of years of compulsory education and levels of schooling available. Tests covering what students were expected to learn (guided by an agreed-upon curriculum) serve a useful purpose — to provide evidence of student effort, of student learning, of what teachers taught, and of what teachers may have failed to teach.

Good tests come in 47 varieties. There are reading-based prompts requiring extensive essays written on the spot, or sets of questions requiring a choice of the best answer from a group of possible answers (both formats can assess conceptual understanding). There are also “product” tests that reflect the application of the knowledge and skills acquired in a particular course of studies. The hysteria about testing per se is unwarranted.

While the number of students ‘passing’ has risen, nothing is happening at the level that should indicate academic success.

More serious questions arise about “teaching to the test.” If the test requires students to do something academically valuable — to demonstrate comprehension of high quality reading passages at an appropriate level of complexity and difficulty for the students’ grade, for example — then, of course, “teaching to the test” is appropriate. That is exactly what we want English or history teachers to do. What is not clear with respect to the New York State tests is the extent to which the English tests actually compel teachers to teach students how to read high quality literature written at an appropriate level of complexity or difficulty for the grade.

Based on a perusal of New York State’s grade 8 reading selections several years ago, I judged that the test was assessing the ability to understand passages more appropriate for grades 4 and 5. And this judgment was independent of where the cut score was set.

Why is this relevant today? The combined scores in New York City show a decline from grade 3 to grade 8 in the percentage of students who are “advanced proficient”: from 17 percent in grade 3, 20 percent in grade 4, 22 percent in grade 5, 16 percent in grade 6, 14 percent in grade 7 and 9 percent in grade 8. These combined scores are heavily influenced by the mathematics scores (which also decline regularly over the grades). If we look at the scores in English separately, we find that the percentage of students who are “advanced proficient: went from 8 percent in grade 3, 5 percent in grade 4, 10 percent in grade 5, 7 percent in grade 6, 4 percent in grade 7, to 3 percent in grade 8. Moreover, the percentages for this level of performance do not appear to have risen at all in the past decade; if anything, they appear to have declined. While the number of students “passing” has risen, nothing is happening at the level that should indicate academic success.

Reading is the crucial subject in the curriculum, affecting all the others, as we know. Teachers should be teaching to a demanding English test, but until we know that they are, one may ask: Should English teachers be teaching to these particular tests?

What We Really Need

James Comer is the Maurice Falk Professor of Child Psychiatry and the founder of the Yale Child Study Center at Yale University. He is the author of “Leave No Child Behind” and the forthcoming “What I Learned in School: Reflections on Race, Child Development and Education.”

Test scores are widely accepted in our country as a measure of student and school effectiveness. But the expressed purpose of education is to help prepare students to be successful in school and in life; to protect and promote their own health, development and learning, to be highly competent workers in school and beyond, to be competent and responsible family members (parents if they choose) community members and citizens capable of finding gratification and meaning in life. These outcomes are the product of a good developmental experience, and varied and rich curricular, instructional and assessment programs, in a caring school environment or culture created by adults who are selected, prepared and supported well in doing so.

There is significant evidence that teachers and administrators, adequately prepared and supported, can provide such experiences, raise test scores and meet the true purpose of education, and in turn the needs of society.

Test driven, or force-fed, learning can not enrich and promote the traits necessary for life success.

An almost exclusive focus on raising test scores usually leads to teaching to the test, denies rich academic content and fails to promote the pleasure in learning, and to motivate students to take responsibility for their own learning, behavior, discipline and perseverance to succeed in school and in life. Test driven, or force-fed, learning can not enrich and promote the traits necessary for life success. Indeed, it is dangerous to focus on raising test scores without reducing school drop out, crime and dependency rates, or improving the quality of the workforce and community life.

Students, families and groups that have been marginalized in the past are hurt most when the true purposes of education are not addressed. The focus on raising test scores without an even greater focus on preparation and support of the education workforce will continue to distract from providing those in greatest need and the nation with the world-class education system we all need.

Beware the Mayor’s Claims

img src=”https://www.pacificresearch.org/wp-content/uploads/2017/06/bruce_fuller.50.jpg” alt=”” width=”50″ height=”50″ class=”alignright size-full wp-image-12016″ />

Bruce Fuller is professor of education and public policy at the University of California, Berkeley.

New York City’s schools are getting better, according to multiple barometers, thanks to a variety of gutsy reform efforts. But the exaggerated trumpeting of modest progress — what’s become a yearly ritual in the mayor’s office — undercuts the credibility of Michael Bloomberg and his schools chief, Joel Klein.

Mayor Bloomberg claims that more than two-thirds of the city’s students are now proficient readers. But, according to federal education officials, only 25 percent cleared the proficient-achievement hurdle after taking the National Assessment of Education Progress, a more reliable and secure test in 2007.

When New York lowers standards and the mayor hypes the progress, it’s no surprise that parents and employers remain skeptical over the schools’ true efficacy.

This gap is precisely why few informed analysts still take seriously where state and city officials peg “proficient” student performance. Magically, even more students in Mississippi are “proficient” readers than allegedly high-flying New York City pupils.

The major lesson is that officials in all states — from New York to Mississippi — have succumbed to heavy political pressure to somehow show progress. They lower the proficiency bar, dumb down tests and distribute curricular guides to teachers filled with study questions that mirror state exams.

This is why the Obama administration has nudged 47 states to come around the table to define what a proficient student truly knows. Somehow the mayor’s back-slapping press release failed to mention this test-score inflation that has raged in New York and across the nation ever since No Child Left Behind let state officials define cut-points signaling when a student is proficient or not.

Mayor Bloomberg’s claim of dramatic achievement growth is dubious when placed in historical context. The mayor claims that the share of eighth graders proficient in math has climbed from 29 percent in 2002 to more than 82 percent this past spring — an eye-popping 52-point increase in just seven years, three times that detected by the more reliable federal assessment over the last two decades.

The city’s youngsters are certainly acquiring basic literacy skills more effectively now than a decade ago. But when Albany lowers standards and the mayor hypes the progress, it’s no surprise that parents and employers remain skeptical over the schools’ true efficacy.

What’s key in moving forward is to depoliticize student testing and hold public officials accountable when they grossly overstate progress. Given Mayor Bloomberg’s faith in education markets — forcefully backing charter schools and competition among reform groups that help lift the schools — he should know that parental choice works only when families have sound and reliable information about school quality.

The mayor’s self-congratulatory interpretation of student progress prompts a feeling of disbelief, not one of confidence.

The Problem for Low-Income Students

Richard D. Kahlenberg is a senior fellow at the Century Foundation and author of “All Together Now: Creating Middle-Class Schools through Public School Choice,” and “Tough Liberal: Albert Shanker and the Battles Over Schools, Unions, Race and Democracy”.

Test score gains among New York City students are important because research finds that how well one performs on cognitive tests matters more to one’s life chances than ever before. Mastery of reading and math, in particular, are significant because they provide the gateway to higher learning and critical thinking. But test score results can also be easily overblown and obscure significant disadvantages still faced by children in New York City’s high poverty schools.

Whatever the score, children in high poverty are still cut off from networks of students, and students’ parents, who can ease access to employment.

Consider, for example, the over-the-top coverage provided to gains in New York state exams by students at an overwhelmingly low-income school that is part of the Harlem Children’s Zone. According to The Times columnist, David Brooks, by eighth grade, in math, the Zone’s middle-school, the Promise Academy, “eliminated the achievement gap between its black students and the city average for white students.” The “approach works,” Mr. Brooks wrote, implying that separate schools for rich and poor and black and white can, in fact, be equal after all. But this conclusion raises two problems, which illustrate the limitations of test score results.

First, just because students are trained to do well on a particular test doesn’t mean they’ve mastered certain skills. As the Columbia University professor Aaron Pallas pointed out, on a different assessment — the Iowa Test of Basic Skills — eighth-grade Promise Academy students scored at the 33rd percentile for a national sample in math. This is important, Mr. Pallas notes, because if the New York State test score gains are real, and not just the result of test prep, the success should transfer to other tests.

Second, whatever the test score results, children in high poverty schools like the Promise Academy are still cut off from networks of students, and students’ parents, who can ease access to employment. This is important given research finding that more than half of jobs are filled through connections.

By all means, let’s celebrate test score gains in New York City, and the narrowing of the achievement gap. But hold the champagne until we can show more.

Preparation for Life

Veda Jairrels, a professor of exceptional education at Clark Atlanta University, is the author of “African Americans and Standardized Tests: The Real Reason for Low Test Scores.”

Reliable and valid standardized tests can be one way to measure what some students have learned. Although they may be indicators of future academic success, they don’t “prepare” students for future success.

There are other methods of assessment that could be more effective.

There are other methods of assessment (role-playing, for instance) that could be effective and representative of real life problem-solving scenarios that the students could actually encounter as adults. Just as some universities have used alternative methods for evaluating at least some students for admission, perhaps K-12 schools (and the federal government) should consider alternative methods of evaluation for more than just students with severe disabilities.

The reliance on standardized tests troubles me for personal reasons. I attended public schools with students who worked diligently, exhibited model classroom behavior, made good grades and yet still managed to score low on at least some standardized tests. These former schoolmates of mine are successful contributing members of society today. Fortunately for them, standardized testing in K-12 schools had not reached the gatekeeper status that it has today.

In spite of my reservations about standardized testing, I am a realist and believe that this type of assessment will continue. I also admit that alternative methods of evaluation take time and money. Yes, it probably is cheaper and less time consuming to shove the bubble-in answer sheet in front of every child.

The key to improving the performance of many African-American students on standardized tests involves more reading, beginning at birth with parents reading to their infants and continuing to do so as their children grow older. Parents should do everything possible to encourage their children to read for pleasure. And it’s not just parents who should do this, educators, church leaders and concerned citizens should work together to achieve this goal.

What Tests Can and Should Do

Lance T. Izumi is the senior director of education studies at the Pacific Research Institute.

In the wake of widespread state testing following the enactment of the No Child Left Behind Act, critics have claimed that teachers are simply teaching to the tests. Yet this argument is overly simplistic and ignores the benefits that come from good tests.

Opponents of testing try to have it both ways. When test scores are low they argue for a holistic view of student achievement that focuses on non-test indicators of performance rather than teaching to the test. When results are high, as in the recent rise in New York City scores, they counter that the scores are suspect because teachers are just teaching to the test. If a state test is well conceived, both these arguments fail to hold water.

If tests are reliably aligned with rigorous state academic content standards, then teachers are right to teach to the test.

Susan Philips, a professor of education at Michigan State University and one of the nation’s leading testing experts, has testified that well-developed standardized multiple-choice tests give more individual examples of student knowledge and skills, are more consistent in scoring, are capable of measuring higher-order thinking and are fairer than other non-standardized assessments. Since standardized testing can accurately assess the “whole” student, low test scores can be a real indicator of student knowledge and deficiencies.

If tests are reliably aligned with rigorous state academic content standards, then teachers who teach to the standards are teaching to the test, and there is nothing wrong with that. E.D. Hirsch, author and University of Virginia education professor, notes that “grade-by-grade standards and some form of fair grade-by-grade tests are logically necessary for monitoring and attaining grade-by-grade readiness.” Many teachers at high-performing, high-poverty schools have said they use student test scores as diagnostic tools to address student weaknesses and raise achievement.

While inappropriate use of test materials should not be countenanced, a valid standardized test linked to tough standards is a critical tool for measuring and improving student performance. Assuming New York has such a test, when Joel Klein says that if test prep means “teaching people to read and understand paragraphs, that’s what I think education is about,” then he’s right.

Ignoring the Needs of English-Language Learners

Marcelo Suárez-Orozco and Carola Suárez-Orozco are the co-directors of the immigration studies program at New York University and the co-authors of “Learning A New Land: Immigrant Students in American Society.”

This article overlooks the elephant in the American classroom: the educational progress of English-language learners, the fastest-growing group of students in American schools. This neglect may not be surprising given that the ever more Byzantine academic testing architecture has never been designed with the educational needs of this population in mind.

This test regime has huge implications for dropout rates and access for immigrant children.

The current high-stakes testing and accountability systems create unintended consequences for immigrant English-language learners, which outweigh whatever benefits standardized tests may have. Because too many immigrant students attend highly segregated and impoverished schools, are not exposed to quality curricula and undergo multiple school and programmatic transitions, their performance on such tests is often compromised. Is it any surprise then that in the “gold standard” (not what The Times calls “the dumbed down” New York State Test) National Assessment of Educational Progress reading assessment for 2007, 71 percent of English-language learners in the eighth grade scored “below basic” in reading and zero percent scored at the “advanced” level?

The high-stakes testing context is proving to be extremely challenging to newcomers. Not only are many immigrant children tested before their academic language skills have adequately developed, but all too often their day-to-day educational experiences are shaped by instruction that teaches to the test, which is far from an adequate measure of what it takes to succeed in the complex and challenging economies and societies of the 21st century.

This eye on the omnipresent “adequate yearly progress” is more often than not at the expense of more engaging, broader academic knowledge. What’s more, this test regime has huge implications for dropout rates as well as college access for children of immigrant families.

Making the Best of a Flawed System

Marcus Winters is a senior fellow at the Manhattan Institute, where he has done several studies on high-stakes testing, school report cards and the effects of vouchers on the public school system.

One of the pitfalls of standardized tests, perhaps the most important accountability-focused reform, is their elevation of scores over genuine learning. In high-stakes testing jurisdictions, anxious teachers, in order to avoid earning bad grades themselves, “teach to the test,” as the saying goes. In doing so, fortunately, some teachers adopt classroom techniques that produce real increases in student proficiency, particularly among the lowest-performing students. Tough proctoring rules can deter less well-motivated teachers from raising test scores in more underhanded ways.

The bigger problem with standardized tests is their emphasis on the achievement of only minimal proficiency.

The bigger problem with standardized tests is their emphasis on the achievement of only minimal proficiency. In most programs, the proficiency benchmarks that students must pass are levels of literacy and numeracy so low that only the most academically troubled students will find themselves better prepared for the outside world. High-achieving students, by contrast, will have already far exceeded them. While it is imperative that even the least accomplished students have sufficient reading and calculating skills to become self-supporting, these are nonetheless the students with, overall, the fewest opportunities in the working world. Meanwhile, limited resources are relocated away from the most promising students. If the premise of our educational system is that all students must be able to crawl before we help others to run, then such a policy is a worthy one.

Regardless of how high or low we choose to set the proficiency bar, standardized test scores are the most objective and best way of measuring it. Still, they are flawed. On a multiple choice exam, a child can demonstrate whether he can read and grasp the gist of a piece of writing, but he cannot usually demonstrate the depth or thoroughness with which he comprehends it. The gap between proficiency and true comprehension would be especially wide in the case of the brightest students. These would be the ones least well-served by high-stakes testing.