What SAT Critics Miss
January 13, 2015
Jan 13, 2015
12 Min read time
Lani Guinier's The Tyranny of Meritocracy.
The Tyranny of Meritocracy: Democratizing Higher Education in America
Beacon Press, $24.95 (cloth)
In Lani Guinier’s portrait of higher education in the United States, the students at our most selective colleges and universities are preening mercenaries with “bad manners” who have no redeeming characteristics, apart from their ability to “perform well on a test.” This sorry state of affairs, according to Guinier, is the inevitable result of a higher education system slavishly devoted to standardized testing, a “testocracy” that has thoroughly confused test performance with “merit.” It reigns supreme on college campuses, where admissions committees base all of their decisions on a cursory glance at test scores. At the bottom of it all is the SAT—a “wealth test” that measures nothing more than rote memorization and worthless test-taking strategies and tricks.
In the face of this “nightmare” system, Guinier argues that we desperately need a culture shift in the academy, away from “testocratic” merit, which rewards wealth and privilege, toward “democratic” merit, which rewards “peer collaboration, leadership [and] drive.” She would like to see fewer students at top institutions hail from the gated communities of “We the Self-Appointed Leaders of Tomorrow” and more from the land of “We the People.” If you can look past the caricatures of elite college students as self-serving brats, you will discover that Guinier’s preeminent concern is with promoting more “meaningful race and class diversity” in higher education, a laudable objective that deserves serious consideration. By attacking standardized testing as the primary obstacle to more diverse student bodies, however, Guinier ignores the much more significant racial and economic disparities that are deeply entrenched in our educational system, starting in kindergarten.
• • •
No critique of standardized testing would be complete without a soliloquy on the evils of the multiple-choice format. “The world . . . provides us with more than one correct answer to most questions,” Guinier says, and nods to Bard College President Leon Botstein who tells us that no professional “pursues her vocation” by choosing the “right” answer from “a set of prescribed alternatives that trivialize complexity and ambiguity.” Incisive points, to be sure, but there are alternatives to the multiple-choice format. Many standardized tests, for instance, now include “open-response” items, which require students to fashion their own answers rather than simply choosing the one “correct” answer from a ready-made list. In my view, however, the limitations of standardized testing with respect to prefabricated questions are far more important than the shortcomings associated with prefabricated answers. The ability to formulate a significant question is a hugely important skill, especially for college-level work, and one that no standardized test even attempts to measure. Standardized tests, then, too often reinforce the dreary lesson taught by many schools that it is the job of students to answer rather than to ask questions.
I never thought I would feel compelled to defend the integrity of the College Board or the number-crunchers at U.S. News and World Report, but a few corrections of the kind of fanciful exaggerations favored by anti-testing crusaders are in order. It has been over twenty years since the SAT ceased to be an acronym but it seems the SAT will always be known as the Scholastic Aptitude Test in the popular imagination, forever associated with the attempt to measure native intellectual ability. Guinier only reinforces this common misconception, stating that the SAT “doesn’t even pretend to measure achievement.” But as the College Board website explains, the SAT “doesn’t test logic or abstract reasoning.” Rather, “it tests the skills you’re learning in school: reading, writing and math.” In other words, today’s SAT is meant to be an achievement rather than an aptitude test.
Guinier, like many critics of the SAT, is dismissive of the test’s predictive power, claiming that the correlation between SAT scores and first-year college grade-point-average is “very, very slight.” In fact, most studies put the figure in the neighborhood of .45, which is a shade higher than the correlation between rates of smoking and incidences of lung cancer. It is also only a tad lower than the correlation between cumulative high school GPA and first-year college GPA.
Finally, according to Guinier, the U.S. News annual college rankings “rely heavily on SAT scores for their calculations.” Admissions test scores actually account for just over 8 percent of a school’s ranking. U.S. News takes into account both SAT and ACT scores—although more college-bound students now take the ACT rather than the SAT, Guinier barely acknowledges the existence of the ACT, to say nothing of its increasing prominence in the higher education landscape.
• • •
Guinier has been arguing for years that the SAT is a “wealth test.” Is she right? Money indisputably matters. The correlation between socioeconomic status and SAT scores is around .40. (If the SAT were nothing but a wealth test, as Guinier maintains, this figure would be 1.00.) For high school graduates from the class of 2013, students from families earning more than $200,000 a year had an average combined SAT score of 1,714 (out of 2400) compared to an average combined score of 1,326 for students from families earning less than $20,000 a year. These averages, of course, obscure the enormous variation within different income brackets—many poor students ace the test while many rich ones bomb it.
Guinier and other anti-testing campaigners identify test prep as the main culprit behind economic disparities in SAT scores. Test prep programs, according to Guinier, ramp up scores and the ability to afford test prep makes all the difference in terms of pushing an application into the acceptance pile. Most studies, however, have found that test prep only boosts a student’s SAT score by an average of twenty to thirty points. This relatively small bump should not be decisive in most cases. There are plenty of good reasons to be concerned, even disturbed, by the negative consequences associated with test prep (it breeds anxiety, drains wallets, and wastes time, to name a few), but unswervingly tipping the admissions scales in favor of the affluent is not one of them.
Lani Guinier has been arguing for years that the SAT is a “wealth test.” Is she right?
It is not just unequal access to test prep, according to Guinier, that makes the SAT such an unfair test. The SAT, Guinier maintains, reflects the “values and culture” of “white, upper middle-class” students. Many scholars in the field of education readily endorse this claim. I know because I used to be one of them. Two years ago, though, after reading through the empirical research carried out by psychologists and psychometricians for a course I created on standardized testing and American education, I concluded that the charges of bias do not stand up to closer scrutiny. The overwhelming majority of leading measurement experts contest the notion that the SAT systematically underestimates the academic skills and knowledge of poor students and students of color. Indeed, Guinier is unable to provide a single specific example of racial or class bias on the test. Consider the fact that Asian Americans significantly outperform whites on the SAT. Is there nothing distinctive about the cultural heritage of Americans of Asian descent? Following Guinier’s logic, the tens of thousands of high-scoring first- and second-generation immigrants from India and China somehow share the “values and culture” of upper middle-class whites, even if they are working-class or their parents do not speak English as a native language.
The SAT is not the only measure where money matters. As University of California, Santa Barbara Professor Emerita Rebecca Zwick has shown, every measure of academic achievement is at least in part a “wealth test,” with higher-income students enjoying consistent performance advantages over their less well-off peers. On average, more affluent students have higher scores on annual tests mandated by the No Child Left Behind legislation; on tests with no coaching available such as the National Assessment of Educational Progress; as well as on high-school exit exams such as New York’s Regents Examination. The same goes for grade-point-averages and the ACT. So no matter the achievement metric, as a group, more affluent students always do better.
Just to be clear, for Guinier and many of the other most vociferous critics of standardized testing, the SAT is a meaningless metric. The only skills it measures are those necessary to succeed on the SAT itself. Access to test prep allows students to learn how to “game” the test and a white, upper-middle class cultural background somehow provides a key to unlock the test’s otherwise mystifying content. In her mind, nothing separates the high from the low-scoring students, apart from bank account balances and the color of their skin. But while the SAT has many noteworthy flaws (the essay section, for instance, is farcical and an insult to the craft of writing), it does a decent job fulfilling its central purpose, which is to measure “college-readiness.” Those students who do well on the SAT really are more likely to succeed in college than those who do poorly.
The school of thought that deems the SAT meaningless is deeply misguided. If you care about expanding educational opportunity for poor students and students of color, charges of class and racial bias turn out to be red herrings. And test prep is the biggest red herring of all. College admissions tests, by and large, register rather than create socio-economic and racial disparities. Weeks or even months of test prep are dwarfed by the lifetime of accrued advantages associated with wealth. Think high-quality nutrition and healthcare, as well as access to museums, travel, and extended social and professional networks. Consider too a dozen plus years of attending the best public schools or private schools with well-qualified teachers and a rigorous high school curriculum that includes a rich array of foreign language offerings, Advanced Placement courses and extracurricular activities. By the time they are seventeen, more affluent students are more likely to succeed on the SAT—and in college—because they have received better educations than their less well-off peers.
To deny the real differences in the linguistic, mathematical and analytical skills between the typical low-income student and the typical high-income student is inexcusable. The test prep industry and “biased tests” make for easy targets—but they distract us from the much more important inequalities that are embedded in our racially and economically stratified K–12 educational system. Across the country, for example, poor students and students of color are disproportionately overrepresented in the lowest academic tracks, and disproportionately underrepresented in “gifted and talented programs” and honors and Advanced Placement classes. We should concentrate on students’ broader educational trajectories, rather than obsessively honing in on the drama of college admissions. We need to stop thinking of college admissions as a point of departure and start looking at it as the culmination of a long journey.
In the spring of 2016, the College Board will roll out a newly redesigned SAT, a development Guinier breezily dismisses in two sentences. The changes to the test are some of the most dramatic in the history of the SAT, including a shift away from a focus on rare vocabulary words such as florid, lugubrious, and sagacity and a new emphasis on evidence-based reading and writing and “real-world” math problems.The new SAT represents a concerted effort by the College Board to address specific issues raised by critics of the SAT, including Guinier herself. For example, the College Board recently announced a partnership with Khan Academy, a nonprofit online resource that has helped students of all ages master material ranging from fractions to art history. “A future determined by merit, not money” is the tagline for this initiative, which will provide free online SAT test-prep. It is just one of several initiatives the College Board has announced to address the issues of equity, access, and opportunity. These issues are of paramount importance to Guinier and it is a shame that she takes no notice of the College Board’s attempt to reckon with them.
• • •
The subtitle of Guinier’s book is “Democratizing Higher Education in America.” She contends that we should open up the admissions process to move beyond test scores to include measures of character such as “leadership, emotional intelligence [and] the capacity to work with others to contribute to society.” This is a specious argument: if the “testocracy” truly dominated the higher education landscape in the way Guinier claims, schools such as Stanford would not routinely reject more than two-thirds of applicants with perfect SATs; nor would there be a burgeoning “test-optional” movement in which more than fifty selective colleges and universities—including Bowdoin, Wake Forest, and Wesleyan—have left the submission of test scores up to individual applicants. Selective colleges and universities, of course, already use a holistic approach to evaluating candidates, with careful consideration of an applicant’s high school academic record, personal essays, letters of recommendation, and participation in extracurricular activities. It is precisely in the fuzzier qualitative arenas that the more affluent students can most press their advantage. Whether it’s taking up fencing, volunteering to teach English in Ecuador, or participating in an expensive science competition, wealthier students have far more opportunities than their less privileged peers to construct the kind of eye-popping college applications that are most attractive to admissions committees.
One particularly noteworthy idea emerges: our educational system should re-orient itself around collaboration and peer learning.
Guinier’s vision of a more democratic system of higher education is more a hastily drawn sketch than a detailed blueprint. Nonetheless, out of an assortment of case studies focusing on exemplary schools where “everyone is in a good mood" and a brief review of psychological research concentrating on intelligence, peer learning, and diversity, one particularly noteworthy idea emerges: our educational system should re-orient itself around collaboration and peer learning. Guinier draws on her experience as an attorney, underscoring that the practice of law is an inescapably collaborative enterprise. She puts her finger on an important problem, which is the disconnect between the work students do in school and the work they will be required to do in the world beyond school.
Have you ever completed a job in a professional context that necessitates working alone with nothing more than a number two pencil? The message that we send to students through standardized testing is often perverse: we are going to assess your abilities in a vacuum, without access to the books, Internet, or peers that you will almost always have access to in the working world. In this respect, standardized testing promotes an antiquated model of teaching, learning, and knowledge. While collaboration in the workplace is rewarded, collaboration on a test is penalized as cheating. It is not just testing, however, that prizes individual achievement. The conventional classroom is organized around individual performance, with students laboring away at solitary desks. The push for more open, collaborative classrooms has always faced stiff resistance from “traditional” teachers and school administrators. If you agree with Guinier that education should be a more cooperative enterprise, then the crucial question is how to incentivize schools to embrace this cultural shift.
• • •
The Tyranny of Meritocracy is a timely book. With more colleges and universities adopting test-optional admissions policies, strident criticism of the new Common Core tests emerging, and major revisions to the No Child Left Behind legislation looming on the horizon, the role that standardized testing will play in the future of American education is genuinely up for debate. Voices like Guinier’s that imagine alternatives to an educational system oriented around testing are a welcome addition to the conversation.
While we have you...
...we need your help. Confronting the many challenges of COVID-19—from the medical to the economic, the social to the political—demands all the moral and deliberative clarity we can muster. In Thinking in a Pandemic, we’ve organized the latest arguments from doctors and epidemiologists, philosophers and economists, legal scholars and historians, activists and citizens, as they think not just through this moment but beyond it. While much remains uncertain, Boston Review’s responsibility to public reason is sure. That’s why you’ll never see a paywall or ads. It also means that we rely on you, our readers, for support. If you like what you read here, pledge your contribution to keep it free for everyone by making a tax-deductible donation.
January 13, 2015
12 Min read time