Paul Osterman

Susan Sturm and Lani Guinier are driven by a concern for continued racial and gender-based inequities and are worried that affirmative action, as we know it, is proving an inadequate solution. Rather than use affirmative action to make amends around the edges, they want to rethink selection processes in general. Their characterization of the current state of play is that the "present system measures merit through scores on paper-and-pencil tests," and this is broadly unfair in both educational and employment settings. They believe that the approach they propose will not only lead to more equitable outcomes but will also help overcome the political opposition to affirmative action as practiced today.

I am very sympathetic to the objectives and underlying premises of this article. I agree that the distribution of rewards, economic and otherwise, in America is blatantly unfair and that an important element of this unfairness lies in racial and gender-based discrimination. I also agree that current selection and sorting processes should not be taken for granted. Having said this, I also think that the arguments made by Sturm and Guinier suffer from some problems of fact and logic that undermine their case.

My first worry is that their characterization of testing is wrong. It is weak both for its mischaracterization of the validity of tests and also for its exaggeration of the role tests play in admissions and in hiring. My second concern is that the solution they propose ignores some important lessons from research on organizational processes and is open to very substantial abuse against the groups they seek to protect.

Their argument that tests such as the SAT do a poor job of predicting performance is, I think, wrong. The Armed Forces Qualifying Test (AFQT), an SAT-like test that measures achievement (not ability or IQ) validly predicts job performance in the military, and a recent National Academy of Science report found that its reliability did not vary by race. With respect to the SAT itself, the research evidence shows that within a given institution (e.g. Yale) the SAT does not predict who does well. However, it seems problematic to argue that the SAT is not useful in predicting who will do well at Yale versus at Slippery Rock. That is, if I ask you about two people, one of whom has an SAT score of 1500 and the other 1000, and you knew nothing else about these people, it would not be hard to guess who has the higher probability of getting better grades at Yale if they were both admitted.

But why should we be denied all other knowledge about the applicants? Isn’t this exactly the authors’ point, that more subtle acceptance processes should be used? But, of course, they already are. No selective college admits simply on the basis of SAT scores and almost all would consider other factors–such as family circumstances, the nature of the high school the students attended, and motivation–that might mitigate the poor SAT. This is part of the reason why within an institution the SAT has little predictive power. Nonetheless, at the end of the day and in conjunction with these other considerations the SAT will still help discriminate between candidates for Yale and for Slippery Rock.

This overstatement is reinforced by the other canonical example Sturm and Guinier use, the civil service test. In this world, at least in the stereotyped view, the only hiring criterion is test scores, with hiring taking place from the top of the list, and affirmative action is thus vulnerable to being seen as unfair. However, it is difficult to think of any other jobs with a hiring procedure like this (and even the civil service doesn’t really work in this manner–consider veterans’ preferences). For virtually all jobs, particularly the "high-stakes" jobs the authors are concerned with, the hiring process already considers a mix of factors, just as colleges do in their admissions process. In the vast majority of today’s labor market, formal tests play a much weaker role in hiring than the author’s imply.

While much of the set-up for the article is centered on tests and the problems surrounding them, the fact is that in most private sector jobs tests are not at the core of hiring or promotion. Instead, the procedures are much more informal–as is illustrated by the case of Bernice cited by Sturm and Guinier. They argue that through a combination of luck and openness to informal consideration of new criteria, Bernice was given a chance that she would not have had under the old system. But the old system, as the authors describe it, was also informal and judgment-based. It’s just that a different set of criteria was part of the informality. So, in effect, the authors want to substitute one set of informal criteria and procedures for another.

The standards the authors want to deploy are attractive and are in tune with an important current of modern ideas about assessment and multiple intelligences. If we use the idea of tests in a more generic sense, to refer to hiring criteria, then I agree with the authors that organizations should be more flexible about what they consider. In fact, many employers are already increasingly using non-traditional criteria. Research on hiring practices in so called "high-performance work organizations" shows that those firms which are implementing work teams and quality circles put heavy emphasis on interpersonal skills and initiative, and that they get at these via multiple interviews and role playing, including the involvement of current employees in selecting new ones.

But there are some further dangers, which should give us pause. First, research on organizations, such as the work of Barbara Reskin, suggests that outcomes are better for women and minorities when hiring and promotion procedures are formalized and not left to informal processes, which are vulnerable to abuse. Yet, in the authors’ words, their proposals are "interactive and informal" and "rely explicitly on discretion and subjectivity." Not only does this run the risk highlighted by the research on organizational processes, but also seems to endanger the criteria of transparency and fairness that Sturm and Guinier lay out in their attack on traditional affirmative action and their sketch of the characteristics of a more desirable system. I think much more careful thought is needed to reconcile these concerns with the advantages of the approach advocated by the authors and those being adopted by cutting-edge firms.

For all these reasons, Sturm and Guinier fail to make as compelling a case as they might wish for their conclusions. Nevertheless, I must say that I basically think that their bottom line is right. Outcomes are unfair. So are the processes that generate them. Diversity has very substantial payoffs both for individuals and organizations. We need to find ways to legitimize diversity as one consideration, among others, in school admissions and in job market decisions. What we also need to do, however, is to think through carefully how to get there. It is possible that the current practice of, in effect, giving some additional "points" to target groups in admissions or hiring is a reasonable way to proceed.

Paul Osterman is professor at MIT’s Sloan School of Management and the author of Securing Prosperity.