At a campaign event for a Democratic candidate in October, Joe Biden embarked down a classically Bidenesque aside. During a soliloquy about civility, history, and change in U.S. politics, Biden lamented that the contemporary scene was “such a mean-spirited political environment.” He recalled a different era: “I’ve been around so long, I worked with James Eastland”—a notorious white supremacist from Mississippi and longtime chairman of the Senate Judiciary Committee. “In the days when I got [to the Senate], the Democratic Party still had seven or eight old-fashioned Democratic segregationists,” Biden said. “You’d get up and you’d argue like the devil with them. Then you’d go down and have lunch or dinner together. The political system worked. We were divided on issues, but the political system worked.”

His remarks drew caustic responses from liberals unmoved by nostalgic paeans to the days of backslapping bonhomie with racists, but in true Biden Gaffe fashion the reminiscence conveyed something instructive to those who share the former Vice President’s discontent with the rancor and dysfunction of contemporary politics. That bygone era of relative partisan comity was real, but it was also rooted in some of the darkest iniquities of U.S. history. In other words, the benefits of a system that “works” in the manner of Biden’s depolarized mid-twentieth century are profound—and so are the costs.

Biden’s assessment that politics today are broken echoes across the culture. Party polarization dominates public discussion of our system and appears to define its core dysfunctions. Scholarly volumes appear regularly with cheery titles such as American Gridlock (2016), Deeply Divided (2014) and Why Washington Won’t Work (2015) that diagnose the pathologies of polarization and fitfully offer solutions. A recent Yale conference themed around the death of democracies—and the growing danger of it happening in the United States—featured polarization among a list of systemic ills. And when Thomas Ricks of Foreign Policy queried several national security experts about the probability of civil war breaking out in the United States within the next fifteen years, their average estimate was 35 percent.

To be human and sentient is to be dissatisfied with current U.S. politics. But realistic alternatives to today’s polarization may offer a cure worse than the disease.

There is irony in this. More than half a century ago, some leading scholars, journalists, and practicing politicians surveyed the U.S political landscape and similarly identified dysfunctions and pathologies that cried out for reform. They too produced alarming volumes with downer titles like The Deadlock of Democracy (1963) and House Out of Order (1965). But what those analysts identified as the central problem in U.S. politics was excessive bipartisanship—and they prescribed polarization as its solution.

The system they critiqued was the very one that Biden walked into in the early seventies: a system in which each party contained within its ranks a vast ideological range that overlapped significantly with the other party’s. Such fuzzily indistinct parties, so the critics’ argument went, served not only to hinder the passage of desirable public policy but also to deny voters meaningful democratic choice and accountability. This argument had impact. It informed the actions of figures on the left and right who worked consciously during the second half of the twentieth century to reshape the parties and their operations around internally cohesive and mutually distinct ideologies. As we know today, they succeeded.

Yet the fruits of their work would likely not satisfy polarization’s mid-century champions any more than they satisfy us. Indeed, the champions of polarization got several big things wrong. Eschewing the pitfalls of misguided nostalgia and false hope in easy solutions, however, also means acknowledging how much they got right. As we grapple with an era of seemingly permanent crisis, we would be wise to reassess the place of polarization in our analysis. The era of ideologically-charged partisan combat likely is not going anywhere soon. The plausible alternatives could likely be worse. Though we may not like it, when it comes to some of the key problems in twenty-first-century U.S. politics, polarization may very well, still, be part of the solution.

•••

“We ought to have two real parties,” Franklin Roosevelt remarked to an aide in 1944, “one liberal, and the other conservative.” This was a wish rather than an observation given the sizeable contingent of conservative Democrats who allied with Republicans to thwart New Deal legislative initiatives. Roosevelt had tried to address this problem in 1938 when he stumped, unsuccessfully, for liberal primary challengers to some of his chief Democratic congressional critics. And six years later, he made his comment about “real parties” in the midst of secret preparations for another attempt at a top-down realignment: forging an alliance with his moderate Republican opponent of 1940, Wendell Willkie, on behalf of a new party combining the liberal wings of the existing Democratic and Republican parties. Willkie had responded favorably to the idea, lamenting that “both parties are hybrids,” but his untimely death that year scotched the effort.

Alternatives to party polarization are not necessarily improvements on it.

In the wake of such failed gambits, the political system that developed during the New Deal years and flourished in the two decades following World War II demarcated a distinct and unusual era of depolarization in national politics. Because the major ideological divides of the period cross-cut rather than reinforced the partisan divide, most lawmaking was carried out via bipartisan coalitions. With parties divided internally (especially the electorally dominant Democrats), congressional power was decentralized from party leaders to powerful and autonomous committee chairs who, thanks to seniority, were disproportionately longtime incumbents from the one-party South.

Norms of civility and across-the-aisle camaraderie were powerful. “Integrity crosses party lines,” a Republican told one scholar in the 1950s. “You rely on some of your Democratic colleagues equally.” The strategic logic of constructive engagement during periods of divided government was epitomized by Democratic congressional leaders’ posture toward Dwight Eisenhower’s presidency. “We are going to look upon the president’s recommendation with kindliness,” Speaker of the House Sam Rayburn said in 1955, “because he is the leader of our country. We are not going to be against [his program] just because a Republican president has recommended it.” Remarks such as this contributed to a lasting impression of the era as a time of staid consensus, but deep and ideologically charged disagreements still abounded in U.S. life and politics. The party system, however, served to muzzle rather than clarify those disagreements.

Such a system facilitated legislative productivity—of a particular kind. A system of ad hoc bipartisan coalitions and decentralized lawmaking via committees accentuated the localism and incrementalism intrinsic to Congress. The era’s clubby deal making proved conducive to small-bore legislation serving parochial interests and was usually less friendly to comprehensive and coherent programmatic agendas. That very parochialism, in turn, helped office-holders to build localized support and to disconnect themselves from their party’s national reputation when necessary. In 1972, the election year that won twenty-nine-year-old Joe Biden a Senate seat, the rate of split-ticket voting in the United States reached a twentieth-century high.

That world is, of course, long gone. Rates of party discipline and polarization in Congress began slowly to rebound just a few years into Biden’s Senate career, followed later by a resurgence of partisanship within the mass electorate. Though a large library could be filled with the scholarly investigations of why and how that transformation took place, most analysts would agree that close to the heart of the process was the ideological sorting out of the two parties. By the late twentieth century, we finally had the “real” parties that Roosevelt had called for.

•••

In my new book, The Polarizers (2018), I highlight the specific actors who helped bring this change about. In the early postwar years, for example, a group of political scientists led by Wesleyan University’s E.E. Schattschneider provided intellectual ballast for the project by reviving a Progressive-era doctrine called “responsible party government.” Proponents of responsible party government sought to nationalize the party structures that had long been patchworks of state and local organizations. They promoted programmatic parties, organized around substantive issues rather than ties of tradition, patronage, or personality. And to secure democratic accountability, they sought to ensure that the two parties’ respective programs were at once coherent and mutually distinct. The goal, as a famous Schattschneider-led committee of the American Political Science Association (APSA) wrote in 1950, was a system in which the parties “bring forth programs to which they commit themselves and . . . possess sufficient internal cohesion to carry out these programs.”

Most of these scholars were themselves frustrated liberal Democrats, just as Roosevelt had been, and they found an eager audience in the ranks of organized postwar liberalism. Officeholders like Hubert Humphrey engaged with responsible-party scholars and championed party discipline in speeches. Ideological advocacy groups like Americans for Democratic Action featured discussions of the doctrine in their literature while the progressive wing of organized labor took up the cause of cohesive party discipline. Meanwhile, an ascendant generation of issue-driven “amateur” activists—thrilling to Adlai Stevenson’s presidential campaigns in the 1950s—battled to wrest control of Democratic organizations from traditional machines in the north while attacking the outsized national power of the conservative leaders in the south. They valorized party discipline in Congress and majority rule within national party affairs, hoping that many disempowered southerners would leave the party.

Today, identity-based hostilities might disrupt rigid ideological polarization. Racism undergirded the depolarized mid-century system as well.

Liberals had allies on the postwar right in advocating this vision. Conservative Republicans inside and outside of office, ranging from South Dakota Senator Karl Mundt to South Carolina politico Greg Shorey, advocated a partisan realignment that would formalize an ideological coalition of southern whites and northern Republicans. The circle of writers and intellectuals who launched National Review in 1955 promulgated a coherent national agenda for conservatism that included opposition to aggressive federal intervention on civil rights. William F. Buckley and his allies’ ideological project developed in the course of arguing against the programmatic blurriness and accommodation they associated with Eisenhower’s approach to Republican politics. Their project culminated in an insurgent presidential nomination victory by Barry Goldwater in 1964—the same year that the Arizona senator voted against the Civil Rights Act. Goldwater rode to the GOP nomination on a wave of resentment at Republican “me-tooism” and a commitment to offer, as he put it, “a choice, not an echo.” 

The following decade witnessed a wave of institutional reforms that further enabled liberal and conservative activists to redraw the lines of ideology and partisanship. Liberals in the early 1970s pursued congressional changes that ended the sanctity of seniority and made committee chairmanships subject to approval by the Democratic rank and file. Simultaneously, reformers instituted sweeping changes to the parties’ presidential nominating procedures that took control of state delegate selection away from formal party actors and opened them up to popular participation. Both reforms rendered the political system more permeable and responsive to ideological activism.

Capitalizing on those reforms, activists on both sides worked to channel the tumult of the 1960s into reformulated coalitions at the respective bases of the two parties. Left-liberals like the socialist writer Michael Harrington and the feminist community organizer Heather Booth forged organizational coalitions between a liberalizing labor movement and the rising forces of feminist, civil rights, and cultural activism. Just as importantly, they oriented those coalitions toward work within the Democratic Party. At the same time, New Right architects within the GOP such as Paul Weyrich and Phyllis Schlafly brokered a lasting alliance between traditional conservatives and a newly mobilizing Christian Right that would power Ronald Reagan’s electoral triumph in 1980. In the decades that followed, a new system of ideologically defined partisanship gradually sharpened through recurrent cycles of intraparty conflict and inter-party sorting.

The world the polarizers made—today’s world—reflects many of the systemic changes that responsible party advocates wanted six decades ago. Contemporary parties are not only more cohesive and distinct than at mid-century, they are also more disciplined when in power. At least during periods of unified partisan control of the federal government, party cohesion can facilitate the passage of major reform, as seen in the big-league regulatory and social legislation passed during Barack Obama’s first two years in office.

During those periods of unified government, lines of responsibility and accountability are also clearer—something that Republicans are realizing as they struggle to get major but unpopular bills over the legislative finish line while eyeing their midterm election prospects next year. Voters now regularly face high-stakes choices over fundamentally different public-policy directions, and their views of the parties have sharpened accordingly. Americans are much more likely than they used to be to tell surveyors that there are meaningful differences between the parties’ stances and to correctly match the right party with its ideological disposition.

•••

So, mission accomplished?

For almost everyone, our current political system feels less than satisfactory, to put it mildly. What did responsible party advocates—who were not fortune-tellers, after all—get wrong? Their core oversights concerned psychology and institutions, respectively.

First, they underestimated the virulence of party polarization in practice. Political psychology and public opinion research confirms what so many of us have experienced. When partisan team spirit becomes reinforced by shared substantive beliefs on core issues, peoples’ partisan identities become a more intensely felt component of their self-identities. Righteous passion for one’s own side intensifies while distrust of and hostility toward the other side deepens. Motivated reasoning and perceptual blinders hinder our ability to deliberate, to learn from those we disagree with, to change our minds or accept compromises.

There are ways to make partisanship ‘work’ better in the U.S. context

The same dynamics that affect ordinary partisans shape the behavior of office holders—amplified enormously by the high political and policy stakes of their collective work. The rancor, incivility, and rampant norm-breaking that we bemoan in our representatives are the byproducts of a structural transformation of the party system.

Sadly, that makes the prospects of mitigating the nasty behavioral dynamics of polarization difficult. The responsible-party advocates of mid-century presumed it would be possible to organize partisan conflict around contrasting issue agendas without inflaming political warfare. As the Schattschneider-led APSA committee hopefully insisted, “increasing concern with their programs” would not “cause the parties to erect between themselves an ideological wall.” The committee members were at least partly bluffing with such reassurances, in the hopes of persuading a would-be national audience. But they were also betraying the assumptions of an era in which political norms of civility and cooperation were more clearly foregrounded in public life.

Those norms are precisely what Joe Biden and millions of other Americans yearn to restore to politics now. But they, too, were byproducts of structural arrangements, in which partisan attachments and substantive agendas cross-pressured many politicians and enabled fluid coalition building. If restoring the mid-century’s clubby spirit depends on reverse-engineering the ideological sorting of the parties themselves, then the cold comfort of nostalgia might be all we can resign ourselves to.

Some analysts, however, have argued that such a structural shock to our system is underway now. Donald Trump’s unorthodox candidacy and campaign appeals, these observers argue, presaged a broader realignment brewing under the surface, one that will see a segment of the existing Democratic coalition drift into a Trumpified nationalist-populist GOP while more cosmopolitan “globalists,” in Steve Bannon’s terminology, defect to the Democrats. “But this will take a while,” political scientist Lee Drutman wrote last year in Vox. “During the transition period . . . the political system will become more fluid and dynamic, with more shifting coalitions and more opportunity for deal-making, and less clear partisanship.”

So far in Trump’s presidency, little of the sort has taken place. He and congressional Republicans have sustained an uneasy but unbroken partnership on a shared—and quite orthodox—policy agenda, while Democrats remain united in opposition. But it is worth pondering for a moment what such a realignment might look like. Bannon, Trumpism’s self-styled ideologist, has talked about building a diverse coalition united on issues of economic nationalism, but his work in advising Trump and running Breitbart News points to a different realigning project: building a cross-class coalition of whites united on racial, ethnic, and national resentments, even as they disagree on, say, economic and social policy. A realignment forged through inflaming identity-based hostilities might indeed disrupt the rigid ideological polarization of the current parties. Racism, remember, undergirded the depolarized mid-century system as well. But rather than renewed bipartisan dynamism, such an effort might more likely lead to further systemic instability and democratic decline, along the lines discussed by those worried political scientists conferring at Yale. Alternatives to party polarization are not necessarily improvements on it.

If the mid-century responsible-party advocates underestimated polarization’s behavioral spillovers, they also failed to anticipate the dysfunctions that would result from disciplined programmatic parties operating within the country’s Madisonian constitutional system. Our political structure is famously fragmented and laden with “veto points.” Legislation must survive an obstacle course of hurdles, bottlenecks, and deathtraps to become law, from the committee process through passage by two coequal legislative chambers right up to the president’s decision to sign or veto. The proliferation of actors able to block policy and the requirement for legislation to receive concurrent majority support in highly distinct legislative bodies (one of which normally requires a filibuster-proof supermajority) means that legislation typically depends on some minority-party support. Parliamentary-style party discipline makes that kind of legislating more difficult, especially by enabling and encouraging minority parties to obstruct the process rather than participate in it.

The responsible party advocates were perfectly aware of these institutional obstacles. But they anticipated that strong programmatic parties would work to transform those institutions to serve their purposes. They hardly anticipated, for example, that a tool of minority obstruction like the Senate filibuster, rather than being abolished outright by ambitious majority parties seeking to achieve collective goals, would instead grow much more prevalent in use over time.

Most importantly of all, these analysts failed to anticipate that divided government would become a frequent occurrence after World War II. This was an understandable oversight given the rarity of divided government in U.S. history until then, but it was a momentous one. As the latter six years of Barack Obama’s tenure most recently showed, giving hostile and disciplined parties simultaneous control over coequal parts of the government is a recipe for grinding stalemate at best and chronic crisis—budget shutdowns, constitutional showdowns, default scares—at worst.

Is the bad “fit” between polarized parties and U.S. political institutions any more amenable to reform than polarization’s toxic psychological dynamics? The prospects for scrapping the separation of powers in favor of a parliamentary system are, of course, close to non-existent. Lyndon Johnson touted a more incremental constitutional amendment to synchronize the terms and election calendars of representatives, senators, and presidents in order to diminish the frequency of divided government, but such a proposal seems similarly pie-in-the-sky now. (Though perhaps ripe for reappraisal by reform-minded citizens ready to think big.)

But institutional changes less sweeping and radical than a wholesale upending of the Constitution can still prove consequential, as illustrated by the story of twentieth century congressional reform, in which the House and Senate became more centralized and party-driven bodies. In the contemporary era, further reforms of procedures not found in the Constitution itself, such as the Senate’s 60-vote requirement for cloture, should be priorities for those—including our filibuster-decrying president—who are frustrated by political gridlock.

It is important to note, however, that such reforms are intended to allow partisan majorities to more easily implement their agenda when in power, to accommodate polarized partisanship rather than mitigate it. In other words, they are ways to make partisanship “work” better in the U.S. context—a solution that many Americans would surely resist. A common fear even among those who aren’t knee-jerk anti-partisans is that a more majoritarian system would allow for undesirable swings in policy from one election to the next.

Consider, however, Republicans’ current difficulties passing ambitious conservative legislation despite unified control. Whatever else is contributing to this, a fear of the electoral consequences of passing unpopular bills is a key factor. Conversely, years spent in the opposition, free of the expectation that voters would hold them responsible for outcomes, arguably encouraged rather than mitigated extremism in rhetoric, tactics, and promises. As Pennsylvania Representative Charlie Dent put it in February when describing the House’s repeal-and-replace health care bill after years of casting symbolic anti-Obamacare votes, “we’re playing with live rounds this time.” The bullets are real because the lines of collective responsibility for federal policy are now clear. And so Republicans have grown more gun-shy.     

•••

To be human and sentient is to be dissatisfied with current U.S. politics. Party polarization is centrally implicated in many of the era’s core failings. But because trade-offs among democratic goals are unavoidable in a political system—pragmatic bargaining versus coherent policymaking, elite comity versus democratic participation and accountability—grappling with polarization presents paradoxes. The story of polarization’s mid-century advocates and architects suggests not only that the road back to a depolarized era may be all but impassable, but also that such an era had limits and pathologies in its own right. Likewise, realistic alternatives to today’s polarization may offer a cure worse than the disease.

Worsening polarization in U.S. politics might just save U.S. democracy.

Most paradoxically of all, the disease may itself be a cure. We could lessen political paralysis, for example, most easily by reforming the processes to accommodate polarization. At the Yale conference on democratic decline, analysts also pointed to rising inequality and stagnant median incomes as indicators that the “class compromise” undergirding popular commitment to the system is starting to unravel. How best to respond politically to inequality is hardly a settled question, but given the GOP’s existing commitments, it is likely that part of the answer will involve a more rather than less disciplined, ideological, and aggressive left-of-center political party. If the Democratic Party became the vehicle for meaningfully redressing inequality that would itself be a polarizing development.   

Extending the point further, a two-party system leaves little room to avoid the fray in pursuing change—try as so many do. The renaissance of civic activism that “resistance” to Trump inspired after the election, for example, has come in countless forms, but one noticeable tendency in much of it has been an aversion to explicitly partisan forms of organization-building and mobilization. But meaningful resistance requires power, and the chief means of winning that power are electoral and partisan ones. To effectively counter the very hatreds and pathologies that seem so endemic to the age, in other words, progressives have no choice but to get into the thick of the action and fight. Doing so would only “worsen” polarization in U.S. politics, but it might just save U.S. democracy.