Robert Cahaly of Trafalgar explains why his polls show Republicans competitive even in some of the bluest places.
Robert Cahaly of Trafalgar explains why his polls show Republicans competitive even in some of the bluest places.
T here are a bunch of big debates to tackle in analyzing what is happening in the 2022 election. Will the president’s poor approval ratings lead, as usual, to a wave election against his party? Are Hispanic voters realigning towards Republicans, or have they at least soured on Biden? Will Republicans be done in by weak Trump-backed candidates, or by Democratic anger at the Dobbs decision? Are the polls even telling us the real story?
There are certainly reasons for Republicans to be optimistic. As I explained at length in a 2014 study (which I’m overdue to update, but which was richly illustrated by that year’s elections), there is a long-standing tendency of Senate races in particular to break hard after mid September in one direction. That direction is indicated by the national trend reflected in presidential-approval ratings and the generic ballot. At this writing, Joe Biden’s job approval in the RealClearPolitics poll average is a grim 12 points underwater (42 percent approval, 54 percent disapproval), and the generic ballot, which has tended historically to overstate Democratic support, favors Democrats by just one point (45 percent to 44 percent). As David Byler of the Washington Post details, another sign of Republican strength is higher primary turnout than Democrats this cycle, traditionally an indicator of disparate enthusiasm.
There are, however, also reasons rationally cited by Democrats for optimism, or at least for them to be less depressed than they were in June or July. Biden’s bad job-approval numbers are not as catastrophic as when he was 20 points underwater in late July, and they have improved as gas prices climbed down from their early-summer high and Biden got something passed through Congress. Republicans led by as much as four points in the generic ballot polls from November 2021 until their advantage evaporated at the end of July. Democrats have done well in multiple special elections, anti-Dobbs turnout defeated a pro-life ballot initiative in red (but often moderate) Kansas, and Donald Trump’s being back in the news in August helped Biden refocus attention away from his own shortcomings and towards the man against whom he won a popular majority. Some polls show the Democrats retaining a durable advantage with Hispanic voters.
Compare the RCP averages from July 4 until this weekend, a good comparison point between the mood entering the summer and the mood entering the fall stretch that commences in mid September. While the national environment is plainly more favorable to Democrats than it was on July 4, it is hard to see a clear trend in individual races. Republicans have gained ground in a number of races, mainly those that were still in the midst of contested primaries in early July or just exiting them; Nevada is looking worse for both Democratic incumbents. Florida looks tighter for the Republican incumbents, although few observers believe that Ron DeSantis or Marco Rubio faces any real threat. Probably the ugliest trend for Republicans is in the Minnesota governor’s race, in which incumbent Tim Walz appears to have consolidated support once the Republicans chose Scott Jensen as their nominee. In some races, such as the Ohio governor’s race, a lot of undecideds have fallen in behind each candidate without altering the shape of the race.
In the polls published since Sunday, Ron Johnson in particular seems to be gaining strength.
The Blind Leading the Blind
Before we draw too many conclusions from the polls, however, the fundamental question lingers: Are the polls misleading us? That may sound like an “unskewed polls” conspiracy theory, until you consider how badly the polls have failed repeatedly in recent cycles — especially 2016 and 2020 — eroding the near-mythic trust that pollsters and poll analysis built in 2012. Nate Cohn of the New York Times is already warning that pollsters haven’t really done anything to fix the methodological problems that caused them to miss a lot of working-class voters in recent elections:
That warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016. . . . Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.
The pattern of Democratic strength isn’t the only sign that the polls might still be off in similar ways. Since the Supreme Court’s Dobbs decision on abortion, some pollsters have said they’re seeing the familiar signs of nonresponse bias — when people who don’t respond to a poll are meaningfully different from those who participate — creeping back into their surveys. . . . If the polls are just as wrong as they were in 2020, the race for the Senate looks very different. The apparent Democratic edge in Senate races in Wisconsin, North Carolina and Ohio would evaporate.
Using Cohn’s numbers, the pattern in 2016 and 2020 was fairly clear — at least, when Donald Trump was on the ballot:
The news could be even worse for Democrats than that. Cohn cites Nevada as an example of a state where the polling error was less consistently in one direction, and Colorado as one where it underestimated Democrats. We could see some similar patterns of polls underestimating Democrats in Arizona and Texas in 2018. What do all those states have in common? A lot of working-class Hispanic voters. Those voters, often hard to poll, were a reliable Democratic bloc for a decade through 2018, but less so in 2020. If they are shifting in a Republican direction this year, the polls may be understating the size of that effect on the electorate — which would be very bad news for Democrats in the crucial Senate races in Arizona, Nevada, and Colorado.
The Dissenter
One pollster who has dissented from the pack since 2016 remains bullish on Republican fortunes and skeptical of his competitors: Robert Cahaly of Trafalgar. As I and others at National Review have detailed previously, Trafalgar has racked up a string of polling successes in recent years by talking to the very sorts of voters that other pollsters have missed, many of whom have responded to Trump and Trumpish candidates.
Trafalgar was ahead of the curve in Virginia in 2021, and alone in seeing a tight race in New Jersey in 2021. In 2020, even when Trafalgar got the outcome wrong in projecting narrow Trump victories in some states, his forecasts were often closer to the result than competitors who projected comfortable wins for Biden in states that Biden carried by his fingernails. Trafalgar has also turned in strong performances in this year’s Republican primary polls. There are no infallible oracles in the businesses of polling or poll analysis, as anyone could tell you from the ups and downs of the careers of John Zogby, Nate Silver, and Sam Wang. Trafalgar has had the occasional clunker, such as projecting a tight race in the 2021 California recall well after other pollsters showed it breaking hard for Gavin Newsom. The very nature of polling counsels us that even the best polling methodology will sometimes produce outliers. But, as is true of analysts such as Silver or Sean Trende, Trafalgar’s track record commands respect, especially compared with pollsters who have faced persistent problems and not corrected them. FiveThirtyEight, which gave high marks to Trafalgar for its accuracy in 2020, currently gives the firm an A- rating for polls conducted within 21 days of an election, ahead of the vast majority of pollsters rated.
Trafalgar is out on a limb once again, projecting good news for Republicans across the board compared with what other pollsters are reporting. The most recent Trafalgar polls released between August 31 and September 18 show Kari Lake winning by 4 points in Arizona, with Blake Masters down by just 2; Dr. Oz and Doug Mastriano each down 2 points in Pennsylvania; Lee Zeldin down 5 in the New York governor’s race; Tiffany Smiley down 3 in the Washington Senate race; Brian Kemp up 7 and Herschel Walker up 1 in Georgia; Ted Budd up 3 in North Carolina; Gerald Malloy down 7 in the Vermont Senate race, with governor Phil Scott up by 53; and Republicans up 6 in the generic ballot.
My natural skepticism of too much good news from a single source leads me to ask: Is this for real? So, I caught up with Cahaly and asked why he’s getting different results from other pollsters. His answer is straightforward: Polling is broken, it is not contacting a representative sample of Americans, and other pollsters aren’t doing enough to fix it. He favorably cited Cohn’s warnings, and offered four major ways in which Trafalgar tries to avoid the same pitfalls:
First, draw up shorter questionnaires. Pollsters such as YouGov that ask 99 questions and 54 subparts may provide writers with a wealth of public feedback to digest, but at the expense of accuracy. As Cahaly told me, the most common question he gets from poll respondents is, “How long will this take?” Ordinary, average Americans may be willing to answer a seven-question survey while they’re making dinner for the kids or between innings of the ballgame, but the longer a survey is, the more the sample will be dominated by people who are highly politically engaged, highly educated, extremely online, or just lonely and happy to talk to a stranger on the phone. Cahaly scoffed at seeing polls reporting that half or more of the sample has a college degree, when no statewide electorate in the country will look like that.
He also noted that Republicans who were hostile to Trump were likelier to talk to pollsters. He thinks the “shy Trump voter” effect of people not wanting to admit their Trump support was real in 2016, but less so in 2020; it just got harder to get people to talk at all.
Second, find more ways to locate voters. Cahaly is particularly scornful of online panels that enlist the same participants over time. Trafalgar’s website says that it “delivers our polling questionnaires utilizing a mix of six different methods: live callers, integrated voice response, text messages, emails, [and] two other proprietary digital methods we don’t share publicly.” The proof is in Trafalgar’s results.
In particular, Cahaly stresses that he has to be more creative in approaching voters not only due to self-selection effects and the diminishing share of Americans who have landlines and answer the phone, but also due to what he calls “submerged” voters: people who are afraid to give their political opinions to strangers for fear of being doxxed or, in the post–January 6 environment, federally investigated. Needless to say, voters with these sorts of concerns are disproportionately not Biden supporters. “I’ve never seen anything like that,” Cahaly says of the number of responses he is getting now from people who need persuading that they are talking confidentially to a legitimate pollster. “This is real. People are scared.”
Third, don’t skimp on sample sizes. Trafalgar never publishes a statewide poll with fewer than 1,000 respondents. Cahaly argues that smaller sample sizes present problems especially when pollsters are weighting particular sub-samples more heavily. Pollsters commonly use weighting to get a model electorate that matches the projected electorate in characteristics such as age, gender, and race — and possibly other characteristics as well. But the smaller the sample, the smaller the sub-samples will be. A poll of 600 respondents in a state where the electorate is 10 percent black may need 60 black respondents — but if it only gets 48, those 48 are weighted more heavily in the published sample, such that each response is worth 25 percent more. The problem is, how does the pollster have confidence in a poll of 48 people?
Fourth, don’t trust self-reported party identification. Cahaly commonly encounters people who describe themselves as independents but who plainly are regular voters in one party or the other. He notes as well that voters are also prone to lie about their income and education. That makes weighting polls to get a target mix of party ID or any other self-reported characteristics — rather than basic demographics — a fool’s errand. He also puts little stock in generic ballot polls.
Given all of this, what does Cahaly see in the electorate? He thinks there was a larger-than-usual turnout of low-propensity voters in the high-turnout Republican primaries, and that people who don’t vote as regularly were likelier to support Trump-backed candidates. He argues that those voters are likelier to turn out this fall than in past midterms. He notes that there are still more voters out there than people think who support Trump himself but are otherwise not regular Republican voters.
On the issue environment, Cahaly says that he hasn’t seen any issue in a long time that motivates people as much as opposition to Biden’s student-loan forgiveness — “nothing touches it.” But he has also heard from people who scoffed at its comparatively small size relative to their overall debts. He sees a lot of general anti-incumbent sentiment, and believes that safety issues (crime, border security) are particularly driving opposition to Democrats in blue states. He specifically cites Lee Zeldin’s race against Kathy Hochul as one he expects to end up looking more like last year’s close race in New Jersey than the cakewalk much of political media projects.
If Cahaly and Trafalgar are right yet again, Democrats could be in for a rough November across the map.