An analysis of the data revealed that Donald Trump fares best among voters who don’t regularly participate in primary elections.

Share story

Ever since Donald Trump has risen to the top of the polls, Republican strategists have questioned whether those polls might be overestimating his support.

There is evidence to support that theory.

A recent New York Times article about Trump’s support used polling data provided by Civis Analytics, a Democratic data firm founded by the 2012 Obama campaign’s analytics director, Dan Wagner. The data showed Trump faring worse than in many recent public polls, but not enough to call his lead into question.

An analysis of the data revealed that he fares best among voters who don’t regularly participate in primary elections. Nevertheless, he still leads the other Republican candidates, even among the most frequent voters.

The analysis also suggests that although Trump fares better among irregular voters, it is not by an unusual amount. And some other candidates also do better with infrequent voters.

Civis polls differ from the surveys sponsored by news organizations or universities. Nearly all public polls try to interview adults by randomly calling telephone numbers, a technique known as random digit dialing. They adjust the responses to match the demographic characteristics of the adult population, then remove those people who say they’re not registered to vote.

The technique is well grounded in statistical theory, but it has shortcomings for election polling. One is that the adult population does not necessarily reflect the electorate, which is especially true in primaries. Most adults, after all, do not vote in primary elections. People tend to self-report that they’re voters, even when they’re not.

The public polls can be misleading if nonvoting adults have substantially different views from primary voters. Usually, the difference isn’t huge. But some analysts have argued that Trump’s celebrity might give him an unusual advantage among irregular voters, who might not pay much attention to political news.

The candidates’ campaigns, for whom more accurate results really matter, have moved to a different approach to polling: They draw their sample from voter-registration files, which contain information on voting history. That gives campaigns a powerful tool for examining the likely electorate.

If Trump had a big advantage among unlikely voters, a poll using a listed sample — like the Civis data — would be the way to find out. The Civis poll was conducted Aug. 10-19 and had a sample of 757 respondents.

The results showed Trump with 16 percent of the vote, below any of his poll results in a month. But much of the difference was because 22 percent of voters in the Civis poll were undecided — much more than in many recent public surveys.

The proportion of respondents who “don’t know” often varies from pollster to pollster, a phenomenon called a “house effect.” That’s because the wording of the questions varies or the interviewers have been trained to push undecided voters into making a choice. Sometimes it depends on whether pollsters asked all respondents or only those registered to vote.

The number of undecided voters can affect the rest of the results in a survey. For instance, if undecided voters were allocated proportionally until the number of undecided voters fell to the level of other polls, Trump would hold 21 percent of the vote. It’s not a good idea to assume that undecided voters break proportionally, but it gives a sense of how much undecided votes could explain the difference between the Civis data and the public polls. (Trump leads in a national polling average, at 26 percent.)

The big gap between the Civis results for Trump and other surveys also hints at the possibility that these voters, pushed to make a choice in public polls, might be breaking for Trump by a disproportionate margin.

At the very least, the large number of undecided voters suggests that many who tell pollsters they support a candidate might have weak preferences, and that Trump might be the beneficiary.

The large number of undecided voters makes it difficult to directly compare the Civis numbers with other public polls. But the Civis analysis, using voter-history data, nonetheless offers good reason to believe that Trump might fare best among voters with little history of voting.

Civis tried different methods to measure the effect of voter participation. All the methods showed the same basic story, with Trump faring better among irregular voters, but not by an unusual amount.

It was also able to look at the effect on other candidates. Jeb Bush, Ben Carson and Chris Christie fared better among irregular voters by a similar or even greater amount.

Civis used a simple model to estimate the relationship between candidate preference and vote history — measured by the total number of elections respondents had voted in since 2000.

Civis took another step and weighted its sample under two scenarios: one reflecting the adult population of Republicans, the other of primary voters. Primary voters aren’t just likelier to vote; they’re also older and likelier to be registered Republicans. Trump fared 2 percentage points worse among primary voters than Civis estimated he would have among all adults. Other candidates, like Bush and Carson, lost more ground. Like Trump, Bush seems to be benefiting from name recognition.

On the other hand, Fiorina gained around 3 points in the primary electorate — although she still held no more than 5 percent — while John Kasich, Scott Walker, Mike Huckabee and Rand Paul each picked up about 1 point.

Trump had 15 percent support among voters who had participated in a primary since 2008, but he had 22 percent of the vote among Republicans who did not vote in the 2012 general election.

Overall, the data is consistent with the view that Trump’s support might be overstated by public polls. But he leads among voters who have participated in one or 12 elections. His challenge among likely voters isn’t necessarily unique. His lead might be modestly overstated, but it’s not a mirage.