In Idaho, independent political polls are rare ahead of elections. Here’s why.

How competitive is the race to represent Idaho in the U.S. Senate? It depends which poll you look at.

According to a poll released last week by the campaign for Todd Achilles, an independent running for U.S. Senate, Republican incumbent Jim Risch has only a six-point lead over Achilles. Eight percent of voters were undecided, the poll found.

The poll buoyed a claim by the Achilles campaign: Victory could be within reach. 

“That tight margin confirms what we’re hearing in every corner of Idaho: voters are tired of career politicians controlled by their party bosses,” Achilles, 58, said in a statement with the poll’s release on Oct. 23.

But the latest poll didn’t ask about support for the other two candidates in the race, which has drawn one of the most active attempts in years at unseating Risch, 83, who’s seeking a fourth, six-year-term. And Risch’s campaign polling — featuring the full line up of candidates — shows him pummeling Achilles.

In the run up to Election Day in a state without much independent polling, that’s the sort of divide in polling outcomes that some pollsters expect voters to see.

“Usually about this time, the candidates or the campaigns for ballot issues are publishing polls that show you that they’re going to win,” Stuart Elway, a longtime independent pollster in Washington state, said in an interview. 

In Idaho, independent political polls — which aren’t backed by a campaign or a special interest group — are hard to come by. They would offer a different perspective, Elway said. 

“An independent poll that’s not sponsored by any party or candidate or interest group gives you an objective measure of how the election is going, and I think more importantly, what the people are thinking about when they’re thinking about the election,” he said. 

Some polling experts say that Idaho lacks independent polls because the state is reliably Republican controlled, and independent pollsters aren’t willing to invest the high funds needed to conduct reliable polling. 

Instead, there are candidate and special interest groups polling in Idaho, like the back and forth of polls released by the campaigns of Risch and Achilles. 

The poll released last week by Achilles’ campaign, conducted by VoteNexus, had info on other races — including 54% opposition for Proposition 1, a ballot initiative to end Idaho’s strict abortion ban. In other key races, like for governor, and two U.S. House seats, Republicans were in the lead. The survey, conducted via online polls sent by text message of 700 likely Idaho voters, had a margin of error of plus or minus 4.5 percentage points. 

So, what makes for a good poll in politics?

How pollsters set up their surveys, from the questions they ask and the order they ask them in, is critical. That’s because certain wording or prefaces can push people toward certain answers. 

But there’s also a host of other survey design decisions to consider, such as whether the poll was done using a preselected online sample, which can have biases, or was done by phone calls, the most traditional method, or other avenues. 

Where are the debates? Idaho incumbents largely forgo debates and forums this year. 

If you’re looking at a poll yourself, you’ll also want to make sure that the survey is telling you its margin of error. That’s a value that tells you how much pollsters think that the results they reported could differ from the public’s actual opinions on the issues. Without that, a poll is “suspect,” said University of Idaho political science department chair Florian Justwan. 

Part of the draw of independent pollsters is, in theory, you have a firm without a vested interest in the issue gathering information. But good polling can be done by groups that have a perspective on an issue or candidate, said Jeffrey Lyons, Boise State University political science professor. 

“I would be hesitant to universally say ‘These polls, like issue group polls, are bad, or you can’t interpret them,’ because I don’t think that’s the case. I think they can be valuable. You just have to be a little careful with them,” he said. “And the same goes for polls done by political campaigns. … There’s a lot of questions to ask about that and what we can do with that information. For example, how many other polls did they do where they didn’t like the numbers and they just didn’t release them, right? So if they’re releasing the poll, they obviously like what they see.”

Campaigns also don’t benefit that much from inaccurate polling, said Lauren Goldstein, principal researcher at Change Research, which polled for independent Idaho gubernatorial candidate John Stegner’s campaign.

“While there might be some differences in goals and incentives of polls that are conducted by independent media outlets versus commissioned by campaigns, at the end of the day, accuracy is paramount for all purposes,” she said in an interview.

Another big hurdle for election polling is trying to ensure that the people they are surveying are actually going to turn out to vote in the election. To determine whether someone is a likely voter, pollsters may use different methods. One method might, Justwan said, count everyone who took the poll and voted in the last three elections, but another might just count anyone who indicates they will likely vote in the next election.

That’s part of the trouble in comparing the results of election polls.

“Using these different filters as a way to remove all non-likely voters from the sample, depending on what your filter is, is going to give you very different responses,” Justwan said. 

Polling has also gotten a lot more difficult and expensive since the advent of cell phones and the shift away from landline phones, pollsters say.

But polls aren’t only helpful for horserace election coverage; they’re for understanding the public’s attitudes. 

The Idaho Public Policy Survey, conducted every year by Boise State University, releases information about how Idahoans feel about a range of issues, including their top policy priorities for the Legislature to address. The last iteration of the survey — which featured responses from 1,000 people, half of whom were through phone calls — cost $46,500, said Matthew May, the survey research director at Boise State’s School of Public Service. 

In the backdrop, many members of the public are losing faith in the polls, a decade after 2016 polls mispredicted that President Donald Trump would lose his first general election for the presidency. A national poll released in August by Rasmussen Reports found that half of voters don’t trust most polls about politics.

Lyons, with Boise State, acknowledged the huge gap in the results between Risch and Achilles’ polls. But he said he’d looked closely at the polls released in August and saw them as trying to do things differently. 

“I don’t view the two polls as being like they did the same thing and came to a different answer,” he said. “I tend to view them as more like they did different things and came to a different answer.”

Are Achilles and Risch in a heated race for an Idaho US Senate seat?

In August, the Achilles campaign released results from an internal poll that made it seem like he has a chance against Risch. When likely voters were asked who they’d vote for, almost a third chose Achilles. Here’s how that broke down at the time:

• Risch: 34%
• Achilles: 30%
• Matt Loesby, a Libertarian Party candidate: 4%
• And unsure: 32%
• *the fine print: The survey had a margin of error rate of plus or minus 3.97 percent, indicating how much actual public opinion could vary from the survey results it reports out. It sampled 608 likely voters. 

That poll — and the candidate options listed — was different from how Risch’s campaign pollster conducted its. For one, Achilles’ campaign’s pollster relied on largely text and online responses to the poll, rather than phone calls, the more traditional, but expensive polling method. And Achilles’ campaign’s poll didn’t include another independent candidate in the race, Natalie Fleming.

Achilles’ campaign manager Crispin Gravatt said in an email that the campaign was focusing on the big picture when asked about not including Fleming’s name on the candidate list.

“If we had unlimited time and resources, we would measure every possible combination and dozens of topics in the state [as I’m sure every data nerd would like!],” Gravatt said. “We do believe the trends in polling provide more insight than any individual poll- our polls and other polls we have seen show Todd’s voter share growing in Idaho, while even Jim Risch’s published polls show him losing steam.”

Brett Lloyd, head of the Bullfinch Group that conducted the survey for Achilles, said the only questions that came before poll respondents before they were asked who they’d vote for were typical demographic and screening questions.

Risch’s campaign, in a poll released days later in August, sought to clear the air. It found support for Risch was much higher:

• Risch: 52%
• Achilles: 18%
• Fleming: 9%
• Loesby: 2%
• None: 1%
• *the fine print: The survey had a margin of error rate of plus or minus 4 percent, indicating how much actual public opinion could vary from the survey results it reports out. It sampled 500 likely voters through phone interviews.

The pollster behind Risch’s survey, Erik Iverson who leads Peak Insights, did not release the question itself, or the full question list, which pollsters say is important to see to understand if pollsters might have pushed likely voters to respond in ways they might not have without a nudge. Iverson said only screening type questions, such as people’s likelihood of voting and favorable images of various candidates, came before the question of who they’ll vote for in the Senate race. 

“Our polling is not designed to shape any sort of a public narrative. It has one singular focus, and Sen. Risch has always been very clear about this,” Iverson said in an interview. “‘I need the most accurate polling possible so that we can develop a detailed campaign plan. We know exactly what we need to do, where we need to spend our money, and then go execute it.’”

A poll commissioned by Risch’s campaign conducted Sept. 28-30 found that Risch was still in the lead, but his support fell to 49%, and Achilles’ rose to 24%.