Opinion / Politics

Hard Look at Political Polls – What They Miss

pollsBrace yourself for wave upon wave of … political polls.  The latest show President Trump lagging presumptive Democratic presidential candidate Joe Biden, as usual.  But stop. Let us do what no one does – look BEHIND these polls, understand the process and bias in polling.

Take two recent polls, by NPR/PBS/Marist (NPR) and ABC/Washington Post (ABC).  Both report new anti-Trump findings.  Take a moment and unpack them.

MSN.com – like may non-technical, left-leaning sites – summarizes both.  A stinging line punctuates a leading paragraph.  Says MSN, NPR’s poll “has Biden beating Trump 60 percent to 35 percent among suburban voters.”  ABC’s poll “is the latest to show former Vice President Joe Biden on a roll … up by a 52 percent to 43 percent margin among suburban voters …”

Okay, so narrative is set – “Biden beating Trump,” “Biden on a roll,” stark numbers.  Most readers go no further, but MSN inserts links to suggest you can – so let us do it.  See:  https://www.msn.com/en-us/news/politics/new-polls-show-joe-biden-is-winning-suburbanites-by-a-historic-margin/ar-BB16Vu0y.

Go to links.  What you see is “methodology,” on NPR’s poll some 61 categories defining 1,640 interviewees.  Who is going to study 61 categories?  Not many – but we are.  Here is what you learn.

On process, the survey was done in English and Spanish (no word how many of each), demographic data lags to 2017, interviews were not all “registered voters,” sample size of small groups were unreported, cell phones for business were not accepted, and no response rate is listed.  That is, no report of how many calls resulted in no contact, or “no thank you, not interested in NPR’s survey.”

Any bias here?  What if the survey was just English – our official language? How were nationalized Americans distinguished from illegals? What if demographics were current?  Doesn’t old data change findings?  Which subgroups were “too small to report?”  What if they tipped for Trump?

What effect did omitting “business cell phones” have on findings?  What if owners of the nation’s 30.2 million small businesses have one cell for both business and personal use?  After all, 50 percent of small businesses operate from “home.”  What if these small businesses are pro-Trump entrepreneurs?

Surprise: They are.  Another survey reports “President Trump’s approval rating among small business owners hit an all-time high of 64 percent” in 2020.  If cells used by business owners are omitted, does that affect results? See:  https://www.cnbc.com/2020/02/20/trumps-approval-rating-among-small-biz-owners-hits-64percent-survey-shows.html#:~:text=Small%20business%20owners%20are%20a%20consistent%20source%20of,%E2%80%9CPart%20of%20this%20is%20just%20partisanship%2C%E2%80%9D%20Wronski%20said.

Dig deeper.  In the NPR sample, 38 percent are Democrats, only 31 percent Republicans, with 29 percent Independents.  Does that overrepresent Democrats v. Republicans, and Independents v. both?  What effect does that have on accuracy?

Look closer.  Among Democrats in NPR’s survey, one third more are women than men.  Do Democrat men favor Trump more than Democrat women?  Maybe.  Slice the pie differently.  In NPR’s survey, 52 percent are women, 48 percent men.  Does that affect findings?

How about the fact that older voters – who tend to be more conservative – vote more often?  How does NPR account for that?  Pew Research reported in 2020, while “older voters accounted for 43 percent of eligible voters, they cast 49 percent of ballots.” See: https://www.pewsocialtrends.org/essay/an-early-look-at-the-2020-electorate/.  Moreover, 54 percent of older Americans approve of Trump, compared to 30 percent of youngest voters.  See: https://www.pewsocialtrends.org/2019/01/17/generation-z-looks-a-lot-like-millennials-on-key-social-and-political-issues/.

With respect to age, only 26 percent of NPR’s respondents were aged 45 to 59.  This may be a major flaw, as 35 percent of voters in 2016 were over 50, and NPR’s survey appears to wholly omit voters over 59.  This is particularly meaningful, as 25 percent of all voters in 2016 were over 65.

Another unreported fact should trigger critical thinking – especially on this NPR survey.  The June 26th NPR survey says Trump’s approval fell from 43 percent in March 2020 to 40 percent in June 2020. But Trump’s overall approval rating was 37 percent in March 2017 – five months after winning election.

Pivot to ABC’s latest survey, more bias.  The media does not expect you to ask, but let us ask. ABC’s headline – to which MSN links: “Pandemic Surge Damages Trump, Boosting Biden’s White House Bid.”  But is that what ABC’s data really says?

To get methodology and qualifiers, you must search.  Stressing credibility, they note: “We do accept some probability-based surveys that do not meet our own methodological standards … recommend cautious use of such data.”  Then on shorter survey methodology, the link just pops an error message.

We learn 65 percent of interviews were cell phones, 35 percent landlines.  Taking that as gospel, demographic data show landlines are held by older Americans, cells held by younger Americans.  Does a 2:1 bias for cells tip results?  Maybe.  See, e.g., https://nypost.com/2017/05/04/why-nearly-46-percent-of-household-still-have-landlines/.

How about this:  ABC uses a “cleaning process” that “excludes respondents who have home-based business-listed phones.” Recall more than 50 percent of small businesses operate from “home,” and 64 percent favored Trump.  Does omitting such lines affect survey results? Maybe.

Other biases are unaccounted – or just undercounted.  The ABC poll omits phones in “institutional” settings, but what of senior living communities? Most are conservative – and residents vote.   Numbers are not small.  In fact, the nation has thousands of apparently undercounted senior living communities. See, e.g. https://www.statista.com/statistics/895322/senior-housing-communities-usa-by-region/.

Last, like other surveys, this one is ambiguous on “response rate” – that is, number of failed contacts, and calls ending in “no, thank you, click.”  Despite contrary arguments, response rates matter.  For several years, telephone survey response rates have been falling.  They now stand at a shocking – six percent.  See: https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/.

What does that mean?  Most people just do not answer.  If they answer, they hang up. Those who talk – when told NPR or ABC is calling – might have a motive.  So, net-net do not put great faith in polls.

Even when luminaries at NPR, ABC, and elsewhere say they have 2020 figured out – they probably do not.  They have results they want, based on questions, demographics, and methods supporting the narrative – and a nice, juicy headline.

So, brace yourself for wave on wave … of polls.  But do not put great stock in them.  The reservations often swallow their findings.  Only one poll really counts.  You know it – November 3, 2020.

If You Enjoy Articles Like This - Subscribe to the AMAC Daily Newsletter!

Sign Up Today
Read more articles by Robert B. Charles
Subscribe
Notify of
98 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
JohnH
2 months ago

Polls are interesting to note, but you are right when you say the only thing that counts are the Nov 3 poll. Get out & vote.

Tina
2 months ago

Poll headlines are meant for one thing: to sway opinion. The problem is stated in this piece: “Most readers go no further” than the headline that summarizes the results of the poll.

The headlines are structured to either lull you into complacency (i.e. My guy is winning!) so you sit on your butt, or they are meant o annihilate your morale (i.e. It’s hopeless. My guy can’t win.), so you sit on your butt.

Don’t be foolish enough to mind the polls. MAGA, KAG, TRUMP 2020. Get out & VOTE RED.

VOTE!!!!!!!

William Priest
2 months ago

Please keep posting this type of information. Fox News, which is supposed to be conservative, posts these polls as if they were fair and unbiased.

David Spade
2 months ago

They forgot to add the 10% favorable Trump rating factor for being tough on just about everything. How many candidates could ever be so wrongly hammered by the press day in and day out since before the election in 2016 and still be standing today? I would say that the voters like a fighter for no other reason than that. People will vote for Donald J. Trump because they hate the leftist Democrat party and the leftist media that attack him every day.

JerryH
2 months ago

This is the kind of factual reporting that journalists need to be doing instead of just parroting bullet points and sounding off headlines loosely based on facts. Thank you for posting this well-documented article.

Bruce Kniegge
2 months ago

polls can be skewed any way a person or organization wants to show their view point is best!

Patriot Will
2 months ago

Personally, I am very suspicious of the samples that are used in making these polls. Consequently, as the article so clearly points out, many of the people who favor President Trump are not represented fairly in the polling used to make predictions on who is going to win the 2020 presidential election. Also, many President Trump supporters are not sharing their honest views with many of these socialistic polls. A lot of independents, moderates, conservatives, and even democrats are keeping their true feelings and opinions to themselves.

KJHanover
2 months ago

No one with any brains believes these polls.

David Midkiff
2 months ago

I can’t see how Joe Biden stands a chance of being the next President. He has done nothing but come out of hiding a few times to tell the fake news what he would do in certain situations. He doesn’t have a good track record or personality to win the election. He’s had 40 plus years to prove himself of any worth and has miserably failed. The fake polls had Hillary ahead in 2016 and you see what happened there.

Fairfield54
2 months ago

Thank you. Keep it up. The message needs to get out to everyone that just because the polls say Trump is behind, does not make it so. What about those who fib? There are many. I was called numerous times for the last election and as soon as I heard who the pollster was – I told them what they wanted to hear – Hillary, Hillary, Hillary. My guy won and I pray he does again.

Ed J
2 months ago

During my graduate MBA program, we were required to take a course in Operations Research. One of the significant components of that course concerned structuring of surveys (in politics, they call them polls) and how easy it was to make significant structural errors in the construction and wording of a survey, resulting in skewing and/or invalidation of results. This article points out many of the same pitfalls we studied in that course. The short answer is that most of these polls are significantly deficient in their construction, their sampling techniques and ultimately in the statistical interpretation of their so-called results.  In effect,… Read more »

Phil Hammersley
2 months ago

Have any of you members EVER been polled? I know that in over 40 years, I haven’t been! Where do they get the ones they poll? NYC, Chicago, Seattle? Or do they get those of us in “flyover country?” I can’t believe there are over 50% of voters so stupid as to vote for anarchy led by Communists!!

Kristine Wright
2 months ago

good article! I like the research.

Roger K
2 months ago

Trump shall win!

Jack
2 months ago

We all know how predictable polls are. Look at 2016. I don’t pay any attention to any of the polls by the biased media. And they are all biased toward the Dims. Propaganda machines is all they are and I’m surprised they aren’t dumping them out of airplanes. Yet. They don’t have the ability to hypnotize the public like Hitler did but they are trying to brainwash everyone by repeating the same thing over and over. They think if it’s repeated often enough it will come true and the American citizens will take it as fact. I’m not biting.

Judy craver
2 months ago

Thank you for the information. I knew the pols had to be wrong. I know Trump in on his way to a second term.

George Moe
2 months ago

Knowing where left leaning people are you can get any results you want. Polls are not reliable now days with the computer power that now exists.

Felix
2 months ago

In a sense the poll that counts will be November 3, however if these polls are correct all conservatives know that the main reason that Biden is leading is because the main stream media’s influence by only broadcasting negative or incomplete out of context stories and we know that the networks, CNN and MSNBC combined viewership is far more than FOX News. Example- President Trump’s roll out on his fight against crime yesterday, which included some very compelling information and testimonies from families effected by the horrific killings in some cities (I channel surfed CNN and MSNBC) was not broadcast… Read more »

Stephen Russell
2 months ago

1 sided polls, not wide enough to encompass voters, rigged thus lies etc day 1, cant trust polls. Unless AMAC can do own polling of members nationwide & rate results, then who can, cant trust AP etc other sites, Be daring

John Walker
2 months ago

A couple of years ago, I read that a week before the 2016 election, at least one poll had Donald Trump’s approval rating at 39%

98
0
Would love your thoughts, please comment.x
()
x