Polls

[bobover3]

It can be confusing reading the Presidential polls reported by Real Clear Politics. Some show Obama ahead, others Romney. Is there any difference in method which helps explain the different results? In another thread, we've looked at assumptions made about the distribution of groups within the US population. But there's something more basic: does a poll sample the general population, registered voters, or registered voters who say they are likely to vote? Decades of data suggest that polls of registered voters who say they are likely to vote produce the most accurate results. Polls of the general population, while interesting, fail to predict voting.

Now let's take a look at the Presidential polls reported by Real Clear Politics. Here are the polls, their results, and who the polls sampled.

Rasmussen Tracking, Romney +7, registered and likely to vote
Gallup Tracking, Romney +3, registered
Associated Press GfK, Obama +8, general population
Reuters/Ipsos, Obama +7, general population
Politico GWU/Battleground, Romney +1, registered and likely to vote
IBD/CSM/TIPP, Obama +3, registered
Democracy Corps (D), tie, registered and likely to vote

I'll let my wise peers study this and draw their own conclusions.

[A-1]

It is a hopeless hedge-podge and exquisite cluster right now. Let us hope that eventually it is sorted out.

...but don't hold your breath...

[bobover3]

A-1, I hope you won't mind my publishing an English translation of your post.

Results favor Romney = "hopeless hedge-podge and exquisite cluster"
Results favor Obama = indisputable fact

[Cainanite]

I think the reasons you see such a swing, and polls becoming less and less reliable are three factors.

1) Technology. Less people have a home phone, and instead have cell phones. Many of my friends have gone this route. A lot of them actually have several phone numbers. Add to that, caller ID. If I do not recognize the number, I simply do not answer. I let it go to voice-mail.

So if you are conducting a phone poll it is going to be very difficult to actually get a true sampling, because most people will not even answer. The only people who will answer will be people who have only one phone line (a physical line, not a cell-phone), and do not have caller ID. This generally tends to be older folks on a fixed income, or people of lesser means.

Still in the technology category, I will call this "Polling from home." What I mean by that is, you run a conservative or democratic website, and create an online poll on your own site. You are going to get pretty skewed results, because the only people participating already think the way you think. There is extraordinarily little crossover. People gravitate to what they already believe. You will have similar results even if it is just linking to an independent poll. If only democrats are linking their traffic to a certain online poll, the results will come out in favor of the democrats.

Technology is increasingly pushing people towards only participating in things that they already believe. If you like dogs, you will only read websites that favor dogs. If you put up a "Do you like dogs?" poll on a cat appreciation site, you'll get a different answer from if you put it on a dog appreciation site.

2) Extreme Positions. People are increasingly moving to more and more extremes of their opinions. The only people who are going to be willing to participate in these polls are people who feel very strongly one way or another about the issue. If you have a position in the middle of an idea, you either will not be willing to participate at all, or the polling bias will not be able to categorize you.

I remember getting polled about my beliefs on abortion. They wanted a strong position on either for or against. I suggested better access to contraceptives, better sexual education for our young people, and improvements to our medical technology. I asked them, if early enough detection could be made, could those unwanted fetuses could be removed and stored for people who did want them? We can already store fertilized embryos, could that technology be adapted to unwanted pregnancies?

The question they wanted me to answer was, "Am I for, or against abortion?" My answer was neither, with a lot of alternate suggestions for addressing the issue. There was simply no way for them to process my answer. The woman conducting the poll, was getting angry with me, because I wouldn't commit to one answer or the other. She ended up telling me, she would put me down for against abortion, and as I was telling her, that that was not what I felt, she hung up on me.

So either the extreme position is on the part of the one answering, or the extreme position is one the part of the one asking. Either way, it is not going to give you an accurate picture of what people are thinking.

3) Apathy. I get so many calls a day from different credit card companies, consumer research companies, telemarketers, political robo-calls, et cetera, that I just tune them all out.

It has become a knee jerk reaction. "Hello. My name is (insert fake name). I'm with (insert fake research group). We're very concerned about (insert your own polling bias). We'd like you to participate in a short survey of (too many damn questions). We'd like to get your opinion of (badly worded and confusing question)."

I've usually hung up, long before they get to their question. I have no delusions that my opinion even matters to these people, so why bother even talking to them?

Have you ever noticed most polls have three answers? Yes, No, and Other. Polling companies, and people reporting on the polls will only ever talk about the Yes and No parts. Why don't they talk about the Other category? The reason, the Other category is where the discussion is happening. It is where real independent thinking happens. The Other category might contain thinking that makes the two sides of an issue superfluous.

Let's try our own poll. Do you like pickles?

Your choices are, Yes, No, and Other.

My answer would be, I like garlic dill pickles, and sweet pickles, but not pickled beets, or pickled herring. I like pickled hot peppers, but they keep me up at night, and upset my stomach.

So am I for or against pickles? Based on my answer, I get shoved into the Other category for my answer. Yet hidden in the Other category is my real answer.

Some people might say they don't like pickles, but forgot when they answered that the olives they enjoy on their salad are a form of pickle.

Some people might say they like pickles, but weren't thinking when they answered about pickled pigs feet, which disgust them.

The real answer can only be discovered in the Other category, but it is never talked about.

Add those three factors up. Technological limitations, Apathy, and Extremity of positions, and you'll see why modern polling means jack squat. It is a great talking point for pundits on MSNBC, Fox or CNN, but in the end, doesn't mean a darn thing, if the real issues are not being talked about.

You won't ever see me quote a poll to help my position. I know polls are all bull. If you feel the need to quote a poll to promote your position, I'm guessing you really don't know all the sides of the issue you are concerned about. Educate yourself a little more, and come up with your own opinion. Just because one poll somewhere agreed with you, is meaningless.

Modern polling will need to change drastically to keep up with modern people. As it currently sits, the way polling works today, you can get any answer you set out to get. Using polls today to predict elections is no more useful than trying to interpret the future from tea leaves. And you don't even get to enjoy a nice cup of tea first.

[hazbalz]

We're six months out. My question is, who cares?

[devi]

It ain't over until the fat lady sings. S-o-o-o... Next November I'll just stick a pillow in my t-shirt, put on my church hat and start singing away at the top of my lungs.

[bobover3]

Hazbalz, those running campaigns care, because poll results tell them how well their strategies are working. Campaigns are all designed, and polls are comparable to market research by advertisers. The polls shape the messages you receive, the priority given to issues, and the candidates' positions.

From the other side, polls give us a sense of what other people think, not just the people we know or who are like us. With all their variability, polls are still the closest thing we have to an objective view of our society, free of bias. It may shape our own perceptions (or it should) to know what others think.

[Cainanite]

Hazbalz, those running campaigns care, because poll results tell them how well their strategies are working. Campaigns are all designed, and polls are comparable to market research by advertisers. The polls shape the messages you receive, the priority given to issues, and the candidates' positions.

From the other side, polls give us a sense of what other people think, not just the people we know or who are like us. With all their variability, polls are still the closest thing we have to an objective view of our society, free of bias. It may shape our own perceptions (or it should) to know what others think.

There is a shift happening in the world of knowing what people think. There are Word Clouds, Focus Groups. Non-Directive Probing (meaning there are no defined answers.) and algorithms being used to interpret trends out of complex answers.

Polls are becoming something of the past. And we are seeing the emergence of new technology, and new ways of tracking information, because polling no longer works.

There is an effort to decode more complex answers than just Yes, No, and Other, because those answers do not paint a full picture of what people are thinking.

We have been seeing examples of polling failures for over a decade. Is that a coincidence that polling failures have tracked with the rise of social media, and and a technologically savvy generation of voters? I don't know for sure, but you don't have to look far or very hard to see where the polls favor one outcome, but the voting favors another.

You are absolutely correct that political organizations need to know where the voters stand. We do need a sense of what people think to make correct decisions. We are seeing technology that looks to trending topics, and monitors popularity levels of interactive media. They are trying to address the very real need to cover the growing gaps that simple polling fails to cover.

I think this is where the more successful strategies will come from in the future. Polling (at least in the way we currently know it) is on its way out.

Just for fun, and in a non-political way, have a look at the movie tracking site Rotten-Tomatoes. It is a unique site because it tracks trends in positive or negative language derived from multiple sources and multiple authors to arrive at a "freshness level" of different movies. It then assigns a percentage from Rotten to Certified Fresh. The higher the percentage, the better the movie is. It is basing the results not on a straight-up/straight-down method, but on all the complex language factors from all the different movie reviewers.

This is an example showing the beginnings of how we will arrive at reliable data from the populace in the future.

[sduyck_2000]

its because people like me lie to the pollsters


i have been polled every week for 5 straight weeks now

i give a different answer every week

it depends on my mood and if they call me away from supper they get nothing from me

[Cainanite]

Here is a Word Cloud I made of this very thread.

CLICK!

I wonder what it tells us?

[Jesus]

Once, and only once, did my department allow me to teach the required course on formal research methods. I warped the minds of the students such that they would never let me near the course again.

For the unit on opinion polls, the class decided on a topic and then we wrote questions that were very subtly different for each point. We predicted how the wording would skew the results and were rewarded (after quite a bit of telephone polling) with results that matched our predictions. The questions all "seemed" neutral on the surface, but were clearly not.

I doubt that any of my students has ever trusted a poll since....

[bobover3]

Cainanite, many people sign up in advance to participate in polls run by various organizations. I signed up for Zogby polls a few years ago, and get them by e-mail about twice a month. Phone use is shifting to cellular, but pollsters can call cell numbers as well as land lines, can they not? Cell numbers are not published in the traditional phone book, but they are compiled by QSent (now TransUnion) and are available in a directory which campaigns can buy. Someone with Caller ID who sees an incoming call from a poll may be eager to answer. As someone who got his first cell phone only a year ago, and uses it exclusively for brief outgoing calls while away from home (because of the proven health threat of cell phone use), I might question your characterization of me as "older folks on a fixed income, or people of lesser means." The demographics of phone use are not so simple.

The polls we're discussing are not self-selecting. Pollsters contact subjects, not vice versa.

Your anecdote about a poll on abortion shows how polls may be poorly designed. The intention of a poll is to learn how people respond to the choices put before them in the real world. For example, in the coming election a person may vote for Obama, Romney, a third party, or not vote at all, or be undecided. If I strongly support Herman Cain, it makes no difference, since his name will not be on the ballot. It is NOT the purpose of a poll to capture the nuances of subjects' thoughts, which may be infinite. The purpose is to learn how subjects will ACT, given the available choices. For example, on that abortion question, if a particular bill is before the legislature, the question is whether you support or oppose that bill. Saying you would support another bill of your own conception is not relevant, since that bill does not exist.

There's no reason to suppose people without extreme positions would avoid polls. People participate because they want to be heard, because they want their opinions to count.

The "other" category is seldom discussed because very few subjects choose it. If they did, that would be a tip that the poll had been misconceived and should be redesigned. Again, pollsters are interested in certain behaviors, e.g., who will you vote for, and they want to know what the majority will do. "Independent thinking" tends to vanish as statistical noise. Pollsters want to know what most people think, not what Cainanite thinks. That may frustrate Cainanite, but unless he, personally, decides the outcome of elections, pollsters are correct to make him one of many.

Even those who have independent opinions, reached after much learning and thought, may be concerned with what is likely to happen because of prevailing opinion. If I strongly support a political position, I want to know the chance that position has of winning elections and becoming policy. It isn't enough to think myself "right," if I'm right alone.

There are polls which have been consistently accurate for many years. If you claim poll results are meaningless, you'll need to explain this.

[bobover3]

Polling bias is addressed by multiple independent polls. If several polls produce consistent results for comparable samples, then the direction of the results is probably correct.

I'd like to ask posters about the alternatives they propose. We want and need to know what the public thinks. How will we do so? If not polls, then what? Polls are the popular voice, despite the frequent lack of clarity. The alternative is rule by an elite without concern for what most people think.

[Dave]

I used this page on Real Clear Politics:
http://www.realclearpolitics.com/epo...bama-1171.html
None of these firms are using stupid questions or mis-ordered and misleading questions. That's a quick way to getting no business. No one wants a poll goofed up like that even if you are the winner in the poll. Too many people understand statistics and probability to get away with a polling company putting out garbage.

1) there is some advantage in what RCP does in averaging all the polls. That's what that jagged, squarish graph below the table of polls shows over time. It's a simple and straightforward measure of where all the polls are. It's not predictive. It's as good as any poll that is reported. All it does is show the results. It doesn't predict. Polling does not produce data that can have a neat and precise line drawn through the points to get a mathematical expression. You can't do regression analysis on poll results to predict an election. These are not "those type" of results.

As for the table presented at the top of the page -- things you could do:

2) I would consider the two tracking polls separate from the other results. Why? Because they use a "frame" of "likely voters" and they average the poll over three or four days. It's a bigger sample of the public that will turn out to vote. Tracking Polls are extremely expensive and manpower intense because they get 700 to 1000 responses per day and use a moving average of three or four days. I say 700 to 1000 because it takes roughly that to get between 350 and 500 usable responses a day for a tracking poll. Yes, the unusable rate can be 50%.

3) Separate the results of polls using "Likely Voters" from "Registered Voters" ... Why? because that's two different populations.

4) never listen to a poll of the general public because that would include people who don't vote and in order to elect a president you need to survey voters. A poll of the general public is an OPINION and show general attitude. Since between 50% and 60% of the people actually vote, at least 40% of the participants will never see a ballot or a voting booth.

Now you can scroll down to a very large table and play with lots of results. BUT, be careful of comparing the same things. For instance, poll dates affect results. Don't compare a poll from months ago to a recent one and declare a winner. That, I hope, is obvious.
Try comparing "Rasmussen Reports" with "PPP" for a better comparison - why? They both use the same methodology and get cellphones and they occur roughly at the same intervals. One is Republican and uses the most favorable Republican Frame and the other is Democratic and uses the most favorable Democratic Frame. I can locate Rasmussen at Romney +3 and PPP at Obama +3 and they are one day apart. What does that tell me? It's a tie.

However, if you plot Rasmussen Reports, it fluctuates around that tie point. That's the weakness of a single poll. Go through the table and pick out the results. I'm using Rasmussen only as an example that was easy to find.

Which is where I will leave you. Six months out from an election is a long time.

Personally, I'd search out an analysis of STATE polls for swing states and work back from there.
The national polls are

[Cainanite]

Cainanite, many people sign up in advance to participate in polls run by various organizations. I signed up for Zogby polls a few years ago, and get them by e-mail about twice a month. Phone use is shifting to cellular, but pollsters can call cell numbers as well as land lines, can they not? Cell numbers are not published in the traditional phone book, but they are compiled by QSent (now TransUnion) and are available in a directory which campaigns can buy. Someone with Caller ID who sees an incoming call from a poll may be eager to answer. As someone who got his first cell phone only a year ago, and uses it exclusively for brief outgoing calls while away from home (because of the proven health threat of cell phone use), I might question your characterization of me as "older folks on a fixed income, or people of lesser means." The demographics of phone use are not so simple.

The polls we're discussing are not self-selecting. Pollsters contact subjects, not vice versa.

Your anecdote about a poll on abortion shows how polls may be poorly designed. The intention of a poll is to learn how people respond to the choices put before them in the real world. For example, in the coming election a person may vote for Obama, Romney, a third party, or not vote at all, or be undecided. If I strongly support Herman Cain, it makes no difference, since his name will not be on the ballot. It is NOT the purpose of a poll to capture the nuances of subjects' thoughts, which may be infinite. The purpose is to learn how subjects will ACT, given the available choices. For example, on that abortion question, if a particular bill is before the legislature, the question is whether you support or oppose that bill. Saying you would support another bill of your own conception is not relevant, since that bill does not exist.

There's no reason to suppose people without extreme positions would avoid polls. People participate because they want to be heard, because they want their opinions to count.

The "other" category is seldom discussed because very few subjects choose it. If they did, that would be a tip that the poll had been misconceived and should be redesigned. Again, pollsters are interested in certain behaviors, e.g., who will you vote for, and they want to know what the majority will do. "Independent thinking" tends to vanish as statistical noise. Pollsters want to know what most people think, not what Cainanite thinks. That may frustrate Cainanite, but unless he, personally, decides the outcome of elections, pollsters are correct to make him one of many.

Even those who have independent opinions, reached after much learning and thought, may be concerned with what is likely to happen because of prevailing opinion. If I strongly support a political position, I want to know the chance that position has of winning elections and becoming policy. It isn't enough to think myself "right," if I'm right alone.

There are polls which have been consistently accurate for many years. If you claim poll results are meaningless, you'll need to explain this.

You can do a very simple experiment on accuracy of polls. Take a large group of people, and have them simply guess whether a coin flipped will come up heads or tails.

Chances are, about half will get it right. Eliminate the half that got it wrong. Take the remaining half that got it right, and perform the experiment again.

Keep doing that until you arrive at a single person. That single person has correctly guessed the flip of the coin for every iteration of your experiment. However once it gets down to a single person, his chances of getting it right, are still only fifty-fifty.

I'm not really using this as a mathematical proof of statistical accuracy, there are plenty of holes I can poke in this argument if I try that. Feel free to point them out if you with, but I'll admit they exist here and now.

My point is that there are bound to be very accurate predictors in certain polling operations. However, they will still be bound by a lot of assumptions that are simply out of their control.

When we get into polling, the variables start to stack up. You choose to participate in the Zogby polls. Is the distribution of Zogby contributors an accurate sample of the populace? Is the age range of voters fairly represented? Are the political views of the contributors a fair representation of the populace? Do you have to do polls on top of polls to find out what the real distribution is? What is it that makes the average Zogby contributor choose to offer their opinion, when someone like me chooses not to?

Another dividing factor in polls is, are all the options being fairly represented?

If you are using the polls to determine who will win an election, how do you account for the likely voters who choose not to participate? I once interviewed for a telemarketing job, and the interviewer informed me that perhaps one in twenty calls would actually be answered. From that only another one in twenty would participate with the conversation. What is the difference in attitude that causes a person to not just answer the call but participate in the conversation? How can you tell if they differ from the rest of the population, if you cannot even talk to those who don't participate?

If you have the simple question of who are you going to vote for, you may be closer to the mark, but still fail, because you have people who will simply lie. If an election is close, the margin of error will reduce the results to no more accurate than a simple toss of a coin.

As to cell-phones, yes. More and more polling companies are hitting up people with cell phones. However, talking to a friend of mine who actually does telemarketing calls, he tries to avoid getting the cell list on his shift, because the failure rate is a lot higher. The person you are calling has to pay for those minutes if they answer, and get very mad if they think for a moment you are wasting them.

Polling really fails the test, when, as you say, the ones using the polls are trying to get an idea of what the populace thinks about certain issues. Where the poll is trying to test the waters, rather than asking exactly what is on an established ballot. That's when the Other category becomes incredibly important, yet is still ignored.

Sadly, polls are not unbiased. Bias can and does influence the outcome. The chances of getting people to participate in polls, is entirely dependent on the idea that the sample is a fair representation of the populace.

There was a time not long ago that polls were very accurate, but that time is getting further and further behind us. New technology, and new methods of data collection have come to be, simply because there is a recognized need for it. If polling were perfect, we wouldn't need the newer methods.

You can try to interpret the data, but even your interpretation can be flawed by bias.

I don't wish to completely discredit the act of polling. It can give you a sense of which way the wind is blowing. My point is that you cannot use it for as a proof of your position. You cannot just say 55% of the people we polled agree same-sex marriage is wrong, and walk off into the sunset. All it really means, is 55% of the people who chose to talk to you agreed it was wrong. The reality may be very different come time for people to cast their votes, and voting is the only real poll that matters.