February 20, 2012

How to Create Momentum By Changing Your Sample

Public Policy Polling is out with a new Michigan Poll showing a significant shift from Santorum +16 to Santorum +4. But is the momentum really with Romney? Not to the extent to which Public Policy Polling might suggest.

In comparing the sample from the February 13th poll, it becomes apparent that there’s a great deal of difference in the sample composition:

Demographic Group 2/13 Poll 2/17 Poll
Evangelicals 48% 41%
Very Conservative 38% 31%
Somewhat Conservative 35% 38%
 Moderate 18% 22%

The 2/17 sample is probably closer to accurate as a representation of the Michigan electorate that will show up on Tuesday than the 2/13 poll. This moves PPP from outlier to right in the mainstream of its polls. When Real Clear Politics updates its polling averages, the PPP Poll will replace the Detroit News poll that also showed Santorum up 4% and the RCP average will remain at Santorum+6.5%.

While there may be some movement of undecided towards Romney, PPP overstates its case because it bobbled its 2/13 sample. Properly examined, this poll falls in alignment with the vast majority of polls taken over the last week.  To say that there’s serious movement towards Mr. Romney we’ll need to see more than one poll, preferably with a consistent polling sample.

by @ 12:00 am. Filed under Poll Analysis
Trackback URL for this post:
http://race42012.com/2012/02/20/how-to-create-momentum-by-changing-your-sample/trackback/

34 Responses to “How to Create Momentum By Changing Your Sample”

  1. SixMom Says:

    The 2/17 break-down is also more in keeping with the public breakdown at large.

  2. Reginald from texas Says:

    Even the 2/17 poll has a much more conservative breakdown than did the 2008 exit polls

  3. Sir David Says:

    The name of the article implies that PPP purposely tried to portray momentum, when in fact it was just a botched early poll.

  4. Sir David Says:

    Question to Sant-heads:

    What makes any of you think that Santo can maintain his lead, even tho no one but Romney has held onto a 25% base?

  5. Reginald from texas Says:

    The debate will probably decide everything anyway. when Mitt destroyed Newt in the debates after SC, it made all of the difference

  6. Adam Graham Says:

    #5:

    To bad, Romney got rid of his Jedi debate coach.

  7. Watchinitall Says:

    6. Not really. He’s going to get the credit or the blame. That’s better for Mitt and the voters, and Mitt has done extremely well in most of the debates.

    Hard to overstate the impact of Wed. Night’s bout.

  8. My Man Mitt 4 President Says:

    So.. in other words evangelical (very conservative) do not like Romney.. and if you want to change the momentum of a candidate you mess with the numbers and produce the desired outcome and if you leave it there long enough it becomes the truth because everyone wants to vote for the winner.

  9. Heath Says:

    Lol keep telling yourself that Ads! It will be close but Mitt wins by 1-5%! And Arizona by 10%!

    Nomination effectively over (phew!).

  10. Teemu Says:

    and the RCP average will remain at Santorum+6.5%.

    Which is due to couple other botched polls still remaining on the list, without them it would be lower. Mitchell 55% evangelical, 38% “very conservative” (2008 MI R primary exit polls ev: 39%, vc, 24%)

    Inside Michigan/MRG 47% strong tea party supporters, at exit polls Florida had that at 35% (and very conservative were 33%), South Carolina 33% (very conservatives were 36%). This means MRG sample is likely somewhere between 44-50% “very conservative”

  11. Barney Says:

    I’m not so sure that they are trying to control the survey to make sure that they get a certain percent of evangelicals, a certain percent of very conservative, somewhat conservative, or moderate. They might have done this on age, sex, party affiliation, but they can only control for so much. It’s important to also realize that the margin of error in the cross tabs can drop significantly. They might, however, test these results to see if these percents are statistically similar to the general population.

    I think sometimes we, myself included, are a little quick to say that a pollster is trying to get a certain response and dismiss or rationalize the results. I can’t say that this doesn’t happen and it probably does with some organizations and some issues. What is even worse, is when we try to justify or explain away the results of surveys to fit our own perspective. There are certain very prominent conservative talk show hosts that do this all the time and it bugs the bat-snot out of me. Survey results that show what they want are somehow more accurate than survey results that might not show what we are trying to promote.

  12. penny Says:

    Which proves my point that PPP created a false narrative that Santorum had a huge surge!!!!!

    And RCP is playing that game too. They have completely ignored polls showing Mitt and Rick tied at the end of hte week.

    This stupid pretense that Santorum can get votes is laughable… the sample of all the polls showing his surge were just as off.

    The stupid media is trying to shove that kook Santorum down our throats.

  13. Willard Crapweasel Mittens Rombot Says:

    6. Mitts spanking of Nooot in the last debates had nothing to do with his debate coach. Maybe Mitt will berate the moderator for some easy points with the crowd like carnival barker nooot.

  14. penny Says:

    11 Barney wake up smell the coffee the whole lie that Santorum had support is a way to keep the race going and get ratings…

    And to damage our nominee for the General.

    You are naive if you can’t figure out the motives for throwing a Presidential race.

    And the ones of you supporting these ABRs… have played Obama’s puppet theater just as planned.

    Could you be more dumb.

  15. Heath Says:

    It’s painfully obvious that PPP just want to make news! Any way they can.

  16. Barney Says:

    Penny,

    Could I be more dumb? Watch this…I don’t think that Santorum’s surge was manufactured by polling companies. Could the results have been an outlier and the surge exaggerated, maybe. But I don’t think that the pollsters decided that they would suddenly sample a higher percent of very conservative in one poll verses another just to extend a story.

    I do agree with you on the ABR’s. But I don’t think it is due to a polling conspiracy, but rather that polling at this point is somewhat meaningless as there is not a clear nominee yet. Head to head, however, I think Romney is clearly the stronger candidate.

  17. Katechon Says:

    Could it be that PPP is sensing a higher turnout amongst non Evangelicals than the turnout they were projecting last week?

  18. Teemu Says:

    17:
    I think that they got it wrong last week not really sensing, of the three 2012 primaries only primary where evangelicals where significantly up was South Carolina,65% compared to 60%, so 65/60=1.083333, 8.3 percent increase, which for Michigan would be equivalent of the evangelical turnout increasing from 39% to 42%. Also I think there was specific reason for the evangelical boost for Newt, many Christians confuse forgiveness and trust. After the CNN debate question and ABC ex-wife interview, the initial gut reaction and subconscious thinking went for them something like this:

    “Oh, that was so mean, I’ll forgive him, I’m such a good and forgiving person for voting that Newt Gingrich”

    Psychologically, voting for Newt was same for many of them as voting for Barack Obama because he was the first African American presidential nominee was for many people. Will there be event giving Santorum similar boost as Newt got in South Carolina, or has the socon issue discussion already brought that, or has Santorum gone to far in socon issues for Michigan primary voters, hard to say.

    In 2008 Michigan Republican primary was more moderate than 2008 Florida Republican primary. Florida had the largest conservative increase in the primary exit polls, South Carolina staid same, though 2 percent more “very conservative”, 2 percent less “somewhat conservative”, New Hampshire was slightly more moderate this year.

    Let’s say Michigan’s conservative turnout proportionally was up by as much as Florida, that would mean “somewhat conservative” 34% (32% in 2008), “very conservative” 29% (from 24% in 2008).

    Now why I don’t think it is going to be up that much:

    Florida had in 2008 ballot also vote on making the constitutional limits on property tax increases more strict, this motivated more moderates, Michigan didn’t have anything special.

    In 2008 the Florida primary was competitive to the end, in 2012 it was not, Romney winning by double digits was obvious because of early and absentee vote and polls, and the system was winner-take-all, pragmatic moderate voters more likely staid in home in greater numbers on the election day due to this. A moderate voter was more likely to stay home because of this obviousness, and moderate voter was more likely to come to the conclusion from the more moderate media sources that the results were obvious, whereas some “very conservative” news sources were probably more likely to keep up the hope of defeating Romney in Florida primary.

    Florida primary, unlike New Hampshire and South Carolina, required you to be registered Republican, having to register as Republican probably prevented any significant moderate trickling from the lack of primary on the Democrat side, in Michigan you don’t need to be registered.

    Also the union household turnout is as high in this sample as in 2008. In 2008 Romney lost the union member vote by 16 points, when he won the state by 9 points, this was his worst category, he did much better among people having just union member in the household, but not union member themselves, or non-union member households. PPP doesn’t separate union members and people who have union member in their household, Romney did somewhat worse in their previous poll among union households than Santorum, and gets crushed in this poll among the union member households, he actually wins non-union member households. Michigan moved from open to semi-open primary. Semi-open means that you have to declare your party identity and this goes on to list, on which parties and unions, or whoever wants to access, can see it. This suppresses union member turnout more than anything else, I’ve heard from some Michigan folks, that in many unions, you really don’t want to end up in any list listed as Republican. I don’t think the union member/union household vote will be 27% this year.

    Also the listing might suppress the cranky paranoid vote, hard to say whether that hurts Santorum or Paul vote more :D

  19. Katechon Says:

    Teemu—very interesting, as always.

    Many, many thanks

  20. Not Your Severely Constipated Daddy Says:

    Mittmentum baby!

  21. Irish Right Says:

    Of course, Adam, you miss the larger point. Santorum never had the kind of support that PPP suggested he had.

  22. Adam Graham Says:

    #21:

    I said pretty clearly last week’s poll was an outlier which indicates it wasn’t accurate.

  23. Adam Graham Says:

    #18:

    It’s hard to say definitively that South Carolina was the only state with higher Evangelical turnout, because we don’t really know what happened in Missouri, Colorado, and Minnesota in that regards. South Carolina may be the only state higher turnout’s been documented in, but its possible in the other states given the result.

  24. Jeff Fuller Says:

    U can’t explain away a +12% shift in Romney’s favor by a -7% shift in Evangelicals. Even if 0% of those Evangrlicals switched over to Romney (ie stuck with Rick or Newt) that would only explain half the surge.

    Look at the undecideds. They’re breaking hard for Mitt and that’s the real take away.

  25. Teemu Says:

    23:
    Evangelicals were also down in Iowa and staid same in Nevada though. Anyway, Missouri beaty contest was pretty unique, turnout down 60% from last year, Colorado, Minnesota and Maine were the lowest turnout events this year, I don’t think their turnout matters that much when it comes to trying to figure turnout of real primaries.

  26. Matthew Kilburn Says:

    “U can’t explain away a +12% shift in Romney’s favor by a -7% shift in Evangelicals. ”

    Bingo! True, PPP recalculated their sample, and, accordingly, Romney made up some ground by default. But you can’t simply write it off like you’re trying to.

    And, even if it WAS entirely due to the sample change, that would just show you that Romney was never in as bad of shape as a lot of people were saying.

  27. Massachusetts Conservative Says:

    The new sample has 8% more men and 8% less women.

    That’s a 16% difference in the gender breakdown.

    Mitt always does worse among men than among women, and for him to make up so much ground when the gender breakdown goes 16% hard to Santorum’s wheelhouse is truly remarkable.

  28. Matt C Says:

    In other words, PPP proves themselves to be a massively unreliable pollster yet again. Big surprise there.

    You have to wonder why PPP all of a sudden decided to weight their sample differently. They accomplished exactly what they wanted to accomplish: they were the first ones out of the gate with a MI poll, and showing Santorum +15 set the narrative nicely. Every piece of media for the next several days was framed from the standpoint on Santorum having a huge lead in MI. A democratic pollster employed by DailyKos and the SEIU set the narrative for the Republican primary with just one crap poll.

    Then other polls starting coming out, though. Suddenly, Santorum +15 started looking like the joke it was. Two polls had Santorum at +3 or +4, other had it at Santorum +8 or so. Nobody else came close to Santorum +15. So, panic at the disco (aka PPP headquarters): “Boss, the race is a lot closer than +15! What do we do?” “Don’t worry, we’ll just weight the next one a little differently to be in line with everybody else’s.”

    You could worry about all of this stuff and analyze all the PPP crosstabs… or just follow my advice from the past three years and ignore PPP polls as the garbage that they are.

    (How’d that 8 point Romney win in Florida go again, PPP? Hahahaha…)

  29. Massachusetts Conservative Says:

    28

    Yes, it’s easy to disagree with you because PPP releases so much stuff and got NH and SC right, but you have convinced me. They have screwed everything up.

  30. OHIO JOE Says:

    Good catch Mr. Graham.

  31. Katechon Says:

    28– non sense of a paranoid mind.

    I really believe in their integrity. They’re transparent. And they’ve helped me a bunch of cash.

  32. Massachusetts Conservative Says:

    31

    They helped you make money because Intrade folks who bet rely on their numbers. So it’s understandable, then, that you defend them. If their credibility sours, that keeps PPP numbers from moving the betters on Intrade.

  33. machtyn Says:

    28. Would it not also be as damaging to have Santorum appear so far ahead and then a few days later have such support suddenly drop?

    If the poll was set up to influence sheep, then those sheep would be as equally devastated by the decrease of support for the candidate the poll attempted to assist. Which, if this turns out to favor Romney – I’m for it! (But not really. I’d prefer a polling company to not have a bias in either its polling process or in its release of data.

  34. cure panic, cure panic attacks, cure anxiety, cure anxiety attacks Says:

    I do trust all of the ideas you’ve presented to your post. They’re really convincing and can definitely work. Nonetheless, the posts are too brief for novices. May just you please extend them a little from next time? Thank you for the post.

Leave a Reply

State of the Race


Obama Approval


Support R4'12

Meta

Recent Posts

Buy This Book

Categories

Archives

Search

Blogroll

Site Syndication

Main