loquacious

Election 08 Prediction: will poll averaging work? Obama wins easily if so.

Hi. It's 7am here on the Pacific coast. I've analyzed the available polling for the past week in each of the swing states. If the method I developed works (for all the reasons it might or might not listed in this post), here's what it looks like the results will be:

Obama: 353 electoral votes
McCain: 185 electoral votes

My method actually isn't good at predicting ND (3 EV's, there weren't enough recent polls), so I could be off by 3.

Here is my breakdown on who wins which swing states:

Obama:
NC
FL
VA
PA
OH
NH
OH
IA
CO
NM
NV

McCain:
GA
IN
MO (this is the closest race)
MT

and like I said, ND has insufficient data.

It's worth reiterating that this predictive model is only as good as the polling data (and the manipulation/weighting the pollsters do to adjust the data). It will actually be interesting if this method does not work this year, because it worked perfectly to predict the winner of each state in 2004 (though not the margin of win).

Given the problems that surfaced in the Democratic primaries (noted here and here), it's not at all clear that current polling methods are effective at getting to the entire electorate. Rural minorities seems to be especially undercounted.  And by now, I'm guessing everyone has heard about the cell-phone-user-only undercount, as well as the so-called "Bradley effect" - which many have recently discounted, but as we saw in the primaries, it was incontrovertible (I think) that certain white voters seemed to prefer Hillary Clinton to such a degree that you could not help but think that race had something to do with it. There was also recently the fascinatingly complex poll that estimated that Obama would likely lose 6% of his potential vote due to racial bias.

UPDATE: well, obviously, the system worked. Except for IN. Nobody got that right. Interesting to consider why....

Predictive power of polls (aka, who will win on Tuesday?)

So, in true nerd fashion, I wondered whether the 2004 presidential election polls actually had any accuracy in regards to who would win, on a state by state basis.

So, I downloaded all the non-partisan polls from 2004 from each of the 'swing states' in that election, from www.electoral-vote.com (a truly excellent polling site).

What did I learn? Well, my technique was to see what would happen if one averaged all the polls for a given state that were conducted in the 7 days prior to the election. I wanted to see if the average of these polls would accurately predict who would win that state.

I ran this analysis for all the 2004 swing states, and all those that ended up with a 5% or less spread between the candidates (CO, FL, IA, MI, MN, NV NH, NM, OH, OR, PA, WI). In 100% of cases, the averaged poll lead did go to the eventual winner of that state. I'm not saying the margin of win was correct, but the actual winner was accurately predicted by the average of all polls in the state.

What does this mean for the coming election? Well, with some caveats (which follow) it means we should be able to average all the results of polls available on Tuesday morning and predict the winner, state by state, and therefore the overall election.

The caveats here are: 1) this only works if the way polling is done has not changed substantively since the last election; 2) if voters are being as honest this year as 4 years ago about who they will vote for; 3) if there is not some other unaccounted for factor that has not been taken into account here (such as more young/first time voters this time around, or more cell phone users who weren't captured by the pollsters, etc).  Still, overall, given that this method worked 100% for the last election, it seems probable to me that it will work this time as well.

We'll see.

Fundamentalism Dot [The Battle for God: A History of Fundamentalism]

Karen Armstrong
2000
370 pp.

I haven't been blogging the books I've been reading lately (wow, for a year and a half!), but this one, The Battle for God, intersects with so many other books I've read, I had to put some notes down on paper screen.

The Battle for God, briefly, is the story of fundamentalist movements of the major monotheisms -- Christianity, Islam, and Judaism. Her thesis, which I will not do justice to here, is that fundamentalism arose as a reaction to the Enlightenment. She traces the history since then (as well as events leading up to the Enlightenment) with a special concentration on the emergence of fundamentalism as a political force in the 20th century. Basically, she argues that for the first 1000 years of all these religions, no one would have dreamed to try and take the holy books literally. That ancient people's knew the difference between mythos (metaphor) and logos (reason/science), and that they were comfortable using them side by side. Only as reason came to dominate life in Europe did some religious believers start to confuse mythos and logos. She follows how that has played out over the centuries, eventually getting us to where we are in terms of Christian fundamentalists in the US, religious Zionists in Israel, and the Muslim jihadists seen in some Arab countries.

One of my long-term projects, and a major reason for this blog, is to try and connect the dots out there. There are a lot of complex, interacting forces at play in the world, which makes it very difficult to understand the reasons why things are as they are, and why things play out the way they do. I keep thinking there must be a way to isolate and define a few key drivers. Hence, the dots. I think that fundamentalism, as explained by Armstrong, probably ranks as one of the dots. [Other dots covered so far are Globalization and Framing/Linguistics. A dot that I still need to work on is Masculinity, mainly based on Susan Faludi's brilliant book Stiffed.]

The rise of fundamentalism pretty neatly fits into these other cultural happenings. Fundamentalism and Globalization are deeply intertwined, and by globalization in this case I mean exploration/colonization that started hundreds of years ago. Presumably, fundamentalism wouldn't be the force it is if Europeans hadn't both come to the Americas as well as done the Crusades, as well as economically controlled/colonized the Middle East in the 20th Century. The kind of literalist backlash that is fundamentalism is of course exactly what Lakoff is describing in Don't Think of an Elephant . . . a kind of thinking that some find basically impossible to understand. Armstrong's book goes a long way to explaining the (religiously) conservative mind set: it's a fundamental focus on the past as the best possible time, which therefore needs to be recreated. Liberal thinkers (or Enlightenment-thinkers/believers in science/technology) focus on the best times being ahead in the future. If you are focused on recreating the past, of course more change is bad... especially unknown change.

As Armstrong explains it, fundamentalism is basically a by-product of the speedy progress that started happening with the Enlightenment. It might have been inevitable, but it was definitely hastened (and deepened?) by people feeling very threatened by the new world that was envisioned by the elite few at the time that could see the power of science. Basically, science did a really bad job of bringing people along, and we're still feeling the backlash. [Also explained much more eloquently than I have here.]

Armstrong doesn't write about this, but of course it's pretty much all men both pushing the enlightenment side, as well as leading the reactionary fundamentalist side. I haven't really thought through the why of this -- there could be many causal or non-causal relationships in there -- I'm just saying. There's some kind of drive for power/control that's very male... or maybe as Faludi would put it, there's some male reaction to loss of power/control that's very dangerous for our society.

Anyway, I think the book goes a long ways to explaining why things are as screwed up today as they are. Why there are fundamental splits within societies (ie. 50/50 split elections) as well as between societies (US hegemony vs. Muslim self-rule, for eg). Why people 'vote against their own interests' (they aren't!). How it is that the balance of power can be held by a small, radical few in the world we've built.

TX and OH

Well, it's time to try out our algorithm to see if it will work for the upcoming Texas and Ohio primaries.

If the method holds, here is what the results should look like, based on polling from the past week only (18 polls in OH, 13 in TX):

Ohio: The polls average to a Clinton win of 6 percentage points (so, Clinton 53% / Obama 47%). Given what we've seen, which is that the method is accurate only to +/- 5%, that would lead to somewhere in the range of a 1% to 11% Clinton win in Ohio.

Texas: The polls there show an average Obama victory of 1%. So given our range, that could be anywhere from a 6% Obama victory to a 4% Clinton victory.

As you'll recall, we were guessing that the polls (and my method) break down in states with larger numbers of rural minority voters, especially rural African American voters. So if that is true, the method should work for OH but perhaps not so well for Texas.

In terms of actual delegates, because of the delegate allocation system in both OH and TX (for the primary-based allocated delegates), it seems likely that the delegate count will be tied. TX also allocates some delegates through their caucuses, so Obama might come out with a lead there. There's not enough polling in VT and RI, but the conventional wisdom there is that they will split those two States, which have about the same # of delegates.   So at the end of the day, maybe a slight lead in delegates for Obama from the night, which is super-bad news for Clinton. But she will likely 'win' at least two states, so maybe that's enough to keep going.

VA and MD polls vs. actual vote

Well, the results are in  for VA and MD. As you might recall, we were going to test our theory of polling on these two contests. They more or less worked out.  Here are the results:

MD: Obama won by 23 percentage points, exactly as predicted.
VA: Obama won by 29 percentage points, the prediction was 19 points.

Virginia has much much more rural areas, and as we've seen and speculated, the polling for rural folks, perhaps especially rural people of color, might be flawed seriously, given how wrong the polls were in AL and GA.

So, in the upcoming contests, we might expect the WI and OH polls to be accurate, while TX may not be.

VA, MD predictions

So, if my previously described methods work, Obama should win VA by 19 points and MD by 23 points (+/-5 for both). This should be especially true for MD which doesn't have much rural population.

Why you should caucus (for Obama)

Hello Washington State friends:

Your democratic caucus is on Saturday. I will not be there to go, as I remain in exile outside the US, but I hope that you all will participate in the only process, the electoral process, that stands any decent chance of making life better for myself and other bi-national couples. In short, even though the system has me down, I think the system is the best chance we have. So please don't abdicate your vote to others, and go. [and yes, there is a Washington primary, but no delegates are awarded from the primary. None, it's for show only, what a waste of time and money.]

Worried, never gone before/apprehensive? Read this nice piece from The Stranger on what a caucus is and how it works, including how to find yours. I really found it fun and not intimidating, if you've never done it, you should try at least once.

Okay, lecture complete.

Here's why I'd caucus for Obama if I were there:  He's the most likely candidate to beat John McCain. I think the differences between Clinton and Obama on policy issues are infinitesimal, so all we are really left with is personality and electability. Personality is not inconsequential, but there are bigger issues in play here, so I'm concentrating on electability.

Obama rates better than Clinton in all the head-to-head races where they are put up against McCain. {see real clear politics}  In fact, he wins bigger and by more.

Clinton has higher and much more persistent/won't go away negative public opinion ratings than Obama.

The republicans have a lot of dirt on Hilary, and they are just waiting to drag it out. Remember all that Rose Law Firm crap and the land scandal she was involved in? What have they got on Obama?

Plus, how do they really go after Obama? Not on race. On what?  He's just a more slippery target (which yes is good and bad, but for the purposes of this argument, is good).

Finally, McCain is a centrist. He will pull independent/moderate republican votes. Clinton does not pull independents as well as Obama, and has absolutely no appeal to republican moderates. With McCain in the race, this will be a battle royale for the middle. McCain can claim the middle much more than Clinton can, but I think Obama can hold his own for the middle.

Good luck at the caucuses.

An uneasy analysis

Well, I have to say that the following analysis is gleaned somewhat anecdotally, from watching last night's coverage, reading a lot of polls over the past two weeks, and especially from some of the county-by-county breakdown of the vote (particularly in Missouri).

Basically, it's this: older voters and rural voters (especially rural souther voters) are for Clinton in a huge way. As are Southern whites.

Men are much more likely to vote for Obama.

While it would be much much better to put these candidate choices into their more positive aspects (young and urban voters for Obama, women for Clinton), the sad fact remains that the opposites remain. And that it's hard not to think older Americans might not be carrying around some lingering racist attitudes, the same with, unfortunately, rural southern Whites.  And that men's sexist attitudes will, in the end, decide their preference between a male and female candidate. In short, this primary is showing us something about America we all knew, but didn't see pushed into our face every time there is an exit poll.

Missouri is a classic case. There are like 60 counties, and three major urban areas. Clinton won every rural (more white) county in the state and Obama won the 5 urban counties (more African-American).  This phenomena overall explains Clinton wins in Tenn. and Oklahoma, while Obama wins Georgia (thanks to Atlanta and the black vote), SC, AL...

I haven't yet seen anyone really start to call out this racism and sexism in the Democratic electorate yet. Will anyone?

Also, I'm not yet prepared, nor am I sure I will ever be, addressing the issues around why Hispanics overwhelmingly prefer Clinton to Obama, or why Asian voters in CA really overwhelmingly (like 80% I think or more) went for Clinton....

Anyone want to touch that one?

 

Momentum/Upsets (or, reality v. polls)

So, it's interesting that one thing that the media concentrates on, rather than the actual delegate counts, are the perceived 'upset' victories. As if that actually matters.

What's more interesting to me, though, are the times when the polling is so very wrong. There were quite a few of those that happened yesterday.

Obama upsets:
The (nonpartisan) polls on average had Clinton ahead in these states, which Obama ended up winning:

• Alabama
• Connecticut
• Missouri
• Minnesota

Clinton didn't have any upsets, but she did have some wins that were much larger than especially the last polls were indicating:

• Massachusetts
• Tennessee
• California
• New Jersey

The final count (minus NM which is still too close to call) is that Obama took 13 states to Clinton's 9. However, Clinton did better in two of the three big states (NY and CA). But, and this is important, Obama won much larger in his big state (IL) than Clinton did in her two, so that might have evened things out. Obama pretty much wins every caucus state.... interesting that Clinton isn't even trying to win there. Her strategy seems to be to get the delegate count by working the party system in the large/delegate rich states, whereas Obama seems to be trying to add up a lot of small states by utilizing volunteers and motivating new voters to come out.

Delegate Counts (where are they?)

Given that the whole point of yesterday was to accrue delegates, I find it odd that the media is really really lagging in telling us (or at least estimating) the number of delegates won.

Recall that Clinton started yesterday with a lead (entirely due to having more superdelegates) of 232 to 158 (by CNN's way of counting). Basically, in the incomplete counts I can find so far, she basically maintains that lead, indicating that yesterday was a draw in the search for delegates. To wit:

MSNBC:
Clinton has an overall 70 delegate lead according to their newscast, when all is said and done and all 1500+ delegates from yesterday are accounted for. Their website says Clinton 582 to Obama 485 currently.
(MSNBC is the only network that did a credible job talking about who was winning what delegates last night... their political director seemed to know beforehand how the delegates would be divided)

NBC:

Another blog reports that NBC is estimating at the end of the super Tuesday count, Obama will have 840 to 849 delegates, versus 829 to 838 for Clinton.

AP:
Clinton: 845
Obama: 765

CNN:
Clinton: 783 (includes 193 superdelegates) -- 335 super tues delegates so far
Obama: 732 (includes  106 superdelegates)  -- 301 super tues delegates so far

NPR:
Clinton: 803 total -- 542 super tues

Obama: 742 total -- 540 super tues

The MSNBC count is the only one that accounts for/estimates all the delegates that were available Tuesday. Assuming that is the case, then Obama actually won more delegates on Tuesday than Clinton.  And it means that Obama still leads on delegates won in contests.