I occupy an unusual niche, writing and performing quantitative analysis. So I’d like to craft a long post on Nate Silver, the election quant at the New York Times.
Unfortunately, the election is tomorrow, and I still have no power – thanks! PSE&G. So I’ll have to tap this out on a cell phone; forgive the Androidian typos. Mainly I want to point out that Silver’s daily column is a great example of how to use a predictive model to gain insight, and how to communicate the model’s strengths and weaknesses to a general audience.
From my phone I can’t link to the best example. Three weeks ago, national polls showed Romney tied or ahead, but state polls favored Obama. Silver patiently explained how his model, more or less split the difference between the two, with a slight edge to the state polls.
Here’s another example, again discussing national vs. state polls:
The FiveThirtyEight forecast has Mr. Obama leading the popular vote along with the Electoral College, because it uses both state and national polls to calibrate its estimate of where the vote stands. Also, however, Mr. Obama’s state polls were adjusted slightly downward because his national polls remain middling.
Still, our state-by-state forecasts are extremely similar to those issued by our competitors. For example, we had Mr. Obama projected to win Ohio by 2.4 percentage points as of Friday. That compares to a 2.3 percentage-point lead for Mr. Obama in the Real Clear Politics average of Ohio polls, a 2.9-point advantage for him in the Huffington Post Pollster model, and a 2.7-point edge for him according to Talking Points Memo’s Poll Tracker.
How often does a lead of two or three points in the polling average, with 10 days to go until the election, translate into a victory in the state?
This is the sort of question that the FiveThirtyEight forecast is designed to address. But a simpler method is to just look at what happened when candidates held similar advantages in the past….
Notice he’s not spending a lot of time explaining the model. If you read other posts, you’ll know it’s a bit of a black box – at various times he has discussed how it tweaks registered voter polls to behave like likely voter polls; how it contains an econometric forecast; and how it weights various polls on their previous predictive ability. But he doesn’t discuss the model’s guts unless he has to.
First he shows how his model resembles other forecasts. Then he discusses an easy-to-understand test that reaches the same result as the model.
Lesson: The model acts as a guide, but needn’t drive the explanation. The examples from outside sources make outside parties comfortable with the results. The simple test does the same.
Often we love our models so much, we want to lay out every detail about them. But sometimes we need to let others do the talking.
Silver has taken some heat for confidently predicting a close Obama victory – around 90% probability, he asserts, based on his proprietary predictive model.
I think many of his detractors misunderstand what he is saying. He isn’t saying Obama will win by a comfortable margin. He is saying Obama’s slim margin will probably hold. The two statements are not identical.
Postscript: A nuanced criticism appears here.
Posted from WordPress for Android