Linzer slides-barug
-
Upload
ishi-von-meier -
Category
Documents
-
view
59 -
download
0
Transcript of Linzer slides-barug
![Page 1: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/1.jpg)
Forecasting the 2012 Presidential Electionfrom History and the Polls
Drew Linzer
Assistant ProfessorEmory University
Department of Political Science
Visiting Assistant Professor, 2012-13Stanford University
Center on Democracy, Development,and the Rule of Law
votamatic.org
Bay Area R Users GroupFebruary 12, 2013
![Page 2: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/2.jpg)
The 2012 Presidential Election: Obama 332–Romney 206
But also: Nerds 1–Pundits 0
Analyst forecasts based on history and the polls
Drew Linzer, Emory University 332-206Simon Jackman, Stanford University 332-206Josh Putnam, Davidson College 332-206Nate Silver, New York Times 332-206Sam Wang, Princeton University 303-235
Pundit forecasts based on intuition and gut instinct
Karl Rove, Fox News 259-279Newt Gingrich, Republican politician 223-315Michael Barone, Washington Examiner 223-315George Will, Washington Post 217-321Steve Forbes, Forbes Magazine 217-321
![Page 3: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/3.jpg)
The 2012 Presidential Election: Obama 332–Romney 206
But also: Nerds 1–Pundits 0
Analyst forecasts based on history and the polls
Drew Linzer, Emory University 332-206Simon Jackman, Stanford University 332-206Josh Putnam, Davidson College 332-206Nate Silver, New York Times 332-206Sam Wang, Princeton University 303-235
Pundit forecasts based on intuition and gut instinct
Karl Rove, Fox News 259-279Newt Gingrich, Republican politician 223-315Michael Barone, Washington Examiner 223-315George Will, Washington Post 217-321Steve Forbes, Forbes Magazine 217-321
![Page 4: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/4.jpg)
The 2012 Presidential Election: Obama 332–Romney 206
But also: Nerds 1–Pundits 0
Analyst forecasts based on history and the polls
Drew Linzer, Emory University 332-206Simon Jackman, Stanford University 332-206Josh Putnam, Davidson College 332-206Nate Silver, New York Times 332-206Sam Wang, Princeton University 303-235
Pundit forecasts based on intuition and gut instinct
Karl Rove, Fox News 259-279Newt Gingrich, Republican politician 223-315Michael Barone, Washington Examiner 223-315George Will, Washington Post 217-321Steve Forbes, Forbes Magazine 217-321
![Page 5: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/5.jpg)
What we want: Accurate forecasts as early as possible
The problem:
• The data that are available early aren’t accurate:Fundamental variables (economy, approval, incumbency)
• The data that are accurate aren’t available early:Late-campaign state-level public opinion polls
• The polls contain sampling error, house effects, and moststates aren’t even polled on most days
The solution:
• A statistical model that uses what we know about presidentialcampaigns to update forecasts from the polls in real time
What do we know?
![Page 6: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/6.jpg)
What we want: Accurate forecasts as early as possible
The problem:
• The data that are available early aren’t accurate:Fundamental variables (economy, approval, incumbency)
• The data that are accurate aren’t available early:Late-campaign state-level public opinion polls
• The polls contain sampling error, house effects, and moststates aren’t even polled on most days
The solution:
• A statistical model that uses what we know about presidentialcampaigns to update forecasts from the polls in real time
What do we know?
![Page 7: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/7.jpg)
What we want: Accurate forecasts as early as possible
The problem:
• The data that are available early aren’t accurate:Fundamental variables (economy, approval, incumbency)
• The data that are accurate aren’t available early:Late-campaign state-level public opinion polls
• The polls contain sampling error, house effects, and moststates aren’t even polled on most days
The solution:
• A statistical model that uses what we know about presidentialcampaigns to update forecasts from the polls in real time
What do we know?
![Page 8: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/8.jpg)
1. The fundamentals predict national outcomes, noisily
Election year economic growth
Source: U.S. Bureau of Economic Analysis
![Page 9: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/9.jpg)
1. The fundamentals predict national outcomes, noisily
Presidential approval, June
Source: Gallup
![Page 10: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/10.jpg)
2. States vote outcomes swing (mostly) in tandem
Source: New York Times
![Page 11: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/11.jpg)
3. Polls are accurate on Election Day; maybe not before
May Jul Sep Nov
40
45
50
55
60
Florida: Obama, 2008O
bam
a vo
te s
hare
Actualoutcome
Source: HuffPost-Pollster
![Page 12: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/12.jpg)
4. Voter preferences evolve in similar ways across states
May Jul Sep Nov
40
45
50
55
60
Florida: Obama, 2008
Oba
ma
vote
sha
re
May Jul Sep Nov
40
45
50
55
60
Virginia: Obama, 2008
Oba
ma
vote
sha
re
May Jul Sep Nov
40
45
50
55
60
Ohio: Obama, 2008
Oba
ma
vote
sha
re
May Jul Sep Nov
40
45
50
55
60
Colorado: Obama, 2008
Oba
ma
vote
sha
re
Source: HuffPost-Pollster
![Page 13: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/13.jpg)
5. Voters have short term reactions to big campaign events
Source: Tom Holbrook, UW-Milwaukee
![Page 14: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/14.jpg)
All together: A forecasting model that learns from the polls
Publicly available state polls during the campaign
Months prior to Election Day
Cum
ulat
ive
num
ber
of p
olls
fiel
ded
12 11 10 9 8 7 6 5 4 3 2 1 0
0
500
1000
1500
20002008
2012
Forecasts weight fundamentals ←→ Forecasts weight polls
Source: HuffPost-Pollster
![Page 15: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/15.jpg)
First, create a baseline forecast of each state outcome
Abramowitz Time-for-Change regression makes a national forecast:
Incumbent vote share = 51.5 + 0.6 Q2 GDP growth+ 0.1 June net approval− 4.3 In office two+ terms
Predicted Obama 2012 vote = 51.5 + 0.6 (1.3)+ 0.1 (-0.8)− 4.3 (0)
Predicted Obama 2012 vote = 52.2%
Use uniform swing assumption to translate to the state level:
Subtract 1.5% for Obama from his 2008 state vote shares
Make this a Bayesian prior over the final state outcomes
![Page 16: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/16.jpg)
First, create a baseline forecast of each state outcome
Abramowitz Time-for-Change regression makes a national forecast:
Incumbent vote share = 51.5 + 0.6 Q2 GDP growth+ 0.1 June net approval− 4.3 In office two+ terms
Predicted Obama 2012 vote = 51.5 + 0.6 (1.3)+ 0.1 (-0.8)− 4.3 (0)
Predicted Obama 2012 vote = 52.2%
Use uniform swing assumption to translate to the state level:
Subtract 1.5% for Obama from his 2008 state vote shares
Make this a Bayesian prior over the final state outcomes
![Page 17: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/17.jpg)
First, create a baseline forecast of each state outcome
Abramowitz Time-for-Change regression makes a national forecast:
Incumbent vote share = 51.5 + 0.6 Q2 GDP growth+ 0.1 June net approval− 4.3 In office two+ terms
Predicted Obama 2012 vote = 51.5 + 0.6 (1.3)+ 0.1 (-0.8)− 4.3 (0)
Predicted Obama 2012 vote = 52.2%
Use uniform swing assumption to translate to the state level:
Subtract 1.5% for Obama from his 2008 state vote shares
Make this a Bayesian prior over the final state outcomes
![Page 18: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/18.jpg)
First, create a baseline forecast of each state outcome
Abramowitz Time-for-Change regression makes a national forecast:
Incumbent vote share = 51.5 + 0.6 Q2 GDP growth+ 0.1 June net approval− 4.3 In office two+ terms
Predicted Obama 2012 vote = 51.5 + 0.6 (1.3)+ 0.1 (-0.8)− 4.3 (0)
Predicted Obama 2012 vote = 52.2%
Use uniform swing assumption to translate to the state level:
Subtract 1.5% for Obama from his 2008 state vote shares
Make this a Bayesian prior over the final state outcomes
![Page 19: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/19.jpg)
Combine polls across days and states to estimate trends
States with many polls States with fewer polls
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Florida: Obama, 2012
Oba
ma
vote
sha
re ●●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Oregon: Obama, 2012
Oba
ma
vote
sha
re
![Page 20: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/20.jpg)
Combine with baseline forecasts to guide future projections
Random walk (no)
Mean reversion
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Florida: Obama, 2012
Oba
ma
vote
sha
re
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Florida: Obama, 2012
Oba
ma
vote
sha
re
Forecasts compromise between history and the polls
![Page 21: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/21.jpg)
Combine with baseline forecasts to guide future projections
Random walk (no) Mean reversion
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Florida: Obama, 2012
Oba
ma
vote
sha
re
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
May Jun Jul Aug Sep Oct Nov
44
46
48
50
52
54
56
Florida: Obama, 2012
Oba
ma
vote
sha
re
Forecasts compromise between history and the polls
![Page 22: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/22.jpg)
A dynamic Bayesian forecasting model
Model specification
yk ∼ Binomial(πi [k]j[k], nk) Number of people preferring Democratin survey k , in state i , on day j
πij = logit−1(βij + δj) Proportion reporting support for theDemocrat in state i on day j
National effects: δjState components: βijElection forecasts: π̂iJ
PriorsβiJ ∼ N(logit(hi ), τi ) Informative prior on Election Day, using
historical predictions hi , precisions τiδJ ≡ 0 Polls assumed accurate, on average
βij ∼ N(βi(j+1), σ2β) Reverse random walk, states
δj ∼ N(δ(j+1), σ2δ ) Reverse random walk, national
Estimated for all states simultaneously
![Page 23: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/23.jpg)
A dynamic Bayesian forecasting model
Model specification
yk ∼ Binomial(πi [k]j[k], nk) Number of people preferring Democratin survey k , in state i , on day j
πij = logit−1(βij + δj) Proportion reporting support for theDemocrat in state i on day j
National effects: δjState components: βijElection forecasts: π̂iJ
PriorsβiJ ∼ N(logit(hi ), τi ) Informative prior on Election Day, using
historical predictions hi , precisions τiδJ ≡ 0 Polls assumed accurate, on average
βij ∼ N(βi(j+1), σ2β) Reverse random walk, states
δj ∼ N(δ(j+1), σ2δ ) Reverse random walk, national
Estimated for all states simultaneously
![Page 24: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/24.jpg)
Results: Anchoring to the fundamentals stabilizes forecasts
●
● ●●●
●
●● ● ● ●
● ●● ● ● ●● ●● ● ● ●●●● ●●●
●●
●●●●●●
●● ●●
●●●●●●●●●●●●
Jul Aug Sep Oct Nov
40
45
50
55
60
Florida: Obama forecasts, 2012O
bam
a vo
te s
hare
Shaded area indicates 95% uncertainty
![Page 25: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/25.jpg)
Results: Anchoring to the fundamentals stabilizes forecasts
Electoral Votes
Jul Aug Sep Oct Nov
150
200
250
300
350
400 OBAMA 332
ROMNEY 206
![Page 26: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/26.jpg)
There were almost no surprises in 2012
On Election Day, average error = 1.7%
Why didn’t the model do more?
![Page 27: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/27.jpg)
There were almost no surprises in 2012
On Election Day, average error = 1.7%
Why didn’t the model improve forecasts by more?
![Page 28: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/28.jpg)
There were almost no surprises in 2012
On Election Day, average error = 1.7%
Why didn’t the model improve forecasts by more?
![Page 29: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/29.jpg)
The fundamentals and uniform swing were right on target
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
30 40 50 60 70
30
40
50
60
70
State election outcomes
2008 Obama vote
2012
Oba
ma
vote
AL
AK
AZ
AR
CA
CO
CTDE
FL
GA
HI
ID
IL
IN
IA
KSKY
LA
ME
MDMA
MIMN
MSMO
MT
NE
NVNH
NJ
NM
NY
NC
ND
OH
OK
OR
PA
RI
SC
SDTN
TX
UT
VT
VA
WA
WV
WI
WY
2012=2008 line
![Page 30: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/30.jpg)
Aggregate preferences were very stable
Percent supporting: Obama Romney
![Page 31: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/31.jpg)
Could the model have done better? Yes
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
0 20 40 60 80 100
−6
−4
−2
0
2
4
6
Difference between actual and predicted vote outcomes
Number of polls after May 1, 2012
Ele
ctio
n D
ay fo
reca
st e
rror
AL
AK
AZ
AR
CACO
CT
DE
FLGA
HI
ID
IL
IN
IA
KS
KY
LA MEMD MA MI
MN
MS
MO
MT
NE
NV NH
NJ
NMNY
NC
ND
OHOK
OR
PA
RISC
SDTN
TX
UT
VT
VAWA
WV
WI
WY
↑ Obama performedbetter than expected
↓ Romney performedbetter than expected
![Page 32: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/32.jpg)
Forecasting is only one of many applications for the model
1 Who’s going to win?
2 Which states are going to be competitive?
3 What are current voter preferences in each state?
4 How much does opinion fluctuate during a campaign?
5 What effect does campaign news/activity have on opinion?
6 Are changes in preferences primarily national or local?
7 How useful are historical factors vs. polls for forecasting?
8 How early can accurate forecasts be made?
9 Were some survey firms biased in one direction or the other?
![Page 33: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/33.jpg)
House effects (biases) were evident during the campaign
![Page 34: Linzer slides-barug](https://reader034.fdocuments.in/reader034/viewer/2022051400/559a98571a28ab7a3d8b45a0/html5/thumbnails/34.jpg)
Much more at votamatic.org
@DrewLinzer