SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND...
Transcript of SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND...
![Page 1: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/1.jpg)
1
SKEWING THE DATA:POLITICAL POLLING AND CONTROVERSY
Andrew S. Tanenbaum
The Votemaster at (www.electoral-vote.com)
![Page 2: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/2.jpg)
THE ELECTORAL COLLEGE
• The president is not elected by popular vote• The president is elected by the electoral college• Each state has a certain number of electors• California has 55, Wyoming has 3• In total there are 538 electoral votes• In 2004, I started a site that kept track of the state polls
2
![Page 3: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/3.jpg)
SITE ON 6 NOV. 2012
3
![Page 4: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/4.jpg)
HOW DO POLLSTERS DECIDE WHO TO CALL?
• Random digit dialing– Phone numbers are of the format (212) 695-xxxx– Computer picks random xxxx– Problems: you call businesses, modems, fax machines, …
• Pollster buys lists of voters (e.g., registered voters)
4
![Page 5: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/5.jpg)
WHAT DO THEY ASK?
• Candidate favorability• Horse race questions• Demographic questions
– Age– Party identification– Gender– Race– Education– Income
5
![Page 6: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/6.jpg)
BACKGROUND ON POLLING
• A poll using live interviewers costs about $15,000• Strong bias on who will talk to interviewer• Response rates are below 10%• So what do polling firms do?• Make small samples and do statistical correction
6
![Page 7: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/7.jpg)
THE POLLS ARE IN. NOW WHAT?
7
Feb Mar Apr May Jun Jul Aug Sep OctJan
60%
55%
50%
45%
40%
Poll
![Page 8: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/8.jpg)
THE POLLS ARE IN. NOW WHAT?
8
If candidate is above 50%, he wins
If candidate is below 50%, he loses
Feb Mar Apr May Jun Jul Aug Sep OctJan
60%
55%
50%
45%
40%
![Page 9: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/9.jpg)
THE POLLS ARE IN. NOW WHAT?
9
Feb Mar Apr May Jun Jul Aug Sep OctJan
60%
55%
50%
45%
40%
If we count only the mostrecent poll, the candidate loses
![Page 10: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/10.jpg)
THE POLLS ARE IN. NOW WHAT?
10
Feb Mar Apr May Jun Jul Aug Sep OctJan
60%
55%
50%
45%
40%
If we average the last 2,The candidate wins
* Average
![Page 11: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/11.jpg)
THE POLLS ARE IN. NOW WHAT?
11
Feb Mar Apr May Jun Jul Aug Sep OctJan
60%
55%
50%
45%
40%
Should this poll count?It was in June
*Average
![Page 12: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/12.jpg)
HOW TO AVERAGE POLLS
• How you handle this is controversial• If the window is too long, you miss rapid changes• If the window is too short, not so accurate• Should partisan pollsters be included?• Should pollsters be weighted by previous accuracy?
12
![Page 13: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/13.jpg)
POLLING CONTROVERSIES
• Is the sample really random?– RDD: Homes with 0 landlines? 3 landlines?– Lists: Which ones? Are they current?– Who to poll? All adults? Registered voters? Likely voters?– Nonresponse bias (e.g. young mothers vs. old widows)– It is illegal for a computer to call a cell phone
• Methodology– Live interviewer vs. robo vs. Internet polls– Question order and wording
13
![Page 14: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/14.jpg)
POLLING CONTROVERSIES
• Polling Non-English-speaking voters– Most firms just ignore them
• Dealing with lying respondents– Bernie-or-bust voters– Bradley effect– Trump does better in robo polls than live interviewer polls
• Some pollsters just made up the data! No real calls
14
![Page 15: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/15.jpg)
DATA CORRECTION
• Pollsters know the demographics of the population• If the sample demographics are different, they correct• Suppose it is known that 52% of voters are women• But in some poll, only 47% were women• Then they weight each woman in the sample as 52/47• They also correct for political party identification• Suppose they know 38% of voters are Democrats• But in this sample only 32% were• Each Democrat then is weighted as 38/32
15
![Page 16: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/16.jpg)
UNSKEWED POLLS
• In 2012, a man named Dean Chambers thought Mitt Romney would beat Barack Obama
• The polls showed otherwise• He assumed the polls were wrong• But how?• He blamed it on the data correction• He re-corrected the raw data based on a much
more Republican electorate• His Website predicted a Romney win
16
![Page 17: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/17.jpg)
UNSKEWEDPOLLS.COM
17
Romney got 206 electoral votes and lost
![Page 18: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/18.jpg)
RIGHT-WING MEDIA BELIEVED IT TOTALLY
• Right-wing media & Websites took this as the truth• Millions of people believed Chambers• Fox News saw him as a rising star• Mitt Romney expected to win based on this• On election night, Karl Rove refused to accept results
18
![Page 19: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/19.jpg)
WERE THE 2016 POLLS “SKEWED”?
• RealClear Politics had Clinton +3.3% ± 2%• Final result nationally was Clinton +2.1%• The national polls were spot on• The state polls showed it too close to call in 12 states
19
![Page 20: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/20.jpg)
CONCLUSION
• Somebody can take the data and modify it• People & media might prefer the “modified” data• Millions of people believed the “modified” data
20
![Page 21: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/21.jpg)
21
THANK YOU !
![Page 22: SKEWING THE DATA - Amsterdam Data Science€¦ · SKEWING THE DATA: POLITICAL POLLING AND CONTROVERSY Andrew S. Tanenbaum The Votemaster at () ... – Bernie-or-bust voters – Bradley](https://reader035.fdocuments.in/reader035/viewer/2022081323/5f05610a7e708231d412ab31/html5/thumbnails/22.jpg)
ELECTORAL-VOTE.COM
22