Taking Fact-checks Literally But Not Seriously? The Eects ...nyhan/trump-corrections.pdfcandidates....

Taking Fact-checks Literally But Not Seriously?The E�ects of Journalistic Fact-checking onFactual Beliefs and Candidate Favorability⇤

Forthcoming at Political Behavior

Brendan Nyhan†

University of MichiganEthan Porter‡

George Washington University

Jason Reifler§

University of ExeterThomas J. Wood¶

The Ohio State University

January 11, 2019

AbstractAre citizens willing to accept journalistic fact-checks of misleading claims from can-didates they support and to update their attitudes about those candidates? Previousstudies have reached conflicting conclusions about the e�ects of exposure to counter-attitudinal information. As fact-checking has become more prominent, it is thereforeworth examining how respondents respond to fact-checks of politicians — a questionwith important implications for understanding the e�ects of this journalistic formaton elections. We present results to two experiments conducted during the 2016 cam-paign that test the e�ects of exposure to realistic journalistic fact-checks of claimsmade by Donald Trump during his convention speech and a general election debate.These messages improved the accuracy of respondents’ factual beliefs, even amonghis supporters, but had no measurable e�ect on attitudes toward Trump. These re-sults suggest that journalistic fact-checks can reduce misperceptions but often haveminimal e�ects on candidate evaluations or vote choice.

⇤We thank Kim Gross, John Pfa�, and D.J. Flynn for comments and Kyle Dropp for fielding study 1. Thisresearch received funding support from the European Research Council (ERC) under the European Union’sHorizon 2020 research and innovation program (grant agreement No 682758). We also received supportfrom the School of Media and Public A�airs at George Washington University. All errors are our own.

†Professor, Ford School of Public Policy; [email protected]‡Assistant Professor, School of Media and Public A�airs; [email protected]§Professor, Department of Politics; [email protected]¶Assistant Professor, Department of Political Science; [email protected]

Journalistic fact-checking is an important new form of political news coverage (e.g., Spivak

2011; Graves 2016). However, little is known about its e�ects on citizens. Do they accept

fact-checks that conflict with their political a�liations or shrug o� those that contradict

the claims of their preferred candidates? These questions have important implications

for debates over citizen competence and the quality of governance in democracies (e.g.,

Hochschild and Einstein 2015; Jamieson 2015).

Concerns about people’s willingness to accept unwelcome factual information like

counter-attitudinal fact-checks have become so widespread that Oxford Dictionaries named

post-truth the word of the year after the 2016 U.S. elections (BBC 2016). These concerns

are well-justified. Some research indicates, for instance, that people can be highly resis-

tant to journalistic fact-checks. Nyhan and Reifler (2010) find that corrective information

in mock news articles frequently fails to reduce salient misperceptions and can even in-

crease the prevalence of misperceptions among ideologically vulnerable groups compared

to those who read an article with no correction — a “backfire e�ect.” Other studies us-

ing relatively balanced formats have also found sti� resistance to uncongenial journalistic

fact-checks (e.g., Nyhan, Reifler, and Ubel 2013; Garrett, Nisbet, and Lynch 2013; Jarman

2016). Citizens may be especially resistant to unwelcome fact-checks during campaigns,

which frequently stimulate motivated reasoning (e.g., Lenz 2012).

By contrast, other studies find that fact-checking and other types of factual information

can partly overcome directionally motivated reasoning and reduce, or “debunk,” misper-

ceptions (e.g., Weeks 2015; Nyhan and Reifler N.d.; Wood and Porter 2018; Chan et al.

2017). Notably, Wood and Porter (2018) examine 52 issue areas and observe no evidence

of backfire e�ects. They do, however, find widespread evidence of motivated reasoning —

for approximately 80% of issues tested, responsiveness to corrective information varied

by ideology (unlike Nyhan and Reifler N.d.).

We thus confront conflicting expectations about how people might respond to journal-

1

istic fact-checks during a general election campaign. Citizens may resist fact-checks that

conflict with their partisan or ideological commitments and maintain (or even strengthen)

their misperceptions. Alternately, people might accept journalistic fact-checks and update

their beliefs to be at least somewhat more accurate.

We present results of two studies conducted during the 2016 U.S. presidential cam-

paign that help illuminate this debate. Both studies examine actual misstatements that

were made by candidates and their proxies and fact-checked by the media during the cam-

paign — a time when partisan commitments are activated and the influence of partisan

leaders is likely to be especially strong (e.g., Zaller 1992; Lenz 2012). Specifically, Study

1 is a preregistered survey experiment that evaluates the e�ects of a journalistic fact-check

of misleading claims about crime made by Donald Trump at the GOP convention. To

increase the realism of the study’s evaluation of the e�ects of journalistic fact-checking

in a campaign, Study 1 also includes experimental conditions in which a political elite

attempts to denigrate and undermine the fact-check in question. Study 2 tests the e�ect

of fact-checking a claim Trump made about unemployment during the first presidential

debate among subjects experimentally induced to have watched the debate.

The design of these studies was intended to address two potential explanations for

conflicting findings in the literature. In many cases, respondents in survey experiments

may lack motivation to engage in e�ortful resistance to unwelcome factual information

about relatively obscure topics or lack su�cient context to form counter-arguments. The

lack of such motivations or context could explain the null or mixed results described above.

To address these concerns, we administered both studies at the height of a presidential

general election campaign and designed them to maximize partisan directional motiva-

tions. In Study 1, we tested corrections of a presidential candidate’s convention accep-

tance speech shortly after it was delivered; in study 2, we tested corrections of a candidate

on the night of a presidential debate among a sample that was encouraged to watch the

2

debate. In addition, while many facts are not the subject of political controversy, we tested

fact-checks of claims that one candidate (Trump) used to criticize the other (Clinton), a

context in which directionally motivated reasoning may be common.

Second, given that counter-arguing of unwelcome political information may be greater

when relevant considerations are available, the first experiment experimentally manipu-

lates the availability of messages denigrating the fact-check. In this study, we randomly

paired a fact-check of a Trump statement with a statement by Trump campaign chairman

Paul Manafort denying the correction and provided an additional Manafort statement as-

cribing a political motivation to the FBI, the source of the data in the fact-check. These

messages should reduce the cognitive demands of counter-argument for low-information

respondents and provide additional considerations upon which high-information Trump

supporters can draw, increasing the realism of the information environment in which the

fact-check is delivered and the likelihood of observing motivated resistance.

The design of these studies also allows us to investigate other important theoretical

questions about the e�ects of fact-checking. First, we consider whether people are willing

to not only revise their factual beliefs in response to a fact-check but to change their atti-

tudes toward the candidate who has made a claim that has been fact-checked — an e�ect

that would likely increase the reputational threat that fact-checking poses to politicians

3

(see, e.g., Nyhan and Reifler 2015).1 In addition, our first study tests whether people are

willing to accept attitude-inconsistent information but instead change their interpretations

of that information in a directionally motivated manner. For instance, Gaines et al. (2007)

find that Democrats and Republicans updated their beliefs about the Iraq war relatively ac-

curately over time; it was interpretations of those facts that diverged along partisan lines.

Khanna and Sood (2018) similarly find that incentivized respondents provide more correct

answers but perceived greater unfairness in the study when doing so.

Our results indicate that exposure to fact-checks reduced misperceptions among sup-

porters of both major party presidential candidates but did not a�ect attitudes toward those

candidates. In Study 1, exposure to fact-checking reduced misperceptions about crime

rates even when respondents were provided a message by a Trump sta�er disparaging

the fact-check. Similarly, providing a fact-check of Trump just after a debate in Study 2

reduced misperceptions about unemployment in Michigan and Ohio, even among Trump

supporters. In short, journalistic fact-checks can overcome directionally motivated reason-

ing and bring people’s beliefs more in line with the facts even when the counter-attitudinal

information is disparaged by a co-partisan. However, neither Clinton nor Trump support-

ers changed their attitudes towards either candidate after receiving fact-checks, suggesting

that voters’ preferences during a presidential election are not contingent on their percep-

1Wintersieck (2017) is a notable exception. There are crucial di�erences between our

study and hers, however. First, whereas Wintersieck looks at candidates deemed “honest”

by fact-checkers, our studies examine statements flagged by fact-checkers for being false.

Second, while Wintersieck focuses on a statewide election and recruits student subjects at

a university, we enroll broader pools of participants in two experiments about a national

election. This sampling distinction is particularly relevant here given that students might

be more disposed to engage in the e�ortful cognition required to counterargue unwelcome

information such as fact-checks (Krupnikov and Levine 2014).

4

tions of the factual accuracy of the candidates. In other words, factual corrections can

achieve the limited objective of creating a more informed citizenry but struggle to change

citizens’ minds about whom to support.

Theory and hypotheses

Our theoretical approach builds on research suggests that people have competing goals

in information processing (e.g., Kunda 1990; Molden and Higgins 2005). When people

have stronger accuracy goals (e.g., they are provided a reward for each correct answer

as in Hill 2017), respondents will process information more dispassionately and seek to

maximize the accuracy of their beliefs. When directional motivations are more salient

(e.g., party identity or a preferred candidate), respondents will instead tend to process

information in a manner that is consistent with that preference rather than maximizing

accuracy or considering information in a dispassionate manner (Taber and Lodge 2006;

Bolsen, Druckman, and Cook 2014).

These findings suggest that directionally motivated reasoning may be an especially

salient factor when people process factual information about controversial political issues

in a highly partisan context such as a presidential election campaign. As discussed above,

however, researchers have reached mixed conclusions about the extent to which direc-

tionally motivated reasoning a�ects belief updating in response to fact-checking in such

contexts (e.g., Nyhan and Reifler 2010; Nyhan, Reifler, and Ubel 2013; Garrett, Nisbet,

and Lynch 2013 versus Weeks 2015; Nyhan and Reifler N.d.; Wood and Porter 2018; Chan

et al. 2017; Porter and Kirby 2018). Other research suggests that directional motivations

may have greater influence on interpretations of contested facts (Gaines et al. 2007) than

on factual beliefs themselves; we also consider this possibility.

To adjudicate among these claims, we test three preregistered hypotheses based on

5

the empirical literature and theoretical considerations discussed above.2 In each case,

we determine whether information is pro- or counter-attitudinal by whether respondents

indicate supporting Trump or Hillary Clinton on a pre-treatment vote choice measure.

First, we propose to test whether fact-checking is ine�ective at increasing the accuracy

of factual beliefs among people for whom it is counter-attitudinal (H1; see Nyhan and

Reifler 2010) or whether fact-checks increase belief accuracy but acceptance is greater

among those for whom the information is pro-attitudinal (H2; see Wood and Porter 2018)3

H1 (motivated resistance): Respondents will resist unwelcome facts about controversial

issues. As a result, people exposed to journalistic fact-checks that are counter-

attitudinal will not come to hold more accurate views. In some cases, their views

could even become more inaccurate. (Evaluated in studies 1 and 2.)

H2 (di�erential acceptance): Respondents will accept journalistic fact-checks that are

counter-attitudinal and update their beliefs to become more accurate, though the

extent to which they update their beliefs may vary based on their prior attitude.

(Evaluated in studies 1 and 2.)

2Our preregistration for Study 1 documents our hypotheses and analysis plan (URL

omitted for peer review). Unless otherwise noted, all Study 1 analyses are consistent with

this document. Study 2 was conducted too rapidly to be preregistered (it was fielded im-

mediately after the debate) but our analysis follows Study 1 to the greatest extent possible.3As discussed above, previous findings are mixed on both hypotheses. For H1, see

Nyhan and Reifler (2010) (backfire on two of five studies) versus Wood and Porter (2018)

(no cases of backfire). For H2, compare Wood and Porter (2018), which finds a consistent

pattern of ideological di�erentials in belief updating, with Nyhan and Reifler (N.d.), which

finds no evidence of di�erential acceptance when fact-checks are pro-attitudinal.

6

We also consider the e�ects of fact-checks on “interpretations” of factual claims. We de-

fine the term consistent with Gaines et al. (2007) who use it to include evaluations (“the

crime rate is high”), explanations (“crime has increased because of a decline in moral val-

ues”), and inferences (“Obama’s acceptance of this crime rate reveals indi�erence to our

plight”). In this context, respondents may be willing to accept a fact-check which contra-

dicts a favored politician but form an attitude-consistent interpretation of the information

that they have accepted. For instance, if conservative respondents are asked to accept a

contradiction of Trump’s claim about increase in violent crime, they might ascribe the

decline in crime to a factor that is consistent with their beliefs (e.g., tougher policing and

longer sentences). We therefore propose to test whether interpretations of factual claims

are formed and updated in a directionally motivated manner (H3; see Gaines et al. 2007):

H3 (attitude-consistent interpretation): Respondents will accept journalistic fact-checks

that are counter-attitudinal, but interpret them in an attitude-consistent manner.

(Evaluated in study 1 only.)

Finally, as noted above, we also evaluate a key research question — will people not only

revise their factual beliefs, but alter their attitudes toward a candidate who has made a

false or unsupported claim? We therefore also measure attitudes toward the candidate,

including vote preference.4 The voluminous literature on candidate evaluation and vote

choice in a presidential election provides unclear expectations. For instance, Funk (1999)

finds that perceived traits such as honesty condition overall evaluation of candidates in the

expected direction — candidates who are perceived as having integrity are more warmly

regarded. However, Rahn et al. (1990) and Pierce (1993) find no relationship between

perceived traits and vote choice even among sophisticated participants who are thought to

4Findings for two other preregistered research questions are described below and in the

online appendix.

7

be capable of connecting potentially distal considerations. Accordingly, we have unclear

expectations for the e�ect of fact-check exposure on subsequent candidate evaluations.

Stimuli

To maximize the external validity of our experiments, we adopted actual candidate state-

ments from the 2016 presidential election. In our experiments, we focus on misleading

claims made by Donald Trump in two high-profile candidate appearances — a suggestion

of dramatically rising crime in his nomination acceptance speech (study 1) and a claim

about the loss of manufacturing jobs during one of the presidential debates (study 2).

Both claims were fact-checked by journalistic outlets at the time. The treatments we use

in the studies aim to match the fact checks published for corresponding misstatements.5

For instance, our fact-check of Trump’s convention statements about the prevalence of

violent crime in Study 1 is very similar to the approach used by journalistic fact checkers.

Here is the Associated Press fact-check of the crime claims Trump made in his convention

speech (Sullivan and Day 2016):

THE FACTS: Violent crime has dropped dramatically since the early 1990s.

According to FBI data , the national violent crime rate last peaked in 1991 at

758 reported violent crimes per 100,000 people. In 2014, the latest year for

which full data is available, the rate was 366 per 100,000 people.

Our fact-check for the same issue in Study 1 is very similar:

5As we describe below, we also seek to maximize the realism of the treatments we use

to test the e�ects of elite messages denigrating a fact-check. Study 1 tests the e�ects of

exposure to actual statements made by Paul Manafort, Trump’s campaign chairman at the

time, challenging the fact-checking of Trump’s convention speech.

8

According to FBI’s Bureau of Justice Statistics, the violent crime rate has

fallen dramatically and consistently over time. According to their estimates,

the homicide rate in the U.S. in 2015 was half that recorded in 1991.

Study 2 considers the e�ect of fact-checking Trump’s claims about job loss in Michi-

gan and Ohio during the first presidential debate. Politico’s “wrongometer” wrote the

following in response to that claim:

In fact, the state’s unemployment rate has declined in recent years, according

to the Bureau of Labor Statistics. That figure now stands around 4.5 percent,

down from a 14.9 percent unemployment rate in June 2009 (Politico 2016).

Similarly, the New York Times wrote the following about Trump’s claim:

Ohio and Michigan have, indeed, su�ered major manufacturing job losses

over the past generation. But in the past year, Ohio has gained 78,300 jobs,

and Michigan has gained 75,800 jobs. In August, the unemployment rate was

4.9 percent in Michigan and 4.7 percent in Ohio, both in line with the national

rate (New York Times 2016).

Our fact-check for this same issue (described further below) reads as follows:

In fact, according to the Bureau of Labor Statistics, unemployment has fallen

in both Michigan and Ohio. Both states each saw 70,000 new jobs over the

last year.

We believe our treatments are faithful representations of the type of journalistic fact-

checking that is now widely disseminated by the media. In other words, our experiment

9

is not a test of authoritative and logically infallible attempts to debunk false claims. It is

instead an analysis of how journalistic fact-checks a�ect mass beliefs and attitudes.6

Study 1: Crime perceptions

Study 1 tested the e�ects of journalistic fact-checks about changes in levels of crime over

time. Respondents were randomly assigned to one of five conditions. In a control condi-

tion, respondents read an article without any political content. Participants in the treatment

conditions read a mock news article featuring misleading claims by Donald Trump about

crime rates based on an actual article that originally appeared on CNN.com. In one treat-

ment condition, participants read an article that omits any corrective information, allowing

us to test the e�ect of exposure to a candidate’s statement alone. Others were assigned to

read versions of the article that include neutral corrective information in the style of jour-

nalistic fact-checking, allowing us to test its e�ects versus a no-correction version of the

6It is important to note that journalistic fact-checks do not always logically contradict a

speaker (e.g., Marietta and Bowser 2015; Uscinski and Butler 2013). Fact-checkers often

seek instead to address possible inferences that listeners might draw from a candidate’s

statement. For instance, Trump’s nomination speech described an “epidemic” of violent

crime. He did not directly state that crime has increased, but a listener might infer as much

(indeed, Trump made clear statements about increasing crime rates at other times). Like

other journalistic fact-checks, our treatment thus cites FBI data on the long-term decline

in violent crime. Similarly, Trump’s debate statement emphasized factory jobs leaving

Ohio and Michigan. While he did not directly say that employment in Michigan and Ohio

is su�ering because of trade policy, he implied that widespread job loss was taking place.

Consequently, our fact-check, like several in the media, provided data on changes in jobs

and unemployment in those states.

10

article as well as the control condition. One condition tested the fact-check alone, while

two other conditions tested whether elite political actors can cultivate resistance to factual

information. In these conditions, the fact-check was followed by a statement from a Trump

surrogate disparaging the validity of the information or a statement by the surrogate dis-

paraging the validity of the information and attributing a political motive to the source of

the information.

Experimental design and instrument

After a series of demographic and attitudinal questions, participants were assigned to one

of five conditions.7 The treatments were based on actual news events during the 2016

Republican National Convention. In his speech, Trump described an America ridden by

increasing crime (uncorrected claim). As the media pointed out, however, these depictions

were contradicted by FBI crime data showing a long-term secular decline (journalistic

fact-check). When Paul Manafort, Trump’s then-campaign chairman, was pressed about

this discrepancy (Schleifer 2016), he questioned the validity of FBI statistics (rejection

of the fact-check) and suggested the FBI could not be trusted because it did not recom-

mend indicting Hillary Clinton in her email scandal (conspiracy theory/fact-check source

derogation).

The specific treatments shown to participants are as follows:

• Control: A birdwatching article.

• Rising crime message: A news article summarizing Trump’s claims

• Rising crime message + fact-check: A news article summarizing Trump’s claims

with a fact-check citing FBI statistics.

7The full instrument is in Online Appendix A.

11

• Rising crime message + fact-check + denial: A news article summarizing Trump’s

claims with a fact-check citing FBI statistics and a statement from Manafort rejecting

the statistics.

• Rising crime message + fact-check + denial + source derogation: A news article

summarizing Trump’s claims about crime with a fact-check citing FBI statistics, a

statement from Manafort rejecting the statistics, and Manafort’s contention that the

FBI was not to be trusted.

Outcome variables

After treatment, subjects were asked two article recall questions to measure receipt of

treatment. We then measured several outcome variables of interest. To assess motivated

resistance, we asked people’s beliefs about changes in the crime rate. We also asked

about perceptions of the accuracy of federal crime statistics and the treatment article and

whether/how the treatment article was biased. In addition, respondents were asked to

choose among interpretations of crime trends, which were coded for belief consistency.8

Finally, we measured evaluations of Trump and other politicians.

Results

Responses were collected on September 30, 2016 by Morning Consult (n = 1,203) and on

Mechanical Turk (n = 2,983).9 To assess our hypotheses, we performed a series of OLS

8Per our preregistration, respondents who indicated crime was up due to inequality or

unemployment were coded as -1 (liberal), those who said crime was up due to moral de-

cline or down due to tougher policing were coded as 1 (conservative), and other responses

were coded as 0.9Demographic and balance data for both samples are provided in Online Appendix C.

12

regressions with robust standard errors.10 All treatment e�ect estimates are unweighted

intent to treat e�ects.11 In the pre-treatment vote choice questions, a small percentage of

respondents did not support Trump or Clinton; per our preregistration, we exclude them

from subsequent analyses.12

To evaluate motivated resistance, we test whether the marginal e�ect of exposure to

the fact-check conditions on misperceptions about crime is null or positive for Trump sup-

porters and whether the di�erence in e�ects is significant compared to Clinton supporters.

To evaluate di�erential acceptance, we test whether the marginal e�ect of exposure to the

conditions including a fact-check is negative for Trump supporters and whether the di�er-

ence in treatment e�ects is significant versus Clinton supporters. We also measure source

derogation and counterargument to understand how respondents respond to fact-checks.

Table 1 presents the e�ects of the manipulation on beliefs about changes in crime where

the control condition is the excluded category. These estimates are calculated among

Trump and Clinton supporters (the latter are thus the excluded category for the Trump

support indicator). The table also reports auxiliary quantities representing di�erences

versus the uncorrected statement condition.

Our results indicate that journalistic fact-checking had a pronounced e�ect on fac-

10All analyses in this section are consistent with our preregistration unless otherwise

indicated. OLS models are replicated using ordered probit where applicable in Online

Appendix C.11Mean scores on two attention checks were 1.62 and 1.92 for controls and 1.59 and

1.87 for the treatment groups on Morning Consult and Mechanical Turk, respectively. (See

Online Appendix A for wording.) We therefore deviate slightly from our preregistration

to omit consideration of response time as a measure of attention.12We report equivalent but more complex models estimated on the full sample in Online

Appendix C.

13

Table 1: Message exposure e�ects on beliefs about changes in crime

Morning Consult Mechanical TurkTrump support 0.52*** 0.81***

(0.16) (0.11)Uncorrected statement -0.34* -0.05

(0.18) (0.09)Uncorrected ⇥ Trump support 0.35 0.16

(0.23) (0.16)Fact-check -0.94*** -0.97***

(0.19) (0.10)Fact-check ⇥ Trump support 0.09 0.11

(0.26) (0.18)Fact-check denial -0.74*** -0.94***

(0.19) (0.09)Fact-check denial ⇥ Trump support 0.21 0.38**

(0.27) (0.17)Denial/source derogation -1.12*** -0.77***

(0.19) (0.10)Denial/source derogation ⇥ Trump support 0.61** 0.29*

(0.27) (0.17)Constant 3.64*** 3.01***

(0.12) (0.06)Fact-check � uncorrected statementClinton supporters -0.61*** -0.92***

(0.19) (0.10)Trump supporters -0.86*** -0.97***

(0.18) (0.15)Denial � uncorrected statementClinton supporters -0.41** -0.89***


(0.19) (0.14)Denial/derogation � uncorrected statementClinton supporters -0.78*** -0.72***


(0.20) (0.14)R2 0.14 0.19N 990 2430

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided). OLS models with robust standarderrors. All independent variables are dichotomous indicators. Dependent variable is a five-point scale of perceived change in crime incidence where higher values indicate perceivedgreater incidence. Participants are respondents from Morning Consult or Mechanical Turkwho supported Hillary Clinton or Donald Trump in the 2016 election (reference categoryfor Trump support is thus Clinton support).

14

tual beliefs. Though Trump’s supporters were more likely than Clinton’s to believe that

crime had increased or not declined significantly over the previous ten years, corrective

information reduced misperceptions among supporters of both candidates. Specifically,

respondents exposed to FBI statistics about decreased crime reported significantly lower

misperceptions compared to the uncorrected statement conditions in both samples regard-

less of the candidate they supported (p < .01). Exposure to Trump’s claim did not further

increase misperceptions among his supporters.

Our results are inconsistent with motivated resistance. We do observe some evidence

of di�erential acceptance, however. Both the fact-check denial on Mechanical Turk (p <

.05) and the denial/source derogation in both samples (p < .05 in Morning Consult, p <

.10 in Turk) reduce crime misperceptions less among Trump supporters than Clinton sup-

porters.13

These findings are illustrated in Figure 1, which presents mean crime perceptions by

condition and candidate support from the Morning Consult data. Mean beliefs about crime

change among Trump supporters declined from 4.17 (out of 5) in the control and uncor-

rected statement conditions to 3.31 in the fact-check condition, 3.62 in the fact-check and

denial condition, and 3.65 in the fact-check/denial/source derogation condition. Similar

declines are observed among Clinton supporters.

To understand these responses, we examine how the treatments a�ect judgments of

the accuracy and fairness of the articles. Table 2 demonstrates that fact-checks provoke

di�erent perceptions of accuracy and fairness among Clinton and Trump supporters. More

specifically, this table presents results from statistical models available in Table C6 in

Online Appendix C that estimate the e�ect of the manipulation on the perceived fairness

and accuracy of the stimulus article and the perceived accuracy of federal crime statistics

13These quantities are estimated with respect to the control condition. These di�erences

are not significant relative to the uncorrected condition.

15

Figure 1: Crime perceptions by treatment condition and candidate support

Mean beliefs about crime change by candidate preference and experimental condition.Survey data from Morning Consult.

for Clinton and Trump supporters relative to the uncorrected condition.

Relative to the uncorrected statement condition, Clinton supporters view the article as

more accurate and fair when a fact-check is present. In three of four comparisons, they

even view federal crime statistics as more accurate when Trump’s sta�er questions them.

By contrast, Trump supporters view the article as less accurate and fair when it includes a

fact-check — a contrast with their reported factual beliefs, which became more accurate.

Trump supporters are also less likely to view federal crime statistics as accurate when they

are invoked in a fact-check, especially when questioned by a Trump sta�er (p < .01 in

each denial condition versus the uncorrected condition).

To illustrate these findings, we plot the means of the perceived accuracy of the stimu-

lus article and federal crime statistics by condition and candidate support for the Morning

Consult data in Figure 2. When Trump supporters receive a fact-check, they are less likely

to see the article or federal crime statistics as accurate (mean of 2.7 for both) compared

to when they receive Trump’s uncorrected statement (means of 2.8 and 3.0, respectively).

16

Figure 2: Perceived accuracy of article and federal crime statistics by treatment conditionand candidate support

Mean perceived accuracy and fairness of the stimulus article and perceived accuracy offederal crime statistics by candidate preference and experimental condition. Survey datafrom Morning Consult.

17

Table 2: Message exposure e�ects on perceptions of accuracy and fairness

Article accurate Article fairness Statistics accurateMC MT MC MT MC MT

Fact-check � uncorrectedClinton supporters 0.40*** 0.65*** 0.26** 0.56*** 0.04 0.08*

(0.11) (0.05) (0.11) (0.06) (0.09) (0.04)Trump supporters -0.17* -0.21*** -0.24*** -0.30*** -0.33*** -0.11

(0.10) (0.08) (0.09) (0.07) (0.10) (0.07)

Denial � uncorrectedClinton supporters 0.43*** 0.69*** 0.28** 0.47*** 0.12 0.14***

(0.10) (0.06) (0.11) (0.06) (0.08) (0.04)Trump supporters -0.19* -0.06 -0.37*** -0.26*** -0.46*** -0.21***

(0.11) (0.08) (0.11) (0.07) (0.11) (0.07)

Denial/derogation � uncorrectedClinton supporters 0.47*** 0.68*** 0.24** 0.41*** 0.15* 0.11**

(0.10) (0.06) (0.11) (0.06) (0.08) (0.04)Trump supporters -0.16 -0.10 -0.23** -0.22*** -0.34*** -0.16**

(0.10) (0.07) (0.10) (0.07) (0.10) (0.07)* p < 0.10, ** p < 0.05, *** p < .01. Auxiliary quantities from OLS models with robuststandard errors reported in Table C6. All independent variables are dichotomous indica-tors. For each dependent variable, higher values indicate greater accuracy and fairness(each measured on a four-point scale). Participants are respondents from Morning Con-sult (MC) or Mechanical Turk (MT) who supported Hillary Clinton or Donald Trump inthe 2016 election (reference category for Trump support is thus Clinton support).

The opposite pattern is frequently observed among Clinton supporters, who view the ar-

ticle as more accurate when a fact-check is included (mean of 2.9 versus 2.5). These re-

sponses do not vary in the presence of fact-check denial or source derogation; the response

seems driven by the presence of a fact-check.14

The di�erences in perceptions of the articles that we observe can be interpreted as

consistent with H3. However, as Table C11 in Online Appendix shows, we do not find

evidence that people interpret the changes in crime they perceive in a viewpoint-consistent

manner (e.g., “[t]ougher policing and longer prison sentences” for Trump supporters who

think crime has decreased).

Finally, we estimate the marginal e�ect of fact-checking on evaluations of Trump

14Findings are similar for perceived article bias (see Online Appendix C).

18

among Clinton and Trump supporters in Table 3. Specifically, we asked subjects to eval-

uate both candidates on a 1–5 scale ranging from “Very unfavorable” to “very favorable.”

We find no significant e�ects of the fact-check on favorability toward Trump regardless of

respondents’ candidate preference.15

In sum, the evidence from Study 1 shows that, even if people are inclined to take a

skeptical view of a fact-checking article and the data underlying it, fact-checks can still

spur people to hold more factually accurate beliefs. However, these changes in belief

accuracy do not seem to lead to corresponding changes in attitudes toward the candidate

being fact-checked.

Study 2

One limitation of Study 1 is that we examined the e�ects of a fact-check several weeks after

the misstatement it targeted was made. It is possible that subjects had already been exposed

to fact-checking of the misstatements we studied and that this exposure limited the e�ects

of the fact-check on attitudes toward the candidate. The design of Study 2 addresses this

limitation because it was conducted immediately after the first 2016 presidential debate.

In the first wave of Study 2, 1,546 participants from Mechanical Turk were asked stan-

dard political and demographic questions as well as questions about their access to cable

television and media consumption. They were then instructed to watch the debate and told

15In Table C19 in Online Appendix C, we show that the manipulation had no e�ect on

favorability toward Clinton or Barack Obama either.

19

Table 3: Message exposure e�ects on Trump favorability

Morning Consult Mechanical TurkTrump support 2.50*** 2.42***

(0.14) (0.10)Uncorrected statement -0.03 -0.07

(0.13) (0.06)Uncorrected ⇥ Trump support 0.13 0.12

(0.20) (0.14)Fact-check 0.10 -0.12*

(0.13) (0.06)Fact-check ⇥ Trump support -0.11 -0.03

(0.20) (0.14)Fact-check denial 0.10 -0.05

(0.13) (0.07)Denial ⇥ Trump support 0.01 -0.09

(0.20) (0.14)Denial/source derogation -0.13 -0.05

(0.11) (0.07)Derogation ⇥ Trump support 0.17 -0.06

(0.21) (0.15)Constant 1.42*** 1.36***

(0.09) (0.05)Fact-check � uncorrected statementClinton supporters 0.14 -0.05

(0.13) (0.06)Trump supporters -0.10 -0.21

(0.14) (0.13)Denial � uncorrected statementClinton supporters 0.13 0.02

(0.13) (0.06)Trump supporters 0.01 -0.19

(0.16) (0.14)Denial/derogation � uncorrected statementClinton supporters -0.09 0.02


(0.18) (0.14)R2 0.62 0.58N 989 2430

* p < 0.10, ** p < 0.05, *** p < .01. OLS models with robust standard errors. Allindependent variables are dichotomous indicators. The dependent variable is a five-pointscale where higher values indicate greater favorability. Participants are respondents fromMorning Consult or Mechanical Turk who supported Hillary Clinton or Donald Trump inthe 2016 election (reference category for Trump support is thus Clinton support).

20

they would be invited to take a survey immediately after its conclusion.16 As soon as the

debate ended, participants were invited to complete a survey that included questions about

candidates’ debate performances and respondents’ general attitudes towards the candidates

(Wave 2). It included the following statement from Trump:

Our jobs are fleeing the country to Mexico... they’re building some of the

biggest, most sophisticated plants. Not so much in America. Thousands of

jobs leaving Michigan, Ohio...their companies are just leaving, they’re gone.17

The claim that large numbers of jobs were leaving Michigan and Ohio at the time due to

factories being moved abroad is inaccurate. As fact-checkers pointed out on the night of the

debate, employment had not fallen in either state. The New York Times fact-check pointed

to the number of new jobs created in each state over the previous year (New York Times

2016). A fact-check by National Public Radio (NPR) of this Trump statement directed

readers to the Bureau of Labor Statistics (BLS) for data confirming the fact check—and

rebutting Trump (National Public Radio 2016).

After seeing Trump’s claim, respondents were randomly assigned with probability .5

to receive the following fact-check: “In fact, according to the Bureau of Labor Statistics,

16Because the broader experiment in which Study 2 was embedded was designed to

examine how post-debate news coverage a�ected debate perceptions, participants were

assigned to one of five content consumption conditions that were orthogonal to the ran-

domization we examine here (C-SPAN with no post-debate coverage, Fox News with or

without post-debate coverage, or MSNBC with or without post-debate coverage). We ex-

cluded subjects who did not have access to cable and block-randomized by party and pre-

ferred cable channel. For additional details, consult (reference omitted for peer review).17The instrument was prepared before transcripts were available, so the statement in our

study di�ers slightly from the o�cial transcript.

21

unemployment has fallen in both states. Both states each saw 70,000 new jobs over the

last year.” Like The New York Times’ fact-check on the night of the debate, we pointed

to the number of new jobs created in each state over the previous year; and like NPR’s

fact-check, we explicitly based the claim of the fact check on BLS data.

All respondents then were asked: “Over the last few years, has unemployment gone

up or down in Michigan and Ohio?” Subjects could respond on a five-point scale, from

“Gone down a lot” to “Gone up a lot,” with “Stayed the same” as the middle category. (See

Online Appendix A for exact wording.) This wave was closed by noon the next day. Five

days later, we recontacted participants and measured perceptions of the debate’s winner

and attitudes toward the candidates (Wave 3).18

Study 2 allows us to test for both motivated resistance (H1) and di�erential acceptance

(H2). Under H1 (motivated resistance), Trump supporters exposed to the fact-check would

not only resist, but come to hold more inaccurate views than uncorrected Trump support-

ers. The fact-check treatment undermined Trump’s claims about the economy and his

opposition to foreign trade, which were central to his candidacy. The claim’s political im-

portance and its clear contradiction by government data makes it a useful test of partisans’

18See Online Appendix C for details on participant demographics and experimental

balance. Though we cannot rule out the possibility of post-treatment bias (Montgomery,

Nyhan, and Torres 2018), we find no significant e�ect of treatment assignment at wave 2

on wave 3 participation in a simple OLS model (b = 0.05, p > .10).

22

responsiveness to fact-checking.19 Under H2 (di�erential acceptance), Trump and Clinton

supporters who are exposed to the fact-check would both integrate this information, but

Trump supporters would be less accepting of it.

Analysis

We begin with respondents’ beliefs about unemployment in Michigan and Ohio after the

fact-check in Wave 2. Following Study 1, we estimate OLS models with robust standard

errors that include indicators for fact-check exposure and Trump support in Wave 1 and an

interaction term.20 As in Study 1, we restrict our analysis to respondents who reported sup-

porting Clinton or Trump in Wave 1. The outcome measure is coded so that higher values

indicate belief that unemployment stayed the same or increased rather than decreased.

We present results in Table 4. The first model shows that the fact-check decreased mis-

perceptions about unemployment in Michigan and Ohio among both Clinton and Trump

supporters (p < .01). As with Study 1, the evidence from Study 2 does not support moti-

vated resistance (H1). We also find no evidence of di�erential acceptance, contradicting

H2. The second model shows that these results are consistent when we control for the

19Such fact-checks are common. For instance, more than 60% of the claims rated by

PolitiFact and the Washington Post Fact Checker were found to be mostly or totally false by

both fact-checkers (Lim 2018). Moreover, fact-checkers consider it part of their mission to

check claims against o�cial data sources and frequently do so. Graves (2016, 85) writes,

for instance, that “Fact-checkers always seek o�cial data and often point to examples

like this [a fact-check assessing claims about government spending and job growth using

federal data] to explain what they do.”20The design does not include a control condition or fact-check denial and denial/source

derogation conditions. The omitted category is an uncorrected statement.

23

Table 4: Message exposure e�ects on unemployment beliefs

(1) (2)Trump support 0.44⇤⇤⇤ 0.45⇤⇤⇤

(0.13) (0.13)Fact-check �0.34⇤⇤⇤ �0.34⇤⇤⇤

(0.11) (0.11)Fact-check ⇥ Trump support �0.20 �0.20

(0.18) (0.18)Condition: MSNBC (post-debate coverage) 0.06

(0.14)Condition: MSNBC (no post-debate coverage) 0.03

(0.14)Condition: Fox (post-debate coverage) �0.13

(0.14)Condition: Fox (no post-debate coverage) �0.04

(0.15)Constant 2.70⇤⇤⇤ 2.71⇤⇤⇤

(0.07) (0.12)Fact-check � uncorrected statementClinton supporters -0.34*** -0.34***


(0.14) (0.14)R2 0.05 0.05N 825 825

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided). OLS models with robust standarderrors. Respondents are Mechanical Turk workers who supported Hillary Clinton or Don-ald Trump and were assigned to watch the first presidential debate in 2016 (referencecategory for Trump support is thus Clinton support). All independent variables are di-chotomous indicators. The dependent variable is a five-point scale of perceived change inunemployment in Ohio and Michigan where higher values indicate a perceived increasein unemployment. The omitted category for the media manipulation is C-SPAN with nopost-debate coverage.

orthogonal media manipulation.

Finally, Table 5 considers fact-check e�ects five days later (Wave 3). We again use OLS

models with robust standard errors to estimate whether exposure to a fact-check a�ected

perceptions that Trump won the debate, evaluations of his debate performance, and vote

choice. We again include indicators for fact-check exposure and Trump support and an in-

teraction term. We see no evidence that the fact-check a�ected these outcomes except for

24

perceptions that Trump “won” the debate, which declined slightly among his supporters.

However, vote choice was not a�ected. Echoing Study 1, fact-checking reduced misper-

ceptions but had no discernible e�ects on participants’ candidate preferences, including

supporters of the candidate who had been fact-checked.

Table 5: Fact-check e�ects at Wave 3

Trump won debate Trump favorability Trump voteTrump support (W1) 0.67⇤⇤⇤ 0.47⇤⇤⇤ 0.89⇤⇤⇤

(0.04) (0.02) (0.03)Fact-check 0.02 0.01 �0.001

(0.03) (0.02) (0.02)Fact-check ⇥ Trump vote (W1) �0.17⇤⇤⇤ �0.06 �0.01

(0.06) (0.04) (0.04)Constant 0.02 0.16⇤⇤⇤ 0.03⇤

(0.02) (0.02) (0.02)Fact-check � uncorrected statementClinton supporters -0.02 -0.01 -0.001

(0.03) (0.02) (0.02)Trump supporters -0.15** -0.05 -0.01

(0.05) (0.03) (0.03)R2 0.47 0.53 0.81N 527 527 527

* p < 0.10, ** p < 0.05, *** p < .01. OLS models with robust standard errors. All in-dependent variables are dichotomous indicators. The dependent variables are measuredon a five-point scale where higher values indicate greater belief that Trump won the de-bate, a more favorable attitude toward Trump, or a greater intention to vote for Trump,respectively. Respondents are Mechanical Turk workers who supported Hillary Clinton orDonald Trump in the 2016 election (reference category for Trump support is thus Clintonsupport). All outcome measures were collected in wave 3.

Conclusion

In two studies of fact-checking conducted during the 2016 presidential election, we find

that people express more factually accurate beliefs after exposure to fact-checks. These ef-

fects hold even when fact-checks target their preferred candidate. In Study 1, we exposed

participants to variants of an article covering claims Donald Trump made about crime.

25

Trump supporters were willing to accept a fact-check of those claims and update their be-

liefs, though we observe some evidence of di�erential acceptance (i.e., they revised their

beliefs less than Clinton supporters). Similarly, the accuracy of Trump supporters’ be-

liefs about unemployment increased in Study 2 after seeing a fact-check of a claim Trump

made during the first debate. However, exposure to journalistic fact-checks did not a�ect

attitudes toward him in either study. Ultimately, we find no evidence that changes in fac-

tual beliefs in a claim made by a candidate a�ect voter preferences during a presidential

election.

Our results on interpretations were mixed. Study 1 participants evaluated the fairness

and accuracy of the stimulus article and the accuracy of the federal crime statistics it cited

in a directionally motivated fashion. However, they did not adopt viewpoint-consistent in-

terpretations of the change in crime. Further research is needed to understand how respon-

dents interpret counter-attitudinal fact-checks and other forms of corrective information.

These results help theoretically inform our understanding of both motivated reasoning

and factual belief updating. As with other recent research (Wood and Porter 2018), we find

little evidence of a backfire e�ect on respondents’ factual beliefs. However, our results

also do not suggest that respondents accept fact-checks uncritically; exposure to counter-

attitudinal information decreases perceptions of the accuracy of our stimulus article and

the source of counter-attitudinal information in Study 1. These findings suggest motivated

reasoning can coexist with belief updating, a possible explanation for divergent findings

in the literature (e.g., Flynn 2016); they need not be mutually exclusive phenomena (see

also Coppock and Guess N.d. and Khanna and Sood 2018).

Of course, these studies have several limitations. First, we did not test a fact-check

of a Clinton misstatement and cannot evaluate how her supporters would have reacted.

Second, Trump was infamous for extreme exaggerations and misstatements, which may

have made some respondents receptive to fact-checking but also prepared his supporters

26

to rationalize their continued support for him. Also, we measured feelings toward the

candidates post-treatment and use them as an outcome measure. It is possible that that the

strongest Trump supporters respond to fact-checks di�erently than less ardent supporters,

but we are unable to test this conjecture.

Further research is also necessary to determine the extent to which our results gen-

eralize to other contexts or forms of fact-checking. Our results suggest that fact-checks

are unlikely to meaningfully diminish the strong attachments people have to their party’s

candidate in a campaign context with strong partisan cues. Their e�ects may be stronger,

however, in elections with weaker partisan cues and less well-known figures on the ballot

(e.g., Wintersieck 2017). In addition, fact-checks could have stronger attitudinal e�ects

on feelings toward politicians the more temporally distant they are from an election. In

addition, the scope of our finding is limited to the e�ects of a fact-check of a single mis-

statement, the format most often employed by journalists. Correcting a series of inaccurate

claims might have had larger e�ects on candidate evaluations. Other research might also

consider the e�ects of joint fact-checking of both major-party candidates or of positive

fact-checks corroborating the accuracy of a candidate’s claim. In addition, future research

should examine a�ective and emotional responses to candidate misstatements of fact as

well as the fact-checks that set the record straight.

Finally, as with all studies of this sort, we cannot completely rule out acquiescence

bias or demand e�ects. Recent research indicates that demand e�ects in Internet-based

survey experiments are relatively rare, however (Mummolo and Peterson N.d.).

Despite these limitations, our results provide compelling evidence that citizens can

accept the conclusions of journalistic fact-checks of misstatements even when they are

made by one’s preferred candidate during a presidential election. This information had

little e�ect on people’s attitudes toward the candidate being corrected, however. In other

words, Trump supporters took fact-checks literally, but not seriously enough to a�ect how

27

they felt toward their preferred candidate.

References

BBC. 2016. “‘Post-truth’ declared word of the year by Oxford Dictionaries.” Novem-

ber 16, 2016. Downloaded February 6, 2017 from http://www.bbc.com/news/

uk-37995600.

Bolsen, Toby, James N Druckman, and Fay Lomax Cook. 2014. “The influence of partisan

motivated reasoning on public opinion.” Political Behavior 36 (2): 235–262.

Chan, Man-pui Sally, Christopher R. Jones, Kathleen Hall Jamieson, and Dolores Albar-

racín. 2017. “Debunking: A Meta-Analysis of the Psychological E�cacy of Messages

Countering Misinformation.” Psychological Science 28 (11): 1531–1546.

Coppock, Alexander, and Andrew Guess. N.d. “Does Counter-Attitudinal Information

Cause Backlash? Results from Three Large Survey Experiments.” Forthcoming, British

Journal of Political Science.

Flynn, D.J. 2016. “The Scope and Correlates of Political Misperceptions in the Mass

Public.” Unpublished paper, Dartmouth College.

Funk, Carolyn L. 1999. “Bringing the candidate into models of candidate evaluation.” The

Journal of Politics 61 (3): 700–720.

Gaines, Brian J., James H. Kuklinski, Paul J. Quirk, Buddy Peyton, and Jay Verkuilen.

2007. “Same facts, di�erent interpretations: Partisan motivation and opinion on Iraq.”

Journal of Politics 69 (4): 957–974.

Garrett, R. Kelly, Erik C. Nisbet, and Emily K. Lynch. 2013. “Undermining the corrective

28

e�ects of media-based political fact checking? The role of contextual cues and naïve

theory.” Journal of Communication 63 (4): 617–637.

Graves, Lucas. 2016. Deciding what’s true: The rise of political fact-checking in American

journalism. Columbia University Press.

Hill, Seth J. 2017. “Learning together slowly: Bayesian learning about political facts.”

The Journal of Politics 79 (4): 1403–1418.

Hochschild, Jennifer L., and Katherine Levine Einstein. 2015. Do Facts Matter? Informa-

tion and Misinformation in American Politics. Norman, OK: University of Oklahoma

Press.

Jamieson, Kathleen Hall. 2015. “Implications of the Demise of ‘Fact’ in Political Dis-

course.” Proceedings of the American Philosophical Society 159 (1): 66–84.

Jarman, Je�rey W. 2016. “Motivated to Ignore the Facts: The Inability of Fact-Checking

to Promote Truth in the Public Sphere.” In Truth in the Public Sphere, ed. Jason Hannan.

Lexington Books.

Khanna, Kabir, and Gaurav Sood. 2018. “Motivated Responding in Studies of Factual

Learning.” Political Behavior 40 (1): 79–101.

Krupnikov, Yanna, and Adam Seth Levine. 2014. “Cross-sample comparisons and external

validity.” Journal of Experimental Political Science 1 (1): 59–80.

Kunda, Ziva. 1990. “The case for motivated reasoning.” Psychological bulletin 108 (3):

480–498.

Lenz, Gabriel S. 2012. Follow the leader? How voters respond to politicians’ performance

and policies. Chicago, IL: University of Chicago Press.

29

Lim, Chloe. 2018. “Checking How Fact-Checkers Check.” Research & Politics 5 (3).

Marietta, Morgan, David C. Barker, and Todd Bowser. 2015. “Fact-Checking Polarized

Politics: Does the Fact-Check Industry Provide Consistent Guidance on Disputed Re-

alities?” The Forum: A Journal of Applied Research in Contemporary Politics 13 (4):

577-596.

Molden, Daniel C., and E. Tory Higgins. 2005. “Motivated thinking.” In The Cambridge

handbook of thinking and reasoning, ed. Keith James Holyoak and Robert G. Morrison.

Cambridge University Press.

Montgomery, Jacob M., Brendan Nyhan, and Michelle Torres. 2018. “How Condition-

ing on Posttreatment Variables Can Ruin Your Experiment and What to Do about It.”

American Journal of Political Science 62 (3): 760–775.

Mummolo, Jonathan, and Erik Peterson. N.d. “Demand E�ects in Survey Experiments:

An Empirical Assessment.” SSRN. Downloaded September 30 2018 from https://

papers.ssrn.com/sol3/papers.cfm?abstract_id=2956147.

National Public Radio. 2016. “Fact Check: Trump And Clinton De-

bate For The First Time.” September 26, 2016. Downloaded February

15, 2017 from http://www.npr.org/2016/09/26/495115346/

fact-check-first-presidential-debate.

New York Times. 2016. “Our Fact Checks of the First Debate.” September 26th, 2016.

Downloaded July 27, 2018 from https://www.nytimes.com/2016/09/27/

us/politics/fact-check-debate.html.

Nyhan, Brendan, and Jason Reifler. 2010. “When Corrections Fail: The persistence of

political misperceptions.” Political Behavior 32 (2): 303–330.

30

Nyhan, Brendan, and Jason Reifler. 2015. “The E�ect of Fact-Checking on Elites: A Field

Experiment on US State Legislators.” American Journal of Political Science 59 (3):

628–640.

Nyhan, Brendan, and Jason Reifler. N.d. “Do People Actually Learn From Fact-

Checking? Evidence from a longitudinal study during the 2014 campaign.” Unpub-

lished manuscript. Downloaded June 28, 2017 from http://www.dartmouth.

edu/~nyhan/fact-checking-effects.pdf.

Nyhan, Brendan, Jason Reifler, and Peter A. Ubel. 2013. “The hazards of correcting myths

about health care reform.” Medical Care 51 (2): 127–132.

Pierce, Patrick A. 1993. “Political sophistication and the use of candidate traits in candi-

date evaluation.”.

Politico. 2016. “Trump wrong on Michigan job losses.” September 26, 2016.

Downloaded November 11, 2017 from https://www.politico.com/

blogs/2016-presidential-debate-fact-check/2016/09/

trump-wrong-on-michigan-job-losses-228707.

Porter, Ethan, Thomas J. Wood, and David Kirby. 2018. “Sex Tra�cking, Russian Infiltra-

tion, Birth Certificates, and Pedophilia: A Survey Experiment Correcting Fake News.”

Journal of Experimental Political Science 2 (5): 304–331.

Rahn, Wendy M., John H. Aldrich, Eugene Borgida, and John L Sullivan. 1990. “A Social

Cognitive Model of Candidate Appraisal.” In Information and Democratic Processes,

ed. John A. Ferejohn and James H. Kuklinski. University of Illinois Press.

Schleifer, Theodore. 2016. “Paul Manafort doubts FBI statistics af-

ter agency spared Hillary.” CNN, July 12, 2016. Downloaded February

31

13, 2017 from http://www.cnn.com/2016/07/21/politics/

paul-manafort-fbi-statistics-hillary-clinton/.

Spivak, Carey. 2011. “The Fact-Checking Explosion.” American Journalism Review 32:

38–43.

Sullivan, Eileen, and Chad Day. 2016. “AP FACT CHECK: Crime stats don’t back

Trump’s dire view.” Associated Press, July 13, 2016. Downloaded October 22, 2018

from https://apnews.com/3e132f145e0c44cf96cb7f4fd448b34a.

Taber, Charles S., and Milton Lodge. 2006. “Motivated Skepticism in the Evaluation of

Political Beliefs.” American Journal of Political Science 50 (3): 755–769.

Uscinski, Joseph, and Ryden Butler. 2013. “The Epistemology of Fact Checking.” Critical

Review 25 (2): 162–180.

Weeks, Brian E. 2015. “Emotions, partisanship, and misperceptions: How anger and anx-

iety moderate the e�ect of partisan bias on susceptibility to political misinformation.”

Journal of Communication 65 (4): 699–719.

Wintersieck, Amanda L. 2017. “Debating the truth: The impact of fact-checking during

electoral debates.” American Politics Research 45 (2): 304–331.

Wood, Thomas, and Ethan Porter. 2018. “The Elusive Backfire E�ect: Mass Attitudes’

Steadfast Factual Adherence.” Political Behavior.

Zaller, John. 1992. The Nature and Origins of Mass Opinion. Cambridge University Press.

32

Online Appendix A

Study 1 instrument

The introductory sections of the Morning Consult and Mechanical Turk questionnairesdi�ered slightly, which we noted was a possibility in our preregistration. Here we presentthe pre-experimental components of both questionnaires. The experimental component isidentical between the two studies. To conserve space, it is therefore presented only once.

Mechanical Turk questionnaire introductory section

[Consent text]Do you consent to participate in the interview?-Yes-No

What is your age?-Under 18-18-24-25–34-35–44-45–54-55–64-65–74-75–84-85 or older

In which state do you currently reside? [Drop down menu]

What is your gender?-Male-Female-Other

If the 2016 presidential election were held today and the candidates were Democrat Hillary

A-1

Clinton and Republican Donald Trump, for whom would you vote?-Hillary Clinton-Donald Trump-Don’t know-No opinion

What is your annual household income?-Under 20 thousand dollars-20 to under 35 thousand-35 to under 50 thousand dollars-50 to under 75 thousand dollars-75 to under 100 thousand dollars-100 thousand or more dollars

[If “100 thousand or more dollars” is selected] Is that...-100 to under 150 thousand-150 to under 200 thousand-200 to under 250 thousand-250 thousand or more

Generally speaking, do you think of yourself as...-Republican-Democrat-Independent-Something else

[If “Democrat” is selected] Would you call yourself...-A strong Democrat?-A not very strong Democrat?

[If “Republican” is selected] Would you call yourself...-A strong Republican?-A not very strong Republican?

A-2

[If “Independent” or “Something else” is selected] Do you think of yourself as closer to...-the Democratic Party-the Republican Party-Neither

Thinking about politics these days, how would you describe your political viewpoint?-Very liberal-Liberal-Slightly liberal-Moderate-Slightly conservative-Conservative-Very conservative-Don’t know

What is the last grade or class you completed in school?-None, or grade 1–8-High school incomplete (grades 9–11)-High school diploma or equivalent, no further schooling-Technical or vocational school after high school-Some college, no degree-Associate’s or two-year college degree-Four-year college degree-Graduate or professional school after college, no degree-Graduate or professional degree

Are you yourself of Hispanic origin or descent, such as Mexican, Puerto Rican, Cuban, oror some other Spanish background?-Yes-No

Which term best describes your race or background?-American Indian-Asian American

A-3

-Black-White-Other

Morning Consult questionnaire introductory section

What is your age?-Dropdown menu ranging from “Under 18” to “100 or over”

What is your gender?-Male-Female

What is the last grade or class you completed in school?-None, or grade 1–8-High school incomplete (grades 9–11)-High school diploma or equivalent, no further schooling-Technical or vocational school after high school-Some college, no degree-Associate’s or two-year college degree-Four-year college degree-Graduate or professional school after college, no degree-Graduate or professional degree

Are you yourself of Hispanic origin or descent, such as Mexican, Puerto Rican, Cuban, oror some other Spanish background?-Yes-No

Which term best describes your race or background?-American Indian-Asian American-Black-White

A-4

-Other

Generally speaking, do you think of yourself as...-Republican-Democrat-Independent-Something else

[If “Democrat” is selected] Would you call yourself...-A strong Democrat?-A not very strong Democrat?

[If “Republican” is selected] Would you call yourself...-A strong Republican?-A not very strong Republican?

[If “Independent” or “Something else” is selected] Do you think of yourself as closer to...-the Democratic Party-the Republican Party

[Omitted Morning Consult questions not present in the MT questionnaire]

In which state do you currently reside? [Drop down menu]


If the 2016 presidential election were held today and the candidates were Democrat HillaryClinton and Republican Donald Trump, for whom would you vote?-Hillary Clinton-Donald Trump-Don’t know/No opinion


A-5

What is your annual household income?-Under 20 thousand dollars-20 to under 35 thousand-35 to under 50 thousand dollars-50 to under 75 thousand dollars-75 to under 100 thousand dollars-100 thousand or more dollars

[If “100 thousand or more dollars” is selected] Is that...-100 to under 150 thousand-150 to under 200 thousand-200 to under 250 thousand-250 thousand or more


Thinking about politics these days, how would you describe your political viewpoint?-Very liberal-Liberal-Slightly liberal-Moderate-Slightly conservative-Conservative-Very conservative-Don’t know


[Consent text]Do you consent to participate in the interview?-Yes-No

Experimental component begins here. The Morning Consult and Mechanical Turk ques-

A-6

tionnaires are identical from this point forward.

We would like to know how much you worry about various problems facing the country.

How much do you personally worry about crime and violence?-A great deal-A fair amount-Only a little-Not at all

How much do you personally worry about the availability and a�ordability of health care?-A great deal-A fair amount-Only a little-Not at all

We would like to know how serious you think various problems facing the country are.

Overall, how would you describe the problem of crime in the United States — is it ex-tremely serious, very serious, moderately serious, not too serious, or not serious at all?-Extremely serious-Very serious-Moderately serious-Not too serious-Not serious at all

Overall, how would you describe the energy situation in the United States — is it extremelyserious, very serious, moderately serious, not too serious, or not serious at all?-Extremely serious-Very serious-Moderately serious-Not too serious-Not serious at all

A-7

Overall, how would you describe the problem of obesity in the United States — is it ex-tremely serious, very serious, moderately serious, not too serious, or not serious at all?-Extremely serious-Very serious-Moderately serious-Not too serious-Not serious at all

We would like to know how important various political issues are to you personally.

How important is the issue of crime to you personally?-Extremely important-Very important-Moderately important-Slightly important-Not at all important

How important is the issue of health care to you personally?-Extremely important-Very important-Moderately important-Slightly important-Not at all important

How important is the issue of global warming/climate change to you personally?-Extremely important-Very important-Moderately important-Slightly important-Not at all important

These next few questions ask about politics in Washington, D.C. and are intended to helpus learn what types of information are known to the public. It is important to us that youdo NOT use outside sources like the Internet to search for the correct answer. Will you

A-8

answer the following questions without help from outside sources?-Yes-No

For how many years is a United States Senator elected - that is, how many years are therein one full term of o�ce for a U.S. Senator?-Two years-Four years-Six years-Eight years-None of these-Don’t know

How many times can an individual be elected President of the United States under currentlaws?-Once-Twice-Three times-Unlimited number of terms-Don’t know

How many U.S. senators are there from each state?-One-Two-Four-Depends on which state-Don’t know

Who is the Prime Minister of the United Kingdom?-Nicola Sturgeon-Jeremy Corbyn-Nick Clegg-Theresa May-Don’t know

A-9

For how many years is a member of the United States House of Representatives elected -that is, how many years are there in one full term for a U.S. House member?-One year-Two years-Four years-Six years-Eight years-Don’t know

Please read the article on the next page carefully. (You’ll need to answer some questionsabout it later in the survey.)

[Article condition 1: control]What do you need for birdwatching?

-The most basic equipment required for bird watching is your eyes, though you willsoon need to have more items with you if you intend to make this a pastime or serioushobby. How far you go is a matter of taste and budget.

-The most useful thing that you can carry is a notepad and pencil. Use this to makea note of location, time, date, weather and habitat. Do a list of the birds that you see andknow. Do a drawing or write down a description of those that you don’t. You can lookthem up later in you field guide. Your notebook should become a diary of where you havebeen and what you have seen.

-A field guide is a book that provides descriptions of birds to assist you in their identi-fication. The descriptions use several factors to help you determine the exact bird that youare looking at. As soon as you see a bird that you do not recognize you will need to haveaccess to a good field guide. There are many to choose from.

-Binoculars. These are pretty essential and buy the best that you can a�ord. A goodpair well looked after will last you a lifetime. Take time to choose ones that suit you.

[Article condition 2: Trump rising crime message]Trump, Accepting GOP Nomination, Promises End to Violent Crime “Epidemic”

Closing the Republican convention in Cleveland today, businessman Donald Trump ac-

A-10

cepted his party’s nomination for president in a speech that made crime a central issue ofhis campaign. Trump described himself as the “law and order” candidate and said he wascommitted to lowering crime rates. This is a “moment of crisis” for the country, Trumpsaid, which threatens “our very way of life.”

The Republican nominee, who previously stated that “crime is rising” in the UnitedStates, painted a picture of an “epidemic” of violent crime that he blamed on the Obamaadministration’s “rollback of criminal enforcement.”

Trump’s comments capped o� the concluding day of the 2016 Republican NationalConvention, which took place from July 18-21. By concluding before the Summer Olympicscommence in Rio, both parties’ conventions came earlier in the political season than in pre-vious years. In 2008 and 2012, both parties staged their conventions following the Games,which were held in Beijing and London, respectively.

The speech was attended by approximately 2,470 delegates from all 50 states, the Dis-trict of Columbia and five territories. Each state’s delegate allotment is set by nationalparty rules and includes at-large delegates, congressional district delegates, and nationalparty representatives. Apart from the states, the District of Columbia and the five territo-ries are awarded a specified number of at-large delegates.

[Article condition 3: Trump rising crime message + fact-check]Trump, Accepting GOP Nomination, Promises End to Violent Crime “Epidemic”

Closing the Republican convention in Cleveland today, businessman Donald Trump ac-cepted his party’s nomination for president in a speech that made crime a central issue ofhis campaign. Trump described himself as the “law and order” candidate and said he wascommitted to lowering crime rates. This is a “moment of crisis” for the country, Trumpsaid, which threatens “our very way of life.”


According to FBI’s Bureau of Justice Statistics, the violent crime rate has fallen dra-matically and consistently over time. According to their estimates, the homicide rate inthe U.S. in 2015 was half that recorded in 1991.

Trump’s comments capped o� the concluding day of the 2016 Republican NationalConvention, which took place from July 18-21. By concluding before the Summer Olympicscommence in Rio, both parties’ conventions came earlier in the political season than in pre-

A-11

vious years. In 2008 and 2012, both parties staged their conventions following the Games,which were held in Beijing and London, respectively.

The speech was attended by approximately 2,470 delegates from all 50 states, the Dis-trict of Columbia and five territories.

[Article condition 4: Trump rising crime message + fact-check + rejection of fact-check]Trump, Accepting GOP Nomination, Promises End to Violent Crime “Epidemic”




In response, Trump’s campaign manager, Paul Manafort, said the FBI’s statistics shouldbe viewed skeptically: "People don’t feel safe in their neighborhoods. I’m not sure whatstatistics you’re talking about."

Trump’s comments capped o� the concluding day of the 2016 Republican NationalConvention, which took place from July 18-21. By concluding before the Summer Olympicscommence in Rio, both parties’ conventions came earlier in the political season than in pre-vious years. In 2008 and 2012, both parties staged their conventions following the Games.

[Article condition 5: Trump rising crime message + fact-check + rejection of fact-check+ conspiracy/fact-check source derogation]Trump, Accepting GOP Nomination, Promises End to Violent Crime “Epidemic”


A-12



In response, Trump’s campaign manager, Paul Manafort, said the FBI’s statistics shouldbe viewed skeptically: "People don’t feel safe in their neighborhoods. I’m not sure whatstatistics you’re talking about."

Manafort suggested that the FBI might even have an interest in the election’s outcome:"The FBI is certainly suspect these days after what they just did with Hillary Clinton,"he added. Manafort was referring to the FBI’s recent decision not to prosecute HillaryClinton over the email scandal that has rocked her presidential campaign.

[If assigned to control condition]What activity was the focus of the article you just read?-Bird watching-Hiking-Boating-Swimming

[If assigned to control condition]What item is most useful to carry for that activity according to the article?-Telescope-Notepad and pencil-Hat-Pencil

[If assigned to non-control condition]What event was the focus of the article you just read?-The Democratic convention-The Republican convention-A presidential debate-A vice presidential debate

A-13

[If assigned to non-control condition]What issue did the article you just read focus on?-Crime-Terrorism-Economy-Health care-Environment

We now would like to ask you some factual political questions. Would you say that, com-pared to ten years ago, the violent crime rate has gone up, stayed about the same, or gonedown?-Gone up-Stayed about the same-Gone down

[If “Gone up” is selected]Compared to ten years ago, has the violent crime rate gone up a lot or only somewhat?-Gone up a lot-Gone up somewhat

[If “Gone up” is selected]What would you say is the main reason violent crime has gone up?-The decline in morals and values-Increased economic inequality and helplessness-Some other reason-Don’t know

[If “Gone down” is selected]Compared to ten years ago, has the violent crime rate gone down a lot or only somewhat?-Gone down a lot-Gone down somewhat

[If “Gone down” is selected]

A-14

What would you say is the main reason violent crime has gone down?-Tougher policing and longer prison sentences-The improved economy and reduced unemployment-Some other reason-Don’t know

In your view, how accurate are federal crime statistics?-Extremely accurate-Somewhat accurate-Not very accurate-Not at all accurate

In your view, how accurate was the article you read in reporting federal crime statistics?-Extremely accurate-Somewhat accurate-Not very accurate-Not at all accurate

In your opinion, how fair was the article you read?-Extremely fair-Somewhat fair-Not very fair-Not at all fair

Would you say the article you read was biased or not biased?-Yes, biased-No, not biased

[If “Yes, biased” is selected]Would you say the article had a liberal bias, a conservative bias, or some other bias?-Liberal-Conservative-Other

A-15

[If “Yes, biased” is selected]Would you say the article was extremely biased, somewhat biased, or slightly biased?-Extremely biased-Somewhat biased-Slightly biased

In general, do you think the courts in this area deal too harshly or not harshly enough withcriminals?-Too harshly-About right-Not harshly enough

Next we are going to give you a list of government programs. For each one, please indicatewhether you would like to see spending increased or decreased.

Should government spending on law enforcement be increased, decreased, or kept thesame?-Increased a great deal-Increased a moderate amount-Increased a little-Kept the same-Decreased a little-Decreased a moderate amount-Decreased a great deal

Should government spending on Social Security be increased, decreased, or kept the same?-Increased a great deal-Increased a moderate amount-Increased a little-Kept the same-Decreased a little-Decreased a moderate amount-Decreased a great deal

A-16

Should government spending on protecting the environment be increased, decreased, orkept the same?-Increased a great deal-Increased a moderate amount-Increased a little-Kept the same-Decreased a little-Decreased a moderate amount-Decreased a great deal

Overall, how would you describe the problem of crime in the United States — is it ex-tremely serious, very serious, moderately serious, not too serious, or not serious at all?-Extremely serious-Very serious-Moderately serious-Not too serious-Not serious at all

Do you approve or disapprove of how Barack Obama is handling his job as president?-Strongly approve-Somewhat approve-Neither approve nor disapprove-Somewhat disapprove-Strongly disapprove

Do you feel favorable or unfavorable toward each of these individuals?

Very Somewhat Neither favorable Somewhat Veryfavorable favorable nor unfavorable unfavorable unfavorable

Barack ObamaHillary ClintonBernie SandersDonald TrumpTed CruzJohn KasichGeorge W. Bush

A-17

Study 2 instrument

[Wave 1 Party ID]Please describe your political party a�liation.-Strong Republican-Not Strong Republican-Lean Republican-Undecided, Independent, Other-Lean Democrat-Not Strong Democrat-Strong Democrat

[Wave 2 pre-treatment vote choice]Which candidate do you plan for in the upcoming presidential election?-Hillary Clinton-Donald Trump-Somebody else

[Wave 2 pre-treatment debate winner]Who won the presidential debate?-Clearly Clinton-Mostly Clinton-Neither Clinton nor Trump-Mostly Trump-Clearly Trump

[Wave 2 pre-treatment debate evaluations]Use the sliders below to tell us how you felt about the candidates’ performances during thedebate. A score of 0 means you felt very cold and negative about that candidate’s debateperformance, and a score of 100 means you felt very warm and positive about that candi-date’s debate performance.

-Hillary Clinton’s debate performance-Donald Trump’s debate performance

A-18

[Wave 2 Trump statement shown to all respondents]Our jobs are fleeing the country to Mexico... they’re building some of the biggest, mostsophisticated plants. Not so much in America. Thousands of jobs leaving Michigan,Ohio...their companies are just leaving, they’re gone.

[Wave 2 fact-check shown to those assigned to fact-check condition]In fact, according to the Bureau of Labor Statistics, unemployment has fallen in bothMichigan and Ohio. Both states each saw 70,000 new jobs over the last year.

[Wave 2 post-treatment question shown to all respondents]Over the last few years, has unemployment gone up or down in Michigan and Ohio?-Gone down a lot-Gone down a little-Stayed the same-Gone up a little-Gone up a lot

[Wave 3 vote choice]Which candidate do you plan for in the upcoming presidential election?-Hillary Clinton-Donald Trump-Somebody else

[Wave 3 debate evaluations]Use the sliders below to tell us how you felt about the candidates’ performances duringthe first debate. A score of 0 means you felt very cold and negative about that candidate’sdebate performance, and a score of 100 means you felt very warm and positive about thatcandidate’s debate performance.

-Hillary Clinton’s debate performance-Donald Trump’s debate performance

[Wave 3 debate winner]Who won the first presidential debate?

A-19

-Hillary Clinton-Donald Trump

A-20

Online Appendix B: Preregistered research questions

In addition to the preregistered hypotheses we consider in main body of the paper, we alsoexplore preregistered three research questions (RQs) that address important substantiveand methodological issues where we do not have strong theoretical priors. In this onlineappendix, we describe these preregistered RQs and briefly report their results.

The first research question (RQ1) examines whether responsiveness to treatments dif-fers between a more conventional online survey sample from the polling company Morn-ing Consult and an Amazon Mechanical Turk sample (Study 1 only). Though studies havefound that Mechanical Turk workers provide valid data (e.g., Berinsky, Huber, and Lenz2012; Levay, Freese, and Druckman 2016), concerns continue about social desirabilitybias when researchers pay and evaluate respondents (e.g., Cli�ord and Jerit 2015). Re-spondents may be inclined to endorse the conclusions of fact-checks to demonstrate thatthey read the stimuli and/or were paying attention. We therefore consider whether the re-sults of Study 1 di�er between participants surveyed by Morning Consult, which draws itsrespondents from SSI and other similar panels, and workers on Mechanical Turk.

The second research question (RQ2) tests whether any of our treatments alter policypreferences or the perceived importance of the issue in question (Study 1 only). Decreasingthe perceived importance of an issue is one way respondents might cope with uncongenialinformation. Previous studies have also found mixed results of factual information on issueopinions (e.g., Flynn 2016).

The third research questions (RQ3) evaluates whether fact-checks about crime a�ectevaluations of the incumbent president (Obama), the candidate whose statement is ques-tioned (Trump), or the candidate who is invoked in an elite attack on the fact-check (Clin-ton)?

We first consider RQ1. As Tables 1 and 2 demonstrate, the marginal e�ects of expo-sure to journalistic fact-checks were strikingly parallel between the Morning Consult andMechanical Turk samples (RQ1). Participants on Mechanical Turk are not di�erentiallyresponsive to fact-checks, a finding which is consistent with the results in Mummolo andPeterson (N.d.).1

Turning to RQ2, we estimate the marginal e�ect of fact-check exposure on respon-1Online Appendix C presents fully pooled results with interaction terms testing for di�erences in e�ects

between the Morning Consult and Mechanical Turk samples. Out of 48 coe�cients that interact an exper-imental manipulation with an indicator for a Mechanical Turk respondent, only eight are significant at thep < .05 level. In particular, only one of 24 coe�cients testing for di�erential responsiveness among Trumpsupporters is significant at the p < .05 level.

B-1

dents’ views of the importance of crime and their preferences on the issue for both can-didates’ supporters. As shown in Table C15 in Online Appendix C, we find no consistente�ect across multiple measures and samples.

Finally, we evaluate RQ3 by estimating the marginal e�ect of assignment to a fact-check on respondents’ feelings towards the sitting president (Obama), and the two major-party nominees (Trump and Clinton). These results are presented in Tables C17 and C19.There is scant evidence that the fact-checks a�ected how respondents answered the feelingthermometer questions.

References

Berinsky, Adam J., Gregory A. Huber, and Gabriel S. Lenz. 2012. “Evaluating onlinelabor markets for experimental research: Amazon. com’s Mechanical Turk.” Political

Analysis 20 (3): 351–368.

Cli�ord, Scott, and Jennifer Jerit. 2015. “Do attempts to improve respondent attentionincrease social desirability bias?” Public Opinion Quarterly 79 (3): 790–802.

Flynn, D.J. 2016. “The Scope and Correlates of Political Misperceptions in the MassPublic.” Unpublished paper, Dartmouth College.

Levay, Kevin E., Jeremy Freese, and James N. Druckman. 2016. “The Demo-graphic and Political Composition of Mechanical Turk Samples.” SAGE Open 6 (1):2158244016636433.

Mummolo, Jonathan, and Erik Peterson. N.d. “Demand E�ects in Survey Experi-ments: An Empirical Assessment.” Unpublished manuscript. Downloaded June 16,2017 from https://www.dropbox.com/s/lga66eon2o5npw6/demand_

effects_mummolo_peterson_4_20_2017.pdf?dl=0.

B-2

Online Appendix C: Supplementary tables

Study 1 supplementary results

Table C1: Study 1 demographics/balance (Morning Consult)

Control Uncorrected Fact-check Denial Denial/derogation Total

Age

18-34 29% 25% 23% 27% 27% 26%35-44 17% 22% 16% 15% 16% 17%45-64 34% 31% 47% 41% 34% 38%65+ 21% 22% 14% 17% 23% 19%

Gender

Female 53% 59% 52% 57% 56% 55%Male 47% 41% 48% 43% 44% 45%

Education

High school or less 19% 17% 15% 11% 15% 15%Some college/associate 35% 36% 42% 40% 42% 39%Bachelor’s degree 25% 29% 24% 25% 24% 25%Graduate degree 21% 18% 19% 25% 19% 20%

Race

Nonwhite 14% 16% 16% 18% 16% 16%White 86% 84% 84% 82% 84% 84%

Party

Democrat 38% 39% 35% 42% 42% 39%Republican 31% 29% 30% 24% 28% 28%Independent/something else 31% 32% 34% 34% 30% 32%

C-1

Table C2: Study 1 demographics/balance (Mechanical Turk)

Control Uncorrected Fact-check Denial Denial/derogation Total

Age

18-34 59% 55% 58% 57% 56% 57%35-44 21% 23% 23% 22% 24% 23%45-64 18% 20% 16% 17% 17% 18%65+ 2% 2% 3% 4% 2% 2%

Gender

Female 50% 55% 53% 54% 49% 52%Male 50% 45% 47% 46% 51% 48%

Education

High school or less 8% 9% 8% 8% 8% 8%Some college/associate degree 39% 39% 38% 42% 40% 40%Bachelor’s degree 37% 32% 37% 35% 35% 35%Graduate degree 16% 19% 17% 16% 16% 17%

Race

Nonwhite 20% 18% 20% 18% 21% 20%White 80% 82% 80% 82% 79% 80%

Party

Democrat 44% 43% 43% 41% 41% 42%Republican 19% 22% 22% 22% 21% 21%Independent/something else 37% 36% 35% 37% 38% 37%

C-2

Table C3: Study 2 demographics/balance (Mechanical Turk)

Wave 1 Wave 2 Wave 3Control Treatment Total Control Treatment Total

Age

18-34 56% 52% 53% 52% 50% 55% 52%35-44 22% 25% 24% 24% 27% 22% 25%45-64 19% 20% 22% 21% 19% 21% 20%65+ 3% 3% 2% 3% 3% 2% 3%

Gender

Male 50% 54% 51% 52% 57% 54% 55%Female 50% 46% 49% 48% 43% 46% 45%

Education

High school or less 10% 9% 11% 10% 10% 10% 10%Some college 36% 37% 32% 35% 33% 28% 31%Bachelor’s degree 41% 40% 42% 41% 42% 46% 44%Graduate degree 14% 14% 15% 14% 15% 16% 16%

Race

Nonwhite 21% 20% 22% 21% 19% 22% 21%White 79% 80% 78% 79% 81% 78% 79%

Party

Democrat 50% 47% 51% 49% 54% 58% 56%Republican 25% 29% 27% 28% 24% 24% 24%Independent 25% 24% 22% 23% 22% 18% 20%

C-3

Table C4: Message exposure e�ects on beliefs about changes in crime (ordered probit)

Morning Consult Mechanical Turk

Trump support 0.42*** 0.65***(0.14) (0.09)

Uncorrected statement -0.24* -0.03(0.13) (0.07)

Uncorrected ⇥ Trump support 0.25 0.10(0.20) (0.13)

Fact-check -0.70*** -0.86***(0.14) (0.09)

Fact-check ⇥ Trump support 0.06 0.13(0.21) (0.16)

Fact-check denial -0.54*** -0.81***(0.15) (0.08)

Fact-check denial ⇥ Trump support 0.13 0.32**(0.22) (0.14)

Denial/source derogation -0.82*** -0.65***(0.15) (0.08)

Denial/source derogation ⇥ Trump support 0.41* 0.25*(0.22) (0.14)

Fact-check � uncorrected statementClinton supporters -0.46*** -0.82***





(0.14) (0.08)Trump supporters -0.42** -0.46***

(0.16) (0.11)

N 990 2430

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); ordered probit models with robust standard errors (cut-points omitted). Respondents are Morning Consult respondents and Amazon Mechanical Turk workers whosupported Hillary Clinton or Donald Trump in the 2016 general election (reference category for Trumpsupport is Clinton support).

C-4

Table C5: Message exposure e�ects on beliefs about changes in crime (all respondents)


Trump support 0.08 0.38***(0.20) (0.14)

Clinton support -0.43** -0.44***(0.21) (0.12)

Uncorrected statement -0.10 -0.09(0.25) (0.16)


Uncorrected ⇥ Clinton support -0.24 0.04(0.30) (0.18)

Fact-check -1.07*** -0.80***(0.28) (0.18)

Fact-check ⇥ Trump support 0.22 -0.06(0.33) (0.24)

Fact-check ⇥ Clinton support 0.13 -0.17(0.33) (0.21)

Fact-check denial -0.76*** -1.10***(0.29) (0.16)

Fact-check denial ⇥ Trump support 0.22 0.54**(0.34) (0.22)

Fact-check denial ⇥ Clinton support 0.01 0.16(0.35) (0.19)

Denial/source derogation -0.62** -0.82***(0.26) (0.17)

Denial/source derogation ⇥ Trump support 0.11 0.34(0.32) (0.22)

Denial/source derogation ⇥ Clinton support -0.50 0.05(0.32) (0.20)

Constant 4.07*** 3.44***(0.17) (0.11)







(0.20) (0.14)

R2 0.13 0.18N 1200 2983

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); OLS models with robust standard errors. Respondentsare Morning Consult respondents and Amazon Mechanical Turk workers.

C-5

Table C6: Message exposure e�ects on perceptions of accuracy and fairness


Trump supporter -0.15 -0.00 -0.07 -0.04 -0.18** -0.08(0.13) (0.09) (0.09) (0.07) (0.08) (0.05)

Uncorrected statement -0.17 0.05 -0.37*** -0.38*** -0.08 0.02(0.12) (0.07) (0.10) (0.06) (0.08) (0.04)

Uncorrected ⇥ Trump support 0.48*** 0.50*** 0.41*** 0.49*** 0.20 -0.06(0.16) (0.12) (0.13) (0.10) (0.12) (0.08)

Fact-check 0.23** 0.71*** -0.10 0.19*** -0.04 0.10**(0.11) (0.06) (0.09) (0.06) (0.08) (0.04)

Fact-check ⇥ Trump support -0.08 -0.36*** -0.09 -0.37*** -0.18 -0.24***(0.16) (0.12) (0.14) (0.10) (0.13) (0.08)

Fact-check denial 0.26** 0.74*** -0.09 0.10* 0.04 0.15***(0.11) (0.06) (0.10) (0.06) (0.08) (0.04)

Denial ⇥ Trump support -0.13 -0.25** -0.24 -0.24** -0.38*** -0.40***(0.17) (0.12) (0.15) (0.10) (0.14) (0.08)

Denial/source derogation 0.30*** 0.74*** -0.12 0.04 0.06 0.13***(0.11) (0.07) (0.09) (0.06) (0.08) (0.04)

Denial/source derogation ⇥ Trump support -0.15 -0.28** -0.06 -0.14 -0.29** -0.33***(0.16) (0.12) (0.14) (0.10) (0.13) (0.08)

Constant 2.68*** 2.25*** 3.06*** 2.99*** 3.05*** 3.00***(0.09) (0.05) (0.06) (0.04) (0.06) (0.02)

Fact-check � uncorrected statementClinton supporters 0.40*** 0.65*** 0.26** 0.56*** 0.04 0.08*


(0.10) (0.08) (0.09) (0.07) (0.10) (0.07)Denial � uncorrected statementClinton supporters 0.43*** 0.69*** 0.28** 0.47*** 0.12 0.14***


(0.11) (0.08) (0.11) (0.07) (0.11) (0.07)Denial/derogation � uncorrected statementClinton supporters 0.47*** 0.68*** 0.24** 0.41*** 0.15* 0.11**


(0.10) (0.07) (0.10) (0.07) (0.10) (0.07)

R2 0.04 0.14 0.03 0.06 0.08 0.07N 987 2426 985 2426 987 2429

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); OLS models with robust standard errors. Respondentsare Morning Consult respondents (MC) and Amazon Mechanical Turk workers (MT) who supported HillaryClinton or Donald Trump in the 2016 election (reference category for Trump support is thus Clinton support).

C-6

Table C7: Message exposure e�ects on perceptions of accuracy/fairness (ordered probit)


Trump supporter -0.24 0.01 -0.11 -0.05 -0.32** -0.17(0.17) (0.13) (0.14) (0.11) (0.15) (0.11)

Uncorrected statement -0.26 0.06 -0.54*** -0.58*** -0.16 0.04(0.17) (0.09) (0.15) (0.09) (0.15) (0.08)

Uncorrected ⇥ Trump support 0.67*** 0.67*** 0.59*** 0.73*** 0.35 -0.11(0.23) (0.16) (0.20) (0.15) (0.22) (0.16)

Fact-check 0.30* 0.98*** -0.14 0.30*** -0.07 0.22***(0.16) (0.09) (0.14) (0.09) (0.15) (0.08)

Fact-check ⇥ Trump support -0.11 -0.54*** -0.15 -0.59*** -0.28 -0.50***(0.23) (0.16) (0.21) (0.15) (0.23) (0.16)

Fact-check denial 0.34** 1.07*** -0.10 0.15* 0.07 0.34***(0.16) (0.09) (0.15) (0.09) (0.15) (0.08)

Denial ⇥ Trump support -0.16 -0.42** -0.37 -0.39** -0.61*** -0.83***(0.24) (0.17) (0.23) (0.15) (0.24) (0.16)

Denial/source derogation 0.40** 1.06*** -0.18 0.06 0.12 0.29***(0.16) (0.10) (0.14) (0.09) (0.14) (0.09)

Denial/source derogation ⇥ Trump support -0.22 -0.48*** -0.10 -0.24 -0.50** -0.68***(0.23) (0.16) (0.21) (0.16) (0.22) (0.16)


(0.15) (0.08) (0.16) (0.09) (0.16) (0.09)Trump supporters -0.22 -0.29** -0.34** -0.45*** -0.54*** -0.22

(0.14) (0.12) (0.14) (0.11) (0.17) (0.15)Denial � uncorrected statementClinton supporters 0.60*** 1.01*** 0.43*** 0.72*** 0.23 0.30***

(0.14) (0.09) (0.16) (0.08) (0.15) (0.09)Trump supporters -0.23 -0.07 -0.52*** -0.40*** -0.73*** -0.42***



(0.15) (0.11) (0.15) (0.11) (0.17) (0.14)

N 987 2426 985 2426 987 2429

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); ordered probit models with robust standard errors (cut-points omitted). Respondents are Morning Consult respondents (MC) and Amazon Mechanical Turk work-ers (MT) who supported Hillary Clinton or Donald Trump in the 2016 election (reference category for Trumpsupport is thus Clinton support).

C-7

Table C8: Message e�ects on perceptions of accuracy and fairness (all respondents)


Trump support 0.29* -0.11 0.11 0.05 0.11 0.03(0.16) (0.12) (0.11) (0.09) (0.10) (0.07)

Clinton support 0.44*** -0.11 0.18* 0.09 0.29*** 0.11**(0.17) (0.10) (0.10) (0.07) (0.09) (0.05)

Uncorrected statement 0.42** 0.20* -0.17 -0.14 0.08 -0.06(0.18) (0.11) (0.12) (0.08) (0.12) (0.07)

Uncorrected ⇥ Trump support -0.11 0.35** 0.21 0.25** 0.03 0.02(0.21) (0.14) (0.15) (0.12) (0.15) (0.10)

Uncorrected ⇥ Clinton support -0.59*** -0.15 -0.20 -0.24** -0.16 0.08(0.22) (0.13) (0.15) (0.10) (0.15) (0.08)

Fact-check 0.43** 0.50*** -0.16 0.01 0.07 0.07(0.18) (0.10) (0.16) (0.09) (0.13) (0.08)

Fact-check ⇥ Trump support -0.28 -0.15 -0.03 -0.19 -0.29* -0.22**(0.21) (0.15) (0.19) (0.12) (0.16) (0.11)

Fact-check ⇥ Clinton support -0.20 0.21* 0.06 0.17 -0.12 0.03(0.21) (0.12) (0.18) (0.11) (0.15) (0.09)

Fact-check denial 0.58*** 0.42*** 0.10 -0.10 0.00 -0.00(0.18) (0.11) (0.13) (0.08) (0.12) (0.07)

Fact-check denial ⇥ Trump support -0.45** 0.07 -0.42** -0.04 -0.35** -0.25**(0.22) (0.15) (0.18) (0.12) (0.16) (0.10)

Fact-check denial ⇥ Clinton support -0.32 0.33*** -0.18 0.20** 0.03 0.16*(0.21) (0.12) (0.16) (0.10) (0.15) (0.08)

Denial/source derogation 0.41** 0.51*** -0.33** -0.04 -0.11 -0.05(0.17) (0.10) (0.14) (0.09) (0.12) (0.07)

Denial/source derogation ⇥ Trump support -0.26 -0.06 0.15 -0.07 -0.12 -0.15(0.21) (0.14) (0.18) (0.12) (0.16) (0.09)

Denial/source derogation ⇥ Clinton support -0.11 0.22* 0.20 0.08 0.17 0.18**(0.21) (0.12) (0.17) (0.11) (0.14) (0.08)

Constant 2.24*** 2.36*** 2.88*** 2.90*** 2.76*** 2.89***(0.14) (0.09) (0.09) (0.06) (0.07) (0.05)



(0.10) (0.08) (0.09) (0.07) (0.10) (0.07)Denial � uncorrected statementClinton supporters 0.43*** 0.69*** 0.28** 0.47*** 0.12 0.14***




(0.10) (0.07) (0.10) (0.07) (0.10) (0.07)

R2 0.05 0.13 0.04 0.05 0.08 0.06N 1196 2978 1193 2977 1196 2980

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); OLS models with robust standard errors. Respondentsare Morning Consult respondents (MC) and Amazon Mechanical Turk workers (MT).

C-8

Table C9: Message exposure e�ects on perceptions of article bias


Trump supporter 0.29** 0.19***(0.15) (0.07)

Uncorrected statement -0.81*** -0.77***(0.18) (0.08)

Uncorrected ⇥ Trump support 0.36 0.35***(0.25) (0.12)

Fact-check -0.46*** 0.02(0.16) (0.07)

Fact-check ⇥ Trump support 0.53** 0.56***(0.25) (0.14)

Fact-check denial -0.20 -0.13*(0.17) (0.07)

Denial ⇥ Trump support 0.81*** 0.39***(0.27) (0.13)


Derogation ⇥ Trump support 0.38 0.28*(0.26) (0.15)

Constant -3.95*** -4.06***(0.10) (0.04)

Fact-check � uncorrected statementClinton supporters 0.35* 0.78***

(0.20) (0.09)Trump supporters 0.53*** 0.99***

(0.20) (0.13)Denial � uncorrected statementClinton supporters 0.62*** 0.63***


(0.22) (0.12)Denial/derogation � uncorrected statementClinton supporters 0.47** 0.48***

(0.21) (0.10)Trump supporters 0.49** 0.41***

(0.21) (0.14)

R2 0.10 0.11N 978 2426

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); OLS models with robust standard errors. Respondentssupported Hillary Clinton or Donald Trump in the 2016 election (reference category for Trump support isthus Clinton support). Outcome variable ranges from “Extremely biased conservative” (-3) to “Extremelybiased liberal” (3).

C-9

Table C10: Message exposure e�ects on perceptions of article bias (ordered probit)


Trump supporter 0.25** 0.22***(0.12) (0.08)

Uncorrected statement -0.66*** -0.77***(0.15) (0.08)

Uncorrected ⇥ Trump support 0.26 0.31**(0.20) (0.13)

Fact-check -0.40*** 0.03(0.14) (0.07)

Fact-check ⇥ Trump support 0.43** 0.52***(0.20) (0.14)

Fact-check denial -0.19 -0.12*(0.14) (0.07)

Denial ⇥ Trump support 0.62*** 0.38***(0.21) (0.14)


Derogation ⇥ Trump support 0.28 0.27*(0.21) (0.16)

Fact-check � uncorrected statementClinton supporters 0.27* 0.80***


(0.16) (0.13)Denial � uncorrected statementClinton supporters 0.48*** 0.65***


(0.17) (0.13)Denial/derogation � uncorrected statementClinton supporters 0.37** 0.49***

(0.16) (0.10)Trump supporters 0.39** 0.45***

(0.17) (0.14)

N 978 2426

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); ordered probit models with robust standard errors (cut-points omitted). Respondents supported Hillary Clinton or Donald Trump in the 2016 election (referencecategory for Trump support is thus Clinton support). Outcome variable ranges from “Extremely biasedconservative” (-3) to “Extremely biased liberal” (3).

C-10

Table C11: Message exposure e�ects on factual interpretations


Trump supporter 0.71*** 0.43***(0.10) (0.07)

Uncorrected statement 0.00 0.01(0.10) (0.05)

Uncorrected ⇥ Trump support -0.15 0.01(0.15) (0.10)

Fact-check 0.07 -0.04(0.10) (0.05)

Fact-check ⇥ Trump support -0.31** 0.00(0.14) (0.10)

Fact-check denial 0.27*** -0.00(0.10) (0.05)

Denial ⇥ Trump support -0.36** -0.09(0.14) (0.10)

Denial/source derogation 0.17* -0.08(0.10) (0.05)

Derogation ⇥ Trump support -0.33** 0.05(0.15) (0.09)

Constant -0.24*** -0.27***(0.07) (0.03)

Fact-check � uncorrected statementClinton supporters 0.07 -0.05

(0.10) (0.05)Trump supporters -0.08* -0.05

(0.11) (0.09)Denial � uncorrected statementClinton supporters 0.27*** -0.01


(0.11) (0.09)Denial/derogation � uncorrected statementClinton supporters 0.16* -0.08


(0.12) (0.08)

R2 0.11 0.08N 990 2427

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); OLS models with robust standard errors. Respondentssupported Hillary Clinton or Donald Trump in the 2016 election (reference category for Trump support isthus Clinton support). Outcome variable is coded -1 for liberal interpretations of reported changes in factualbeliefs, 1 for conservative interpretations, and 0 for other interpretations/don’t know or for respondents whosaid crime had not changed.

C-11

Table C12: Message exposure e�ects on factual interpretations (ordered probit)


Trump supporter 1.11*** 0.71***(0.18) (0.11)

Uncorrected statement 0.01 0.01(0.15) (0.08)


Fact-check 0.11 -0.07(0.15) (0.08)

Fact-check ⇥ Trump support -0.52** 0.00(0.23) (0.16)

Fact-check denial 0.40*** -0.00(0.15) (0.08)

Denial ⇥ Trump support -0.58** -0.15(0.23) (0.16)

Denial/source derogation 0.25* -0.13(0.15) (0.09)

Derogation ⇥ Trump support -0.54** 0.10(0.24) (0.16)



(0.17) (0.14)Denial � uncorrected statementClinton supporters 0.40*** -0.01


(0.18) (0.14)Denial/derogation � uncorrected statementClinton supporters 0.25* -0.15


(0.19) (0.14)

N 990 2427

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); ordered probit models with robust standard errors (cut-points omitted). Respondents supported Hillary Clinton or Donald Trump in the 2016 election (referencecategory for Trump support is thus Clinton support). Outcome variable is coded -1 for liberal interpretationsof reported changes in factual beliefs, 1 for conservative interpretations, and 0 for other interpretations/don’tknow or for respondents who said crime had not changed.

C-12

Tabl

eC1

3:Te

sting

forM

orni

ngCo

nsul

t/Mec

hani

calT

urk

di�e

renc

es

Crim

ech

ange

Arti

cle

accu

rate

Arti

cle

fair

Stat

istic

sacc

urat

eA

rticl

ebi

asCr

ime

inte

rpre

tatio

n

Trum

psu

ppor

t0.

52**

*-0

.15

-0.0

7-0

.18*

*-0

.29*

*0.

71**

*(0

.16)

(0.1

3)(0

.09)

(0.0

8)(0

.15)

(0.1

0)M

echa

nica

lTur

k-0

.63*

**-0

.43*

**-0

.07

-0.0

50.

11-0

.03

(0.1

4)(0

.10)

(0.0

7)(0

.06)

(0.1

1)(0

.08)

Trum

psu

ppor

t⇥Tu

rk0.

300.

150.

040.

100.

10-0

.27*

*(0

.20)

(0.1

6)(0

.12)

(0.1

0)(0

.17)

(0.1

2)U

ncor

rect

edsta

tem

ent

-0.3

4*-0

.17

-0.3

7***

-0.0

80.

81**

*0.

00(0

.18)

(0.1

2)(0

.10)

(0.0

8)(0

.18)

(0.1

0)U

ncor

rect

ed⇥

Turk

0.28

0.22

-0.0

10.

10-0

.05

0.01

(0.2

0)(0

.14)

(0.1

1)(0

.09)

(0.2

0)(0

.11)

Unc

orre

cted

⇥Tr

ump

supp

ort

0.35

0.48

***

0.41

***

0.20

-0.3

6-0

.15

(0.2

3)(0

.16)

(0.1

3)(0

.12)

(0.2

5)(0

.15)

Unc

orre

cted

⇥Tr

ump

supp

ort⇥

Turk

-0.1

90.

020.

08-0

.25*

0.00

0.16

(0.2

8)(0

.20)

(0.1

6)(0

.14)

(0.2

8)(0

.18)

Fact

-che

ck-0

.94*

**0.

23**

-0.1

0-0

.04

0.46

***

0.07

(0.1

9)(0

.11)

(0.0

9)(0

.08)

(0.1

6)(0

.10)

Fact

-che

ck⇥

Turk

-0.0

30.

48**

*0.

29**

*0.

14-0

.48*

**-0

.11

(0.2

1)(0

.13)

(0.1

1)(0

.09)

(0.1

8)(0

.11)

Fact

-che

ck⇥

Trum

psu

ppor

t0.

09-0

.08

-0.0

9-0

.18

-0.5

3**

-0.3

1**

(0.2

6)(0

.16)

(0.1

4)(0

.13)

(0.2

5)(0

.14)

Fact

-che

ck⇥

Trum

psu

ppor

t⇥Tu

rk0.

02-0

.28

-0.2

8-0

.07

-0.0

20.

31*

(0.3

2)(0

.20)

(0.1

7)(0

.15)

(0.2

8)(0

.17)

Fact

-che

ckde

nial

-0.7

4***

0.26

**-0

.09

0.04

0.20

0.27

***

(0.1

9)(0

.11)

(0.1

0)(0

.08)

(0.1

7)(0

.10)

Fact

-che

ckde

nial⇥

Turk

-0.1

90.

49**

*0.

18*

0.12

-0.0

7-0

.27*

*(0

.21)

(0.1

3)(0

.11)

(0.0

9)(0

.18)

(0.1

1)Fa

ct-c

heck

deni

al⇥

Trum

psu

ppor

t0.

21-0

.13

-0.2

4-0

.38*

**-0

.81*

**-0

.36*

*(0

.27)

(0.1

7)(0

.15)

(0.1

4)(0

.27)

(0.1

4)Fa

ct-c

heck

deni

al⇥

Trum

psu

ppor

t⇥Tu

rk0.

17-0

.12

-0.0

0-0

.02

0.42

0.26

(0.3

2)(0

.21)

(0.1

8)(0

.16)

(0.3

0)(0

.17)

Den

ial/s

ourc

ede

roga

tion

-1.1

2***

0.30

***

-0.1

20.

060.

35**

0.17

*(0

.19)

(0.1

1)(0

.09)

(0.0

8)(0

.17)

(0.1

0)D

enia

l/sou

rce

dero

gatio

n⇥

Turk

0.34

0.43

***

0.16

0.06

-0.0

6-0

.24*

*(0

.21)

(0.1

3)(0

.11)

(0.0

9)(0

.19)

(0.1

1)D

enia

l/sou

rce

dero

gatio

n⇥

Trum

psu

ppor

t0.

61**

-0.1

5-0

.06

-0.2

9**

-0.3

8-0

.33*

*(0

.27)

(0.1

6)(0

.14)

(0.1

3)(0

.26)

(0.1

5)D

enia

l/sou

rce

dero

gatio

n⇥

Trum

psu

ppor

t⇥Tu

rk-0

.31

-0.1

3-0

.09

-0.0

30.

100.

38**

(0.3

2)(0

.20)

(0.1

7)(0

.15)

(0.3

0)(0

.18)

Cons

tant

3.64

***

2.68

***

3.06

***

3.05

***

3.95

***

-0.2

4***

(0.1

2)(0

.09)

(0.0

6)(0

.06)

(0.1

0)(0

.07)

R20.

200.

120.

050.

070.

110.

11N

3420

3413

3411

3416

3404

3417

*p<

0.10

,**

p<

0.05

,***

p<

.01

(two-

sided

);O

LSm

odel

swith

robu

ststa

ndar

der

rors

.Res

pond

ents

are

Mor

ning

Cons

ultr

espo

nden

tsan

dA

maz

onM

echa

nica

lTur

kw

orke

rsw

hosu

ppor

ted

Hill

ary

Clin

ton

orD

onal

dTr

ump

inth

e201

6el

ectio

n.A

ssuc

h,th

eref

eren

ceca

tego

ryfo

rthe

Mec

hani

calT

urk

indi

cato

risM

orni

ngCo

nsul

tand

ther

efer

ence

cate

gory

forT

rum

psu

ppor

tisC

linto

nsu

ppor

t.

C-13

Tabl

eC1

4:Te

sting

forM

orni

ngCo

nsul

t/Mec

hani

calT

urk

di�e

renc

es(o

rder

edpr

obit)

Crim

ech

ange

Arti

cle

accu

rate

Arti

cle

fair

Stat

istic

sacc

urat

eA

rticl

ebi

asCr

ime

inte

rpre

tatio

n

Trum

psu

ppor

t0.

45**

*-0

.26

-0.1

1-0

.36*

*-0

.30*

*1.

17**

*(0

.14)

(0.1

8)(0

.14)

(0.1

7)(0

.15)

(0.1

9)M

echa

nica

lTur

k-0

.47*

**-0

.63*

**-0

.09

-0.1

20.

12-0

.04

(0.1

1)(0

.15)

(0.1

1)(0

.13)

(0.1

1)(0

.13)

Trum

psu

ppor

t⇥Tu

rk0.

190.

260.

060.

200.

09-0

.49*

*(0

.17)

(0.2

2)(0

.18)

(0.2

0)(0

.16)

(0.2

1)U

ncor

rect

edsta

tem

ent

-0.2

5*-0

.28*

-0.5

4***

-0.1

70.

75**

*0.

01(0

.14)

(0.1

7)(0

.14)

(0.1

7)(0

.17)

(0.1

7)U

ncor

rect

ed⇥

Turk

0.22

0.34

*-0

.04

0.22

-0.0

30.

01(0

.16)

(0.1

9)(0

.17)

(0.1

8)(0

.18)

(0.1

8)U

ncor

rect

ed⇥

Trum

psu

ppo

0.26

0.69

***

0.59

***

0.39

-0.2

8-0

.28

(0.2

1)(0

.23)

(0.2

0)(0

.24)

(0.2

4)(0

.26)

Unc

orre

cted

⇥Tr

ump

supp

ort⇥

Turk

-0.1

7-0

.03

0.14

-0.4

9*-0

.02

0.29

(0.2

4)(0

.28)

(0.2

5)(0

.29)

(0.2

6)(0

.31)

Fact

-che

ck-0

.76*

**0.

30*

-0.1

4-0

.08

0.45

***

0.12

(0.1

5)(0

.17)

(0.1

4)(0

.17)

(0.1

6)(0

.16)

Fact

-che

ck⇥

Turk

-0.0

80.

67**

*0.

44**

*0.

28-0

.48*

**-0

.19

(0.1

7)(0

.19)

(0.1

7)(0

.19)

(0.1

7)(0

.18)

Fact

-che

ck⇥

Trum

psu

ppor

t0.

06-0

.11

-0.1

5-0

.30

-0.4

9**

-0.5

4**

(0.2

2)(0

.23)

(0.2

1)(0

.25)

(0.2

3)(0

.24)

Fact

-che

ck⇥

Trum

psu

ppor

t⇥Tu

rk0.

06-0

.41

-0.4

4*-0

.18

0.01

0.55

*(0

.27)

(0.2

8)(0

.26)

(0.2

9)(0

.27)

(0.2

9)Fa

ct-c

heck

deni

al-0

.59*

**0.

34**

-0.1

00.

080.

200.

43**

*(0

.16)

(0.1

7)(0

.15)

(0.1

6)(0

.16)

(0.1

6)Fa

ct-c

heck

deni

al⇥

Turk

-0.1

90.

72**

*0.

250.

25-0

.08

-0.4

3**

(0.1

8)(0

.19)

(0.1

7)(0

.18)

(0.1

8)(0

.18)

Fact

-che

ckde

nial⇥

Trum

psu

ppor

t0.

14-0

.16

-0.3

7-0

.66*

**-0

.70*

**-0

.61*

*(0

.23)

(0.2

4)(0

.23)

(0.2

6)(0

.25)

(0.2

5)Fa

ct-c

heck

deni

al⇥

Trum

psu

ppor

t⇥Tu

rk0.

17-0

.25

-0.0

2-0

.13

0.35

0.46

(0.2

7)(0

.29)

(0.2

7)(0

.30)

(0.2

8)(0

.29)

Den

ial/s

ourc

ede

roga

tion

-0.8

8***

0.41

**-0

.18

0.13

0.33

**0.

27*

(0.1

6)(0

.17)

(0.1

4)(0

.16)

(0.1

6)(0

.16)

Den

ial/s

ourc

ede

roga

tion⇥

Turk

0.26

0.64

***

0.24

0.15

-0.0

6-0

.40*

*(0

.17)

(0.1

9)(0

.17)

(0.1

8)(0

.18)

(0.1

8)D

enia

l/sou

rce

dero

gatio

n⇥

Trum

psu

ppor

t0.

44*

-0.2

3-0

.10

-0.5

5**

-0.3

2-0

.56*

*(0

.23)

(0.2

4)(0

.21)

(0.2

4)(0

.25)

(0.2

6)D

enia

l/sou

rce

dero

gatio

n⇥

Trum

psu

ppor

t⇥Tu

rk-0

.21

-0.2

4-0

.14

-0.1

00.

070.

66**

(0.2

7)(0

.29)

(0.2

7)(0

.29)

(0.2

9)(0

.30)

N34

2034

1334

1134

1634

0434

17

*p<

0.10

,**

p<

0.05

,***

p<

.01

(two-

sided

);or

dere

dpr

obit

mod

elsw

ithro

bust

stand

ard

erro

rs(c

utpo

ints

omitt

ed).

Resp

onde

ntsa

reM

orni

ngCo

nsul

tres

pond

ents

and

Am

azon

Mec

hani

cal

Turk

wor

kers

who

supp

orte

dH

illar

yCl

into

nor

Don

ald

Trum

pin

the2

016

elec

tion.

Ass

uch,

ther

efer

ence

cate

gory

fort

heM

echa

nica

lTur

kin

dica

tori

sMor

ning

Cons

ulta

ndth

eref

eren

ceca

tego

ryfo

rTru

mp

supp

orti

sClin

ton

supp

ort.

C-14

Table C15: Message exposure e�ects on issue importance and policy preferences

Crime importance Treatment of criminals Law enforcement $MC MT MC MT MC MT

Trump support 0.18 0.22** 0.29*** 0.39*** 0.26 -0.16(0.12) (0.09) (0.10) (0.06) (0.18) (0.67)

Uncorrected statement -0.25** -0.04 -0.01 -0.02 -0.10 -0.26(0.12) (0.07) (0.10) (0.05) (0.17) (0.34)

Uncorrected ⇥ Trump support 0.13 0.09 0.05 0.17* -0.12 0.34(0.17) (0.12) (0.14) (0.09) (0.27) (1.02)

Fact-check -0.26** -0.13** -0.04 -0.03 -0.28 -0.01(0.12) (0.07) (0.10) (0.05) (0.18) (0.10)

Fact-check ⇥ Trump support 0.11 0.22* 0.24* 0.17* 0.12 0.81(0.17) (0.12) (0.13) (0.09) (0.25) (0.68)

Fact-check denial -0.11 -0.11* -0.09 -0.05 -0.06 -0.06(0.11) (0.07) (0.09) (0.05) (0.17) (0.10)

Denial ⇥ Trump support 0.09 0.22* 0.34** 0.19** 0.21 0.82(0.17) (0.12) (0.14) (0.09) (0.26) (0.68)

Denial/source derogation -0.21* -0.07 -0.04 -0.08 0.05 -0.09(0.12) (0.07) (0.10) (0.06) (0.16) (0.10)

Derogation ⇥ Trump support 0.16 0.11 0.22 0.20** 0.06 0.88(0.17) (0.13) (0.14) (0.09) (0.25) (0.68)

Constant 3.95*** 3.35*** 2.15*** 1.95*** 5.41*** 4.50***(0.08) (0.05) (0.07) (0.04) (0.12) (0.07)

Fact-check � uncorrected statementClinton supporters 0.00 -0.09 -0.03 -0.04 -0.18 0.24

(0.13) (0.07) (0.10) (0.15) (0.18) (0.34)Trump supporters -0.02 0.04 0.16* 0.05 0.07 0.71

(0.13) (0.10) (0.09) (0.17) (0.20) (0.70)Denial � uncorrected statementClinton supporters 0.15 -0.07 -0.08 -0.12 0.04 0.20

(0.11) (0.07) (0.10) (0.15) (0.17) (0.34)Trump supporters 0.11 0.06 0.22** 0.37** 0.37* 0.68

(0.14) (0.09) (0.10) (0.17) (0.21) (0.71)Denial/derogation � uncorrected statementClinton supporters 0.04 -0.03 -0.03 -0.05 0.16 0.17

(0.12) (0.07) (0.10) (0.16) (0.17) (0.34)Trump supporter 0.07 0.00 0.14 0.55** 0.34 0.71

(0.13) (0.10) (0.10) (0.22) (0.21) (0.70)

R2 0.03 0.04 0.11 0.12 0.02 0.01N 990 2430 987 2430 991 2430


C-15

Table C16: Message e�ects on issue importance/policy preferences (ordered probit)

Crime importance Treatment of criminals Law enforcement $MC MT MC MT MC MT

Trump support 0.22 0.26** 0.47*** 0.62*** 0.23 0.39***(0.15) (0.10) (0.16) (0.10) (0.15) (0.11)

Uncorrected statement -0.32** -0.05 -0.02 -0.03 -0.10 0.04(0.15) (0.08) (0.16) (0.08) (0.14) (0.08)

Uncorrected ⇥ Trump support 0.16 0.11 0.07 0.30* -0.10 0.03(0.22) (0.14) (0.23) (0.15) (0.22) (0.15)

Fact-check -0.31** -0.16** -0.06 -0.05 -0.21 -0.02(0.15) (0.08) (0.15) (0.08) (0.14) (0.08)

Fact-check ⇥ Trump support 0.13 0.26* 0.43* 0.30** 0.05 0.14(0.21) (0.15) (0.23) (0.15) (0.21) (0.15)

Fact-check denial -0.14 -0.13* -0.14 -0.07 -0.06 -0.05(0.14) (0.08) (0.14) (0.08) (0.14) (0.08)

Denial ⇥ Trump support 0.13 0.27* 0.69*** 0.35** 0.20 0.16(0.22) (0.14) (0.26) (0.16) (0.22) (0.15)

Denial/source derogation -0.27* -0.08 -0.06 -0.12 0.02 -0.08(0.15) (0.08) (0.15) (0.09) (0.13) (0.08)

Derogation ⇥ Trump support 0.20 0.13 0.41* 0.33** 0.07 0.19(0.22) (0.15) (0.25) (0.15) (0.22) (0.15)

Fact-check � uncorrected statementClinton supporters 0.00 -0.11 -0.04 -0.02 -0.11 -0.06

(0.15) (0.08) (0.15) (0.09) (0.14) (0.08)Trump supporters -0.03 0.05 0.32* 0.27** 0.03 0.05

(0.16) (0.12) (0.17) (0.13) (0.16) (0.13)Denial � uncorrected statementClinton supporters 0.18 -0.08 -0.12 -0.04 0.04 -0.09

(0.14) (0.08) (0.15) (0.09) (0.13) (0.08)Trump supporters 0.14 0.08 0.50** 0.24* 0.33* 0.04

(0.18) (0.11) (0.22) (0.12) (0.18) (0.12)Denial/derogation � uncorrected statementClinton supporters 0.05 -0.03 -0.05 -0.09 0.11 -0.12

(0.14) (0.09) (0.16) (0.09) (0.13) (0.08)Trump supporter 0.09 -0.01 0.30 0.27** 0.28 0.04

(0.17) (0.13) (0.20) (0.13) (0.18) (0.12)

N 990 2430 987 2430 991 2430


C-16

Table C17: Message exposure e�ects on Trump favorability (ordered probit)


Trump support 2.28*** 2.12***(0.19) (0.11)

Uncorrected statement -0.05 -0.11(0.20) (0.12)


Fact-check 0.19 -0.23*(0.19) (0.13)

Fact-check ⇥ Trump support -0.23 0.14(0.24) (0.16)

Fact-check denial 0.13 -0.09(0.19) (0.12)

Denial ⇥ Trump supportt -0.05 -0.03(0.24) (0.15)

Denial/source derogation -0.17 -0.11(0.20) (0.12)

Derogation ⇥ Trump support 0.27 0.06(0.26) (0.16)


(0.19) (0.13)Trump supporters -0.10 -0.19*


(0.19) (0.12)Trump supporters 0.01 -0.21*



(0.18) (0.12)

N 989 2430


C-17

Table C18: Message exposure e�ects on Trump favorability (all respondents)


Trump support 2.13*** 2.07***(0.17) (0.13)

Clinton support -0.36** -0.35***(0.17) (0.11)

Uncorrected statement 0.50** -0.06(0.22) (0.13)


Uncorrected ⇥ Clinton support -0.54** -0.01(0.25) (0.14)

Fact-check 0.00 0.19(0.19) (0.14)

Fact-check ⇥ Trump support -0.01 -0.34*(0.24) (0.19)

Fact-check ⇥ Clinton support 0.10 -0.31**(0.23) (0.16)

Fact-check denial 0.14 0.01(0.21) (0.13)

Fact-check denial ⇥ Trump support -0.03 -0.14(0.26) (0.18)

Fact-check denial ⇥ Clinton support -0.04 -0.06(0.25) (0.15)

Denial/source derogation 0.13 0.13(0.20) (0.14)

Denial/source derogation ⇥ Trump support -0.09 -0.24(0.27) (0.19)

Denial/source derogation ⇥ Clinton support -0.26 -0.18(0.23) (0.15)

Constant 1.79*** 1.70***(0.14) (0.10)







(0.18) (0.14)

R2 0.59 0.53N 1198 2983

* p < 0.10, ** p < 0.05, *** p < .01; OLS models with robust standard errors. Respondents are MorningConsult respondents (MC) and Amazon Mechanical Turk workers (MT).

C-18

Table C19: Message exposure e�ects on politician favorability

Obama favorability Clinton favorabilityMC MT MC MT

Trump support -2.67*** -2.43*** -2.77*** -2.26***(0.14) (0.10) (0.12) (0.09)

Uncorrected statement -0.00 -0.02 0.10 -0.01(0.12) (0.08) (0.13) (0.08)

Uncorrected ⇥ Trump support 0.04 0.15 0.04 -0.04(0.21) (0.15) (0.19) (0.13)

Fact-check -0.11 0.05 0.03 -0.09(0.12) (0.08) (0.12) (0.09)

Fact-check ⇥ Trump support -0.07 0.13 -0.04 0.20(0.19) (0.15) (0.17) (0.14)

Fact-check denial -0.08 0.02 -0.06 -0.12(0.12) (0.07) (0.14) (0.08)

Denial ⇥ Trump support -0.05 -0.01 0.14 0.07(0.20) (0.14) (0.19) (0.13)

Denial/source derogation -0.01 0.04 -0.03 -0.03(0.12) (0.08) (0.13) (0.08)

Derogation ⇥ Trump support 0.06 0.19 0.14 0.13(0.22) (0.15) (0.19) (0.14)

Constant 4.38*** 4.17*** 4.15*** 3.65***(0.08) (0.05) (0.09) (0.06)

Fact-check � uncorrected statementClinton supporters -0.11 0.07 -0.07 -0.09

(0.12) (0.08) (0.12) (0.09)Trump supporters -0.21 0.05 -0.14 0.15

(0.17) (0.14) (0.15) (0.10)Denial � uncorrected statementClinton supporters -0.08 0.03 -0.16 -0.11

(0.12) (0.08) (0.13) (0.09)Trump supporters -0.17 -0.13 -0.06 0.00

(0.18) (0.13) (0.16) (0.10)Denial/derogation � uncorrected statementClinton supporters 0.00 0.06 -0.13 -0.03

(0.12) (0.08) (0.13) (0.09)Trump supporters 0.02 0.10 -0.03 0.14

(0.20) (0.13) (0.17) (0.11)

R2 0.64 0.52 0.67 0.48N 990 2430 990 2430


C-19

Table C20: Message exposure e�ects on politician favorability (ordered probit)

Obama favorability Clinton favorabilityMC MT MC MT

Trump support -2.35*** -2.12*** -2.59*** -2.18***(0.17) (0.12) (0.17) (0.13)

Uncorrected statement -0.04 -0.02 0.09 0.01(0.15) (0.08) (0.14) (0.08)

Uncorrected ⇥ Trump support 0.06 0.16 0.10 -0.12(0.25) (0.16) (0.26) (0.19)

Fact-check -0.14 0.09 -0.02 -0.07(0.15) (0.09) (0.13) (0.08)

Fact-check ⇥ Trump support -0.06 0.11 -0.05 0.24(0.23) (0.16) (0.24) (0.18)

Fact-check denial -0.13 0.01 -0.04 -0.08(0.14) (0.08) (0.14) (0.08)

Denial ⇥ Trump support -0.02 0.02 0.19 -0.04(0.24) (0.16) (0.25) (0.19)

Denial/source derogation 0.02 0.06 -0.05 -0.02(0.15) (0.09) (0.14) (0.08)

Derogation ⇥ Trump support 0.03 0.20 0.22 0.14(0.25) (0.16) (0.26) (0.19)

Fact-check � uncorrected statementClinton supporters -0.11 0.11 -0.11 -0.08

(0.15) (0.09) (0.14) (0.08)Trump supporters -0.22 0.06 -0.27 0.28

(0.20) (0.13) (0.22) (0.17)Denial � uncorrected statementClinton supporters -0.09 0.03 -0.13 -0.09

(0.14) (0.08) (0.15) (0.08)Trump supporters -0.17 -0.11 -0.05 -0.01

(0.21) (0.13) (0.23) (0.18)Denial/derogation � uncorrected statementClinton supporters 0.06 0.08 -0.14 -0.02

(0.15) (0.09) (0.14) (0.08)Trump supporters 0.04 0.12 -0.02 0.24

(0.21) (0.13) (0.24) (0.18)

N 990 2430 990 2430


C-20

Table C21: Message exposure e�ects on unemployment beliefs (ordered probit)

(1) (2)

Trump support 0.35⇤⇤⇤ 0.36⇤⇤⇤(0.10) (0.11)

Fact-check �0.31⇤⇤⇤ �0.31⇤⇤⇤(0.09) (0.09)

Fact-check ⇥ Trump support �0.07 �0.07(0.15) (0.15)

Condition: MSNBC (post-debate coverage) 0.05(0.12)

Condition: MSNBC (no post-debate coverage) 0.03(0.12)

Condition: Fox (post-debate coverage) �0.09(0.12)

Condition: Fox (no post-debate coverage) �0.04(0.12)

Constant 2.70⇤⇤⇤ 2.71⇤⇤⇤(0.07) (0.12)


(0.09) (0.09)Trump supporters -0.38** -0.38**

(0.12) (0.12)

N 825 825

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); ordered probit models with robust standard errors (cut-points omitted). Respondents are Mechanical Turk workers who supported Hillary Clinton or Donald Trumpand were assigned to watch the first presidential debate in 2016 (reference category for Trump support isthus Clinton support). The omitted category for the orthogonal media manipulation is C-SPAN with nopost-debate coverage.

C-21

Table C22: Message exposure e�ects on perceptions of Trump performance (probit)

Trump won debate Trump evaluation Trump vote

Trump support (W1) 2.59⇤⇤⇤ 4.86⇤⇤⇤ 3.33⇤⇤⇤(0.27) (0.11) (0.27)

Fact-check 0.28 0.23 �0.01(0.29) (0.16) (0.27)

Fact-check ⇥ Trump support (W1) �0.68⇤ �0.23 �0.03(0.35) (0.17) (0.39)

Constant �2.11⇤⇤⇤ 0.89⇤⇤⇤ �1.89⇤⇤⇤(0.23) (0.11) (0.19)

Fact-check � uncorrected statementClinton supporters 0.28 0.02 -0.01

(0.29) (0.16) (0.27)Trump supporters -0.39* 0.0001 0.04

(0.12) (0.12) (0.28)

N 527 527 527

* p < 0.10, ** p < 0.05, *** p < .01 (two-sided); probit models with robust standard errors. Respondentsare Mechanical Turk workers who supported Hillary Clinton or Donald Trump and were assigned to watchthe first presidential debate in 2016 (reference category for Trump support is thus Clinton support).

C-22

Taking Fact-checks Literally But Not Seriously? The Eects ...nyhan/trump-corrections.pdfcandidates....

Documents

Transcript of Taking Fact-checks Literally But Not Seriously? The Eects ...nyhan/trump-corrections.pdfcandidates....