Day 17: Evaluating Research

Daniel J. Mallinson

School of Public AffairsPenn State Harrisburgmallinson@psu.edu

PUBPL 304

Mallinson Day 17 September 29, 2017 1 / 12

Road map

Discuss finding evaluating the quality of research

Mallinson Day 17 September 29, 2017 2 / 12

Opening Question

What do you think constitutes quality research?

Mallinson Day 17 September 29, 2017 3 / 12

Research vs. Propaganda (Litman 2012)

Research

“Research (or scholarship) investigates ideas and uncoversuseful knowledge.”

Propaganda

“Propaganda (information intended to persuade) issometimes misrepresented as objective research.”“Information, especially of a biased or misleading nature,use to promote or publicize a particular political cause orpoint of view. (Link)”

Mallinson Day 17 September 29, 2017 4 / 12

Examples from American History

Figure: by Phil von Phul, Public DomainFigure: by U.S. National Library of Medicine,Public Domain

Remember what Stone said about symbolsMallinson Day 17 September 29, 2017 5 / 12

Propaganda in the Age of the Internet

Going Viral

Figure: by got insurance? Figure: by got insurance?

Source: http://www.businessinsider.com/brosurance-obamacare-ad-campaign-insurance-colorado-2013-11

Mallinson Day 17 September 29, 2017 6 / 12

How Can Research Be Used AsPropaganda?

Mallinson Day 17 September 29, 2017 7 / 12

Good Research Includes:

A well-defined question

Description of the context and existing information about anissue

Consideration of various perspectives

Presentation of evidence, with replicable data and analysis

Discussion of critical assumptions, contrary findings, andalternative interpretations

Cautious conclusions and discussion of other implications

Adequate references, including original sources, alternativeperspectives, and criticism

Mallinson Day 17 September 29, 2017 8 / 12

Propaganda Can Include:

https://www.uvm.edu/~jleonard/AGRI183/propoaganda.html

Mallinson Day 17 September 29, 2017 9 / 12

Resources for Research Evaluation

Todd Litman’s handbook provides multiple useful toolsand examples: https://www.vtpi.org/resqual.pdf

Mallinson Day 17 September 29, 2017 10 / 12

Practice

Figure: by AFP of Pennsylvania

Mallinson Day 17 September 29, 2017 11 / 12

Homework

Read over this report by the Guttmacher Institute and answer thefollowing questions:

1 Using Litman’s Research Document Evaluation Guidelines, listthree examples of desirable practices and three examples ofundesirable practices represented in the report

2 How can this report be re-written/re-framed so that it is betterin line with Litman’s desirable practices?

3 As a policy analyst, what is the value of the report the way it iscurrently written?

Due Wednesday October 4

Mallinson Day 17 September 29, 2017 12 / 12