Talking Theory
-
Upload
colin-smith-clark -
Category
Technology
-
view
951 -
download
2
description
Transcript of Talking Theory
Talking Theory
Five Perspectives on Theory Construction in Psychology
Voodoo Correlations are Everywhere—Not Only in
NeuroscienceKlaus Fiedler, University of
Heidelberg
Voodoo Correlations in Neuroscience
• Or “Puzzlingly High Correlations,” at least (Vul, Harris, Winkielman, & Pashler, 2009)
• Selecting voxels based on thresholds leads to distorted estimates of correlations
Other Research Biases—Study design
• Sampling stimuli– Generally an intuitive process; researcher
implicitly simulates what is expected to be observed
– Random sampling of stimuli can eliminate some effects, ex. Overconfidence & trivia questions (e.g., Gigerenzer, Hoffrage, & Kleinbölting, 1991)
• Pilot testing of stimuli/tasks– X values are auditioned to correlate with Y
Other Research Biases—Selecting Variables
• Dependent variables– What is chosen to be studied affects theoretical
perceptions• Ex. TV & aggression
• Sampling levels of IVs– Magnitude of ΔY is easily manipulated by ΔX– Effect sizes are useful, but can also be misleading
Other Research Biases—Analysis
• Mediator variables– Trifling mediators (Y’) can be found if they are
correlated enough with Y• Moderator variables– Strong moderators can be overlooked or assumed in
certain paradigms, overestimating actual effects• Publishing– Weak findings will be “optimized”– Analyses will be re-run until they are “successful”– File-drawer problem
Fiedler: Conclusions• Aim for representative designs (Brunswick, 1955)
– When that fails, treat stimuli as random factors, as appropriate, to enhance external validity
• Preplanned comparisons• >2 levels of IV• Convergent validation
– Multiple methods to converge on the existence of an effect– Usually underestimates the true effect size, but can provide
a lower limit• Study boundary conditions
– Can reveal important moderators, enabling conditions– Or rather, make implicit moderators explicit
The Problem of Circularity in Evidence, Argument, and
ExplanationUlrike Hahn, Cardiff University
Question
• How are we to effectively argue in psychology?
Circular Argumentation
• Reassertion: ex. “God exists, because God exists.”– A because A
• Clearly convertible: ex. “Opium induces sleep because it has a soporific quality.”– A because A’
• Self-dependent justification: ex. “God exists because the bible says so, and the Bible is the word of God.”– A because B; B because A
Circular Argumentation• Is it invalid?
– No; on the contrary, it’s always valid• Is it persuasive?
– Sometimes yes, actually– Reassertion and convertible cases are not very persuasive; miss
the point of argumentation: belief change– Self-dependent case is frequently used in hypothesis testing: ex.
“Electrons exist because their signature effects can be seen in cloud chambers”• Modus ponens argument: “if P, then Q”; falsifiable, but not entirely
verifiable– Persuasiveness of argument rests on conditional probabilities
[What’s your P(e|~h)? Eliminating alternative hypotheses]
3 Domains of Circularity
• 1: Circular Explanations– Instead of explaining, merely re-describes the
observed phenomenon– Gigerenzer (2009) makes grand accusations
against much of psychology• Matching bias in Wason selection task is a result of
greater “transparency” – but what does that mean?
Levels of Explanation
Explanation / MechanismExplanation / Mechanism
TheoryTheory
DataData
3 Domains of Circularity• 2: Circular Analysis– Voodoo correlations—practice of pre-selecting data– Is it circular?
• No: not technically an argument• No claim nor evidence, but bias introduced to data• Presents a limit on the “interestingness” of findings
– Is it wrong?• Not really… but it’s not right, either; why bother?• Inferences based on this data are statistically invalid
– Consider model fitting• Model is constructed to accommodate data• Evaluating freedom of model (i.e., minimizing parameters) helps
to guard against false positives
3 Domains of Circularity
• 3: Circular Methods– Refers to Fiedler (2011)– Circular? Not technically, but significant concerns
about “interestingness”
Hahn: Conclusions
• Q: Is psychology a circular science?– A: No; at least, not more than any other science.
• Q: Why write a paper about circularity in psychology when there is none?– A: To be pedantic.
• Be aware at what level you are speaking/thinking about a phenomenon
Much Pain, Little Gain? Paradigm-Specific Models and Methods in
Experimental PsychologyThorstein Meiser, University of
Mannheim
Using Paradigms
• Paradigms can be useful, but they they can also hinder scientific progress
• Wason card selection task was developed to study deductive reasoning, but turns out people don’t reason deductively with it– Inappropriate for studying deductive reasoning,
but fecund paradigm for other ideas
Variables & Constructs• Manifest X & Y variables in a given study map
onto latent constructs ξ & η• Considering a given paradigm as being sui generis
can divorce the real constructs– Example: DRM paradigm purports to be an instance of
false memory, but is also an instance of category learning (e.g., Hintman, 1986)
• Need to bridge gaps in literatures, connect disparate literatures– Like Bobbie Spellman did with category induction &
persuasion (Ranganath, Spellman, & Joy-Gaba, 2010)
Meiser: Conclusions
• We should aspire to whittle down the universe of plausible explanations and constructs, not add to it
• Relying on traditional paradigms (and accounts of paradigms) can obfuscate further knowledge to be learned from them
• Paradigm-independent methodologies, such as signal detection theory, are advocated
Friends and Foes of Theory Construction in Psychological
Science: Vague Dichotomies, Unified Theories of Cognition, and the New
ExperimentalismLeonel Garcia-Marques & Mario B.
Ferreira, University of Lisbon
Dichotomous Theories
• Competing choices– ex. Modal vs. amodal conceptualization, analogical vs.
propositional representation– In reality, two out of an infinite possibility of theories
(e.g., Goodman’s (1946) concept of “grue”)• Textbook experiments—illustration of a particular
theory, leave aside complications, exceptions, etc.• Problem of theoretical mimcry—same result can
be accounted for by modification of alternate theory
Unified Theories of Cognition
• Attempts to explain cognitive architecture at a grand scope– John Anderson’s ACT & ACT-R, SOAR, etc.
• Avoid problems of more specific theories, but suffer their own problems– Task theories—how is the specific instantiation
represented?– Parameter overfitting—what can explain
everything explains nothing
The New Experimentalism• Dichotomies present the possibility of unproductive
theoretical mimics• UTCs are limited by parameter fitting and task theories• Good experiments:
– Lead to productive errors (effetive error-correcting mechanisms)
– Results are only weakly theory-dependent– Tested hypotheses are easily falsifiable– Severe tests of specific hypotheses (low false positive rate)
• The results of good experiments can effectively serve to limit the search space of possible theories
• More novel results must be accommodated by new theories, reducing theory glut
Garcia-Marques & Ferreira: Conclusions
• Generate more useful hypotheses• Have >2 levels of IV• Describe data better; ex. Exploratory analyses,
robust descriptive statistics• Use effect sizes• Test alternate accounts
Why the Cogntive Approach in Psychology Would Profit From a
Functional Approach and Vice VersaJan De Houwer, Ghent University
Extrapolation from Behavior• The cognitive perspective aims to uncover the mental
operations that are responsible for behavior• Behavioral effects are often taken as proxies for mental
events– Ex. Classical conditioning paradigm is taken to mean formation
of association between stimuli in memory• Mental construct is not a necessary component; alternate
causes could explain effects at least part of the time• Could take a tentative view; what’s the harm?
– Researchers could take construct for granted, become conditioned and shut out alternate explanations
– Theories that are invested in particular behavioral effects could be massively disrupted if they are found to be poor proxies
Questions
• Does this entail a difference in how we talk about experiments, or about how we think about them? (Both)
• Is this an argument for a stricter separation between experimentation and theorizing? (Yes)
De Houwer: Conclusions• Don’t skip steps in your explanations
– Before you propose a mental construct as the explanans of a behavioral effect, be sure you know what the environmental explanans is
– This goes double for imaging studies• Five suggestions:
– Use separate functional and mental terms– Define behavioral effects by causal impact of specific
environmental stimuli on specific aspects of behavior– Don’t treat behavioral effects as proxies of mental constructs– First describe empirical findings in functional terms; should be
abstract and general– Research should be directed to advance both mental and
functional explanations