theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html
description
Transcript of theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html
![Page 1: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/1.jpg)
http://theveryclosedcircle.blogspot.com/2012/04/360-meta-analysis.html
![Page 2: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/2.jpg)
Special Topics in Biological Data Analysis:
Meta-analysis
Jarrett ByrnesUMass Boston Spring 2014
![Page 3: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/3.jpg)
Is one study ever definitive?
![Page 4: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/4.jpg)
![Page 5: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/5.jpg)
Huston et al. 2000 Nature
Tilman et al. 1997 Science
![Page 6: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/6.jpg)
How can we determine the answer?
![Page 7: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/7.jpg)
Narrative Reviews
• Author searches and summarizes evidence from their perspective
• Can be biased by author’s own opinion or searching abilities (if they did a search)
![Page 8: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/8.jpg)
Schoener 1983
“Rare until recently, field-experimental studies of interspecific competition now number well over 150. Competition was found in 90% of the studies and 76% of their species, indicating its pervasive importance in ecological systems.”
“In a strictly defined sample of competition studies using controlled field experiments, covering 215 species and 527 experiments competition was found in most of the studies, in somewhat more than half of the species, and in about two-fifths of the experiments.”Connell 1983
![Page 9: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/9.jpg)
Systematic Review
• Defined repeatable criteria for searching
• Clear inclusion/exclusion strategy
• Systematic coding/grading of results from studies
• Not often done outside of medicine
• But see http://www.environmentalevidencejournal.org/
http://www.campbellcollaboration.org/ http://www.cochrane.org/
![Page 10: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/10.jpg)
Vote-Counting
• Says nothing about strength of effect• Weak evidence = no evidence, biased by
experiments with large and small sample sizes
Schoener 1983
“Rare until recently, field-experimental studies of interspecific competition now number well over 150. Competition was found in 90% of the studies and 76% of their species, indicating its pervasive importance in ecological systems.”
![Page 11: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/11.jpg)
![Page 12: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/12.jpg)
One Giant P-Value
• One study contains p(data|Hypothesis) for one data set
• Why not combine them?
For example: ppnormzi, =0, =1)
• Unequal sample sizes, methodologies, assumptions, etc……
![Page 13: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/13.jpg)
Meta-analysis!
An analysis of analyses!
Systematic combination of information from published studies to derive the weight of evidence to a particular question
![Page 14: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/14.jpg)
The First Meta-Analysis
Pearson, K. (1904). Report on certain enteric fever inoculation statistics. British Medical Journal, 3, 1243-1246.
![Page 15: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/15.jpg)
1970's: Dawn of Modern Meta-analysis
• First, medical: Anti-coagulants as a treatment for heart attacks (Chalmers 1977)
• Psychotherapy (Smith and Glass 1977)• Education: Class size and outcomes (Glass and
Smith 1979)• Opposition: "an exercise in Mega-Silliness"
![Page 16: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/16.jpg)
Why would you object to meta-analysis as a tool?
![Page 17: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/17.jpg)
![Page 18: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/18.jpg)
Characteristics of Meta-Analysis
• Systematic search for studies• Strict criteria for inclusion• Common effect size
• Analysis that accounts for difference in power of different studies
Example: Hedge's D
![Page 19: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/19.jpg)
![Page 20: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/20.jpg)
![Page 21: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/21.jpg)
eigenfactor.org
Why did it take so long to get to Ecology?
![Page 22: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/22.jpg)
Example 1: Strength of Competition
Gurevitch et al. 1992
![Page 23: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/23.jpg)
Example 2: Does Diversity Influence Ecosystem Function
![Page 24: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/24.jpg)
Species richness
Standing stockorResource depletion
Monoculture, m Most diverse polyculture, p
LRAve = ln(yp / ym)LRMax = ln(yp / ymax)
93 studies 175 experiments
Cardinale et al., Cardinale et al., Nature 2006 Nature 2006
![Page 25: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/25.jpg)
Eff
ect
size
-2
0
2
Plants (nutrients or water)Herbivores (live plant tissue)Predators (live prey)Detritivores (dead organic matter)
Standing stock of t
Trophic group t (resource)
Resource depletion by t
AquaticTerrestrial
Eff
ect
size
-1
0
1
A.
C.
B.
D.
34 41 34
39
38 32 35
28
LRmLRm LRmLRm LRmLRm LRmLRmLRMaxLRAve LRAveLRMax
![Page 26: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/26.jpg)
0
1
2
3
4
Species richness, S
0 10 20 30 40 80
0
1
2
3
4
(YS / Ym)
Ymax K
0
1
2
3
Ymax K
0
1
2
3SK
SYY
max
Standing stock
Resource depletion
(YS / Ym)
![Page 27: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/27.jpg)
Waycott et al. 2009 PNAS
Example 3: Are Seagrass Beds Changing?
![Page 28: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/28.jpg)
Example 3: Are Seagrass Beds Changing?
Waycott et al. 2009 PNAS
![Page 29: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/29.jpg)
For LOTS more in EEB
• Ecology Volume 80 Issue 4 in 1999
• Ecology Volume 91 Issue 9 in 2010: Is meta-analysis a mega-mistake when it comes do diversity and productivity?
![Page 30: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/30.jpg)
So what are we doing here?
![Page 31: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/31.jpg)
Course Goals
1. Learn how to do a meta-analysis
2. DO a meta-analysis – and publish it!
3. Learn how to teach yourself new statistical skills
4. Learn how to participate in a working group type of environment
![Page 32: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/32.jpg)
Course Outline
• Now-March: Gathering and
Assessing Data
• March-April: Analytic Techniques
• April-May: (not so) Unique Challenges
![Page 33: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/33.jpg)
Class Structure
• 1-2 or 2:15 You lecture on the topic of the week
• 2:15-2:45 Code or other demo (if needed)
• 2:45-4: Working group
![Page 34: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/34.jpg)
I’m going to lecture?!
• You will encounter many new analytic techniques in the future
• This class provides an introduction to learning a new topic
• Meet with me at 1pm on Tuesday to go over your lecture for the week
• I’ll be lecturing on some of the hairy stuff & all of the code
![Page 35: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/35.jpg)
a Requirement
• We will use R for analysis• metafor package for meta-analysis• nlme and lmer for mixed models• MCMCglmm and JAGS for Bayesian methods
http://www.metafor-project.org/
![Page 36: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/36.jpg)
http://swirlstats.com/ By March 1st!
![Page 37: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/37.jpg)
http://www.nceas.ucsb.edu/handbook_of_meta-analysis
![Page 38: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/38.jpg)
Other Books!
• Introduction to Meta-Analysis. 2009. Michael Borenstein, Larry Hedges, Julian Higgins, and Hannah R. Rothstein. Wiley.
• The Handbook of Research and Synthesis and Meta-Analysis, 2nd Edition. Edited by Harris Cooper, Larry Hedges, and Jeffry Valentine. Russel Sage Foundation.
![Page 39: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/39.jpg)
What will you do in this course?
![Page 40: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/40.jpg)
Choose a Question
![Page 41: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/41.jpg)
Search the Literature, with a Protocol
http://agris.fao.org/
![Page 42: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/42.jpg)
(Re)Choose a Question
![Page 43: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/43.jpg)
What will you do in this course?
![Page 44: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/44.jpg)
Harvest Data
http://arohatgi.info/WebPlotDigitizer/app/
![Page 45: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/45.jpg)
Design a Database
![Page 46: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/46.jpg)
Generate Strict Meta-data
![Page 47: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/47.jpg)
Go back to the beginning…
• As you begin to search, you will learn about what information is out there
• Starting over (sometimes more than once) is normal!
![Page 48: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/48.jpg)
Be Circumspect: Assess Bias, Power, Sensitivity, etc.
![Page 49: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/49.jpg)
Analysis!
http://www.metafor-project.org/
![Page 50: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/50.jpg)
Publish?
![Page 51: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/51.jpg)
Is species diversity decreasing at local scales in the sea?
![Page 52: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/52.jpg)
Is species diversity decreasing at local scales in the sea?
![Page 53: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/53.jpg)
![Page 54: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/54.jpg)
![Page 55: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/55.jpg)
![Page 56: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/56.jpg)
Problems with Vellend et al.
• Sampling bias
• Low power of test misses trend
• Didn't test fundamental assumptions of conclusions
• Extrapolation from data they had to conclusions about things not measured
• Terrestrial only
![Page 57: theveryclosedcircle.blogspot/2012/04/360-meta-analysis.html](https://reader036.fdocuments.in/reader036/viewer/2022062803/56814865550346895db572ce/html5/thumbnails/57.jpg)
Is species diversity decreasing at local scales in the sea?