The impact of ecosystem services knowledge on decisions

University of VermontScholarWorks @ UVM

Graduate College Dissertations and Theses Dissertations and Theses

2015

The impact of ecosystem services knowledge ondecisionsStephen Mark PosnerUniversity of Vermont

Follow this and additional works at: https://scholarworks.uvm.edu/graddis

Part of the Natural Resource Economics Commons, Natural Resources Management and PolicyCommons, and the Sustainability Commons

This Dissertation is brought to you for free and open access by the Dissertations and Theses at ScholarWorks @ UVM. It has been accepted forinclusion in Graduate College Dissertations and Theses by an authorized administrator of ScholarWorks @ UVM. For more information, please [email protected].

Recommended CitationPosner, Stephen Mark, "The impact of ecosystem services knowledge on decisions" (2015). Graduate College Dissertations and Theses.413.https://scholarworks.uvm.edu/graddis/413

https://scholarworks.uvm.edu?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages

https://scholarworks.uvm.edu/graddis?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages

https://scholarworks.uvm.edu/etds?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages

https://scholarworks.uvm.edu/graddis?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages

http://network.bepress.com/hgg/discipline/169?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages




https://scholarworks.uvm.edu/graddis/413?utm_source=scholarworks.uvm.edu%2Fgraddis%2F413&utm_medium=PDF&utm_campaign=PDFCoverPages

mailto:[email protected]

THE IMPACT OF ECOSYSTEM SERVICES KNOWLEDGE ON DECISIONS

A Dissertation Presented

by

Stephen Mark Posner

to

The Faculty of the Graduate College

of

The University of Vermont

In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

Specializing in Natural Resources

October, 2015

Defense Date: May 19, 2015

Dissertation Examination Committee:

Taylor Ricketts, Ph.D., Advisor Asim Zia, Ph.D., Chairperson

Jon Erickson, Ph.D. Walter Poleman, Ph.D.

Cynthia J. Forehand, Ph.D., Dean of the Graduate College

ABSTRACT

The need to protect diverse biological resources from ongoing development pressures is one of today’s most pressing environmental challenges. In response, “ecosystem services” has emerged as a conservation framework that links human economies and natural systems through the benefits that people receive from nature. In this dissertation, I investigate the science-policy interface of ecosystem services in order to understand the use of ecosystem service decision support tools and evaluate the pathways through which ecosystem services knowledge impacts decisions. In the first paper, I track an ecosystem service valuation project in California to evaluate how the project changes the social capacity to make conservation-oriented decisions and how decision-makers intend to use ecosystem services knowledge. In a second project, I analyze a global sample of cases and identify factors that can explain the impact of ecosystem services knowledge on decisions. I find that the perceived legitimacy of knowledge (whether it is unbiased and representative of many diverse viewpoints) is an important determinant of whether the knowledge impacts policy processes and decisions. For the third project, I focus on the global use of spatial ecosystem service models. I analyze country-level factors that are associated with use and the effect of practitioner trainings on the uptake of these decision support tools. Taken together, this research critically evaluates how ecosystem service interventions perform. The results can inform the design of boundary organizations that effectively link conservation science with policy action, and guide strategic efforts to protect, restore, and enhance ecosystem services.

ii

CITATIONS

Material from this dissertation has been submitted for review to Proceedings of the National Academy of Science on February 5, 2015 in the following form:

Posner, S.M., E. McKenzie, T. Ricketts. What explains the impact of ecosystem services knowledge on decisions? Proceedings of the National Academy of Science. Material from this dissertation has been submitted for review to Ecosystem Services on May 15, 2015 in the following form:

Posner, S.M., G. Verutes, I. Koh, D. Denu, T. Ricketts. Global use of ecosystem service models. Ecosystem Services.

iii

ACKNOWLEDGEMENTS

This work was made possible by the efforts of many smart, creative, fun, and

hardworking colleagues. I would like to thank my advisor, Dr. Taylor Ricketts, and

my committee members, Drs. Jon Erickson, Walter Poleman, and Asim Zia, for

their guidance throughout this work. I share any credit or blame for this research

with co-authors Emily McKenzie, Christy Getz, Gregory Verutes, Insu Koh, Doug

Denu, and Taylor Ricketts. Gioia Thompson and the UVM Office of Sustainability

provided valuable support and encouragement. Thanks to the staff of the

Rubenstein School of the Environment and Natural Resources (especially Carolyn

Goodwin-Kueffner) and the Gund Institute for Ecological Economics (especially

Deidre Zoll and Jill Cunningham). I appreciate the inspiration and perspective

provided by the beautiful natural environments in Vermont, California, and

Maryland where I spent time during this research. Thanks to my many friends at

UVM and beyond, and to my family for your love and care.

iv

TABLE OF CONTENTS

Page

CITATIONS .................................................................................................................... ii

ACKNOWLEDGEMENTS ............................................................................................ iii

CHAPTER 1: INTRODUCTION AND LITERATURE REVIEW ................................ 1

1.1. Big Picture Summary ............................................................................................. 1

1.2. Natural Capital and Ecosystem Services ............................................................... 3

1.2.1. From Debut to Widespread Popularity ............................................................. 4

1.2.2. Conditions for Ecosystem Services Science Success ....................................... 6

1.2.3. Is Ecosystem Services More/Better/Different Biodiversity Conservation? ..... 8

1.2.4. Limitations to a Natural Capital Approach ..................................................... 10

1.3. The Science-Policy Interface ............................................................................... 13

1.3.1. Advocacy Coalition ........................................................................................ 17

1.3.2. Complexity ..................................................................................................... 18

1.3.3. Knowledge Transfer ....................................................................................... 18

1.3.4. Boundary Organizations ................................................................................. 19

1.3.5. Knowledge Utilization .................................................................................... 21

1.4. Linking Knowledge to Action ............................................................................. 24

1.5. References ........................................................................................................... 26

CHAPTER 2: EVALUATING THE IMPACT OF ECOSYSTEM SERVICE

ASSESSMENTS ON DECISION-MAKERS ............................................................... 35

2.1. Abstract ................................................................................................................ 35

2.2. Introduction ......................................................................................................... 36

v

2.3. Methods ............................................................................................................... 39

2.3.1. Quantitative Methods ...................................................................................... 40

2.3.2. Qualitative Methods ........................................................................................ 42

2.4. Results ................................................................................................................. 43

2.4.1. Quantitative Results ........................................................................................ 43

2.4.1. Qualitative Results .......................................................................................... 44

2.5. Discussion ............................................................................................................ 48

2.6. Conclusion ........................................................................................................... 51

2.7. References ........................................................................................................... 54

2.8. Supporting Information ....................................................................................... 64

2.8.1. Direct Observation Record Form .................................................................... 64

2.8.2. Interview Questions ........................................................................................ 66

2.8.3. Ecosystem Services Survey ............................................................................ 68

CHAPTER 3: WHAT EXPLAINS THE IMPACT OF ECOSYSTEM SERVICES

KNOWLEDGE ON DECISIONS? ................................................................................ 75

3.1. Abstract ................................................................................................................ 75

3.2. Significance Statement ........................................................................................ 76

3.3. Introduction ......................................................................................................... 76

3.3.1. Ecosystem Services Knowledge Use In Decision-Making ............................ 76

3.3.2. Enabling Conditions Framework .................................................................... 78

3.3.3. Research Question And Hypotheses ............................................................... 79

3.4. Methods ............................................................................................................... 79

3.4.1. Sample Of Cases ............................................................................................. 79

3.4.2. Measuring Enabling Conditions (Explanatory Variables) .............................. 80

vi

3.4.3. Measuring Impact (Outcome Variables) ........................................................ 81

3.4.4. Analyses .......................................................................................................... 82

3.5. Results ................................................................................................................. 83

3.6. Discussion and Conclusion .................................................................................. 84

3.7. Acknowledgments ............................................................................................... 88

3.8. References ........................................................................................................... 89

3.9. Supporting Information ..................................................................................... 104

3.9.1. Survey Questions .......................................................................................... 104

3.9.2. Principal Components Analysis .................................................................... 106

3.9.3. Model Selection ............................................................................................ 107

CHAPTER 4: GLOBAL USE OF ECOSYSTEM SERVICE MODELS ................... 108

4.1. Abstract .............................................................................................................. 108

4.2. Introduction ....................................................................................................... 109

4.3. Methods ............................................................................................................. 111

4.3.1. InVEST Data ................................................................................................ 111

4.3.2. Country-level Data ........................................................................................ 113

4.3.3. Analysis ........................................................................................................ 116

4.4. Results ............................................................................................................... 118

4.5. Discussion and Conclusion ................................................................................ 120

4.6. Acknowledgements ........................................................................................... 123

4.7. References ......................................................................................................... 124

4.8. Supporting Information ..................................................................................... 137

vii

4.8.1. Ecosystem Service Models in InVEST ......................................................... 137

4.8.2. Model Use in Countries with Trainings ........................................................ 140

CHAPTER 5: CONCLUSION .................................................................................... 149

5.1. Legitimacy ......................................................................................................... 150

5.2. Capacity Building .............................................................................................. 152

5.3. Lessons Learned ................................................................................................ 153

5.4. Concluding Thoughts ........................................................................................ 156

APPENDIX A: FULL LIST OF REFERENCES ........................................................ 159

viii

LIST OF TABLES

Table Page

Table 2.1: The difference-in-differences method. The two treatment groups of Santa

Clara and Santa Cruz Counties had the ES initiative, whereas the comparison group

composed of individuals from nearby counties did not have a county-wide ES

assessment. YC, pre refers to an outcome variable for the comparison group before the

initiative. ........................................................................................................................ 61

Table 2.2: Difference-in-differences estimates of the impact of the initiative on

decision-maker outcomes. β3 is the coefficient of the interaction term from our

regression models. .......................................................................................................... 62

Table 2.3: Percentages of respondents who agreed or strongly agreed with statements

about ES in our survey questions. .................................................................................. 63

Table 3.1: Enabling conditions that facilitate the success of ecosystem service projects,

as suggested by qualitative reviews of projects. ............................................................ 97

Table 3.2: The sample of 15 global cases in which InVEST was used in a policy

decision context. Each case represents a data point in the analysis. Ruckelshaus et al.

(2013) discusses cases in more detail. ........................................................................... 98

Table 3.3: Summary of predictive factors in three broad categories of enabling

conditions. See s1 for survey questions. ...................................................................... 100

Table 3.4: Questions to assess impact (the response variable) according to the

evaluative framework. .................................................................................................. 102

ix

Table 3.5: Relationships between attributes of knowledge and policy impact. Each

element reports the F1,15 value (and corresponding p-value) for the ANOVA results. 103

Table 3.6: Model selection results with top 5 models and AICc values. Check mark ✓

indicates variables included in each model. ................................................................. 107

Table 4.1: The main hypothesized drivers of ecosystem service model use, predicted

relationships, and justifications. ................................................................................... 133

Table 4.2: Correlation results comparing model use with country level variables. Bold

rows indicate positive and significant correlations. N is the number of countries for

which those data are available. WE calculated the first principal component of

Governance and Biodiversity variables to capture >90% of the variance in the

underlying variables. “Governance PC” includes government effectiveness, regulatory

quality, and control of corruption (for pairwise correlations, rho=0.91, 0.95, and 0.87);

“Biodiversity PC” includes GEF Benefits Index of Biodiversity and mammals

(rho=0.77). ** p < 0.01; * p < 0.05; + p < 0.1 ............................................................ 134

Table 4.3: Model selection results for the top 10 statistical models by AICc value.

Statistical models are listed by row in rank order (the first has the lowest AICc value

corresponding to best fit) with a check mark for variables that are included in the

statistical model. Only the 8 variables that contained data for all countries were

included in model selection. ........................................................................................ 135

Table 4.4: Comparison of ES model use before and after trainings. Change factor is the

ratio of use in 13-week periods after/before training. .................................................. 136

Table 4.5: Ecosystem service types for InVEST models. ............................................ 137

x

LIST OF FIGURES

Figure Page

Figure 1.1: A Natural Capital Project framework for how ecosystem services can be

integrated into decision-making. Based on Daily et al. (2009). ....................................... 7

Figure 2.1: Framework for how ES knowledge leads to impact. Five different pathways

to impact are represented as columns with increasing impact the further one moves to

the right. Our study focuses mainly on pathways 2 and 3. Based on Ruckelshaus et al.

(2013) and modified by Posner et al. (in review). ......................................................... 59

Figure 2.2: Differences in outcomes between the comparison group and the two

treatment groups of Santa Clara and Santa Cruz Counties. Y-axis is mean Likert score.

Data points are for mean outcomes with standard errors before and after the initiative

for the three groups. ....................................................................................................... 60

Figure 3.1: Evaluative framework for how ES knowledge leads to impact. Each column

represents a pathway to different forms of impact, with increasing levels of impact

going from left to right. Columns 2, 3, and 4 (a and b) were the basis for our

measurement of impact in each of the cases. ................................................................. 94

Figure 3.2: Effects of knowledge attributes on policy impact. Bars depict mean levels

of impact for different levels of salience, credibility, and legitimacy in the 15 cases. No

standard error bar indicates 1 case in that category. ** p < 0.01; * p < 0.05; + p < 0.1 95

Figure 3.3: Effects of multiple attributes on policy impact. Points and whiskers

represent model average coefficients with 95% confidence intervals. Predictor variables

are explained in Table 3. The first principal components were used for three groups of

highly correlated predictors (Spearman rank correlation coefficient > 0.80): Interact1

xi

(level of joint production as estimated by interactions in person or by phone/email),

prep1 (decision-making power and stakeholder representation), and CC1 (institutional

capacities to measure baseline ES and human activities, monitor changes to ES and

human activities, and implement policy). ...................................................................... 96

Figure 3.4: PCA bi-plots. We reduced the dataset by combining the following highly

correlated variables. prep: power and representation (correlation coefficient = 0.74);

interact: in-person interactions and electronic interactions (correlation coefficient =

0.72); CC: contextual conditions as institutional capacity to measure baseline

conditions, monitor changes, and implement policies (paired correlation coefficients of

0.70, 0.79, and 0.80). In subsequent analyses, we used PC1 for explanatory variables

(or the inverse of PC1, so that increases in the principal component correlated to

increases in the underlying variables). ......................................................................... 106

Figure 4.1: Growth in the use of InVEST models over time. Model use occurred in 102

different countries with 44% of all use occurring in the U.S. For non-U.S. countries,

there were 14,301 model runs with 43% of use occurring in 5 countries: the UK (1554),

Germany (1491), China (1209), France (1074), and Colombia (780). ........................ 128

Figure 4.2: Types of ecosystem service models used. 46% of all model use is for

regulating services. “Rios” focuses on freshwater provisioning services but includes

other service types as well. .......................................................................................... 129

Figure 4.3: Comparison of average use in non-U.S. countries with and without

trainings. Error bars depict standard error. There was a significant difference in the

average use for countries with (mean = 280, sd = 81) and countries without (mean =

107, sd = 29) trainings ( t(22.6) = 2.0 , p= 0.05). ........................................................ 130

Figure 4.4: Example of a training in the UK. Model use spikes during the training. We

compared the number of models runs and average weekly use for one 13-week period

before and two 13-week periods after trainings. .......................................................... 131

xii

Figure 4.5: The positive effect of trainings on model usage across 9 cases. “Before” is

average weekly model use and standard error for a 13-week period (approximately 90

days) before a training. “After 1” is for a 13-week period after a training and “After 2”

is for the following 13-week period. ............................................................................ 132

Figure 4.6: Argentina 2-day training 9/12/2013 .......................................................... 140

Figure 4.7: Cambodia 3-day training 6/17/2013 .......................................................... 141

Figure 4.8: Canada 2-day training 2/04/2013 .............................................................. 142

Figure 4.9: Chile 3-day training 9/09/2013 .................................................................. 143

Figure 4.10: Korea 1-day training 9/11/2012 .............................................................. 144

Figure 4.11: Peru 3-day training 5/27/2013 ................................................................. 145

Figure 4.12: Spain 2-day training 11/18/2013 ............................................................. 146

Figure 4.13: UK1 3-day training 10/15/2013 and UK2 1-day training 3/07/2013 ...... 148

1

CHAPTER 1: INTRODUCTION AND LITERATURE REVIEW

1.1. Big Picture Summary

The worlds in which scientists conduct research and policymakers craft policy do

not always coexist. Scientists and policymakers can exist in two separate communities,

with limited interactions and infrequent exchanges of information that lead to uninformed

or ineffectual policy decisions. Without an effective interface between science and

policy, good science may not have an impact on real-world decisions. This is a problem

because past decisions based on a partial scientific understanding of how ecosystems

benefit people have undervalued ecosystems and contributed to environmental decline.

We can improve the environmental and long-term economic outcomes of policy

decisions by doing a better job incorporating scientific information about the state or

trend of environmental conditions and how development activities alter the flow of

ecosystem services (the benefits that nature provides to people and that support human

well-being).

When scientists go about their research without understanding how the knowledge

they are producing might be used, the research occurs in a vacuum with limited

connection to real policy decisions. This is especially important to consider given the

urgency of global environmental problems today. Human activity now impacts the entire

planet in drastic ways, leading to the claim that we have entered a period of time

dominated by human-caused influence on the environment called the Anthropocene. One

negative impact resulting from this extensive human activity is the widespread loss of

biodiversity and ecosystem services (Millennium Ecosystem Assessment, 2005). This

2

eroding of the ecological support system of the planet threatens the stability and

sustainability of economic systems (Kates et al., 2001).

If scientists want to make it easier for certain kinds of information to be

incorporated into decisions, then they need a firm understanding of what makes

knowledge useful to decision-makers. There is anecdotal evidence for how ecosystem

services information has an impact on decisions, but there has not yet been much research

about how or why this occurs (Laurans & Mermet, 2014; Laurans et al., 2013; Liu et al.,

2010; McKenzie et al., 2014). Understanding what makes for effective interactions

between scientists and policymakers will benefit those who produce ecosystem service

knowledge as well as those who use it in decision-making.

A more effective science policy interface is needed for the protection,

conservation, and restoration of ecosystem services (Nesshover et al., 2013). As coupled

social-ecological systems undergo changes, we need an adaptive balance between real

human development needs and the ongoing maintenance and health of Earth’s life

support systems (Weichselgartner & Kasperson, 2010). This requires policy informed by

a sound scientific basis. At the same time, it calls for science that is use-inspired, policy-

relevant, and defined according to the problems at hand.

The theory and practice of ecosystem service analysis provides a fertile landscape

for work at the science-policy interface. Much work has been done on understanding the

science of ecosystem services and how ecosystem structure can lead to ecosystem

function with measurable benefits to people (Kremen, 2005). Other areas of research

have focused on policy mechanisms, and how to mainstream ecosystem service concepts

3

into policy discussions (Daily & Matson, 2008; Fisher et al., 2008; Fisher, Turner, &

Morling, 2009).

This doctoral dissertation investigates how ecosystem services science is

incorporated into decision-making and what makes the production, distribution, and

utilization of ecosystem services knowledge effective. My research uncovers how the use

of ecosystem service science enhances the ability of decision-makers to understand 1) the

benefits that people receive from nature and 2) how those benefits are likely to be

affected by alternative decisions. Research objectives include to evaluate how ecosystem

service interventions affect decision-makers and to identify enabling conditions for

effectively translating ecosystem services knowledge into action. The results can inform

how to be more effective in strategic efforts to protect and enhance ecosystem services,

and guide the design of knowledge systems that effectively link conservation science to

policy action.

The following overarching questions drive this dissertation research:

1) How do decision-makers use ecosystem service knowledge?

2) What is the impact of ecosystem service knowledge on decisions?

3) Why does ecosystem service knowledge have an impact? What factors (i.e. attributes

of the science, the decision context, and the policy process) can explain the impact of

knowledge?

1.2. Natural Capital and Ecosystem Services

This section describes the emergence of ecosystem services as a framework for

the conservation of nature and biodiversity. I describe some of the ways this concept has

4

been put to use in various contexts, and discuss theoretical critiques and debates about the

risks of embracing such an anthropocentric and economically based framework.

1.2.1. From Debut to Widespread Popularity

The concepts that underpin “natural capital” and “ecosystems services” emerged

as early as the 1920s (Costanza & Kubiszewski, 2011). E.F. Schumacher in his popular

book Small is Beautiful (1973) framed many of the general ideas and invoked the term

“natural capital.” In the peer-reviewed scientific literature, Ehrlich & Mooney (1983) are

credited with the first use of the term “ecosystem services”, while a paper by Costanza &

Daly (1992) is one of the first to explicitly mention of the term “natural capital.”

Subsequent contributions had significant influence on mainstreaming ecosystem service

concepts, in particular: a research article by Costanza et al. (1997) that estimated the

value of the world’s ecosystem services; work by Lovins, Lovins, & Hawken (1999) that

described the implications of natural capitalism for economies, industries, and businesses;

and books by Daily (1997) and Daily & Ellison (2002) that spoke of the growing

recognition of ecosystems as capital assets, with diverse examples of how people have

benefited from a natural capital approach to management and conservation.

More recently, the notion of ecosystem services has gained wider use among

scientific, business, and political communities. The Millennium Ecosystem Assessment

(2005) defined clear as well as less tangible, though still important, ecosystem service

categories in terms of the benefits that humans obtain from nature. This global

assessment describes four main categories of services. Provisioning services support the

ways that people directly extract benefits from nature (i.e. food, timber, clean water).

Regulating services moderate natural phenomena to people’s benefit (i.e. carbon

5

sequestration, flood control, decomposition). Supporting services provide a more

fundamental basis for the Earth sustaining living things (i.e. photosynthesis and primary

production, nutrient cycling). Non-material cultural services contribute to the cultural and

social development of people (i.e. recreation, spiritual inspiration). Several other

ecosystem service frameworks exist, for example The Economics of Ecosystems and

Biodiversity study (TEEB), which modifies the MEA framework to more explicitly

account for the role of biodiversity (Bishop, 2012; Kumar et al., 2010).

Today, the concepts of natural capital and ecosystem services have moved beyond

scientific research. In the fields of law and policy, (Ruhl, Kraft, & Lant, 2007) describe

detailed case studies that highlight how land use is a driving factor in the creation of

ecosystem services, and how trade-offs between ecosystem goods, ecosystem services,

and other desirable aims require that society choose what it wants from a set of possible

future scenarios. In the business world, organizations such as Businesses for Social

Responsibility (BSR), the World Business Council for Sustainable Development

(WBCSD), and the World Resources Institute (WRI) have all developed approaches to

measuring and assessing the ways that companies interact with ecosystem services, often

through valuing the services in monetary terms.

Governments have also demonstrated interest in the ecosystem services concept,

and in some cases have undergone national-level studies such as the UK National

Ecosystem Assessment (Bateman et al., 2014; Watson, 2012). Other federal governments

have integrated natural capital ideas and approaches into agency frameworks, as in the

USDA Office of Environmental Services, the USGS public domain tool Social Values for

Ecosystem Services (SolVES), and the EPA’s unit on Ecosystem Services Research.

6

Meanwhile, The Economics of Ecosystems and Biodiversity (TEEB) and the

Intergovernmental Platform on Biodiversity and Ecosystem Services (IPBES) indicate

that natural capital continues to gain traction in international planning and governance

settings as well.

1.2.2. Conditions for Ecosystem Services Science Success

A report on The State of Ecosystem Services by Cox and Searle (2009) found that

fragmented knowledge on ecosystem services indicates a need for replicable and

standardized projects. The authors of the report suggest the following four conditions for

ecosystem service conservation success:

• clear science (about ecosystem services, interactions between services, and how

proposed actions may affect services),

• defined benefits to people (with quantifiable values, and providing links from

ecosystems to human well-being),

• a confined system (with clearly identified stewards, perpetrators of negative

impacts, and service beneficiaries), and

• good governance (in terms of clearly defined ownership or tenure, a legal

system, enforcement capacity, monitoring of impacts, and a functioning infrastructure to

support and enable projects).

Several efforts have evolved in the process of establishing standardized scientific

approaches to modeling and measuring ecosystem services, from early efforts like the

Global Unified Meta-model of the Biosphere (Boumans et al., 2002) to more recent

projects that incorporate machine learning and spatial ecosystem service flows such as

ARtificial Intelligence for Ecosystem Services (ARIES) (Bagstad et al., 2013; Villa et al.,

7

2014). The Natural Capital Project with its Integrated Valuation of Environmental

Services and Trade-offs (InVEST) family of tools, has demonstrated success in terms of

replicable and standardized projects across a variety of political and cultural contexts

(Tallis & Kareiva, 2009). Their framework for integrating natural capital into decision-

making shows how biophysical models of ecosystems and the services they produce are

combined with economic and cultural models to describe ecosystem service values. This

information is then mediated through institutions, and decisions are made with impacts

on ecosystems and ecosystem services (Figure 1.1).

Figure 1.1: A Natural Capital Project framework for how ecosystem services can be integrated into decision-making. Based on Daily et al. (2009).

Calls have been issued for ecosystem service research that generates new policies,

incentives, and institutions on a large scale (Daily & Matson, 2008; Daily et al., 2009).

Kinzig et al. (2011) and Wunder, Engel, & Pagiola (2008) describe factors that influence

the effectiveness of various policy approaches, including the cost of alternative built

infrastructure and enforceable actions to ensure service provision. McKenzie and Irwin

(2011) present a table of “enabling conditions for policy instruments” regarding

8

ecosystem service and biophysical attributes, producers of ecosystem services,

beneficiaries, and costs and benefits. Wunder et al. (2008) compare payment for

environmental services (PES) programs and find that more careful analysis is needed to

understand how such policy approaches work. This dissertation builds on and extends

this work by focusing on the enabling conditions for ecosystem service science to have

influence.

1.2.3. Is Ecosystem Services More/Better/Different Biodiversity Conservation?

But does the concept of ecosystem services add anything more, better, or different

to biodiversity conservation efforts? McCauley (2006) argues that ecosystem services is a

new means to the same end goal of biodiversity conservation. He points out that

ecosystem services can at times be “useful bargaining chips in specific conservation

plans,” but he urges a return to the primary mission of protecting nature for it’s own sake

and not necessarily to guarantee a profit. According to this view, ecosystem services may

not provide anything other than a distraction from the previously established end goal of

conserving biodiversity. And while an approach based on ecosystem services can be a

means to achieving biodiversity conservation, there are situations in which it will not,

such as increasing food provisioning services by replacing natural vegetation with crops,

or maintaining a supply of freshwater by building a dam that impacts fish and countless

other species (Reyers et al., 2012).

The viewpoint that ecosystem services is just a new means to the same end often

compares the additional influence that can be achieved by an ecosystem service approach

with the potential risks and drawbacks. Do ecosystem services detract from the moral

arguments underpinning biodiversity conservation? Do they risk reducing people’s

9

relationships with nature to purely economic terms? Supporters of this viewpoint raise

these important questions and voice skepticism that ecosystem services can continue to

live up to the expectations posed by significant recent interest and support from

governments, conservation organizations, and academics.

Others feel that maintaining ecosystem services is a valid end in itself. Costanza

et al. (1997) present a perspective based strongly on the protection of necessary

ecosystem services as a worthy end goal. Maintaining ecosystem services, they argue, is

the goal because these services are the life-support system for humans and the

foundations of our economies. A focus on ecosystem services could ensure the

sustainability of human welfare on the planet. The authors of this study consider “natural

capital as essential to human welfare. Zero natural capital implies zero human welfare

because it is not feasible to substitute, in total, purely ‘non-natural’ capital for natural

capital.” Thus, the effort to conserve and protect ecosystem services needs to be a

focused endeavor in and of itself.

Balmford et al. (2002) also frame the maintenance of ecosystem services as an

end goal. They focus on the economic benefits of conservation efforts and how in most

cases, when considering a full range of factors, the benefits of conservation outweigh the

benefits of alternative decisions such as forest conversion. By maintaining ecosystem

service flows first and foremost, we can ensure the sustainability of human communities

that depend on these services. These arguments are based on connections between

ecosystem services and human well-being, adopting an anthropocentric view of the

importance of nature’s benefits to humans.

10

Lastly, there is the view that ecosystem services and biodiversity conservation are

parallel goals to be done simultaneously but independently. A paper by Chan et al. (2006)

supports this view and explores how well biodiversity and ecosystem services align

within a conservation planning framework. They find that it is important to pursue the

two goals at the same time. Targeting only biodiversity or ecosystem service goals can

lead to trade-offs in which pursuit of one objective occurs at the expense of the other.

Broadening conservation goals to include both biodiversity and ecosystem

services can “sustain critical services, open new revenue streams, and make conservation

broad based and commonplace” (Chan et al., 2006). Indeed, many others, including

Goldman & Tallis (2009) think that parallel goals have the potential to attract additional

funding, engage a broader set of stakeholders, and establish conservation as more

mainstream practice. Reyers et al. (2012) also conclude that the conservation community

needs to move beyond the either/or debate to embrace both intrinsic biodiversity values

and instrumental ecosystem service values in dealing with the problem of biodiversity

loss.

1.2.4. Limitations to a Natural Capital Approach

Many within the conservation community and beyond voice concerns about the

rapid rise in popularity of the ecosystem services concept. Critics point out potential

conflict in situations that have unavoidable trade-offs between biodiversity and

ecosystem services. Reyers et al. (2012) discuss the potential for biodiversity

conservation to decrease the provision of ecosystem services, as well as how

management designed to increase ecosystem services could have negative impacts on

biodiversity. Others point out how emphasizing ecosystem services can have limited or

11

no effect on actual land management decisions if technological or other economic

conditions are stronger drivers of behavior (Ghazoul, 2007).

Beyond the ecological and human behavior outcomes of ecosystem service

applications, there are serious concerns rooted in ethical considerations, including

changes in motivations for conserving nature (or not), the effect of commodification, and

equity implications (Luck et al., 2012; Muradian et al., 2013). For example, Redford &

Adams (2009) describe some of the potential problems with payment for ecosystem

services schemes that arise when economic pricing of services from a landscape

overshadow the intrinsic value that could motivate conservation. Market-based

mechanisms can be a risky diversion from what some believe to be a more important

ethical motivation to conserve nature (McCauley, 2006).

Commodification presents other issues as well. Ecosystem services based on

social values can be difficult to price, such as the spiritual benefits of a forest.

Commodification involves narrowing the complexity of ecosystems, and this can deny

multiplicity in values (Kosoy and Corbera, 2010). Ecosystem services applications do not

always acknowledge the multiple, potentially incommensurable values for nature that

different stakeholder groups hold. Trade-offs in implementation across diverse socio-

ecological contexts may require more deliberative methods for uncovering multiple value

systems (Kosoy and Corbera, 2010). When the process of valuation compresses

information about ecosystem attributes into a single monetary metric, it creates an

unavoidable loss of information about ecosystems (Vatn and Bromley, 1994). This loss of

information associated with pricing and valuation is nonrandom, thus valuation does not

12

reliably generate conclusive results about the “best” individual or collective welfare-

enhancing decisions with regards to nature.

Equity considerations further limit the appropriate use of an ecosystem services

approach. Ecosystem services can propagate or reinforce asymmetries in the distribution

of costs and benefits among different groups. The distribution of some essential

ecosystem goods and services (i.e. food) is unjust because overreliance on competitive

market-based forces can lead to speculation, price instability, and inefficient allocation of

essential resources (Farley et al., 2014). In protected areas and rural communities in

particular, concentrated decision-making power leads to ecosystem service initiatives

based on the needs and desires of some groups at the expense of others (for example,

service providers can directly receive compensation more than service users, who may be

excluded from receiving benefits of development) (Corbera et al., 2007).

Those in positions of power often define the legitimacy of ecosystem services

knowledge. Different knowledge and value systems present in many ecosystem service

contexts require deliberation and collective action to emerge from reciprocal interactions

over time (Kolinjivadi et al., 2015). But this time-intensive process does not always

occur. The complexity of power relations and institutional settings for ecosystem services

projects can lead to a form of knowledge imperialism (Hardy and Patterson, 2012).

Ecosystem services can become a form of projecting power onto developing world

(Muradian et al., 2010). Indigenous groups may have long-established ways to sustain the

forests upon which they depend. But ecosystem services may risk disregarding such

cultural practices, along with traditional ways of knowing and valuing nature (Ernstson

and Sorlin, 2013). The effectiveness of an approach based on ecosystem services depends

13

in part on the political forces at play and the structure of institutions for mediating

information about values, expert and traditional sources of knowledge, and the

distribution of costs and benefits across different populations (Muradian et al., 2013).

It is important to recognize that ecosystem services is only one way to understand

nature and the relationships that people have with the environment (Luck et al., 2012).

Researchers must consider the unique context and place of an application before deciding

whether an ecosystem services approach is warranted. The degree to which ecosystem

services conservation becomes a distinct goal depends on these specific considerations

and limitations, as well as the general ways science interacts with policy.

1.3. The Science-Policy Interface

This section presents a review of the “science-policy interface” literature. It

describes theories of how policy is formulated, how knowledge is used by policymakers,

and the importance of spanning the boundary between science and policy worlds. It ends

with theories of how knowledge links to action that provide intellectual underpinnings for

this dissertation.

The science-policy interface refers to the intersection of scientific assessment and

policy decision-making (Watson, 2005). Van den Hove (2007) defines these interfaces as

“social processes which encompass relations between scientists and other actors in the

policy process, and which allow for exchanges, co-evolution, and joint construction of

knowledge with the aim of enriching decision-making.” Research in this area is

concerned with the role and use of scientific evidence in policy formulation, the ways

14

that scientific research efforts interact with policy processes, and the characteristics of

science that can lead to effective policy.

The conceptual separation of science and policy reflects how, in a conventional

sense, scientists and policy-makers have conducted their work in largely separate spheres.

The interface exists as a conceptual boundary between two separate communities

(Kerkhoff & Lebel, 2006). On one side of the science-policy interface are scientific

communities that investigate, organize, and report knowledge. On the other side, there are

policy communities that make far-reaching decisions based on knowledge, values and

beliefs, and perceived priorities. Huitema & Turnhout (2009) describe how the cultures of

science and policy are often perceived to be very different, from the ways in which they

communicate, to the interests they pursue, to their fundamental levels of objectivity or

subjectivity.

The boundary between science and policy can range from sharp and impermeable

to more blurred and porous (Guston, 2001). The interface is often mediated by

institutions called boundary organizations that determine processes of decision-making

among groups of stakeholders with diverse values. One major disagreement within the

literature has to do with whether it is better to attempt to blur the lines to overcome the

differences between communities of scientists and policymakers (and if so, how best to

do so), or rather to maintain firm boundaries between the worlds of science and policy so

that each can function independently without undue interference by the other.

For example, Leiss (2000) argues strongly for separate and independent scientific

assessments of policy options. He feels that scientific research agendas should not be

constrained by political conflicts, and that a critical role for policy-makers is to manage

15

risk based on information provided by independent science. If a government becomes

overly involved in conducting scientific research itself, then the public could perceive it

as an “interested party” and potentially question the legitimacy of its scientific efforts.

Leiss (2000) rightly points out how problems of risk management have become

increasingly important as human population growth and pressure on the environment

have led to natural resource issues in which we no longer have the same margin of safety

between sustainable and unsustainable resource use. Thus, the process of developing

policy must include managing scientific information to compare relative risks of options.

He stresses, though, that policy choices cannot be dictated by scientific study – that a

firm boundary between science and policy can ensure risks are properly managed based

on objective scientific truths. This would be consistent with the “pure scientist” role

defined by Pielke (2007), in which distance between scientists and policy-makers helps

maintain objectivity of the science.

Others disagree and argue for a more co-mingled process of joint production of

research between scientists and policy-makers (Watson, 2005; Watson, 2012; Driscoll,

Lambert, & Weathers, 2011; Cash et al., 2003; Lemos & Morehouse, 2005). In a co-

production scenario, scientific experts and decision-makers come together to

collaboratively produce research outputs that can be used in formulating policy. Some

claim that it is virtually impossible for researchers to remain insulated from the world of

politics when there is strong political debate about the nature of the problems being

addressed (Hirsch & Luzadis, 2013; Huitema & Turnhout, 2009). For this reason, an

“honest broker” role for scientists may be most appropriate, in which researchers expand

the range of policy options by communicating with policy-makers and providing

16

scientific knowledge about options and their implications (Pielke, 2007). If scientists

were to reduce the range of policy options and align with particular policies, they would

be characterized more as “issue advocates” (such as the potential shift within the IPBES

from more of an investigative function to an advocacy one described by Perrings et al.,

2011). Scientists could also play an “arbiter” role if they engage with policymakers to

provide expert scientific opinion but steer clear of the responsibility to make decisions

about which policies to adopt (Pielke, 2007).

Even while disagreement exists about the extent to which the worlds of science

and policy interact, there is general agreement about important factors that constitute

effective science with regards to its interaction with policy. An open and transparent

peer-review process for science is understood to be an important element for maintaining

trust of scientific knowledge (Watson, 2005; Watson, 2012; Cash et al., 2003; Reid et al.,

2009). Jones, Fischhoff, & Lach (1999) represent another widely-held view in the

science-policy literature: that research results should be relevant to current issues of

interest if they are to be useful.

Ultimately, it is unrealistic to expect an impermeable boundary be maintained

between science and policy. At the same time, it is threatening to the integrity and

credibility of science for scientists to be directly involved in advocating for political

issues. In striking a balance, scientists must strive for an “honest broker” role in which

they acknowledge and interact with the world of policy, but conduct research that

expands the scope of choices to policymakers (rather than narrows it), and systematically

evaluates the implications of the choices. In this way, the science-policy interface can be

17

characterized by a flow of information more about policy options but less about

recommendations, with limited value-based judgments attached to any particular option.

The following topics from the academic literature represent different theoretical

lenses through which researchers conceive of the science-policy interface and the

processes through which knowledge and policy action are connected. The discussion

provides general background and informs the theory of change and methods used in this

doctoral research.

1.3.1. Advocacy Coalition

Sabatier & Jenkins-Smith (1993) have advanced an advocacy coalition framework

as one approach to understanding how science is translated into policy. This framework

acknowledges the role of active attempts to influence government policy decisions

(advocacy) and focuses on the interactions of actors and institutions (coalitions) in this

process (Weible et al., 2011). Sabatier and Jenkins-Smith point out how policies and

programs can be understood as “sets of value priorities and causal assumptions about

how to realize them” (Sabatier & Jenkins-Smith, 1993). Thus learning becomes a

centrally important process as a mechanism for altering a belief system.

Enabling conditions for policy-oriented learning may include: an intermediate

level of conflict between competing belief systems (too little conflict does not provide the

tension needed to catalyze action while too much leads to irreconcilably incompatible

worldviews); analytical tractability (accepted quantitative data and theory about natural

systems that can convince the need for policy); and a forum that is professional and

conducive to different authoritative figures vetting their perspectives.

18

1.3.2. Complexity

Elliott & Kiel (1999) and Geyer & Rihani (2010) present a foundation for social

science and public policy based on complexity. They describe complexity in relation to

modern and post-modern views on science and policy. The modernist approach suggests

that knowledge is essentially order. A social system that increases knowledge also

increases order and thus predictability and potential for control and management. The

world is objectively knowable and more knowledge should be a policy goal. On the other

hand, in the post-modern view, the world is relational and there is no way to accurately

know the “right” policy because all policies can be understood from different, sometimes

mutually incompatible perspectives – policy is always contested and is a response to an

unpredictable world.

The complexity worldview appreciates that science can be used to know more,

but there are always limits to knowledge, and the world is dynamic and full of

uncertainty. Rather than achieving some end goal, policy needs to steadily learn to adapt

to changing conditions. More knowledge is useful, but not necessarily effective due to the

emergence of new issues from the complex interactions of many, dynamic parts of any

system. According to this framework, flexibility and adaptive management in policy

processes are key attributes to pursue. Berkes (2009) further suggest that adaptive

resource management and adaptive governance, in which science and policy are co-

mingled, are critical to the resilience of social-ecological systems.

1.3.3. Knowledge Transfer

19

A static model of how science translates into policy involves the transmission of

scientific facts and results from scientists to policymakers. Policymakers then use the

scientific information they receive to make decisions and formulate policy (Reynolds,

2007). Knowledge transfer suggests that knowledge exists as a tidy bundle that can be

delivered to the appropriate user in a well-defined time frame and with a clear sequence

of discrete steps toward a policy outcome. In reality, a strictly linear conceptual model of

science transferring knowledge to policy does not fit well. Rather, the boundary between

science and policy may be fuzzy, with knowledge use as more of a dynamic process than

an event that takes place at a moment in time (Miller, Jasanoff, & Long, 1997). A

science-policy interface can involve interactions that are iterative, more long-term, and

evolve over time as relationships between scientist, policy-maker, and citizens, giving

rise to the idea that organizations can be boundary-spanning between the worlds of

science and policy (Reynolds, 2007).

1.3.4. Boundary Organizations

Boundary organizations serve to bridge the science and policy worlds. White,

Corley, & White (2013) describe three main aspects of boundary organizations. First,

they facilitate the creation of boundary objects that serve as mutually acknowledged and

used artifacts between science and nonscience communities (i.e. computer models, maps,

patents). Second, they include scientist and decision-maker participants mediated by

professionals. Third, they are accountable in different ways to both the political and

scientific systems.

20

Boundary organizations can facilitate the joint production of ecosystem service

knowledge, which involves knowledge generators and users working together to answer

management and policy questions. When decision-makers are integrated as part of the

scientific enterprise, they can go beyond simply interacting with scientific knowledge –

they can become connected to the scientific process as co-investigators. As a result, they

may come to develop deeper levels of trust of the knowledge outputs and the underlying

science upon which it is built.

Scientific research processes that are in collaboration with decision-makers have

high potential for influencing decisions (Rowe & Lee, 2012). Collaborative processes

that span the boundary provide a forum for scientists and policy-makers to define and

answer questions together. Karl, Susskind, & Wallace (2007) and Andrews (2002) stress

“joint fact finding” and two-way dialogues as key processes for an effective interface of

science and policy. Continuous, iterative processes that involve both scientists and

policymakers can create mutual understanding of problems and increased likelihood of

ecosystem service information being used in crafting solutions (Reid et al., 2009;

Nesshover et al., 2013; Ruckelshaus et al., 2013).

Numerous researchers have found trust to be an important part of effective

participatory science-to-policy processes (Dalton, 2005; Reed, 2008; Voinov & Gaddis,

2008). Strong social capital can facilitate cooperation among scientists and policymakers

by enhancing: trust relationships; reciprocity and exchange; common rules and norms;

and connectedness of individuals in networks and groups (Pretty & Smith, 2004; Pretty &

Ward, 2001). Boundary organizations can cultivate trust that helps actors relate quickly

21

and easily, increasing the chances that any improvements or successes achieved last

beyond the timeframe of the project (Reid et al., 2009).

1.3.5. Knowledge Utilization

The knowledge utilization and agenda setting literature focus explicitly on the

role of scientific knowledge in policy development. Hinkel (2011) considers the

utilization of knowledge in indicators that can be used to monitor policy performance

over time or to justify action/inaction. There is general agreement in the literature that

policymakers can utilize scientific knowledge in distinct ways. However, the terms used

to describe different types of knowledge utilization may differ. Weible, Pattison, &

Sabatier (2010) describe learning, political, and instrumental functions of scientific

information in policy, while Laurans et al. (2013) describe similar functions in terms of

informative, technical, and decisive uses. Of the different uses of scientific environmental

knowledge described in the literature, there is overlap and general agreement about three

clear ways that policymakers utilize knowledge: to build support for their agendas, learn

new information about policy problems and the range of solutions, and/or make specific

decisions based on scientific information. McKenzie et al. (2014) expound these three

primary ways that decision-makers utilize knowledge, based on an extensive review of

the science policy interface literature (Amara, Ouimet, & Landry, 2004; Haas, 1992;

Kingdon, 2011; Lavis et al., 2003; Lee, 1993; Lester, 1993; Oh, 1996; Pannell, 2004;

Roux et al., 2006; Tomich et al., 2004; van Kerkhoff, 2006):

1) Instrumental / problem-solving use – where decision-making or

problem-solving directly uses knowledge from findings and recommendations.

22

2) Conceptual / indirect use – where ideas and concepts filter into

thinking, debate and dialogue; knowledge provides new ideas, theories, interpretations or

hypotheses about the issues and facts related to decision-making context, eventually

catalyzing change in actions.

3) Symbolic / strategic use – where practitioners and decision-makers use

knowledge to legitimate their views.

Groot et al., (2010) find that an ecosystem service approach has changed the

terms of discussion and been most effective in the ‘conceptual enlightenment’ function of

influencing decisions. Hirsch (2011) further indicates the importance of indirect use of

knowledge, highlighting how trade-off thinking shapes language in policy dialogues and

ideas about complex conservation issues. While all three uses can be present in a given

policy or issue context, decisions-makers may utilize ecosystem services knowledge most

effectively through the framing of discussions and debates. Weiss (1977) and Mitchell et

al., (2004) claim that such indirect influence of information on policy development

occurs by affecting who participates in discussions, how discussions are framed, and

what issues receive attention and visibility in discussions.

The science policy interface literature on knowledge utilization also sheds light on

key factors that influence the uptake of science. Landry, Amara, & Lamari (2001)

examine the case of social science research in Canada and find that the behavior of

scientists and decision-making contexts have more influence than the attributes of the

knowledge itself. This indicates the importance of focusing on knowledge-to-action

processes and relationships rather than simply outcomes of either scientific or

policymaking efforts. Andrews (2002) and Karl et al. (2007) highlight how ecosystem

23

service knowledge that is jointly produced by scientists and stakeholders is more likely to

influence the way people conceptually understand the issues.

Anderson (1975) provides a similar theoretical lens for understanding policy

processes. The policy stages framework they describe is helpful for disentangling the

complex processes involved in policy and focusing on distinct subprocesses such as

problem identification, agenda setting, adoption, implementation, and policy evaluation.

This framework is applied to environmental policy by Tomich et al. (2004) in describing

the following stages of an environmental issue cycle: perception by pioneers of problem;

lobbying by action groups; increasing acceptance of impacts; debate cause and effect;

inventory options; negotiate prevention or mitigation of impacts; implement, monitor,

and enforce.

At which of these stages does ecosystem service science have the most influence?

The ‘conceptual enlightenment’ model of knowledge utilization suggests that ecosystem

service science would have most influence in the earlier stages. But recent approaches

such as action research and innovative participatory processes draw into question the

utility of this linear model of subprocesses (Kerkhoff & Lebel, 2006). The stages

framework has serious limitations. For example, it is not a causal model in that it lacks

identifiable linkages, drivers, and influences that exist within and between the distinct

subprocesses. The policy stages framework is also inherently top-down and legal-

focused, and so is likely to overlook important elements outside the purview of legislative

processes – “behind-the-scenes” elements that in reality may be more important to how

policy decisions are made.

24

1.4. Linking Knowledge to Action

This dissertation draws from these theoretical roots and builds on the ideas about

knowledge systems described by Cash et al. (2003). In focusing broadly on how science

and technology can be harnessed for sustainability, this research considers the networks

and organizations of programs, institutional arrangements, and policies that comprise

knowledge systems. Effective knowledge systems apply mechanisms that facilitate

communication, translation, and mediation across boundaries between scientists and

policymakers. Cash et al. (2003) identify three important features of the knowledge

produced by scientific efforts:

• relevance (applicability of scientific assessment to the needs of decision-

makers),

• credibility (“scientific adequacy of technical evidence and arguments”), and

• legitimacy (perception that the production of information and technology has

been unbiased and respectful of diverse viewpoints).

Rowe & Lee (2012) describe a theory of change for how science has an impact on

policy. They suggest that knowledge that is jointly produced by scientists and knowledge

users is more likely to be relevant, credible, and legitimate, and thus more likely to have

influence on real world decisions. These attributes can reinforce one another, but often

they cannot all be optimized. Sarkki et al. (2013) further unpack the trade-offs between

relevance, credibility, and legitimacy. For example, when a scientific output includes

information about uncertainty, it can be viewed as more credible. But this may detract

from the relevance to policymakers, who may find it difficult to understand or

incorporate into policy. Similarly, a thorough and high quality scientific assessment takes

25

time, as does a legitimate process that takes into account plural perspectives. The time

required to develop the credibility or legitimacy of scientific outputs creates a trade-off

when policy work demands a rapid response.

Many opportunities exist for rich interaction between science and policy

communities. This dissertation contributes to the growing body of literature that studies

these rich interactions in detail in the context of ecosystem services and how knowledge

links to action. In the following pages, I explore what constitutes an effective science-

policy interface by: evaluating the impact that ecosystem services assessments have on

decision-makers; analyzing global use of ecosystem service models; and testing how

relevance, credibility, legitimacy, and other proposed conditions facilitate the impact of

ecosystem service science.

26

1.5. References

Amara, N., Ouimet, M., & Landry, R. (2004). New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75-106.

Anderson, J. E. (1975). Public Policy Making. New York: Praeger. Andrews, C. J. (2002). Humble analysis : the practice of joint fact-finding. New

York: Praeger. Bagstad, K. J., Johnson, G. W., Voigt, B., & Villa, F. (2013). Spatial dynamics of

ecosystem service flows: A comprehensive approach to quantifying actual services. Ecosystem Services, 4, 117-125.

Balmford, A., Bruner, A., Cooper, P., Costanza, R., Farber, S., Green, R. E., . . .

Turner, R. K. (2002). Economic Reasons for Conserving Wild Nature. Science, 297, 950-953.

Bateman, I. J., Harwood, A. R., Abson, D. J., Andrews, B., Crowe, A., Dugdale,

S., . . . Termansen, M. (2014). Economic Analysis for the UK National Ecosystem Assessment: Synthesis and Scenario Valuation of Changes in Ecosystem Services. Environmental & Resource Economics, 57(2), 273-297.

Berkes, F. (2009). Evolution of co-management: Role of knowledge generation,

bridging organizations and social learning. Journal of Environmental Management, 90(5), 1692-1702.

Bernard, H. R. (2011). Research methods in anthropology : qualitative and

quantitative approaches (5th ed.). Lanham, Md.: AltaMira Press. Bishop, J. (2012). The economics of ecosystems and biodiversity in business and

enterprise. New York: EarthScan. Boumans, R., Costanza, R., Farley, J., Wilson, M. A., Portela, R., Rotmans, J., . . .

Grasso, M. (2002). Modeling the dynamics of the integrated earth system and the value of global ecosystem services using the GUMBO model. Ecological Economics, 41(3), 529-560.

Cash, D. W., Clark, W. C., Alcock, F., Dickson, N. M., Eckley, N., Guston, D. H.,

. . . Mitchell, R. B. (2003). Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences of the United States of America, 100(14), 8086-8091.

27

Chan, K. M. A., Shaw, M. R., Cameron, D. R., Underwood, E. C., & Daily, G. C. (2006). Conservation Planning for Ecosystem Services. PLoS Biology, 4(11), 1-15.

Corbera, E., Kosoy, N., Tuna, M. (2007). Equity implications of marketing

ecosystem services in protected areas and rural communities: Case studies from Meso-America. Global Environmental Change, 17: 365-380.

Costanza, R., d'Arge, R., Groot, R. d., Farber, S., Grasso, M., Hannon, B., . . .

Belt, M. v. d. (1997). The value of the world's ecosystem services and natural capital. Nature, 387, 253-260.

Costanza, R., & Daly, H. E. (1992). Natural Capital and Sustainable

Development. Conservation Biology, 6(1), 37-46. Costanza, R., & Kubiszewski, I. (2011). Valuing Ecological Systems and Services

F1000 Biology Reports, 3(14). Cox, B., & Searle, B. (2009). The State of Ecosystem Services. Boston: The

Bridgespan Group. Creswell, J. W. (2009). Research design : qualitative, quantitative, and mixed

methods approaches (3rd ed.). Thousand Oaks, Calif.: Sage Publications. Daily, G. C. (1997). Nature's services : societal dependence on natural

ecosystems. Washington, DC: Island Press. Daily, G. C., & Ellison, K. (2002). The new economy of nature : the quest to

make conservation profitable. Washington, DC: Island Press : Shearwater Books. Daily, G. C., & Matson, P. A. (2008). Ecosystem services: From theory to

implementation. Proceedings of the National Academy of Sciences of the United States of America, 105(28), 9455-9456.

Daily, G. C., Polasky, S., Goldstein, J., Kareiva, P. M., Mooney, H. A., Pejchar,

L., . . . Shallenberger, R. (2009). Ecosystem services in decision making: time to deliver. Frontiers in Ecology and the Environment, 7(1), 21-28.

Driscoll, C. T., Lambert, K. F., & Weathers, K. C. (2011). Integrating Science and

Policy: A Case Study of the Hubbard Brook Research Foundation Science Links Program. Bioscience, 61(10), 791-801.

Ehrlich, P. R., & Mooney, H. A. (1983). Extinction, Substitution, and Ecosystem

Services. Bioscience, 33(4), 248-254.

28

Elliott, E. W., & Kiel, L. D. (1999). Nonlinear dynamics, complexity and public policy. Commack, N.Y.: Nova Science Publishers.

Ernstson, H. & Sorlin, S. (2013). Ecosystem services as technology of

globalization: On articulating values in urban nature. Ecological Economics, 86: 274-284. Farley, J., Filho, A. S., Burke, M., & Farr, M. (2014). Extending market allocation

to ecosystem services: Moral and practical implications on a full and unequal planet. Ecological Economics, in press and available online.

Ferraro, P. J., Lawlor, K., Mullan, K. L., & Pattanayak, S. K. (2012). Forest

Figures: Ecosystem Services Valuation and Policy Evaluation in Developing Countries. Review of Environmental Economics and Policy, 6(1), 20-44.

Ferraro, P. J., & Pattanayak, S. (2006). Money for Nothing? A Call for Empirical

Evaluation of Biodiversity Conservation Investments. PLoS Biology, 4(4), 0001-0007. Fisher, B., Turner, K., Zylstra, M., Brouwer, R., Groot, R. d., Farber, S., . . .

Balmford, A. (2008). Ecosystem Services and Economic Theory: Integration for policy-relevant research. Ecological Applications, 18(8), 2050-2067.

Fisher, B., Turner, R. K., & Morling, P. (2009). Defining and classifying

ecosystem services for decision making. Ecological Economics, 68, 643-653. Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J.

(2010). Impact Evaluation in Practice. Washington, DC: The Wolrd Bank. Geyer, R., & Rihani, S. (2010). Complexity and public policy : a new approach to

twenty-first century politics, policy and society. London ; New York: Routledge. Ghazoul, J. (2007). Challenges to the uptake of the ecosystem service rationale

for conservation. Conservation Biology, 21(6), 1651-1652. Goldman, R. L., & Tallis, H. (2009). A Critical Analysis of Ecosystem Services

as a Tool in Conservation Projects: The Possible Perils, the Promises, and the Partnerships. The Year in Ecology and Conservation Biology, Annals of the NY Academy of Science(1162), 63-78.

Groot, R. S. d., Alkemade, R., BRaat, L., Hein, L., & Willemen, L. (2010).

Challenges in integrating the concept of ecosystem services and values in landscape planning, management, and decision making. Ecological Complexity, 7, 260-272.

Guston, D. H. (2001). Boundary organizations in environmental policy and

science: An introduction. Science Technology & Human Values, 26(4), 399-408.

29

Haas, P. M. (1992). Epistemic Communities and International-Policy Coordination - Introduction. International Organization, 46(1), 1-35.

Hardy, D. J. & Patterson, M. G. (2012). Cross-cultural environmental research in

New Zealand: Insights for ecological economics research practice. Ecological Economics, 73: 75-85.

Hinkel, J. (2011). Indicators of vulnerability and adaptive capacity: Towards a

clarification of the science-policy interface. Global Environmental Change, 21, 198-208. Hirsch, P. D., & Luzadis, V. A. (2013). Scientific Concepts and Their Policy

Affordances: How a focus on compatibility can improve science-policy interaction and outcomes. Nature and Culture, 8(1), 97-118.

Huitema, D., & Turnhout, E. (2009). Working at the science-policy interface: a

discursive analysis of boundary work at the Netherlands Environmental Assessment Agency. Environmental Politics, 18(4), 576-594.

Jones, S. A., Fischhoff, B., & Lach, D. (1999). Evaluating the Science-Policy

Interface for Climate Change Research. Climactic Change, 43, 581-599. Karl, H. A., Susskind, L. E., & Wallace, K. H. (2007). A dialogue not a diatribe -

Effective integration of science and policy through joint fact finding. Environment, 49(1), 20.

Kates, R. W., Clark, W. C., Corell, R., Hall, J. M., Jaeger, C. C., Lowe, I., . . .

Svedin, U. (2001). Environment and development - Sustainability science. Science, 292(5517), 641-642.

Kerkhoff, L. v., & Lebel, L. (2006). Linking Knowledge and Action for

Sustainable Development. Annual Review of Environment and Reosurces, 31, 445-477. Khandker, S. R., Koolwal, G. B., & Samad, H. A. (2010). Handbook on impact

evaluation : quantitative methods and practices. Washington, D.C.: World Bank. Kingdon, J. W. (2011). Agendas, alternatives, and public policies (Updated 2nd

ed.). Boston: Longman. Kinzig, A. P., Perrings, C., Chapin, F. S., Polasky, S., Smith, V. K., Tilman, D., &

Turner, B. L. (2011). Paying for Ecosystem Services-Promise and Peril. Science, 334(6056), 603-604.

Kolinjivadi, V., Grant, A., Adamowski, J., Kosoy, N. (2015). Juggling multiple

dimensions in a complex socio-ecosystem: The issue of targeting in payments for ecosystem services. Geoforum, 58: 1-13.

30

Kosoy, N., Corbera, E. (2010). Payments for ecosystem services as commodity

fetishism. Ecological Economics, 69: 1228-1236. Kremen, C. (2005). Managing ecosystem services: what do we need to know

about their ecology? Ecology Letters, 8, 468-479. Kumar, P., et al. (2010). The economics of ecosystems and biodiversity:

ecological and economic foundations. New York: Earthscan. Landry, R., Amara, N., & Lamari, M. (2001). Utilization of social science

research knowledge in Canada. Research Policy, 30(2), 333-349. Laurans, Y., & Mermet, L. (2014). Ecosystem services economic valuation,

decision-support system or advocacy? Ecosystem Services, 7, 98-105. Laurans, Y., Rankovic, A., Bille, R., Pirard, R., & Mermet, L. (2013). Use of

ecosystem services economic valuation for decision making: Questioning a literature blindspot. Journal of Environmental Management, 119, 208-219.

Lavis, J. N., Robertson, D., Woodside, J. M., McLeod, C. B., Abelson, J., & Grp,

K. T. S. (2003). How can research organizations more effectively transfer research knowledge to decision makers? Milbank Quarterly, 81(2), 221.

Lee, K. N. (1993). Compass and Gyroscope: Integrating Science and Politics for

the Environment. Washington, DC: Island Press. Leiss, W. (2000). Between expertise and bureaucracy: Risk management trapped

at the science-policy interface Risky Business: Canada’s Changing Science-Based Policy and Regulatory Regime (pp. 49-74). Toronto: University of Toronto Press.

Lemos, M. C., & Morehouse, B. J. (2005). The co-production of science and

policy in integrated climate assessments. Global Environmental Change-Human and Policy Dimensions, 15(1), 57-68.

Lester, J. P. (1993). The Utilization of Policy Analysis by State Agency Officials.

Knowledge-Creation Diffusion Utilization, 14(3), 267-290. Liu, S., Costanza, R., Farber, S., & Troy, A. (2010). Valuing ecosystem services

Theory, practice, and the need for a transdisciplinary synthesis. Ecological Economics Reviews, 1185, 54-78.

Lovins, A. B., Lovins, L. H., & Hawken, P. (1999). A road map for natural

capitalism. Harvard Business Review, 77(3), 145.

31

Luck, G. W., Chan, K. M. A., Eser, U., Gomez-Baggethun, E.., Matzdorf, B., Norton, B., Potschin, M. B. (2012). Ethical considerations in on-ground applications of the ecosystem services concept. Bioscience, 62(12): 1020-1029.

Margoluis, R., Stem, C., Salafsky, N., & Brown, M. (2009). Design Alternatives

for Evaluating the Impact of Conservation Projects. In M. Birnbaum & P. Mickwitz (Eds.), New Directions for Evaluation (pp. 85-96).

McCauley, D. J. (2006). Selling out on nature. Nature, 443, 27-28. McKenzie, E., & Irwin, F. (2011). Incorporating ecosystem service in decisions.

In P. M. Kareiva, H. Tallis, T. Ricketts, G. C. Daily & S. Polasky (Eds.), Natural Capital: Theory and Practice of Mapping Ecosystem Services. Oxford: Oxford University Press.

McKenzie, E., Posner, S., Tillmann, P., Bernhardt, J. R., Howard, K., &

Rosentha, A. (2014). Understanding the use of ecosystem service knowledge in decision making: lessons from international experiences of spatial planning. Environment and Planning C: Government and Policy, 32(2), 320 – 340

Millennium Ecosystem Assessment (Program). (2005). Ecosystems and human

well-being : synthesis. Washington, DC: Island Press. Miller, C. A., Jasanoff, S., & Long, M. (1997). Shaping knowledge, defining

uncertainty: the dynamic role of assessments. In W. C. Clark, J. J. McCarthy & E. Shea (Eds.), A Critical Evaluation of Global Environmental Assessments. Cambridge, MA: Kennedy School of Government.

Mitchell, R. B., Clark, W. C., Cash, D. W., & Alcock, F. (2004). Science,

scientists, and the policy process: lessons from global environmental assessments for the northwest forest In K. Arabas & J. Bowersox (Eds.), Forest Futures: Science, Politics, and Policy for the Next Century. Lanham, MD: Rowman and Littlefield.

Muradian, R. Corbera, E., Pascual, U., Kosoy, N., May, P. H. (2010). Reconciling

theory and practice: An alternative conceptual framework for understanding payments for environmental services. Ecological Economics, 69: 1202-1208.

Muradian, R., Arsel, M., Pellegrini, L., Adaman, F. … (2013). Payments for

ecosystem services and the fatal attraction of win-win solutions. Conservation Letters, 6(4): 274-279.

Nesshover, C., Timaeus, J., Wittmer, H., Krieg, A., Geamana, N., van den Hove,

S., . . . Watt, A. (2013). Improving the Science-Policy Interface of Biodiversity Research Projects. Gaia-Ecological Perspectives for Science and Society, 22(2), 99-103.

32

Oh, C. (1996). Linking Social Science Information to Policy Making. Greenwich, CT: Jai Press.

Pannell, D. J. (2004). Effectively communicating economics to policy makers.

Australian Journal of Agricultural and Resource Economics, 48(3), 535-555. Perrings, C., Duraiappah, A., Larigauderie, A., & Mooney, H. (2011). The

Biodiversity and Ecosystem Services Science-Policy Interface. Science, 331(4 March ), 1139-1140.

Pielke, R. A. (2007). The Honest Broker. Cambridge: Cambridge University

Press. Pretty, J., & Smith, D. (2004). Social capital in biodiversity conservation and

management. Conservation Biology, 18(3), 631-638. Pretty, J., & Ward, H. (2001). Social capital and the environment. World

Development, 29(2), 209-227. Redford, K. H., & Adams, W. M. (2009). Payment for Ecosystem Services and

the Challenge of Saving Nature. Conservation Biology, 23(4), 785-787. Reed, M. S. (2008). Stakeholder participation for environmental management: A

literature review. Biological Conservation, 141(10), 2417-2431. Reid, R. S., Nkedianye, D., Said, M. Y., Kaelo, D., Neselle, M., Makui, O., . . .

Clark, W. C. (2009). Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa. PNAS, Early Edition, 1-6.

Reyers, B., Polasky, S., Tallis, H., Mooney, H. A., & Larigauderie, A. (2012).

Finding Common Ground for Biodiversity and Ecosystem Services. Bioscience, 62(5), 503-507.

Reynolds, K. M. (2007). Sustainable forestry : from monitoring and modelling to

knowledge management and policy science. Wallingford, UK ; Cambridge, MA: CABI. Roux, D. J., Rogers, K. H., Biggs, H. C., Ashton, P. J., & Sergeant, A. (2006).

Bridging the science-management divide: Moving from unidirectional knowledge transfer to knowledge interfacing and sharing. Ecology and Society, 11(1).

Rowe, A., & Lee, K. (2012). Linking Knowledge with Action: an approach to

philanthropic funding of science for conservation. Palo Alto, CA: David & Lucile Packard Foundation.

33

Ruckelshaus, M., McKenzie, E., Tallis, H., Guerry, A., Daily, G., Kareiva, P., . . . Bernhardt, J. (2013). Note from the field: Lessons learned from using ecosystem service approaches to inform real-world decisions. Ecological Economics.

Ruhl, J. B., Kraft, S. E., & Lant, C. L. (2007). The law and policy of ecosystem

services. Washington: Island Press. Sabatier, P. A., & Jenkins-Smith, H. C. (1993). Policy change and learning : an

advocacy coalition approach. Boulder, Colo.: Westview Press. Sarkki, S., Niemela, J., Tinch, R., Hove, S. v. d., Watt, A., & Young, J. (2013).

Balancing credibility, relevance and legitimacy: A critical assessment of trade-offs in science-policy interfaces. Science and Public Policy, 2013, 1-13.

Schumacher, E. F. (1973). Small is beautiful; economics as if people mattered.

New York,: Harper & Row. Tallis, H., & Kareiva, P. (2009). Mapping and Valuing Ecosystem Services as an

Approach for Conservation and Natural-Resource Management. Annals of the New York Academy of Sciences, 1162, 265–283.

Tomich, T. P., Chomitz, K., Francisco, H., Izac, A. M. N., Murdiyarso, D.,

Ratner, B. D., . . . van Noordwijk, M. (2004). Policy analysis and environmental problems at different scales: asking the right questions. Agriculture Ecosystems & Environment, 104(1), 5-18.

van den Hove, S. (2007). A rationale for science-policy interfaces. Futures, 39(7),

807-826. Vatn, A. & Bromley, D. W. (1994). Choices without prices without apologies.

Journal of Environmental Economics and Management, 26: 129-148. Villa, F., Bagstad, K. J., Voigt, B., Johnson, G. W., Portela, R., Honzak, M., &

Batker, D. ( 2014). A methodology for adaptable and robust ecosystem services assessment. PloS ONE, 9(3), e91001.

Voinov, A., & Gaddis, E. J. B. (2008). Lessons for successful participatory

watershed modeling: A perspective from modeling practitioners. Ecological Modelling, 216(2), 197-207.

Watson, R. (2005). Turning science into policy: challenges and experiences from

the science-policy interface. Phil. Trans. R. Soc. B, 360, 471-477.

34

Watson, R. T. (2012). The science-policy interface: the role of scientific assesments -- UK National Ecosystem Assessment. Proceedings of the Royal Society A, 468, 3265-3281.

Weible, C. M., Pattison, A., & Sabatier, P. A. (2010). Harnessing expert-based

information for learning and the sustainable management of complex socio-ecological systems. Environmental Science & Policy, 13, 522-534.

Weible, C. M., Sabatier, P. A., Jenkins-Smith, H. C., Nohrstedt, D., Henry, A. D.,

& deLeon, P. (2011). A Quarter Century of the Advocacy Coalition Framework: An Introduction to the Special Issue. Policy Studies Journal, 39(3), 349-360.

Weichselgartner, J., & Kasperson, R. (2010). Barriers in the science-policy-

practice interface: Toward a knowledge-action-system in global environmental change research. Global Environmental Change-Human and Policy Dimensions, 20(2), 266-277.

Weiss, C. H. (1977). Research for Policys Sake - Enlightenment Function of

Social-Research. Policy Analysis, 3(4), 531-545. White, D. D., Corley, E. A., & White, M. S. (2013). Water Managers' Perceptions

of the Science-Policy Interface in Phoenix, Arizona: Implications for an Emerging Boundary Organization. Society & Natural Resources: An International Journal, 21(3), 230-243.

Wunder, S., Engel, S., & Pagiola, S. (2008). Taking stock: A comparative

analysis of payments for environmental services programs in developed and developing countries. Ecological Economics, 65(4), 834-852.

Yin, R. K. (2009). Case study research: design and methods (4th ed.). Los

Angeles, Calif.: Sage Publications.

35

CHAPTER 2: EVALUATING THE IMPACT OF ECOSYSTEM SERVICE

ASSESSMENTS ON DECISION-MAKERS

Stephen Posner 1,2, Christy Getz3, Taylor Ricketts 1,2

1 Gund Institute for Ecological Economics, 617 Main St., University of Vermont, Burlington, VT 05405, USA 2 Rubenstein School of Environment and Natural Resources, University of Vermont, Burlington, VT 05405, USA 3 Environmental Science, Policy, and Management, University of California, Berkeley, Berkeley, CA 94720, USA

Key Words: ecosystem services, impact evaluation, land use decisions, conservation

2.1. Abstract

Ecosystem services are declining in many places and are impacted by land use

decisions. Ecosystem service assessments can affect the capacity of decision-makers to

make conservation-oriented land use decisions, but the actual impact of assessments is

almost entirely unstudied. We conducted a difference-in-differences analysis of the

impact of an ecosystem service assessment in California on decision-maker

understanding of and attitudes about ecosystem services. We used surveys to compare

“pre-” and “post-” differences in outcomes among two treatment groups in counties

where assessment occurred and a comparison group without such assessments. Mixed

methods included fitting regression models to estimate the treatment effect of the

assessment and conducting interviews and direct observations to further understand how

decision-makers changed their minds in response to the assessment. Regression results

showed small increases relative to a comparison group in decision-maker understanding

36

of ecosystem services and perceived relevance of ecosystem services to their work.

Analysis of the interviews confirmed that decision-makers became more aware of

specific ways that ecosystem services could be used in conservation and development

decisions, and believed that ecosystem services would improve the outcomes of land use

decisions. Further impact evaluation studies of this type that estimate a counterfactual

and explore rival explanations for observed outcomes are needed to build evidence for

how ecosystem service projects impact relevant decision-makers and, ultimately,

outcomes for environmental and human well-being.

2.2. Introduction

Land use and land management decisions have significant impacts on ecosystems

and ecosystem services (ES), the valuable goods and services that ecosystems provide to

people (Daily, 1997; Polasky et al., 2011). Increasingly, efforts to conserve, protect, or

restore ES aim to influence land use decisions so that they incorporate information about

the values of ES (Chan et al., 2006; Daily et al., 2011; Goldman & Tallis, 2009). Efforts

to incorporate ES information into decisions rest on basic assumptions that this

information will improve decisions and result in improved environmental and human

well-being outcomes.

But there is a lack of sound evidence about the impact ES information has on

decisions, or how decision-makers use ES knowledge (Laurans et al., 2013; Mermet et

al., 2014). Many valuation studies mention prospective or intended roles for ES

knowledge in terms of informative, technical, or decisive uses, but rarely do these studies

describe actual use (Laurans et al., 2013). In a survey of researchers, Fisher et al. (2008)

37

found that ES research was used to inform policy agents, support policy initiatives, and

directly influence government policy and investment. A recent review of three

international case studies describe similar ways ES knowledge is used: conceptually to

raise awareness and reframe dialogues, strategically to build support for plans or policies,

and also instrumentally to make specific decisions (McKenzie et al., 2014). If

conservation science is to inform improved land use decisions, it is critical to better

understand what difference ES knowledge makes to land use decision-makers (McKenzie

et al., 2011).

According to theories of the science-policy interface, knowledge has an important

role in shaping decisions. Theory suggests that decision-makers are more likely to trust

and use knowledge that they perceive as salient (i.e., relevant to the needs of decision-

makers), credible (i.e., based on expert, reliable science), and legitimate (i.e., unbiased

and inclusive of diverse perspectives) (Cash et al., 2003; Cook et al., 2013; Keller, 2010).

A simple, linear model of policymaking described by Meier (1991) includes a role for

knowledge early in the policy process, when it can affect the understanding and attitudes

of policymakers. The more complex stages model (Grindle & Thomas, 1991), the policy

streams model (Kingdon, 2011), and the advocacy coalition model of policy processes

(Sabatier & Weible, 2014) all portray a similar key role for knowledge. These models

share general components in common. Decision-makers and other stakeholders: perceive

a problem, gather and evaluate knowledge about the problem and proposed solutions, and

acknowledge the need to act on policy options.

Knowledge about the value of ES could thus be valuable as an early lens for

identifying problems, a framework for crafting solutions and building support, and a tool

38

for evaluating proposed policy options. For ecosystem services specifically, a conceptual

framework first presented by Ruckelshaus et al. (2013) and built upon by Posner et al. (in

review) describes several pathways through which knowledge impacts policy decisions

(Figure 1). Our study here mainly focuses on pathway 2, when ES knowledge helps shape

the minds of decision makers by raising awareness and providing an ES focus for

stakeholders. We also describe the emergence of pathway 3, through which decision

makers and stakeholders build support for particular policy options and use language

related to ES as a frame within policy dialogues. Lastly, we investigate the potential for

pathway 4 and assess how decision makers envision using ES information to evaluate

projects, compare options, and design new policies and plans.

The health, policy, and international development fields have long included

systematic impact evaluation research, and now researchers and practitioners in

conservation increasingly recognize the need for improved evidence of impact (Ferraro &

Pattanayak, 2006; Fisher et al., 2013). The complexity and scale of real world social-

environmental interactions has made rigorous and quantitative evaluation of impact in

conservation difficult, but recent research is moving beyond anecdotal evidence and

testing specific causal mechanisms through which impact may occur (Andam et al., 2010;

Arriagada et al., 2012; Ferraro & Hanauer, 2014b; Miteva et al., 2012; Naidoo &

Johnson, 2013; Pfaff et al., 2008). In order to understand how conservation programs and

projects lead to improved outcomes for biodiversity and well-being, these studies use

control groups and statistical matching to estimate impact (Ferraro, 2009; Margoluis et

al., 2009).

39

Our study complements this growing body of work focused on the impact of

conservation policy on environmental outcomes (pathway 5 in Figure 1). Our work here

focuses on impact at a different, earlier part of the policymaking process – when ES

knowledge has an impact on the minds of those proposing and making policy decisions

(pathway 2 in Figure 1). We aim to detect whether information about the value of ES

changes the capacity of natural resource managers and conservation decision-makers in

California to make conservation-oriented decisions. In the process, we evaluate the

importance and impact of ES knowledge as a resource for decision-makers.

Specifically, we ask: do ES valuation projects and the knowledge they generate

impact local decision-makers’ 1) understanding of ES and natural capital concepts, and 2)

attitudes about conservation and planning approaches based on these concepts? We

follow ES assessments in two counties in California, and use quantitative methods to

compare changes in decision-maker understanding and attitudes with those in

neighboring counties without assessments. We also use qualitative methods to explore

why understanding and attitudes did or did not change. Tracking change in decision-

makers and their capacity to consider ES is vital in order to link scientific knowledge

with action, and to understand the difference that ES information may make.

2.3. Methods

“Healthy Lands, Healthy Economies” is a regional initiative designed to

demonstrate the economic value of conservation in Sonoma, Santa Clara, and Santa Cruz

Counties in California (herein referred to as the initiative). One of the goals of the

initiative is to measure the tangible effect that protecting natural areas has on local and

40

regional economies. The multi-year project has resulted in ES valuation reports for Santa

Clara County (Batker et al., 2014), Santa Cruz County (Schmidt et al., 2014), and

Sonoma County (forthcoming) that frame the natural resources of each county as capital

assets requiring investment in order to maintain a flow of economic benefits. The project

team consists of ecological economists, ecologists, and conservation planners who work

to identify and quantify the economic and community benefits achieved by investing in

working lands, natural areas, and water resources in the greater San Francisco Bay Area.

We used a mixed methods approach to evaluate the impact of the initiative and

the subsequent ES valuation reports on decision-makers in Santa Cruz and Santa Clara

County (Bamberger, 2012; Creswell, 2009; Wholey et al., 2010). This approach allowed

us to consider multiple sources of quantitative and qualitative evidence in our analysis

and construct a more complete description of impact. Our overall approach involved

before-after analysis of decision-maker understanding and attitudes in a treatment group

in the two counties and a comparison group composed of decision-makers in surrounding

counties that were not part of the initiative.

2.3.1. Quantitative Methods

We surveyed individuals in two treatment groups (Santa Clara and Santa Cruz

counties, where the ES valuation study was conducted and reported) and a comparison

group (8 neighboring counties, where no county-wide ES valuation studies were

occurring). We administered the survey electronically using Survey Monkey before the

initiative was launched in Winter 2013 and after release of the final county-level ES

valuation reports in Winter 2015. Survey respondents were land use and conservation

41

decision-makers identified by our team as the intended audience for the initiative (for

example, general managers of water districts, county planners, and executive directors of

conservation NGOs).

In the comparison group of 63 individuals, we received 10 responses to both the

pre- and post- survey (16% response rate). These 10 individuals were from Alameda,

Contra Costa, Marin, Monterey, Napa, San Mateo, San Lius Obispo, and Solano

Counties. In each of the Santa Cruz and Santa Clara groups, we received 9 responses to

the pre- and post- surveys (18% response rate). Response rates were higher for the pre-

survey, but fewer of these individuals responded to the post- survey and some initial

respondents moved jobs or locations during the initiative. We only used survey results for

individuals who responded to both the pre- and post- survey to maintain a consistent

sample. Our data are based on 28 pre- and 28 post- survey responses: a total of 20 in the

comparison group, 18 in Santa Cruz, and 18 in Santa Clara. The first round of surveys

informed the design of interview questions used in the qualitative analysis.

The survey consisted of open-response and 5-point Likert-scale questions that we

used in the following quantitative analysis (Appendix 3). For outcome variables, we used

mean responses to seven survey questions with standard errors (all part of the first

compound survey question): 1) relevance of ES to one’s organization and 2) relevance to

one’s work; 3) credibility and 4) legitimacy of ES knowledge produced by the initiative;

5) understanding of ES; and 6) the capacity within one’s county to monitor impacts to

ES, and 7) implement policies or plans about ES.

We used a difference-in-differences approach to estimate the effect of the

initiative on decision-makers (Gertler et al., 2010; Khandker et al., 2010). We calculated

42

the before-after differences for outcome variables in the treatment groups; calculated the

before-after differences for the same variables in the comparison group; and estimated the

average treatment effect as the difference between before-after changes between each

treatment group and the comparison group (Table 1).

We then specified time-series linear regression models:

Yit = β0 + β1Pit + β2Git + β3(Pit x Git) + εit

where Yit is a decision-maker outcome of interest, Pit is a dummy variable for time period

where pre-initiative periods are 0 and post-initiative are 1, and Git is a dummy variable

for group where comparison group is 0 and the treatment county is 1 (Branas et al.,

2011). We defined the term Pit x Git as the interaction between a pre-initiative and post-

initiative difference for each decision-maker. The coefficient for the interaction term β3 is

identical to the difference-in-differences term calculated above, and estimates the average

effect of the initiative on the outcome (Meyer, 1995). All data analysis was performed in

R (R, 2011).

2.3.2. Qualitative Methods

We used a case study approach to gather, organize, and analyze data (Yin, 2009).

We conducted direct observations of 10 initiative workshops and meetings in Fall 2012

through Winter 2013. This data collection technique allowed us to systematically observe

decision processes and early dialogue using a structured, pre-designed observation record

form (Appendix 1) (Taylor-Powell & Steele, 1996). We observed decision-makers and

scientists in their natural settings as they engaged with ES and natural capital concepts

during the process of the ES valuation study.

43

We conducted 11 semi-structured interviews with stakeholders before the

initiative was launched and 12 interviews after the county reports were published (23

total interviews) (Appendix 2). Interview questions built upon the results of the pre-

initiative survey. We used content analysis on the interviews and direct observation

records to understand the impact of the initiative.

2.4. Results

2.4.1. Quantitative Results

Difference-in-differences estimates on the outcome variables showed mixed

results (Figure 2, Table 2). Most estimates of impact from the regression models were

weak. We emphasize two results with a p-value < 0.15 (because our small sample sizes

make it difficult to detect differences). In Santa Clara County, we found an increase in

the perceived relevance of ES to one’s work. In Santa Cruz, our results show an increase

in the understanding of ES. Other relationships were non-significant, though, including

positive estimates of impact in Santa Clara for the perceived relevance and legitimacy of

ES and the capacity to monitor ES; and in Santa Cruz for perceived credibility of ES and

relevance of ES to a person’s work, understanding of ES, and capacity to monitor ES. In

both counties, non-significant negative impacts were found for the capacity to implement

policies or plans about ES.

Looking more closely at the raw data, we examined the outcome variables with p

< 0.15 from our regression results. For the survey questions we used to measure

outcomes, we found changes between the pre and post periods in the percentage of

respondents who agreed or strongly agreed (Table 3). For example, this percentage for

44

understanding of ES had the largest increase of all outcomes in both counties relative to

comparison group: in Santa Clara, from 55.6% pre-initiative to 77.8% post-initiative; in

Santa Cruz, from 33.3% to 77.8%; and in the comparison group, no increase from 90%.

Also, the percentage of respondents who agreed or strongly agreed that ES is relevant to

their work went up from 77.8% to 88.9% in both Santa Clara and Santa Cruz, but went

down from 100% to 90% in the comparison group. The decrease in perceived capacity to

implement policies or plans about ES is also evident in the raw data. In Santa Clara, the

percentage of respondents who agreed or strongly agreed declined from 77.8% pre-

initiative to 55.6% post-initiative; in Santa Cruz, from 55.6% to 33.3%; in the

comparison group, from 50.0% to 40.0%.

2.4.1. Qualitative Results

The results of our qualitative analysis provide additional insight into the impact of

the initiative. Again, there were some mixed responses among decision-makers, but the

interviews uncovered stronger evidence of impact. The discussion that follows considers

why we observed different impacts using quantitative and qualitative approaches.

Decision-makers understood ES better, but found it a new concept for themselves and the

public

The interviewees used more technical language about ES and demonstrated

increased understanding of ES topics post-initiative. Interviewees reported being more

comfortable discussing ES concepts. Several people described ES as “a new way of

thinking” and “a new concept for people,” even “a new paradigm.” In pre-initiative

45

interviews, they expected the initiative to “bring added information” to land use

decisions. Post-initiative, interviewees confirmed that this was occurring. Decision-

makers felt that the new information provided in the initiative could inspire a different

way of doing conservation with better environmental and long-term economic outcomes.

Decision-makers were initially skeptical of ES, but acquired ways to deal with their

uncertainties

In 3 of the 12 follow up interviews, decision-makers reported initial skepticism

about ES, but over the course of the initiative came to perceive ES as valuable for

informing decisions. Also, 4 interviewees pre-initiative described the potential for

information about ES to be either good or bad, depending on how the information would

be used. Post-initiative, there was less concern about the potential mis-use of ES

information, for example, to inflate property values prior to open space acquisition.

Decision-makers had questions about the often-wide ranges of ES value

estimates. For example: “to what extent can the ES valuation numbers be accepted as

empirical and exact figures, versus just starting points for needed conversations?” Half of

the interviewees post-initiative reported not having any problem with the value ranges.

Others still felt the ranges could hurt the credibility of the ES value estimates, but most

accepted the value ranges as useful for long-range regional-scale planning.

Several decision-makers mentioned that their organizations were considering

ways to internally evaluate the ES value estimates (for example, by having staff

ecologists review how the ES valuation reports determined which ES were provided by

different land use types). This indicated engagement with the reports. Vetting of the ES

46

assessments, either internally within an organization or publicly, could lead to more co-

production of useful ES knowledge (Cutts et al., 2011; Roux et al., 2006).

Decision-makers used or intended to use the ES knowledge conceptually, strategically,

and instrumentally

Of all the interviewees, only 3 of the total 23 were skeptical about ES and their

utility in land use decisions. There was widespread agreement about the value of ES

knowledge for framing conversations about nature and highlighting trends in

environmental quality. One interviewee described how the initiative “painted a picture to

demonstrate how the value of each [ES] affects people in real ways.” A representative

from a funding organization described how the initiative was helping ES become a

regular part of their “lexicon.” These results indicate conceptual use of the ES

knowledge.

There was also clear evidence of strategic knowledge use. Decision-makers felt

that the ES knowledge provided by the initiative would be used: to inform conversations

about the potential formation of a new open space district; to build public support for

conservation funding measures; and to “change legislators’ thinking about the value of

state parks or open space.” There was also mention of building support within the

business community because of how nature “provides nice views and recreation,

attracting and retaining a qualified workforce.”

Pre-initiative interviews found that people held a range of general ideas for

instrumental use of ES knowledge. Post-initiative, people could bring these ideas into

sharper focus. Decision-makers described more specific intended uses: to inform coastal

47

management and identify vulnerable areas for planned retreat vs. other actions; to

“evaluate priorities and options for watershed stewardship projects (such as habitat

improvement, invasive species removal, fish barrier removal, etc.)”; and to help decide

whether to build desalinization plants to maintain water supply. In particular, people felt

the dollar value estimates from the initiative would improve cost/benefit analysis in

infrastructure decisions. In pre-initiative interviews, people were interested in a more

integrated, coordinated approach to decisions about conservation and infrastructure

(which are often thought about separately in terms of funding and jurisdiction). Post-

initiative interviews found this interest was bolstered by the ES valuation reports. There

was clear intention to use ES knowledge to develop a more balanced investment between

“green” and “grey” infrastructure.

Decision-makers believed the initiative would help shift patterns in land use and

development

An important component of our evaluation study was to ask decision-makers to

consider what would occur in the absence of any ES value assessment in their counties.

Interviewees all generally agreed that without the initiative, things would continue in a

“business as usual” way, meaning that “there would be a continued undervaluing of green

infrastructure and an overemphasis on grey, built infrastructure.” In pondering this

hypothetical question, all interviewees felt that without the initiative, there would be

ongoing loss of nature in the region. Many also mentioned opinions that the initiative

would facilitate including ES in cost-benefit analyses for projects and shift peoples’

perspectives on conservation.

48

2.5. Discussion

We found strong evidence of the initiative’s impact with our interviews, but

weaker evidence of impact with our difference-in-differences analysis. By triangulating

among qualitative and quantitative data, we found slightly different results in our mixed

methods impact evaluation. Overall, the strongest impacts post-initiative were in how

decision-makers came to understand ES more and envisioned more specific ways in

which they could use ES knowledge in their work.

A few factors could explain the different results between Santa Clara and Santa

Cruz Count. There are differences in natural resource management between counties that

affect the dissemination and potential use of ES information (i.e. Santa Clara County has

one large consolidated water district, while Santa Cruz County has many small individual

water districts). Also, there are small differences in how and when the initiative occurred

in either county. The Santa Clara report on Nature’s Value was completed three months

before the Santa Cruz report. And while a similar process was conducted in both

counties, there were inevitable differences in two separate valuation processes. Lastly, ES

knowledge can be expected to have different impacts within the varied cultures of the

Santa Clara and Santa Cruz counties (one county is a technology business hub while the

other is a coastal recreation destination).

The quantitative and qualitative results are complementary, but the differences

need to be examined in the context of the entire study. The quantitative results from our

survey indicate the initiative coincided with an increase in some decision-makers’ general

understanding of ES and perceived relevance of ES to one’s work. In considering rival

49

explanations for the observed outcomes, we highlight four reasons why the difference-in-

differences results were not very significant and posit how likely each is.

First, the quantitative results could mean that the initiative did not make a

difference. This is an unlikely reason for the non-significant results. The qualitative

results indicate that the initiative did make a difference, as they confirm a growing

understanding of ES and show that decision-makers acquired insights into specific ways

they could use ES knowledge to inform decisions. However, the qualitative results show

that the initiative’s impact was stronger for certain outcomes, such as understanding of

ES, than for others, such as credibility of ES valuation methods. This heterogeneous

impact across the outcomes we measured would produce some non-significant results in

our quantitative study, but the qualitative methods uncovered clear impacts from the

initiative.

Second, concurrent factors could have influenced the observed outcomes in

decision-makers. ES has been an increasingly popular topic in conservation, and other

state or national ES programs could have influenced the comparison or treatment groups

(and we assume it would influence them all equally). Another factor was the record-

setting drought that consumed the attention of many California decision-makers during

the timeframe of the study. While some aspects of ES were relevant to freshwater

provision, ES as a whole was not as primary an issue as water shortage and management

issues. Lastly, a spillover effect from treatment counties to nearby comparison counties

could also have influenced our survey results. There were undoubtedly pre-existing

relationships among people in the northern California conservation community. Since

50

this spillover would equalize outcomes between groups, it is a likely explanation of some

of the non-significant difference-in-differences results.

Third, we acknowledge low sample size, selection bias, and other statistical

limitations. By the time we collected post-initiative data, we had small sample sizes of

individuals who responded to both surveys. These small sample sizes contributed to large

p-values. Also, the selection of decision-makers was based on identifying a target

audience for the initiative and counterparts in the comparison counties. Those individuals

who completed both the pre- and post-initiative surveys could have introduced sampling

bias.

Fourth, many of the initiative’s impacts will take time to emerge and may be

subtle, involving shifts in perspective and policy dialogue with respect to nature. Actual

policy decisions and other potential ecological impacts are expected to occur beyond the

timeframe of our study. A fruitful strategy for supporting a perspective shift over time

would be to build connections with local schools systems to bring thinking about ES to

younger generations.

Our qualitative results rule out the possibility that the initiative had no impact on

decision-makers. The statistical limitations of our sample sizes and spillover between

counties are the most likely reasons for non-significant quantitative results. The

heterogeneous impact across outcomes and longer time frames needed for impact to

manifest are also important. As in all such studies, our survey design (i.e. basic choice of

outcomes to measure and wording of questions) introduced an additional measurement

error. If we used other proxies for decision-making capacity in our survey, we may have

found different levels of impacts.

51

The qualitative results shed light on the initiative’s impact. The interviews

uncovered barriers to using ES knowledge that are related to decision-makers reporting

less capacity to implement policies or plans about ES post-initiative. In the interviews,

most decision-makers felt that the methods underpinning the ES valuation reports needed

to be vetted and refined before they could be used to enact new policies. A significant

challenge lies in how to effectively integrate “new” ways of valuing land into existing

decision processes and tools such as cost-benefit analysis. Efforts to implement policies

or plans about ES could be further impaired by academic-style assessments that do not

match the needs of people involved in policymaking, development review, or project

implementation. Lastly, there is a potential mismatch between the scale of county-wide

ES assessments and the scale of individual property-level decisions. These issues warrant

consideration by those involved in ES assessment or policy. Despite the challenges

associated with using ES knowledge, the processes of the county-wide assessments did

affect how decision-makers thought about longer-term, regional planning. Decision-

makers appreciated having additional ways to communicate with people about the value

of conservation.

2.6. Conclusion

The conservation community could benefit from being more rigorous in assessing

impact of ES assessments. Our study highlights the importance of mixing quantitative

and qualitative methods to uncover a more nuanced picture of impact (Smith et al., 2012).

While our study was not amenable to experimental design, we used mixed methods with

a comparison group to understand the impact of the initiative. How can we tell if any

52

observed impacts were due to the initiative? The comparison group allowed us to

consider what we would have observed had the initiative not occurred, and we directly

asked decision-makers in interviews to comment on the hypothetical situation where no

ES assessments had occurred. By estimating counterfactual situations with a comparison

group and through interviews, we can tentatively link observed changes in decision-

makers to the initiative.

There are a variety of ways to do impact evaluation. Counterfactual thinking that

considers the hypothetical situation in which the treatment is absent, and is an important

part of considering rival explanations for observed outcomes (Ferraro & Hanauer,

2014a). Difference-in-differences is a good approach in that it includes quantitative

comparison between a treatment group and an estimate of the counterfactual. It does,

however, rest on the assumption of equal trends in outcomes among all counties in the

absence of the initiative, an assumption that could be tested with surveys at multiple pre-

initiative time periods. There are stronger evaluation designs other than difference-in-

differences, but we were limited by poor data on observable outcomes and unobservable

sources of bias, which is often the case in environmental evaluation (Ferraro & Miranda,

2014). The limited data available means that our findings are best interpreted as

suggestive and useful for guiding future studies.

Future research would benefit from longer-term monitoring to evaluate links

between various interventions and the subsequent impacts. This would contribute to

understanding the full chain of impact from scientific analyses and the knowledge they

generate, to changes in the understanding and attitudes of decision-makers, altered

policies or plans, and ultimately actual improved outcomes for ES and human well-being.

53

Multi-site evaluations with better estimates of counterfactuals (perhaps by statistical

matching on observable characteristics) and randomized study designs are not always

feasible, but should be the aim. At the very least, ES projects need to include impact

evaluation as a core component, as the Healthy Lands, Healthy Economies Initiative has.

To move forward and more effectively incorporate ES into decision-making, the

planning stages of ES projects need to clearly measure baseline conditions and identify

opportunities for knowledge use in collaboration with decision-makers (Waite et al.,

2014). During projects, it is important to track how ES knowledge is produced and used.

After projects, systematic evaluation of impact and knowledge use over different

timescales can provide evidence for what works. This would improve our understanding

of how knowledge links to impact and benefit the design of future ES conservation

programs.

54

2.7. References

Andam, K. S., Ferraro, P. J., Sims, K. R. E., Healy, A., & Holland, M. B. (2010). Protected areas reduced poverty in Costa Rica and Thailand. PNAS, 107(22), 9996-10001.

Arriagada, R. A., Ferraro, P. J., Sills, E. O., Pattanayak, S. K., & Coredero-

Sancho, S. (2012). Do Payments for Environmental Serices Affect Forest Cover? A Farm-Level Evaluation from Costa Rica. Land Economics, 88(2), 382-399.

Bamberger, M. (2012). Introduction to Mixed Methods in Impact Evaluation

Impact Evaluation Notes (Vol. 3, pp. 1-42): InterAction and The Rockefeller Foundation. Batker, D., Schwartz, A., Schmidt, R., Mackenzie, A., Smith, J., & Robins, J.

(2014). Nature's Value in Santa Clara County. Tacoma, WA and San Jose, CA: Earth Economics and the Santa Clara Valley Open Space Authority.

Branas, C. C., Cheney, R. A., MacDonald, J. M., Tam, V. W., Jackson, T. D., &

Have, T. R. T. (2011). A Difference-in-Differences Analysis of Health, Safety, and Greening Vacant Urban Space. American Journal of Epidemiology, 174(11), 1296-1306.


et al. (2003). Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences of the United States of America, 100(14), 8086-8091.

Chan, K. M. A., Shaw, M. R., Cameron, D. R., Underwood, E. C., & Daily, G. C.

(2006). Conservation Planning for Ecosystem Services. PLoS Biology, 4(11), 1-15. Cook, C. N., Mascia, M. B., Schwartz, M. W., Possingham, H. P., & Fuller, R. A.

(2013). Achieving Conservation Science that Bridges the Knowledge-Action Boundary. Conservation Biology, 27(4), 669-678.

Creswell, J. W. (2009). Research design : qualitative, quantitative, and mixed

methods approaches (3rd ed.). Thousand Oaks, Calif.: Sage Publications. Cutts, B. B., White, D. D., & Kinzig, A. P. (2011). Participatory geographic

information systems for the co-production of science and policy in an emerging boundary organization. Environmental Science & Policy, 14, 977-985.

Daily, G. C. (1997). Nature's services : societal dependence on natural

ecosystems. Washington, DC: Island Press.

55

Daily, G. C., Kareiva, P. M., & Polasky, S. (2011). Mainstreaming natural capital into decisions. In P. M. Kareiva, H. Tallis, T. Ricketts, G. C. Daily & S. Polasky (Eds.), Natural Capital: Theory and Practice of Mapping Ecosystem Services. Oxford: Oxford University Press.

Ferraro, P. J. (2009). Counterfactual thinking and impact evaluation in

environmental policy. In M. Birnbaum & P. Mickwitz (Eds.), Environmental program and policy evaluation (Vol. 122, pp. 75–84): New Directions for Evaluation

Ferraro, P. J., & Hanauer, M. M. (2014a). Advances in Measuring the

Environmental and Social Impacts of Environmental Programs. Annual Review of Environment and Resources, 39, 495-517.

Ferraro, P. J., & Hanauer, M. M. (2014b). Quantifying causal mechanism to

determine how protected areas affect poverty through changes in ecosystem serices and infrastructure. PNAS, 111(11), 4332-4337.

Ferraro, P. J., & Miranda, J. J. (2014). The performance of non-experimental

designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark. Journal of Economic Behavior & Organization, 107, 344-365.


Evaluation of Biodiversity Conservation Investments. PLoS Biology, 4(4), 0001-0007. Fisher, B., Balmford, A., Ferraro, P. J., Glew, L., Mascia, M., Naidoo, R., et al.

(2013). Moving Rio Forward and Avoiding 10 More Years with Little Evidence for Effective Conservation Policy. Conservation Biology, 00(0), 1-3.

Fisher, B., Turner, K., Zylstra, M., Brouwer, R., Groot, R. d., Farber, S., et al.

(2008). Ecosystem Services and Economic Theory: Integration for policy-relevant research. Ecological Applications, 18(8), 2050-2067.

Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J.

(2010). Impact Evaluation in Practice. Washington, DC: The Wolrd Bank. Goldman, R. L., & Tallis, H. (2009). A Critical Analysis of Ecosystem Services


Grindle, M. S., & Thomas, J. W. (1991). Public Choices and Policy Change: The

Political Economy of Reform in Developing Countries. Baltimore: The John Hopkins University Press.

56

Keller, A. C. (2010). Credibility and Relevance in Environmental Policy: Measuring Strategies and Performance among Science Assessment Organizations. Journal of Public Administration Research and Theory, 20(2), 357-386.

Khandker, S. R., Koolwal, G. B., & Samad, H. A. (2010). Handbook on impact


ed.). Boston: Longman. Laurans, Y., Rankovic, A., Bille, R., Pirard, R., & Mermet, L. (2013). Use of




McKenzie, E., Irwin, F., Ranganathan, J., Hanson, C., Kousky, C., Bennett, K., et

al. (2011). Incorporating ecosystem service in decisions. In P. M. Kareiva, H. Tallis, T. Ricketts, G. C. Daily & S. Polasky (Eds.), Natural Capital: Theory and Practice of Mapping Ecosystem Services. Oxford: Oxford University Press.


Rosenthal, A. (2014). Understanding the use of ecosystem service knowledge in decision making: lessons from international experiences of spatial planning. Environment and Planning C: Government and Policy, 32(2), 320 – 340

Meier, G. (Ed.). (1991). Politics and Policy Making in Developing Countries:

Perspectives on the New Political Economy. San Francisco: ICU Press. Mermet, L., Laurans, Y., & Lemenager, T. (2014). Tools for What Trade?

Analysing the Utilization of Economic Instruments and Valuations in Biodiversity Management. Paris: Agence Francaise de Developpement

Meyer, B. D. (1995). Natural and Quasi-Experiments in Economics. Journal of

Business & Economic Statistics, 13(2), 151-161. Miteva, D. A., Pattanayak, S. K., & Ferraro, P. J. (2012). Evaluation of

biodiversity policy instruments: what works and what doesn't? Oxford Review of Economic Policy, 28(1), 69-92.

Naidoo, R., & Johnson, K. (2013). Community-based conservation reduces sexual

risk factors for HIV among men. Globalization and Health, 9(27), 1-5.

57

Pfaff, A., Robalino, J. A., & Sanchez-Azofeifa, G. A. (2008). Payment for Environmental Services: Empirical analysis for Costa Rica Working Papers Series SAN08-05. Durham, NC: Terry Sanford Institute of Public Policy at Duke University.

Polasky, S., Nelson, E., Pennington, D., & Johnson, K. A. (2011). The Impact of

Land-Use Change on Ecosystem Services, Biodiversity and Returns to Landowners: A Case Study in the State of Minnesota. Environmental & Resource Economics, 48(2), 219-242.

Posner, S. M., McKenzie, E., & Ricketts, T. H. (in review). What explains the

impact of ecosystem services knowledge on decisions? R. (2011). R: A language and environment for statistical computing: R

Development Core Team. Roux, D. J., Rogers, K. H., Biggs, H. C., Ashton, P. J., & Sergeant, A. (2006).


Ruckelshaus, M., McKenzie, E., Tallis, H., Guerry, A., Daily, G., Kareiva, P., et

al. (2013). Note from the field: Lessons learned from using ecosystem service approaches to inform real-world decisions. Ecological Economics.

Sabatier, P. A., & Weible, C. M. (Eds.). (2014). Theories of the Policy Process.

Boulder, CO: Westview Press. Schmidt, R., Lozano, S., Robins, J., Schwartz, A., & Batker, D. (2014). Nature's

Value in Santa Cruz County. Tacoma, WA and Capitola, CA: Earth Economics and the Resource Conservation District of Santa Cruz County.

Smith, L. C., Khan, F., Frankenberger, T. R., & Wadud, A. K. M. A. (2012).

Admissable Evidence in the Court of Development Evaluation? The Impact of CARE's SHOUHARDO Project on Child Stunting in Bangladesh. World Development, 41, 196-216.

Taylor-Powell, E., & Steele, S. (1996). Collecting Evaluation Data: Direct

Observation. Madison, WI: University of Wisconsin Extension. Waite, R., Kushner, B., Jungwiwwattanaporn, M., Gray, E., & Burke, L. (2014).

Use of coastal economic valuation in decision-making in the Caribbean: Enabling conditions and lessons learned. Ecosystem Services.

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (2010). Handbook of

Practical Program Evaluation (3rd ed.). Hoboken: Wiley.

58

Yin, R. K. (2009). Case study research : design and methods (4th ed.). Los Angeles, Calif.: Sage Publications.

59

Figure 2.1: Framework for how ES knowledge leads to impact. Five different pathways to impact are represented as columns with increasing impact the further one moves to the right. Our study focuses

mainly on pathways 2 and 3. Based on Ruckelshaus et al. (2013) and modified by Posner et al. (in review).

60

Figure 2.2: Differences in outcomes between the comparison group and the two treatment groups of Santa Clara and Santa Cruz Counties. Y-axis is mean Likert score. Data points are for mean

outcomes with standard errors before and after the initiative for the three groups.

61

Table 2.1: The difference-in-differences method. The two treatment groups of Santa Clara

and Santa Cruz Counties had the ES initiative, whereas the comparison group composed of

individuals from nearby counties did not have a county-wide ES assessment. YC, pre refers to an

outcome variable for the comparison group before the initiative.

Comparison Santa Clara or Santa Cruz

Pre YC, pre YSC, pre

Post YC, pre YSC, post

Difference YC, post – YC, pre YSC, post – YSC, pre (YC, post – YC, pre) – (YSC, post – YSC, pre) = estimate of treatment effect

62

Table 2.2: Difference-in-differences estimates of the impact of the initiative on decision-

maker outcomes. β3 is the coefficient of the interaction term from our regression models.

Outcomes Estimate (β3) SE p-value Santa Clara Relevance to org 0.44 0.42 0.30 Relevance to work * 0.86 0.57 0.14 Credibility of ES -0.11 0.63 0.87 Legitimacy of ES 0.19 0.64 0.77 Understanding ES -0.08 0.42 0.85 Capacity to monitor impacts to ES 0.34 0.87 0.70 Capacity to implement policies -0.85 0.77 0.28 Santa Cruz Relevance to org -0.24 0.35 0.50 Relevance to work 0.09 0.56 0.87 Credibility of ES 0.10 0.66 0.89 Legitimacy of ES -0.26 0.70 0.72 Understanding ES* 0.90 0.61 0.15 Capacity to monitor impacts to ES 0.45 0.75 0.55 Capacity to implement policies -0.53 0.76 0.49

63

Table 2.3: Percentages of respondents who agreed or strongly agreed with statements about

ES in our survey questions.

Santa Clara pre

Santa Clara post

Santa Cruz pre

Santa Cruz post

Comparison pre

Comparison post

Relevance to org

88.9% 100.0% 77.8% 100.0% 100.0% 100.0%

Relevance to work*

77.8% 88.9% 77.8% 88.9% 100.0% 90.0%

Credibility of ES

66.7% 66.7% 44.4% 55.6% 60.0% 70.0%

Legitimacy of ES

33.3% 55.6% 33.3% 33.3% 30.0% 60.0%

Understanding ES*

55.6% 77.8% 33.3% 77.8% 90.0% 90.0%

Capacity to monitor impacts to ES

33.3% 33.3% 22.2% 44.4% 60.0% 60.0%

Capacity to implement policies**

77.8% 55.6% 55.6% 33.3% 50.0% 40.0%

64

2.8. Supporting Information

2.8.1. Direct Observation Record Form

Purpose of observations

To provide data for evaluating the impact of this ecosystem services valuation initiative

To understand current issues facing decision-makers and stakeholders

To understand project architecture and processes

Not to report on any individual’s or organization’s performance

Components to observe

Characteristics of participants

• gender, age, vocation, dress, appearance, ethnicity

• attitude toward subject, others, self

• skill and knowledge level

• statements about commitments, intentions, values, changes to be made

Interactions

• level of participation/interest

• power relationships, decision-making, current issues

• general climate for learning, problem-solving

• levels of support, cooperation

Nonverbal behavior

• facial expressions, gestures, posture

• apparent interest and commitment – impressions of meeting

Leaders/presenters

65

• clarity of communication and responses to questions

• group leadership skills, encouraging participation

• awareness of group climate and dynamics

• flexibility/adaptability

• knowledge of subject, use of aids, teaching techniques

• sequence of activities

Physical surroundings

Products and outcomes

Questions to consider

How is ecosystem service knowledge presented?

How do participants engage with ecosystem service knowledge?

What relationship do participants have with the scientific knowledge?

What levels of salience, credibility, and legitimacy are present?

Do decision-makers play a role in co-creating ecosystem service knowledge? What kind

of a role? What ownership do they have over different aspects of the project?

What are the intended uses of the information or project outputs?

What are major differences and similarities among the counties?

How do stakeholders, meeting participants, and Earth Economics team marshal evidence

in making their cases?

66

2.8.2. Interview Questions

1. I would like to get an idea of how you are involved with the HLHE initiative.

Describe in what capacity you know about the initiative and how your work is

related.

2. Do you work directly with anyone involved in the HLHE initiative?

3. How are decisions about conservation and stewardship made in your organization?

Are any decisions made related to Ecosystem Services (ES)?

4. What are the main challenges to incorporating ES into decision-making? How could

they be overcome?

5. What questions do you think the Nature’s Value report sought to answer?

6. How do you expect the results of the study (such as the information presented in the

Nature’s Value reports) to be used?

a. Describe any particular decisions you think can be informed by the study.

b. Describe any particular people or groups you think will use the study.

(for example, hazard mitigation, public goods charges, damage assessment, avoided cost

estimates)

7. What other influence do you think this ES knowledge has had or will have?

67

a. Will it help decide among options (i.e. desal plant or not)?

b. Will it illustrate the value of particular options?

c. Will it bring new information or knowledge to stakeholders, advisors, or policy

makers?

d. Will the SC3 project influence the capacity to monitor impacts or implement policies

specific to ES? (in Santa Cruz, example of potential use is flood management)

8. Thinking about key future projects/decisions that could be affected by this project…

a. In Santa Clara: do you think the report on The Value of Nature in Santa Clara County

affected your vote or other peoples' votes for Measure Q?

b. In Santa Cruz: do you think the report on The Value of Nature in Santa Cruz County

could have future uses (such as to support a local funding measure for protection of

ecosystem services)?

c. In Santa Cruz: do you think the report will influence discussions about the potential

formation of a new open space district? If so, how?

9. In terms of the messaging used in the HLHE Initiative...

a. What worked? What resonated with you or other stakeholders?

b. What didn't work?

10. What do you think would occur if there were no ES valuation studies in these

counties?

11. Is there anything else you would like to add that I haven’t asked about?

68

2.8.3. Ecosystem Services Survey

1. Please tell us how much you agree with the following statements:

Sub-question Don’tKnow

Strongly Disagree

Disagree Neutral Agree Strongly Agree

An ecosystem services approach is relevant to my company’s/ organization’s work.

1 2 3 4 5 6

My company/organization actively considers ecosystem services in making decisions.

1 2 3 4 5 6

The economic value of ecosystem services is relevant to my own work. 1 2 3 4 5 6

Conservation work is central to my company’s/organization’s core mission.

1 2 3 4 5 6

Regional or multi-county projects are high priority for my company/organization.

1 2 3 4 5 6

Barriers outside of my organization impede multi-jurisdictional collaboration.

1 2 3 4 5 6

My organization is involved in projects with other agencies/ organizations.

1 2 3 4 5 6

My organization should be involved in more projects with other agencies/ organizations.

1 2 3 4 5 6

Capacity to monitor impacts to ecosystem services exists in my county/region.

1 2 3 4 5 6

Capacity to implement policies or plans about ecosystem services exists in my county/region.

1 2 3 4 5 6

Ecosystem service knowledge is legitimate – gathered in a way that is complete, correct, and unbiased.

1 2 3 4 5 6

The economic value of ecosystem services can be quantified in scientifically credible ways.

1 2 3 4 5 6

I have a solid understanding of what the term ecosystem services means. 1 2 3 4 5 6

I have been familiar with ecosystem services language for more than a year.

1 2 3 4 5 6

2. Some people do not think the term ecosystem services is effective in non-scientific

settings. Which of the following terms do you prefer as an alternative? (check one)

69

The value of nature The economic value of conservation Conservation’s benefits Nature’s assets Natural capital Nature’s benefits None of the above, I like the term ecosystem services Other_______________________________________

3. Approximately what percentage of your time would you estimate is spent on work

and/or projects that interface with ecosystem services?

(Place a mark at the closest point on the following line or check the box “I don’t know”)

0--------20-------30-------40--------50-------60-------70-------80------90------100 %

I don’t know

4. What percentage of your time do you work in the following places?

Next to each option, please estimate (by marking an X) the percentage of your overall

work in that county/region.

Santa Cruz 0------20-----30-----40-----50-----60----70-----80----90-----100 %

Santa Clara 0------20-----30-----40-----50-----60----70-----80----90-----100 %

Sonoma 0------20-----30-----40-----50-----60----70-----80----90-----100 %

Regional level 0------20-----30-----40-----50-----60----70-----80----90-----100 %

State level 0------20-----30-----40-----50-----60----70-----80----90-----100 %

5. How would you characterize your relationship to conservation work? (check all that

apply)

Providing scientific or technical input Advocating for conservation Mediating relationships

70

Implementing projects Strategic planning Making decisions about funding priorities Developing regulations or crafting policy Enforcing regulations Assessing or mitigating impacts Other__________________________________

6. Approximately what percentage of your time would you estimate is spent on

conservation?

(Place a mark at the closest point on the following line.)

0--------20-------30-------40--------50-------60-------70-------80------90------100 %

7. Approximately what percentage of your time would you estimate is spent on

management of natural resources?

(Place a mark at the closest point on the following line.)

0--------20-------30-------40--------50-------60-------70-------80------90------100 %

8. How important CURRENTLY are the following ecosystems services in your

organization’s work? Please circle one number for each statement, indicating how

important each is to your organization, with 1 being “not important at all”, and 5 being

“highly important.” If you do not know, please circle “0” for “don’t know” (circle one

number for each):

71

Ecosystem Services

Don

’t K

now

Not

impo

rtant

at a

ll

Som

ewha

t un-

impo

rtant

Nei

ther

impo

rtant

or

uni

mpo

rtant

Som

ewha

t im

porta

nt

Hig

hly

impo

rtant

Regulation of greenhouse gases (forests store carbon)

0 1 2 3 4 5

Protection from natural disasters (coastal ecosystems mitigate hazards from storms or severe weather)

0

1

2

3

4

5

Flood control 0 1 2 3 4 5

Water quality protection 0 1 2 3 4 5 Soil retention 0 1 2 3 4 5 Soil formation 0 1 2 3 4 5 Nutrient cycling (promotes

healthy and productive soils)

0 1 2 3 4 5

Pollination 0 1 2 3 4 5 Biological control (pest and

disease control) 0 1 2 3 4 5

Habitat and biodiversity 0 1 2 3 4 5

Food & Agriculture 0 1 2 3 4 5 Timber/forest products 0 1 2 3 4 5

Aesthetic quality (enjoyment of

scenery) 0 1 2 3 4 5

Recreation & Tourism 0 1 2 3 4 5 Public Health 0 1 2 3 4 5

Cultural and historic value 0 1 2 3 4 5 Spiritual value 0 1 2 3 4 5 Science and education 0 1 2 3 4 5

9. How important do you think the following ecosystems services SHOULD BE in your

organization’s work? Please circle one number for each statement, indicating how

important you think each service should be to your organization, with 1 being “not

important at all”, and 5 being “highly important.” If you do not know, please circle “0”

for “don’t know” (circle one number for each):

72

Ecosystem Services

Don

’t K

now

Not

impo

rtant

at a

ll

Som

ewha

t un-

impo

rtant

Nei

ther

impo

rtant

or

uni

mpo

rtant

Som

ewha

t im

porta

nt

Hig

hly

impo

rtant

Regulation of greenhouse gases (forests store carbon)

0 1 2 3 4 5

Protection from natural disasters (coastal ecosystems mitigate hazards from storms or severe weather)

0

1

2

3

4

5

Flood control 0 1 2 3 4 5

Water quality protection 0 1 2 3 4 5 Soil retention 0 1 2 3 4 5 Soil formation 0 1 2 3 4 5 Nutrient cycling (promotes

healthy and productive soils)

0 1 2 3 4 5

Pollination 0 1 2 3 4 5 Biological control (pest and

disease control) 0 1 2 3 4 5

Habitat and biodiversity 0 1 2 3 4 5

Food & Agriculture 0 1 2 3 4 5 Timber/forest products 0 1 2 3 4 5

Aesthetic quality (enjoyment of

scenery) 0 1 2 3 4 5

Recreation & Tourism 0 1 2 3 4 5 Public Health 0 1 2 3 4 5

Cultural and historic value 0 1 2 3 4 5 Spiritual value 0 1 2 3 4 5 Science and education 0 1 2 3 4 5

10. What are three ecosystem services related to your work, in order of importance?

1)_______________________________________________________________

2)_______________________________________________________________

3)_______________________________________________________________

73

10b. Please describe briefly how you would go about estimating the economic value of

#1 (ecosystem service) that you have listed above?

11. What is your job title?

12. What type of organization to your work for? (please check one box below)

Resource Conservation District Parks and Recreation Department Open Space Authority Special District State Regulatory Agency Consulting Company Water Agency/District Private Non-Profit/NGO Funding organization Private Business OTHER_________________

12. How old are you (please check one)

21-30 61-70 31-40 71-80 41-50 81+ 51-60

13. Are you … ? (please check one)

Male Female

14. What is your highest level of education?

Some school Bachelor’s degree or equivalent High School Master’s Degree Some College Ph.D., M.D., J.D., or equivalent

15. What is your name?*

74

* Please also note an email address or a phone number if you’d like to talk with us more

in-depth about these issues.

75

CHAPTER 3: WHAT EXPLAINS THE IMPACT OF ECOSYSTEM SERVICES

KNOWLEDGE ON DECISIONS?

Stephen Posner 1,2, Emily McKenzie 3,4,5, Taylor H. Ricketts 1,2,4

1 Gund Institute for Ecological Economics, 617 Main Street, University of Vermont, Burlington, VT 05405, USA

2 Rubenstein School of Environment and Natural Resources, University of Vermont, Burlington, VT 05405 USA

3 World Wildlife Fund, 1250 24th St NW, Washington, DC 20037, USA 4 The Natural Capital Project, 371 Serra Mall, Stanford University, Stanford, CA

94305, USA 5 WWF-UK, The Living Planet Centre, Rufford House, Brewery Road, Woking,

Surrey, GU21 4LL, United Kingdom

3.1. Abstract

Research about ecosystem services (ES) often aims to generate knowledge that

influences policies and institutions for conservation and human development. Yet, we

have limited understanding of how decision-makers use ES knowledge, or what factors

facilitate use. Here we address this gap and report on the first quantitative analysis of the

factors and conditions that explain the policy impact of ES knowledge. We analyze a

global sample of cases where similar ES knowledge was generated and applied to

decision-making. We first test whether attributes of ES knowledge itself predict different

measures of impact on decisions. We find that legitimacy of knowledge is more often

associated with impact than either the credibility or salience of the knowledge. We also

examine whether explanatory variables related to the science-to-policy process and the

contextual conditions of a case are significant in predicting impact. Our findings indicate

that while many factors are important, attributes of the knowledge best explain the impact

of ES science on decision-making. Our results are consistent with both theory and

76

previous qualitative assessments in suggesting that the attributes and perceptions of

scientific knowledge are important determinants of whether that knowledge leads to

action.

3.2. Significance Statement

Our study introduces a conceptual framework and empirical approach to evaluate

the impact of scientific knowledge on decisions. We illustrate this novel approach with a

sample of 15 international cases involving ecosystem services research, but it has broad

applicability. Our results demonstrate that the perception of research knowledge as

legitimate (unbiased and representative of multiple points of view) is of paramount

importance for impact. More surprisingly, we found that credibility of knowledge is not a

significant factor for impact. To enhance the legitimacy needed for knowledge to

stimulate action, ES researchers must engage meaningfully with stakeholders to

incorporate diverse perspectives transparently. Our results indicate how research can be

designed and carried out to maximize the potential impact on real-world decisions.

3.3. Introduction

3.3.1. Ecosystem Services Knowledge Use In Decision-Making

The ongoing loss of biological diversity and persistence of poverty have sparked

interest in policies that protect, restore, and enhance ecosystem services (ES). In

response, there has been a growth in ecosystem service research that aims to inform

policies, incentives, and institutions on a large scale (Daily & Matson, 2008; Daily et al.,

2009; Hogan et al., 2011). Despite this goal, scientific knowledge about ES continues to

77

have limited impact on policy and decisions (Daily & Matson, 2008; de Groot et al.,

2010; Laurans et al., 2013; Liu et al., 2010; Spilsbury & Nasi, 2006).

The fact that most land and resource use policy decisions still do not take ES into

account stems in part from an ineffective or non-existent interface between ES science

and policy, a lack of attention to decision-making processes, and challenges in clarifying

conflicting stakeholder values (Eppink et al., 2012; Knight et al., 2008; Mermet et al.,

2014; Nesshover et al., 2013). The ES research and policy communities are too often

disconnected from one another, with limited interactions, infrequent exchanges of

information, and different objectives that hinder coordinated science and policy processes

(Weichselgartner & Kasperson, 2010). Many scientists conduct ES research without fully

considering how the knowledge they are producing might be used (Laurans et al., 2013).

If we want ES information to be incorporated into decisions, then we need to understand

how and why decision-makers use certain kinds of information.

Much of the evidence for how and why ES knowledge influences policy decisions

is anecdotal. A few recent studies have focused on this issue with qualitative, in-depth

case studies (Laurans & Mermet, 2014; MacDonald et al., 2014; McKenzie et al., 2014).

To more generally understand this issue, however, we also need quantitative, empirical

research into how and why ES knowledge has an impact on decisions (Laurans &

Mermet, 2014; Laurans et al., 2013; Liu et al., 2010; McKenzie, et al., 2014). This is an

understudied area of research, not least because empirical data on impacts from replicate

cases are difficult to compile.

Here we report on a quantitative approach to understanding the factors and

conditions that enable ES knowledge impact in decision-making. More carefully

78

examining the relationships between impacts and enabling conditions helps us better

understand why an ES approach may generate impacts on decisions and why it may not.

3.3.2. Enabling Conditions Framework

Understanding the factors that explain impact will benefit those who produce ES

knowledge (i.e. by illuminating effective strategies for enhancing knowledge use) as well

as decision-makers (i.e. by encouraging their participation in defining use-inspired

science). Cash et al. (2003) identify salience, credibility, and legitimacy of knowledge as

important enabling conditions for linking sustainability knowledge to action. Salience

refers to the relevance of scientific knowledge to the needs of decision-makers,

credibility comes from scientific and technical arguments being trustworthy and expert-

based, and legitimacy refers to knowledge that is produced in an unbiased way and that

fairly considers stakeholders’ different points of view. Their framework has inspired

others to investigate these three attributes and how they affect decision-makers using

knowledge (Cook et al., 2013; Keller, 2010; Reid et al., 2009; Rowe & Lee, 2012; Sarkki

et al., 2013).

Others focus on process rather than content of environmental management and

policy (Andrews, 2002; Cox & Searle, 2009; Karl et al., 2007; Rosenthal et al., 2014).

They describe the importance of joint fact-finding and iterative processes of engagement

among scientists and policymakers. Another branch of research has focused on more

contextual conditions about the institutions, governance, and culture of places where

environmental policy is successful. Haas et al. (1993) and Wunder et al. (2008) note

institutional capacity to monitor environmental conditions and enforce rules as critical to

effective science-based policies.

79

We organize these perspectives into three categories of enabling conditions for ES

knowledge to lead to action (Table 1). The first category contains variables related to

attributes of the scientific knowledge produced. Variables in the second category focus

on characteristics of the process through which science informs decisions and policy. The

third category contains variables that reflect contextual conditions of the project or place

in which it is located.

3.3.3. Research Question And Hypotheses

Drawing from these theoretical frameworks about linking knowledge with action,

we test quantitatively which enabling conditions can explain the impact of ES science

across a global sample of 15 cases. We analyze this set of science-policy interventions

through the attributes of the 1) knowledge produced, 2) science-to-policy process, and 3)

contextual conditions of each case. In so doing, we address the question ‘What explains

the impact of ecosystem services knowledge on decisions?’ In answering this question,

we aim to identify conditions that enable the impact of ES science.

We hypothesize that

• H1: Higher levels of salience, credibility, and legitimacy of ecosystem service

knowledge are associated with higher measures of impact.

• H2: Predictor variables related to knowledge are more significant than those

related to process or contextual conditions in explaining impact.

3.4. Methods

3.4.1. Sample Of Cases

80

To examine whether certain factors and conditions predict impact, we sought a

sample of cases in which similar scientific tools and approaches were used, but different

levels of impact achieved. We used a global sample of case studies from The Natural

Capital Project, in which a standardized scientific tool, InVEST, was applied to decisions

with the aim of improving conservation, human development and environmental planning

outcomes (Table 2). The Natural Capital Project was formed in 2006 by World Wildlife

Fund (WWF), The Nature Conservancy, Stanford University, and the University of

Minnesota, under the premise that information on biodiversity and ecosystem services

can be used to inform decisions that improve human well-being and the condition of

ecosystems (Ruckelshaus et al., 2013). InVEST (Integrated Valuation of Environmental

Services and Trade-offs) is a suite of software models that can be used to map, quantify

and value ecosystem services (Arkema et al., In Press; Bhagabati et al., 2014; Nelson et

al., 2009).

3.4.2. Measuring Enabling Conditions (Explanatory Variables)

We sent an electronic survey to decision-makers and boundary organization

contacts in each of the demonstration sites presented in Ruckelshaus et al. (2013).

Boundary organizations were NGOs that aimed to create more effective policymaking by

spanning/bridging the science and policy communities (Guston, 2001). We received

survey responses from 15 cases, providing a 40% response rate (Table 2). The survey

collected self-reported, ordinal scale data on the variables that we identified in the

literature as important elements of an effective science policy interface, or that team

members of the Natural Capital Project proposed as conditions that increase the

likelihood of knowledge impact (Table 3). This is not an exhaustive list of all potential

81

enabling conditions, but includes variables that have been identified by multiple sources

as significant. Specific survey questions are included in supplementary information.

3.4.3. Measuring Impact (Outcome Variables)

We use the conceptual framework for impact described by Ruckelshaus et al.

(2013). We modified the framework to include five pathways through which impact is

achieved in ES projects (Figure 1). We added a fifth pathway to reflect recent insights

into ways to define ES knowledge impact to include co-production of knowledge

(Pathway 1), conceptual use (Pathway 2), strategic use (Pathway 3), instrumental use

(Pathway 4), and outcomes for human wellbeing, biodiversity and ecosystems (Pathway

5) (McKenzie et al., 2014). We designed a scoring rubric with a 5-point scale based on

this evaluative framework.

Three reviewers (one of whom is a co-author) analyzed a qualitative review of the

impacts in each case from Ruckelshaus et al. (2013), written documentation of the cases

(including project reports, management plans, or case study summaries), and online

resources pertaining to the cases (such as project websites or presentations to decision-

makers). The reviewers then provided initial impact scores for each case. Through a

Delphi process, the reviewers then gathered and discussed results before independently

revising their scores. We averaged the three reviewer scores to obtain, for each case, an

estimate of impact 3 (build support), impact 4a (generate action: proposed plans and

policies based on ES), and impact 4b (generate action: new policy or finance mechanisms

for ES established). Similar methods have been described by Sutherland (2003) and used

by Sutherland et al. (2011) and Kenward et al. (2011) to evaluate the impact of science

and governance strategies.

82

A fourth measure of impact included in our analysis, impact 2 (awareness and

understanding of ES) was based on survey questions to decision-makers rather than the

expert scores (Table 4). We omitted columns 1 and 5 of the evaluative framework from

this study because all cases involved co-producing and publishing research results, and it

is difficult to show whether biodiversity, wellbeing or ES outcomes were enhanced over

the timescale of these projects (Figure 1). Given the post facto nature of this research, the

design of the study is limited by the inability to use rigorous impact evaluation methods

(Ferraro et al., 2012; Gertler et al., 2010; Margoluis et al., 2009).

3.4.4. Analyses

Inter-rater reliability analysis was used to compare among the three expert

reviewers who measured impact in the cases. We calculated a Krippendorff’s alpha to

measure agreement for ordinal data among three reviewers. From our sample, α = -

0.0544 for Impact 3, α = 0.655 for Impact 4a, and α = 0.619 for Impact 4b. For

conclusions based on the impact data, we took α > 0.6 as acceptable for this study

(Krippendorff, 2004). The low level of agreement among reviewers for Impact 3

indicates that only tentative conclusions should be drawn from these data.

We treated the 15 cases as independent data points in the analysis, each with 4

outcome variables (i.e., measures of impact; Table 4) and 16 explanatory variables (i.e.,

enabling conditions; Table 3). To test H1, we used the 5-point scale to group cases into 3

broader categories (low, medium, high) for salience, credibility, and legitimacy. For

example, the lowest levels of credibility of ES knowledge were labeled as “low,” the

middle two self-reported levels were labeled “medium,” and the highest level assigned to

the cases by survey respondents was labeled “high.” We then used analysis of variance

83

(ANOVA) to test whether higher levels of salience, credibility, and legitimacy are

associated with higher impact for all four impact measures. The unequal number of cases

in each of the groups led to an unbalanced ANOVA design.

To test H2 about which enabling conditions could best explain impact, we used an

information theoretic approach (Kenward et al., 2011). We first reduced the dataset by

using principal components analysis (PCA) on explanatory variables with a Spearman

rank correlation coefficient > 0.80 (see Supplementary Information for PCA plots)

(Dormann et al., 2013). Using the first principal component for groups of highly

correlated variables allowed us to focus on 12 explanatory variables. We used the R

package MuMIn to conduct multi-model inference (Barton, 2014). We tested all possible

linear models with these 12 variables, for each of the 4 measures of impact. To determine

which predictors best explain variability in impact, we ranked the top models by AICc

values and calculated model average coefficients with 95% confidence intervals for each

of the predictors. Model average coefficients represent the average coefficient for each

explanatory variable across all models, weighted for goodness of fit of the models

(Burnham & Anderson, 2002).

3.5. Results

For almost all impact measures, we find that impact tends to increase with higher

levels of credibility, salience, and legitimacy (Figure 2). With legitimacy, this effect is

significant for three of the four measures of impact; with salience, two measures; and

with credibility, none (Figure 2; Table 5).

84

We also find that certain attributes of the ecosystem service knowledge explain

impact better than characteristics of the process or contextual conditions (Figure 3).

Again, legitimacy emerges as a strong predictor of impact; averaging coefficients across

all possible models, we find that legitimacy of the ecosystem service knowledge is better

at explaining impact than any other included variable. For all measures of impact, the top

models include legitimacy as the strongest variable for explaining impact (see

Supplementary Information for table of model selection results with AICc values for top

models). Other variables included in best models (ranked by lowest AICc value) are the

number of interactions between scientists and decision-makers, the institutional capacities

in the cases, and the degree to which local knowledge was incorporated into decisions.

3.6. Discussion and Conclusion

We develop a quantitative approach to examine the conditions under which

scientific knowledge about ecosystem services most influences policies and decisions.

Using four measures of impact in a global sample of ES cases, we find that legitimacy of

scientific knowledge explains impact more than any of the other explanatory variables we

tested. Interestingly, higher levels of credibility are not associated with higher levels of

impact, however measured. Credibility, or the trustworthy and expert base of the

scientific information, is the factor that scientists are most responsible for, while salience

and legitimacy are established by both scientists and decision-makers (Rowe & Lee,

2012). The scientific adequacy of ES knowledge is undoubtedly important, perhaps as a

necessary precondition to policy processes, but this study finds that it is not significantly

associated with higher levels of impact.

85

The finding that legitimacy appears to matter more than credibility puts great

responsibility on researchers to engage with stakeholders. Researchers need to do more to

make the knowledge they produce legitimate, and research institutions need to put in

place the incentives and time required for researchers to do this.

We also find that different factors are important for different stages of impact

(Figure 3). Evidence from practitioners in the field and qualitative studies claim that

salience, credibility, and legitimacy are important to generate policy action, even while

they recognize tradeoffs may be necessary among these attributes (Cash et al., 2003;

Keller, 2010; Reid et al., 2009; Sarkki et al., 2013). Our results indicate that these

attributes are not equally important for each stage of impact we considered.

Salience appears to be important at early stages of the policy process, in shaping

people’s ideas and discussions about ES (Figure 2). Knowledge perceived to be salient –

relevant to the needs of decision-makers – is more likely to increase awareness. Greater

perceived legitimacy of ES knowledge is significantly associated with greater impact for

three measures of impact, including changing awareness, building support, and drafting

plans and policies that consider ES (Figure 2). This reinforces the idea that it is important

for decision-makers to view ES knowledge as unbiased, and based on a fair consideration

of different stakeholder values at all stages of decision-making. Transparently

incorporating key diverse perspectives surrounding an issue can build trust and improve

decision-makers’ acceptance of knowledge as legitimate (Young et al., 2013).

Regular interactions between scientists and decision-makers are important for

impact 3 (building support). Building support is a political process of aligning shared

interests behind particular positions, and interactions among decision-makers and

86

stakeholders also take place at many of those same events/meetings. Decision-makers are

likely to perceive the resulting scientific knowledge as salient when relevant policy

questions, which they help frame, inspire science. Scenarios of future conditions and

collaborative processes among scientists and decision-makers can also ensure the salience

and legitimacy of knowledge-producing efforts (Rosenthal et al., 2014).

Interestingly, local knowledge and institutional capacities are also important, but

with negative coefficients, indicating an inverse relationship with this measure of impact.

This is due to a few cases where low impact was achieved despite local knowledge being

included, and where high impact was achieved despite low institutional capacities.

Scientists and decision-maker interactions, local knowledge, and institutional capacities

are also important for explaining impact 4a (draft plans and policies consider ES) and

impact 4b (new plans and policies for ES are established).

Viewed as a whole, these results indicate that early on in a science-to-policy

cycle, the perception of ES knowledge as legitimate and salient is important to help shape

conversations and raise awareness. Later on in the science-to-policy cycle, the contextual

conditions that are outside of scientists’ control gain importance. According to these

findings, the factors that best predict the final stage of impact (when a project results in

draft or established policies that consider ES) are the degree to which decision-makers

perceive ES knowledge as legitimate, the institutional capacities in the place where the

project occurs, the use of local knowledge, and the amount of interaction between

scientists and decision-makers (Figure 3 and Table S3). However, there could be

interactions among these variables or effects that our analysis did not uncover because of

our sample size (for example, legitimacy only matters when credibility is high). And, in

87

the final stages of a policy process when a new policy or finance mechanism actually

becomes established, there are a multitude of variables at play, including many not

measured here.

While our study illustrates a potentially powerful empirical approach to these

issues, several limitations should be kept in mind. First, in exploring these relationships,

it is difficult to link a policy change to any specific causal factor, because so many

competing variables influence policy development (Ferraro et al., 2012). Studies with

multiple case study comparisons complement our results by taking into account many of

the subtle issues at play within the context of each unique case (Creswell, 2009;

McKenzie et al., 2014; Yin, 2009). Second, a sample of only 15 cases limits our

statistical power and ability to infer general relationships. Nevertheless, consistent trends

observed across several impact measures (e.g., Figure 2) instill some confidence in our

overarching results. Assembling larger datasets and measuring impact with a standard

framework across researchers will allow future studies to strengthen confidence in

general findings. Third, expert opinion carries inherent potential for bias and error, but is

increasingly well understood and supported as an empirical approach for research at the

intersection of science and policy (Manos & Papathanasiou, 2008). While observer bias

remains an issue, the relative differences observed among cases are more robust.

Despite these limitations, our study advances our understanding of enabling

conditions, use of ecosystem service knowledge, and the elements that lead to an

effective science-policy interface (Kenward et al., 2011; Mermet et al., 2014; Waite et al.,

2014). Understanding the factors that tend to enhance the policy impact of ES knowledge

is critical. Unless we consider the relationships that decision-makers have with the

88

products and process of science, the impact of ES knowledge will be haphazard, and will

not prevent the continued declines in ecosystems, biodiversity, and the benefits they

provide to people.

3.7. Acknowledgments

We thank the many government representatives, non-governmental

organizations, and individuals who provided data. Thanks to Becky Chaplin-Kramer,

Steve Polasky, and Duncan Russell for helpful comments on an earlier version of this

manuscript, and to Insu Koh, Alicia Ellis, Eduardo Rodriguez, Matthew Burke, Rebecca

Traldi, and Amy Rosenthal. SP and TR were supported by a WWF Valuing Nature

Fellowship.

89

3.8. References

Andrews, C. J. (2002). Humble analysis : the practice of joint fact-finding. Westport, Conn.: Praeger.

Arkema, K. K., Verutes, G., Wood, S. A., Clarke, C., Rosado, S., Canto, M., et al.

(In Press). Improving the margins: Modeling ecosystem services leads to better coastal plans. PNAS.

Barton, K. (2014). MuMIn: R package for Multi-model inference. USA: CRAN

project. Bhagabati, N. K., Ricketts, T., Sulistyawan, T. B. S., Conte, M., Ennaanay, D.,

Hadian, O., et al. (2014). Ecosystem services reinforce Sumatran tiger conservation in land use plans. Biological Conservation, 169, 147-156.

Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel

inference: a practical information-theoretic approach (2nd ed.). New York: Springer-Verlag.



Cook, C. N., Mascia, M. B., Schwartz, M. W., Possingham, H. P., & Fuller, R. A.


Cox, B., & Searle, B. (2009). The State of Ecosystem Services. Boston: The

Bridgespan Group. Creswell, J. W. (2009). Research design : qualitative, quantitative, and mixed

methods approaches (3rd ed.). Thousand Oaks, Calif.: Sage Publications. Daily, G. C., & Matson, P. A. (2008). Ecosystem services: From theory to



L., et al. (2009). Ecosystem services in decision making: time to deliver. Frontiers in Ecology and the Environment, 7(1), 21-28.

de Groot, R. S., Alkemade, R., Braat, L., Hein, L., & Willemen, L. (2010).

Challenges in integrating the concept of ecosystem services and values in landscape planning, management and decision making. Ecological Complexity, 7(3), 260-272.

90

Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carre, G., et al.

(2013). Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 27-46.

Eppink, F. V., Werntze, A., Mas, S., Popp, A., & Seppelt, R. (2012). Land

Management and Ecosystem Services: How Collaborative Research Programmes Can Support Better Policies. Gaia-Ecological Perspectives for Science and Society, 21(1), 55-63.

Ferraro, P. J., Lawlor, K., Mullan, K. L., & Pattanayak, S. K. (2012). Forest



(2010). Impact Evaluation in Practice. Washington, DC: The World Bank. Guston, D. H. (2001). Boundary organizations in environmental policy and

science: An introduction. Science Technology & Human Values, 26(4), 399-408. Haas, P. M., Keohane, R. O., & Levy, M. A. (1993). Institutions for the earth :

sources of effective international environmental protection. Cambridge, Mass.: MIT Press.

Hogan, D., Arthaud, G., Brookshire, D., Gunther, T., Pincetl, S., Shapiro, C., et al.

(2011). Developing an Institutional Framework to Incorporate Ecosystem Services into Decision Making - Proceedings of a workshop (Vol. 1221). Reston, Virginia: USGS.

Karl, H. A., Susskind, L. E., & Wallace, K. H. (2007). A dialogue not a diatribe -

Effective integration of science and policy through joint fact finding. Environment, 49(1), 20-.

Keller, A. C. (2010). Credibility and Relevance in Environmental Policy:

Measuring Strategies and Performance among Science Assessment Organizations. Journal of Public Administration Research and Theory, 20(2), 357-386.

Kenward, R. E., Whittingham, M. J., Arampatzis, S., Manos, B. D., Hahn, T.,

Terry, A., et al. (2011). Identifying governance strategies that effecively support ecosystem serivces, resource sustainability, and biodiversity. PNAS, 108(13), 5308-5312.

Knight, A. T., Cowling, R. M., Rouget, M., Balmford, A., Lombard, A. T., &

Campbell, B. M. (2008). Knowing but not doing: Selecting priority conservation areas and the research-implementation gap. Conservation Biology, 22(3), 610-617.

91

Krippendorff, K. (2004). Content Analysis: an introduction to its methodology. (Second ed.). Thousand Oaks, CA: Sage.

Laurans, Y., & Mermet, L. (2014). Ecosystem services economic valuation,

decision-support system or advocacy? Ecosystem Services, 7, 98-105. Laurans, Y., Rankovic, A., Bille, R., Pirard, R., & Mermet, L. (2013). Use of


Liu, S., Costanza, R., Farber, S., & Troy, A. (2010). Valuing ecosystem services


MacDonald, D. H., Bark, R. H., & Coggan, A. (2014). Is ecosystem service

research used by decision-makers? A case study of the Murray-Darling Basin, Australia. Landscape Ecology.

Manos, B., & Papathanasiou, J. (2008). GEM-CON-BIO Governance and

Ecosystem Management for the Conservation of Biology. Greece: Aristotle University of Thessaloniki.



McKenzie, E., Posner, S., Tillmann, P., Bernhardt, J. R., Howard, K., & A., R.

(2014). Understanding the use of ecosystem service knowledge in decision making: lessons from international experiences of spatial planning. Environment and Planning C: Government and Policy, 32(2), 320 – 340



Mermet, L., Laurans, Y., & Lemenager, T. (2014). Tools for What Trade?


Nelson, E., Mendoza, G., Regetz, J., Polasky, S., Tallis, H., Cameron, D. R., et al.

(2009). Modeling multiple ecosystem services, biodiversity conservation, commodity production, and tradeoffs at landscape scales. Frontiers in Ecology and the Environment, 7(1), 4-11.

92

Nesshover, C., Timaeus, J., Wittmer, H., Krieg, A., Geamana, N., van den Hove, S., et al. (2013). Improving the Science-Policy Interface of Biodiversity Research Projects. Gaia-Ecological Perspectives for Science and Society, 22(2), 99-103.

Reid, R. S., Nkedianye, D., Said, M. Y., Kaelo, D., Neselle, M., Makui, O., et al.

(2009). Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa. PNAS, Early Edition, 1-6.

Rosenthal, A., Verutes, G., McKenzie, E., Arkema, K. K., Bhagabati, N., Bremer,

L., et al. (2014). Process matters: A framework for conducting decision-relevant assessments of ecosystem services. International Journal of Biodiversity Science, Ecosystem Services and Management.




al. (2013). Note from the field: Lessons learned from using ecosystem service approaches to inform real-world decisions. Ecological Economics, in press.

Sarkki, S., Niemela, J., Tinch, R., Hove, S. v. d., Watt, A., & Young, J. (2013).


Spilsbury, M. J., & Nasi, R. (2006). The interface of policy research and the

policy development process: challenges posed to the forestry community. Forest Policy and Economics, 8(2), 193-205.

Sutherland, W. J. (2003). Predicting the ecological consequences of

environmental change: a review of the methods. Journal of Applied Ecology, 43, 599-616.

Sutherland, W. J., Goulson, D., Potts, S. G., & Dicks, L. V. (2011). Quantifying

the Impact and Relevance of Scientific Research. PloS ONE, 6(11), 1-10. Waite, R., Kushner, B., Jungwiwwattanaporn, M., Gray, E., & Burke, L. (2014).


Weichselgartner, J., & Kasperson, R. (2010). Barriers in the science-policy-

practice interface: Toward a knowledge-action-system in global environmental change research. Global Environmental Change-Human and Policy Dimensions, 20(2), 266-277.

93

Wunder, S., Engel, S., & Pagiola, S. (2008). Taking stock: A comparative analysis of payments for environmental services programs in developed and developing countries. Ecological Economics, 65(4), 834-852.


Angeles, Calif.: Sage Publications. Young, J. C., Jordan, A., Searle, K. R., Butler, A., Chapman, D. S., Simmons, P.,

et al. (2013). Does stakeholder involvement really benefit biodiversity conservation? Biological Conservation, 158, 359-370.

94

Figure 3.1: Evaluative framework for how ES knowledge leads to impact. Each column represents a pathway to different forms of impact, with increasing levels of impact going from left to right.

Columns 2, 3, and 4 (a and b) were the basis for our measurement of impact in each of the cases.

95

Figure 3.2: Effects of knowledge attributes on policy impact. Bars depict mean levels of impact for different levels of salience, credibility, and legitimacy in the 15 cases. No standard error bar indicates

1 case in that category. ** p < 0.01; * p < 0.05; + p < 0.1

96

Figure 3.3: Effects of multiple attributes on policy impact. Points and whiskers represent model average coefficients with 95% confidence intervals. Predictor variables are explained in Table 3. The

first principal components were used for three groups of highly correlated predictors (Spearman rank correlation coefficient > 0.80): Interact1 (level of joint production as estimated by interactions in person or by phone/email), prep1 (decision-making power and stakeholder representation), and CC1 (institutional capacities to measure baseline ES and human activities, monitor changes to ES

and human activities, and implement policy).

97

Table 3.1: Enabling conditions that facilitate the success of ecosystem service projects, as

suggested by qualitative reviews of projects.

Reference Attributes of knowledge

Process Contextual conditions

Cox and Searle (2009)

Clear science about ES, interactions between services, and how proposed actions may affect services

A confined system with clearly identified stewards, perpetrators of negative impacts, and service beneficiaries

Good governance in terms of clearly defined ownership or tenure, a legal system, capacity to enforce laws and monitor impacts, and a functioning infrastructure to support projects

Waite et al. (2014) A clear policy question; A clear presentation of methods, assumptions, and limitations

Strong stakeholder engagement; Effective communications and access to decision-makers

Good governance; Local demand for valuation; Economic dependence on resources; High levels of threats to coastal resources

Rosenthal et al. (2014); McKenzie et al. (2014); Ruckelshaus et al. (2013)

Policy question; Pertinent data; Integration of local and traditional knowledge

Meaningful participation and engagement with diverse groups; Joint knowledge production; Iterative process; Scenario development

Capacity to measure ES; Established planning process; Policy window

98

Table 3.2: The sample of 15 global cases in which InVEST was used in a policy decision

context. Each case represents a data point in the analysis. Ruckelshaus et al. (2013) discusses cases in

more detail.

Location Decision Context Organizations of survey respondents 1 Belize Spatial planning Coastal Zone Management Authority and

Institute (national government) 2 Canada - West

Coast Vancouver Island

Spatial planning West Coast Aquatic (board with representation from local and provincial government, nine First Nations, conservation NGOs, and businesses)

3 Colombia - Cauca Valley Water Fund

Water Funds Cauca Valley Water Fund, The Nature Conservancy

4 Colombia - Cesar Department

Permitting & mitigation

The Nature Conservancy

5 China - Baoxing County, Hainan Island, Upper Yangtze River Basin

Spatial planning for Ecosystem Function Conservation Areas

Chinese Academy of Sciences, The Natural Capital Project

6 Himalayas (Bhutan, Nepal, India)

Spatial planning WWF Eastern Himalayas Program

7 Latin America Water Funds Platform

Water funds The Nature Conservancy

8 Indonesia - Borneo

Spatial planning, policy advocacy

WWF Indonesia

9 Indonesia - Sumatra

Spatial planning, Strategic Environmental Assessment

WWF Indonesia

10 Tanzania - Eastern Arc Mountains

PES & REDD planning, policy advocacy

Valuing the Arc project at Cambridge University and WWF

11 United States – Ft. Lewis-McChord and Ft. Pickett

Spatial planning for military installation activities

US Army and Air Force (Department of Defense)

12 United States -Galveston Bay in Texas

Hazard mangement, spatial planning, climate adaptation

SSPEED (Severe Storm Prediction, Education and Evacuation from Disasters) Center led by Rice University, The Nature Conservancy marine science

13 United States -Monterey Bay in California

Climate adaptation Santa Cruz County, Moss Landing Marine Labs of California State Universities

99

14 United States - North Shore of O`ahu in Hawai`i

Spatial planning Kamehameha Schools (private land-owner)

15 Virungas Landscape - DRC, Uganda, Rwanda

Permitting & mitigation, Strategic Environmental Assessment

Institute of Tropical Forest Conservation

100

Table 3.3: Summary of predictive factors in three broad categories of enabling conditions.

See s1 for survey questions.

Variable Description Survey question

Reference

Attributes of knowledge Salience The relevance of scientific

knowledge to the needs of decision-makers

Q1 Cash et al. (2003); Hirsch and Luzadis (2013)

Credibility How trustworthy and expert-based are the scientific and technical arguments

Q2 Cash et al. (2003)

Legitimacy Unbiased knowledge that fairly considers stakeholders’ different points of view

Q3 Cash et al. (2003)

Traditional knowledge

Whether local knowledge and experience was included in ES assessment and planning

Q4 Watson (2005)

Process Level of joint production *

Scientists, stakeholders, and decision-makers working together to produce ES information; amount of interaction by phone/email or in person

Q5, Q6, and Q7 as separate variables

Cash et al. (2003); Karl, Susskind, Wallace (2007); Lee and Rowe (2013)

Stakeholder representation **

Proportion of people impacted by decisions about ES that were included

Q8 Beierle and Konisky (2001); Young et al. (2013); Lee (2003); Reed (2008)

Presence of conflict/consensus

Level of perceived conflict or disagreement among stakeholders

Q9 Karl, Susskind, Wallace (2007)

Trust Amount that trust between stakeholders increased during project

Q10 Pretty and Ward (2001); Pretty and Smith (2004)

Length Length, in years, of project Natural Capital Project

Contextual conditions Capacity to measure baseline ES and

Assessed for at the start of the project

Q11 Ferraro, Hanauera, Sims (2011); Smit

101

human activities ***

and Wandel (2006)

Capacity to monitor changes to ES and human activities ***


Q12 Levy, Keohane, Haas (2001); Smit and Wandel (2006)

Capacity to implement policy ***


Q13 Levy, Keohane, Haas (2001); Smit and Wandel (2006)

Decision-making power **

Concentrated or shared distribution of decision-making power among stakeholders

Q14 Natural Capital Project

Year Year project began Natural Capital Project

Note: Multi-model inference was conducted with the following first principal components based on the variables noted in the table: * Interact1; ** Prep1; *** CC1.

102

Table 3.4: Questions to assess impact (the response variable) according to the evaluative

framework.

Impact pathway

Measured with

Questions for survey respondents (for impact 2) or expert reviewers (for impact 3, 4a, and 4b)

2: Change perspectives

Survey What proportion of stakeholders was aware of and understood ES before the ES project? After the project?

3: Build support

Expert scoring

How much were the science and InVEST results used to build support for ES among stakeholders? Did the ES knowledge help develop a common language among stakeholders, articulate different positions, or mediate differences?

4a: Generate action

Expert scoring

How much did draft plans or policies emerge that consider ES? Did proposed plans or policies consider ES?

4b: Generate action

Expert scoring

Did a plan, policy, or finance mechanism to enhance, conserve, or restore ES become established?

“Stakeholders” refers to people or groups with an interest or concern in policy decisions that affect ES (for example, individual landowners, conservation NGOs, private businesses).

103

Table 3.5: Relationships between attributes of knowledge and policy impact. Each element

reports the F1,15 value (and corresponding p-value) for the ANOVA results.

Impact 2 Impact 3 Impact 4a Impact 4b Salience 6.04 (0.029)* 3.23 (0.095)+ 1.29 (0.277) 1.29 (0.276) Credibility 2.14 (0.167) 2.42 (0.144) 1.23 (0.288) 0.36 (0.559) Legitimacy 8.89 (0.011)* 10.25 (0.007)** 4.60 (0.052)+ 2.78 (0.119)

** 0.01; * 0.05; + 0.1.

104


3.9.1. Survey Questions

Q1. How relevant and timely was the ecosystem service information? Not at all, only a little, somewhat, very, a great deal Q2. Was the ecosystem service information scientifically credible? (“scientifically credible” refers to whether the information was reliable and based on scientific expertise) Not at all, only a little, somewhat, very, a great deal Q3. Did the decision-making process represent many diverse views on the management or policy issues? Not at all, only a little, somewhat, a lot, a great deal Q4. How much was local knowledge and experience included… a) in the ecosystem service assessment? b) in management or planning decisions? Not at all, only a little, somewhat, a lot, a great deal (responses to a and b were averaged together) Q5. How much did scientists, stakeholders, and decision-makers work together to co-produce ecosystem service information? Not at all, only a little, somewhat, a lot, a great deal Q6. How many time did you interact with scientists by phone or email? 0, 1-5, 6-10, 11-15, 16-20, more than 20 times Q7. How many time did you interact with scientists in person? 0, 1-5, 6-10, 11-15, 16-20, more than 20 times Q8. About what proportion of the people impacted by decisions about ecosystem services were represented in the process of… a) producing ecosystem service information? b) using ecosystem service information in decisions? None at all, some, about half, most, all (responses to a and b were averaged together) Q9. How much conflict or disagreement was there among stakeholders during the process of… a) producing ecosystem service information? b) using ecosystem service information in decisions? None at all, only a little, some, a lot, a great deal (responses to a and b were averaged together)

105

Q10. How much did trust between stakeholders increase during the process of… a) producing ecosystem service information? b) using ecosystem service information in decisions? None at all, only a little, somewhat, a lot, a great deal (responses to a and b were averaged together) Q11. Was there institutional capacity to measure baseline ecosystem services and human activities before the project? None at all, only a little, somewhat, a lot, a great deal Q12. Was there institutional capacity to monitor changes to ecosystem services and human activities before the project? None at all, only a little, somewhat, a lot, a great deal Q13. Was there institutional capacity to implement policies that impact ecosystem services and human activities before the project? None at all, only a little, somewhat, a lot, a great deal Q14. How was decision-making power distributed among stakeholders? Decision-making power very concentrated, somewhat concentrated, evenly concentrated and shared, somewhat shared, decision-making power very shared Q15. How long was the ecosystem service project? Less than 1 year, 1 year, 2 years, 3 or more years Q16. What year was the project completed?

106

3.9.2. Principal Components Analysis

Figure 3.4: PCA bi-plots. We reduced the dataset by combining the following highly correlated variables. prep: power and representation (correlation coefficient = 0.74); interact: in-person

interactions and electronic interactions (correlation coefficient = 0.72); CC: contextual conditions as institutional capacity to measure baseline conditions, monitor changes, and implement policies (paired correlation coefficients of 0.70, 0.79, and 0.80). In subsequent analyses, we used PC1 for

explanatory variables (or the inverse of PC1, so that increases in the principal component correlated to increases in the underlying variables).

107

3.9.3. Model Selection

Table 3.6: Model selection results with top 5 models and AICc values. Check mark ✓

indicates variables included in each model.

IMPACT 2 Model\Var legitimacy length power &

representation salience - AICc

1 ✓ 30.0 2 ✓ ✓ 31.5 3 ✓ ✓ ✓ 32.0 4 ✓ 32.1 5 ✓ 32.1 IMPACT 3 Model\Var legitimacy local

knowledge interactions institution

capacity - AICc

1 ✓ ✓ 22.3 2 ✓ ✓ 22.7 3 ✓ ✓ ✓ 22.9 4 ✓ ✓ 23.5 5 ✓ 24.0 IMPACT 4a Model\Var legitimacy local

knowledge interactions institution

capacity - AICc

1 ✓ 45.0 2 ✓ ✓ 45.1 3 ✓ ✓ 45.2 4 ✓ ✓ 47.2 5 ✓ 47.4 IMPACT 4b Model\Var legitimacy local

knowledge disagreement institution

capacity salience AICc

1 ✓ ✓ 56.1 2 ✓ 56.3 3 ✓ ✓ 57.0 4 ✓ 57.5 5 ✓ 57.8

108

CHAPTER 4: GLOBAL USE OF ECOSYSTEM SERVICE MODELS

Stephen Posner 1,2, Gregory Verutes 3,4, Insu Koh 1,2, Doug Denu 3, Taylor Ricketts 1,2 1 Gund Institute for Ecological Economics, 617 Main St., University of Vermont, Burlington, VT 05405, USA 2 Rubenstein School of Environment and Natural Resources, University of Vermont, Burlington, VT 05405, USA 3 The Natural Capital Project, 371 Serra Mall, Stanford University, Stanford, CA 94305, USA 4 World Wildlife Fund, 1250 24th St NW, Washington, DC 20037, USA Key Words: ecosystem services, modeling, decision support, capacity building, conservation

4.1. Abstract

Spatial models of ecosystem services are increasingly important for informing

land use and development decisions. Understanding who uses these models and the

conditions associated with model use is critical for increasing their impact. We tracked

the use of The Natural Capital Project’s ecosystem service models (InVEST) over a 25-

month period and observed that 19 different models were run 43,363 times in 104

countries. We analyzed the 25,431 models runs that were not tied to model development

activity. Models for regulating services (e.g., carbon and sediment retention) were the

most commonly used. We analyzed relationships between country-level variables and use

of ecosystem service models and found that capacity (population, GDP per capita,

Internet and computer access, and InVEST trainings), governance, biodiversity, and

conservation spending are positively correlated with use. Measures of civic involvement

in conservation, carbon project funding, and forest cover are not correlated with use.

Using multivariate statistical models, we analyzed which combinations of country-level

variables best explain the use of InVEST and found further evidence that variables

109

related to capacity are the strongest predictors. Finally, we examined InVEST trainings in

detail and found a significant effect of trainings on subsequent use of InVEST models (p

< 0.001). Our results indicate that the general capacity of a country may limit uptake and

use of these decision support tools. Trainings are thus more likely to have a large impact

if other capacity aspects are present. Thoughtfully tracking and analyzing the use of

decision support models can help us understand the user audience and context, and design

better tools.

4.2. Introduction

Ecosystem services, the benefits that people receive from nature, are degraded

and projected to decline further over the first half of this century (Millennium Ecosystem

Assessment Program, 2005). Since many current policy and economic decisions do not

account for the values of ecosystem services (ES), planners and decision-makers are

increasingly focused on the management of ES as a viable way to understand and manage

human interactions with ecosystems (Braat & Groot, 2012; Holzman, 2012). The concept

of ES is becoming essential in many of today’s largest conservation organizations and

academic research about the topic has increased steadily over the past two decades

(Abson et al., 2014; Fisher et al., 2009).

As the ES concept grows more popular, there is more demand for ES information

that has the potential to affect policy decisions (Daily & Matson, 2008; Daily et al., 2009;

Mermet et al., 2014; Ruckelshaus et al., 2013). In particular, computer models that

generate spatially explicit information about ES are commonly used to inform decisions

(Burkhard et al., 2013; Crossman et al., 2012). The information these tools produce often

110

illustrates how landscapes provide different amounts and patterns of ES under different

future alternative scenarios (Goldstein et al., 2012; Lawler et al., 2014).

Several spatially-based decision support tools have emerged for ES assessment

(Bagstad et al., 2013). Freely available tools such as InVEST (Integrated Valuation of

Ecosystem Services and Tradeoffs), ARIES (ARtificial Intelligence for Ecosystem

Services), EVT (Ecosystem Valuation Toolkit), TESSA (Toolkit for Ecosystem Service

Site-based Assessment), and SolVES (Social Values for Ecosystem Services) have been

developed and tested in private and public environmental decision contexts (Bagstad et

al., 2014; Peh et al., 2013; Ruckelshaus et al., 2013; Sherrouse et al., 2011; Villa et al.,

2014). But there has not yet been a comprehensive, systematic appraisal of the use of

these tools. In order to improve user support and expand the reach of ES tools, it is vital

to track how, where, and when they are being used.

This study examines the emerging user network of one particular tool – InVEST.

We analyze where users are, which models they run, and country-level factors associated

with model usage over a 25-month period. InVEST, developed by the Natural Capital

Project, provides a suite of software models that can be used to map ES values and

compare trade-offs among development alternatives (Kareiva et al., 2011; Nelson et al.,

2009).

Research has found global patterns in where certain kinds of conservation

activities and needs occur. Countries in which conservation activities are more likely to

happen (based on lower protected area management costs, high numbers of endangered

species, and identification as important for conservation) also score poorly on measures

of corruption (McCreless et al., 2013; Smith et al., 2003, 2007). Other studies have found

111

that national characteristics such as number of threatened species, quality of governance,

and deforestation rates are associated with the location of REDD demonstration sites and

forest carbon projects (Cerbu et al., 2011; Lin et al., 2012; Lin et al., 2014). These studies

suggest that country-level factors could be associated with conservation science uptake.

Building from this theory, we hypothesize that countries with high use of

ecosystem service models also tend to have more capacity, more effective governance,

lower environmental quality (Cerbu et al., 2011; Lin et al., 2012; Lin et al., 2014), more

conservation spending, and more civic involvement in conservation (Table 1). We also

make two more specific hypotheses: first, that biodiversity-related InVEST model usage

is associated with lower environmental quality; and second, that carbon-related model

usage occurs in countries with more forests and more overall conservation spending.

Finally, we explore in more detail the effect that formal trainings have on use of ES

models. We hypothesize that the average use in countries with trainings is higher than in

countries without trainings, and that usage increases for a prolonged time period

following trainings.

Support of these hypotheses would indicate certain conditions that facilitate the

adoption and use of science-based tools. This understanding can help to predict patterns

of uptake for new tools and target capacity building efforts to increase scientific and

policy impact.

4.3. Methods

4.3.1. InVEST Data

112

When an InVEST model is run on a computer connected to the Internet, a log is

created with date, IP address, InVEST version, and model type. We analyzed 25 months

of these logs from June 2012 through June 2014. These data represent a network of

InVEST usage, however they do not include models runs for computers not connected to

the Internet, model activity done from outside the user interface (for example, through

Python scripts), or certain specific models that were not reporting usage information

during the timeframe of this study (such as coastal protection). We estimate that our

dataset of 43,363 model runs represents most InVEST model runs during the study

period.

We used IP addresses and GeoIP2 Precision Services provided by MaxMind to

identify the country in which each model run occurred. We used model type to identify

which type of ES model was run and we collapsed all possible model types into a concise

list of primary ES models (Supporting Information).

We used information about the InVEST version of each model run to exclude

model development and testing activity. For part of the analysis, we also screened out use

that occurred in the U.S. because a) much of this use was likely internal Natural Capital

Project scientists, and b) we did not want our results to be skewed by the fact that the

bulk of use was in the U.S. We focused on terrestrial/freshwater models rather than

marine because many of the marine models were not tracked over the entire study period

and the available country-level environmental quality data are about forests and terrestrial

biodiversity. The use of marine models at trainings in Portugal, Korea, Mexico, and

Canada was not included in our data, so these are conservative estimates of the ES model

use that occurred in those places.

113

4.3.2. Country-level Data

We gathered 19 country-level variables from global datasets (roman numerals

below) and grouped them into 5 categories that we hypothesize are associated with use of

ES models.

Capacity

We used estimates of i) population and ii) GDP/capita (in current US dollars) for

the year 2013 available online from World Bank Open Data (http://data.worldbank.org/).

iii) We used 2013 estimates of the number of Internet users per 100 people from World

Bank Indicators available online (http://data.worldbank.org/indicator/IT.NET.USER.P2).

iv) We used the percentage of households with a computer for the most recent year

within the 2008-12 range for which data are available. These data were collected from

national statistical offices by the International Telecommunication Union, the UN

specialized agency for information and communication technologies. More information

about the Internet and Computer Technology Data and Statistics Division is available

online (http://www.itu.int/en/ITU-D/Statistics). v) We included a variable to indicate

whether a country had a training prior to or during the study period. This variable was 0,

1, 2, or 3 based on the length of the training (0 if there was no training, 3 if there was a

training for 3 or more days).

Governance

The World Bank estimates Worldwide Governance Indicators at the country level

through surveys and consultations with citizens, experts, businesses, and international

organizations (Kaufmann et al., 2011). We used three governance indicators relevant to

the management of ESs: vi) government effectiveness (the quality of policy formulation

114

and implementation, vii) regulatory quality (the ability of governments to create policies

and regulations to promote private industry), and viii) control of corruption. We used

2013 estimates of percentile (0-100%) rank among all countries for these two indicators.

Data and background information for the indicators are available online

(www.govindicators.org).

Environmental Quality

The general level of environmental quality for a country is difficult to measure

and quantify. We focused on three main datasets for quantifiable, comparable

information about environmental quality for countries. Using information from multiple

sources allowed us to minimize the bias associated with any one dataset.

ix): The Global Environment Facility Benefits Index for Biodiversity is “a

composite index of relative biodiversity potential for each country based on the species

represented in each country, their threat status, and the diversity of habitat types in each

country. The index has been normalized so that values run from 0 (no biodiversity

potential) to 100 (maximum biodiversity potential)” (Pandey et al., 2006). Data and

background information are available online from World Bank Open Data.

x-xii): The Environmental Performance Index (EPI), estimated by the Yale Center

for Environmental Law and Policy, ranks how well countries protect ecosystems and

protect human health from environmental harm (Hsu et al., 2013). We used x) overall

EPI estimates for 2014, as well as two sub-indicators related to xi) forests (percent

change in forest cover between 2000 and 2012 in areas with greater than 50% tree cover)

and xii) biodiversity and habitat (an averaged composite of indices for critical habitat

protection, terrestrial protected areas with national biome weight, terrestrial protected

115

areas with global biome weight, and marine protected areas). Data and background

information for the indicators are available online (http://epi.yale.edu/).

xiii): Threatened mammal species includes the number of mammal species

(excluding whales and porpoises) classified by the International Union for Conservation

of Nature (IUCN) as endangered, vulnerable, rare, indeterminate, out of danger, or

insufficiently known. We used 2014 estimates for each country provided by the United

Nations Environmental Program, the World Conservation Monitoring Center, and the

IUCN Red List of Threatened Species. Data and background information are available

online from World Bank Open Data.

Conservation Spending

Two datasets provided information on country-level spending on conservation:

xiv) We used information about total average annual spending (in $ US million 2005)

from 2001-2008 as estimated by Waldron et al. (2013). This includes all flows of funding

estimated: international donors, domestic governments, trust funds, and self-funding via

user payments. We also used a database of REDD+ projects sourced through a number of

dedicated multilateral and bilateral climate funds to include xv) the amount of climate

finance and xvi) the amount of REDD funding countries received from 2003-2013. Data

and background information are available online

(http://www.climatefundsupdate.org/data).

Civil Society Involvement in Conservation

We followed McCreless et al. (2013) and focused on three datasets that measure

the extent to which civil society is involved in conservation efforts for many countries.

xvii) BirdLife International (BLI) is the largest global partnership of conservation

116

organizations in the world. We used data on citizen membership in BLI partner

organizations available online (www.birdlife.org/worldwide/national/index.html). We

standardized NGO membership numbers by country population to represent the

proportion of a county’s population that belongs to a leading local conservation NGO.

xviii) IUCN is the largest global environmental organization in the world. The

Environmental Sustainability Index provides a country level estimate of the number of

IUCN organizations per million people. These data are available online

(www.yale.edu/esi/c_variableprofiles.pdf). xix) Local Agenda 21 initiatives are

“measures undertaken and overseen by local authorities to address problems of

environmental sustainability, and represent the involvement of civil society in

environmental governance.” The Environmental Sustainability Index provides a country

level estimate of the number of local Agenda 21 initiatives per million people available

online (www.yale.edu/esi/c_variableprofiles.pdf).

4.3.3. Analysis

We examined the relationships that use of ES models has with each of the

country-level variables. We also examined relationships that carbon model use and

habitat model use have with particular variables. We used nonparametric Spearman rank

correlations because the individual datasets did not meet the assumptions required for

parametric correlations such as data having normal distributions (Crawley, 2007). All

data analyses were conducted in the statistical platform R (R, 2011).

We tested for correlation among country-level variables and used principal

components analysis (PCA) to reduce two groups of highly correlated variables with

117

correlation coefficients > 0.70: Governance and Biodiversity (Dormann et al., 2013). We

used the first principal component to capture over 90% of the variance for two groups of

highly correlated variables (Table 2). We then used model selection to rank all possible

linear combinations of variables to identify those statistical models that could best

explain the outcome variable of InVEST model usage. We used the R package MuMIn

(multi-model inference) for model selection, and ranked models based on AICc values to

identify the explanatory variables present among the top models (Barton, 2014). We did

model selection with only the variables that have data for all countries so that submodels

would not be fitted for different datasets. All possible combinations of our 8 predictor

variables resulted in 256 models being evaluated.

For our analysis on trainings, we initially narrowed our focus to 9 trainings that

occurred within our study period and that had at least 10 model runs within 30 days

before and after the training (Table 4). We defined InVEST trainings as organized

meetings of Natural Capital Project staff with registered event participants in order to

introduce and train people in the use of InVEST models. We gathered information on

trainings from Natural Capital Project records, including meeting agendas, participant

lists, training evaluation surveys, and facilitator notes.

We defined a “Before” period of time as the 13 weeks (approximately 90 days)

before a training, “After 1” as the 13-week period following a training, and “After 2” as

the next 13-week period (Figure 4). We computed a change factor as the ratio of model

runs between the “Before” and “After 1” periods. We also calculated average weekly

usage in these time periods to estimate the prolonged effect of trainings on InVEST

model usage.

118

To evaluate the effect of trainings statistically, we created a generalized linear

mixed model with country as a random effect and period as a fixed effect. We assumed a

Poisson distribution for weekly usage data and tested whether usage was different in the

periods “Before”, “After 1”, and “After 2” with a type III Wald chisquare test.

Finally, we combined our quantitative results with a qualitative analysis of

documents from these trainings, including detailed facilitator notes and participant

surveys (Creswell, 2009). This review focused on lessons learned by the training

facilitators and the following open-ended questions given to participants:

- Please identify two things you found the most useful from this course (favorite parts).

- What recommendations do you have for improving the course (least favorite parts)?

- What subject(s) would you like to see offered in future training sessions?

- Do you have any additional comments?

4.4. Results

The use of InVEST models increased over the 25-month study period (Figure 1).

A significant amount of overall use occurred in the U.S., but most of the growth over

time occurred in non-U.S. countries.

Across all countries, ES models related to habitat, water yield, carbon, sediment,

and nutrients were used most often (Figure 2). Based on the ES classifications provided

by the Millennium Ecosystem Assessment (Program) (2005), we identified each service

as provisioning, regulating, supporting, or cultural and found that 46% of model use was

for regulating services (see SI for how InVEST models represent each service type).

119

We found a number of positive and significant correlations between total use of

ES models and the country-level variables (Table 2). These relationships exist in all of

the hypothesized categories, especially for variables related to Capacity. For carbon

model use, none of the hypothesized relationships showed positive correlations. For

habitat model use, the Biodiversity PC and EPI variables had positive and significant

correlations. We examined whether trainings were more likely to occur in particular

kinds of places and did not find that trainings were significantly correlated with any of

the other country-level variables.

Model selection shows which variables are present in the statistical models that

best explain InVEST usage (Table 3). “Population,” “Training,” and “Internet” were

found to be the most common and important variables in the top 10 models. Other

variables such as those for biodiversity and the Environmental Performance Index appear

in some statistical models, but not with the same consistency.

In analyzing the effect of trainings, we found that the average use of ES models in

countries with trainings was over two times larger than in countries without trainings

(Figure 3). For the 9 trainings we analyzed, we typically observed a burst of usage during

the training and then more activity in the After 1 periods than the Before periods (Figure

4 and Supporting Information). The change factors (ratio of total model runs in After 1

period to model runs in Before period) showed that all but one of the cases had an

increase in model use following a training (Table 4). The average change factor for 1-day

trainings was 0.80 (n=2), 2-day trainings was 6.2 (n=3), and 3-or-more-day trainings was

3.5 (n=4). Using the generalized linear mixed model with country as a random effect, we

120

found a positive and significant effect of trainings on model use (Figure 5. χ22 = 154.8;

p<0.001).

The qualitative review of training evaluations further illuminated place-specific

factors than help explain use of ES models, such as connections to a university course or

a funded project with deadlines. Among the 7 cases reviewed, change factors were

highest for trainings that offered case studies and demonstrations relevant to participants’

on-going projects and when a specific deliverable using InVEST was due soon after the

event. Training attendees valued the opportunity to interact with experienced analysts and

developers of the InVEST models. This in-person support served to narrow the user-

developer divide that is a known barrier to decision-support tool uptake (Haklay &

Tobon, 2003). After establishing a rapport with training facilitators, participants felt more

comfortable applying the tool and requesting support following the event.

4.5. Discussion and Conclusion

Using a unique global dataset, we conduct the first quantitative analysis of the use

of ecosystem service modeling tools. We show that general capacity to use these kinds of

tools (i.e., population, GDP/capita, computer technology) as well as specific capacity

building for the tool in question (i.e., trainings) are the strongest predictors of model use

in a given country. While the effect of trainings varied widely among countries, in

general trainings had a positive and enduring effect on tool use. Understanding the

factors that encourage uptake and use of scientific tools can help target trainings, improve

tool design, and improve impact of scientific knowledge on decisions.

121

Formal trainings can build on a country’s existing capacity to use decision support

tools. Why do some trainings have a bigger impact than others? Trainings that are longer

than 1-day probably have a larger effect because they allow trainers and participants to

spend more time learning the details of how to use the tools. Extended trainings include

time for repeatedly running models and practice working through the different steps of an

ES assessment (Rosenthal et al., 2014). Other, site-specific factors related to a training

can explain the places where we observe larger impact, such as countries with reliable

Internet access and more spending on conservation-related research. Regardless of these

local factors, problem-based exercises that are simple, well designed and include detailed

guidance (e.g., step-by-step tutorials) show promise as an entry point for a range of

potential tool users (Verutes & Rosenthal, 2014). Further research on effective ways to

sequence introductory to more technical content for a diverse range of audiences can

inform creative approaches to building local capacity that actively engage participants in

learning a new technology.

Can country-level variables help us identify new, underserved audiences for

computer-based decision support tools, or understand which technologies may be most

relevant to a local place? We found evidence that country-level conditions can be used to

estimate the capacity for using InVEST models generally, but did not find that country-

level conditions were highly correlated with the use of carbon or biodiversity ES models.

It is likely that other variables beyond those we tested are associated with tool use. These

include some of the site-specific factors uncovered in our qualitative assessment, such as

a funded ES project with deadlines, as well as factors related to the presence of ES

122

concepts in government policies, overall amounts of science research activity, and levels

of education in a population.

Limitations to this study are worth noting. First, as mentioned above, some ES

models (i.e. marine models) were not included in our data set, so we almost certainly

underestimated the impacts of trainings that focused in part on these models. Second, our

data capture where a given model is used, not the location where ES are being evaluated.

Researchers in one country can run InVEST models focused on another, and some

InVEST trainings included participants from other countries, who likely went on to use

InVEST outside the country where the training was held. Excluding the US from our

analyses removes the largest source of this issue, but further subsetting to countries where

we are certain models are being used in-country reduces the size of our dataset rapidly.

Our analyses therefore pertain to where models are used rather than where they are

applied. Third, our analyses focused on InVEST, but of course several other ES models

and computer-based tools exist. We are not aware of equivalent tracking data for any

other tool, but we expect they would display patterns associated with national-scale

capacity and training opportunities.

Future research in this area would benefit from in-depth qualitative analyses to

better understand the factors that lead to differences in effectiveness of trainings. In

addition, understanding the relationships among users (e.g., through surveys of users)

could help to illuminate how technology diffuses through social networks

(Haythornthwaite, 1996). User surveys could also help to clarify user demographics, the

decision contexts in which the tools are used, and the ways in which outputs are used to

inform decisions (McKenzie et al., 2014). Finally, model developers could make several

123

simple additions to the information reported for each model use, including location of the

region being assessed, and saving records for later reporting if the computer is not

connected to the Internet.

People may use an ES model for any of a number of reasons: they receive

training, they find decision support tools useful for a particular context, they have project

deadlines or an academic adviser nudging them to produce results, etc. Efforts to increase

use of these models should therefore focus equally on understanding these drivers,

building capacity generally, and providing specific training in the tools. Formal training

opportunities that provide locally-relevant demonstrations of the tool and follow-up

activities to reinforce what was learned are an effective way to support the continued

usage of spatial models in ES assessments. If country-level factors can predict use along

with trainings, then we need to be aware of which countries have the basic capacity to use

these models. And we need to think about more than just trainings – having key

conditions in place, such as the capacity variables illuminated in this study, will enable

people to use what they learn. Tracking and explaining tool use can lead to more strategic

deployment of technology and smarter applications of these models for informing real

world decisions..

4.6. Acknowledgements

Thanks to Rebecca Chaplin-Kramer, Gillian Galford, and Richard Sharp for

helpful comments on an earlier version of this manuscript. SP was supported by the Gund

Institute for Ecological Economics and the Rubenstein School of Environment and

Natural Resources at the University of Vermont during this research.

124

4.7. References

Abson, D. J., Wehrden, H. v., Baumgartner, S., Fischer, J., Hanspach, J., Hardtle, W., et al. (2014). Ecosystem services as a boundary object for sustainability. Ecological Economics, 103, 29-37.

Bagstad, K. J., Semmens, D. J., Waage, S., & Winthrop, R. (2013). A

comparative assessment of decision-support tools for ecosystem services quantification and valuation. Ecosystem Services.

Bagstad, K. J., Villa, F., Batker, D., Harrison-Cox, J., Voigt, B., & Johnson, G.

W. (2014). From theoretical to actual ecosystem services: mapping beneficiaries and spatial flows in ecosystem service assessments. Ecology and Society, 19(2).


project. Braat, L. C., & Groot, R. d. (2012). The ecosystem services agenda: bridging the

worlds of natural science and economics, conservation and development, and public and private policy. Ecosystem Services, 1(1), 4-15.

Burkhard, B., Crossman, N., Nedkov, S., Petz, K., & Alkemade, R. (2013).

Mapping and modelling ecosystem services for science, policy, and practice. Ecosystem Services, 4, 1-3.

Cerbu, G. A., Swallow, B. M., & Thompson, D. Y. (2011). Locating REDD: A

global survey and analysis of REDD readiness and demonstration activities. Environmental Science & Policy, 14(2), 168-180.

Crawley, M. J. (2007). The R Book. West Sussex, England: John Wiley & Sons

Ltd. Creswell, J. W. (2009). Research design : qualitative, quantitative, and mixed

methods approaches (3rd ed.). Thousand Oaks, Calif.: Sage Publications. Crossman, N. D., Burkhard, B., & Nedkov, S. (2012). Quantifying and mapping

ecosystem services. International Journal of Biodiversity Science, Ecosystem Services and Management, 8(1-2), 1-4.

Daily, G. C., & Matson, P. A. (2008). Ecosystem services: From theory to


125

Daily, G. C., Polasky, S., Goldstein, J., Kareiva, P. M., Mooney, H. A., Pejchar, L., et al. (2009). Ecosystem services in decision making: time to deliver. Frontiers in Ecology and the Environment, 7(1), 21-28.

Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carre, G., et al.

(2013). Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 27-46.

Fisher, B., Turner, R. K., & Morling, P. (2009). Defining and classifying

ecosystem services for decision making. Ecological Economics, 68, 643-653. Goldstein, J. H., Caldarone, G., Duarte, T. K., Ennaanay, D., Hannahs, N.,

Mendoza, G., et al. (2012). Integrating ecosytem-service tradeoffs into land-use decision. PNAS, 109(19), 7565-7570.

Haklay, M., & Tobon, C. (2003). Usability evaluation and PPGIS: towards a user-

centered design approach. International Journal of Geographic Information Science, 17(6), 577-592.

Haythornthwaite, C. (1996). Social network analysis: An approach and technique

for the study of information exchange. Library & Information Science Research, 18(4), 323-342.

Holzman, D. C. (2012). Accounting for nature's benefits: the dollar value of

ecosystem services. Environmental Health Perspectives, 120(4), 152-157. Hsu, A., Johnson, L., & Lloyd, A. (2013). Measuring Progress: A Practical Guide

From the Developers of the Environmental Performance Index (EPI). New Haven: Yale Center for Environmental Law & Policy.

Kareiva, P. M., Tallis, H., Ricketts, T. H., Daily, G. C., & Polasky, S. (2011).

Natural Capital: Theory and practice of mapping ecosystem services. Oxford, UK: Oxford University Press.

Kaufmann, D., Kraay, A., & Mastruzzi, M. (2011). The Worldwide Governance

Indicators: Methodology and Analytical Issues. Hague Journal on the Rule of Law, 3(2), 220-246.

Lawler, J. J., Lewis, D. J., Nelson, E., Plantinga, A. J., Polasky, S., Withey, J. C.,

et al. (2014). Projected land-use change impacts on ecosystem services in the United States. PNAS, 111(20), 7492-7497.

Lin, L., Pattanayak, S. K., Sills, E. O., & Sunderlin, W. D. (2012). Site selection

for forest carbon projects. In A. Angelson (Ed.), Analysing REDD. Indonesia: CIFOR.

126

Lin, L., Sills, E., & Cheshire, H. (2014). Targeting areas for Reducing Emissions from Deforestation and forest Degradation (REDD+) projects in Tanzania. Global Environmental Change-Human and Policy Dimensions, 24, 277-286.

Martinez-Harms, M. J., & Balvanera, P. (2012). Methods for mapping ecosystem

service supply: a review. International Journal of Biodiversity Science, Ecosystem Services and Management, 8(1-2), 17-25.

McCreless, E., Visconti, P., Carwardine, J., Wilcox, C., & Smith, R. J. (2013).

Cheap and Nasty? The Potential Perils of Using Management Costs to Identify Global Conservation Priorities. PloS ONE, 8(11).



Mermet, L., Laurans, Y., & Lemenager, T. (2014). Tools for What Trade?


Millennium Ecosystem Assessment (Program). (2005). Ecosystems and human

well-being : synthesis. Washington, DC: Island Press. Nelson, E., Mendoza, G., Regetz, J., Polasky, S., Tallis, H., Cameron, D. R., et al.


Pandey, K. D., Buys, P., Chomitz, K., & Wheeler, D. (2006). Biodiversity

Conservation Indicators: New Tools for Priority Setting at the Global Environmental Facility. Washington, DC: World Bank, Development Economics Research Group and Environment Department.

Peh, K. S. H., Balmford, A., Bradbury, R. B., Brown, C., Butchart, S. H. M.,

Hughes, F. M. R., et al. (2013). TESSA: A toolkit for rapid assessment of ecosystem services at sites of biodiversity conservation importance. Ecosystem Services, 5, 51-57.

R. (2011). R: A language and environment for statistical computing: R

Development Core Team. Rosenthal, A., Verutes, G., McKenzie, E., Arkema, K. K., Bhagabati, N., Bremer,


127



Sherrouse, B. C., Clement, J. M., & Semmens, D. J. (2011). A GIS application for

assessing, mapping, and quantifying the social values of ecosystem services. Applied Geography, 31(2), 748-760.

Smith, R. J., Muir, R. D. J., Walpole, M. J., Balmford, A., & Leader-Williams, N.

(2003). Governance and the loss of biodiversity. Nature, 426(6962), 67-70. Smith, R. J., Muir, R. D. J., Walpole, M. J., Balmford, A., & Leader-Williams, N.

(2007). Revisiting the links between governance and biodiversity loss. Conservation Biology, 21(4), 899-900.

Verutes, G. M., & Rosenthal, A. (2014). Using Simulation Games to Teach

Ecosystem Service Synergies and Trade-offs. Environmental Practice, 00(0), 1-11. Villa, F., Bagstad, K. J., Voigt, B., Johnson, G. W., Portela, R., Honzak, M., et al.

(2014). A Methodology for Adaptable and Robust Ecosystem Services Assessment. PloS ONE, 9(3).

Waldron, A., Mooers, A. O., Miller, D. C., Nibbelink, N., Redding, D., Kuhn, T.

S., et al. (2013). Targeting global conservation funding to limit immediate biodiversity declines. PNAS, 110(29), 12144-12148.

128

Figure 4.1: Growth in the use of InVEST models over time. Model use occurred in 102 different

countries with 44% of all use occurring in the U.S. For non-U.S. countries, there were 14,301 model runs with 43% of use occurring in 5 countries: the UK (1554), Germany (1491), China (1209), France

(1074), and Colombia (780).

129

Figure 4.2: Types of ecosystem service models used. 46% of all model use is for regulating services. “Rios” focuses on freshwater provisioning services but includes other service types as well.

130

Figure 4.3: Comparison of average use in non-U.S. countries with and without trainings. Error bars depict standard error. There was a significant difference in the average use for countries with (mean

= 280, sd = 81) and countries without (mean = 107, sd = 29) trainings ( t(22.6) = 2.0 , p= 0.05).

131

Figure 4.4: Example of a training in the UK. Model use spikes during the training. We compared the

number of models runs and average weekly use for one 13-week period before and two 13-week periods after trainings.

132

Figure 4.5: The positive effect of trainings on model usage across 9 cases. “Before” is average weekly model use and standard error for a 13-week period (approximately 90 days) before a training. “After

1” is for a 13-week period after a training and “After 2” is for the following 13-week period.

133

Table 4.1: The main hypothesized drivers of ecosystem service model use, predicted

relationships, and justifications.

Category Relationship with use of ES models

Justification

Capacity + Places with more people, trainings, and access to technology have more basic capacity to use ES models

Governance + Stronger systems of governance enables more use of sophisticated decision support tools

Environmental quality

– Worse environmental quality makes it more likely that people will use tools to inform environmental decisions

Conservation spending

+ Places with higher levels of conservation spending have an established presence of environmental organizations and a higher likelihood that ES models will be used

Civic engagement in conservation

+ People in places with higher rates of involvement in conservation organizations are more likely to use ES models

134

Table 4.2: Correlation results comparing model use with country level variables. Bold rows

indicate positive and significant correlations. N is the number of countries for which those data are

available. WE calculated the first principal component of Governance and Biodiversity variables to

capture >90% of the variance in the underlying variables. “Governance PC” includes government

effectiveness, regulatory quality, and control of corruption (for pairwise correlations, rho=0.91, 0.95,

and 0.87); “Biodiversity PC” includes GEF Benefits Index of Biodiversity and mammals (rho=0.77).

** p < 0.01; * p < 0.05; + p < 0.1

Variable comparison N Spearman’s rho 95% CI p-value Total model use ~ … CAPACITY GDP per capita 94 0.347 (0.155, 0.513) 0.00062 ** Population 94 0.482 (0.309, 0.624) 1.8E-7 ** Internet 94 0.316 (0.121, 0.487) 0.0019 ** Computers 86 0.237 (0.0269, 0.428) 0.028 * Trainings 94 0.273 (0.0748, 0.451) 0.0077 ** GOVERNANCE Governance PC 94 0.335 (0.142, 0.504) 0.00095 ** ENVIRONMENTAL QUALITY Biodiversity PC 94 0.317 (0.122, 0.488) 0.0019 ** EPI 94 0.262 (0.062, 0.441) 0.011 * EPI Forests 86 -0.0179 (-0.229, 0.195) 0.87 EPI biodiversity 94 0.131 (-0.0732, 0.325) 0.21 CONSERVATION SPENDING Conservation spending 92 0.603 (0.454, 0.719) 2.1E-10 ** REDD 31 0.0580 (-0.136, 0.540) 0.76 Climate finance 94 0.00992 (-0.193, 0.212) 0.92 CIVIC ENGAGEMENT IN CONSERVATION BLI 62 -0.248 (-0.468, 0.00241) 0.052 + IUCN 89 0.0885 (-0.122, 0.291) 0.41 Agenda 21 74 0.0867 (0.000496, 0.435) 0.46 Carbon model use ~ … EPI Forests 86 0.0252 (-0.188, 0.236) 0.82 REDD 31 0.127 (-0.238, 0.461) 0.50 Climate finance 94 0.0311 (-0.173, 0.232) 0.77 Habitat model use ~ … Biodiversity PC 94 0.244 (0.043, 0.425) 0.018 ** EPI 94 0.236 (0.0352, 0.419) 0.022 * EPI Forests 86 0.0153 (-0.197, 0.226) 0.89 EPI biodiversity 94 0.153 ( -0.0509, 0.345) 0.14

135

Table 4.3: Model selection results for the top 10 statistical models by AICc value. Statistical

models are listed by row in rank order (the first has the lowest AICc value corresponding to best fit)

with a check mark for variables that are included in the statistical model. Only the 8 variables that

contained data for all countries were included in model selection.

Model Biod

iver

sity_

PC

EPI

EPI b

iodi

vers

ity

GD

P

Gov

erna

nce_

PC

Inte

rnet

Popu

latio

n

Trai

ning

df Log likelihood ΔAICc

1 - - - - - ✔ ✔ ✔ 5 -652.62 0.00 2 - - ✔ - - ✔ ✔ ✔ 6 -652.25 1.55 3 ✔ - - - - ✔ ✔ ✔ 6 -652.27 1.58 4 - ✔ - - - - ✔ ✔ 5 -653.47 1.70 5 - ✔ - - - ✔ ✔ ✔ 6 -652.52 2.09 6 - - - - ✔ ✔ ✔ ✔ 6 -652.61 2.26 7 - - - ✔ - ✔ ✔ ✔ 6 -652.62 2.27 8 - - - - - ✔ ✔ - 4 -654.89 2.31 9 - - - - ✔ - ✔ ✔ 5 -654.04 2.83

10 - - ✔ - - ✔ ✔ - 5 -654.23 3.22

136

Table 4.4: Comparison of ES model use before and after trainings. Change factor is the ratio

of use in 13-week periods after/before training.

Country Date of training

Model use before training

Model use after training

Change factor

Argentina** 9/12/2013 14 52 3.71 Cambodia*** 6/17/2013 6 13 2.17 Canada** 2/04/2013 64 92 1.44 Chile*** 9/09/2013 17 82 4.82 Korea* 9/11/2012 46 54 1.17 Peru*** 5/27/2013 18 40 2.22 Spain** 11/18/2013 13 175 13.46 UK1*** 10/15/2013 137 661 4.82 UK2* 3/07/2013 50 23 0.42

* 1 day training, ** 2-day, *** 3-day or longer

137


4.8.1. Ecosystem Service Models in InVEST

Table 4.5: Ecosystem service types for InVEST models.

InVEST model category

Ecosystem service type

Specific models included

Agriculture Provisioning "agriculture" Blue carbon Regulating "blue_carbon"

"blue_carbon_biophysical" "blue_carbon_preprocessor"

Carbon Regulating "carbon_biophysical" "carbon_biophysical\" "carbon_combined" "carbon_valuation"

Coastal vulnerability Regulating "coastal_vulnerability" Finfish Provisioning "finfish_aquaculture" Habitat Supporting "habitat_quality"

"biodiversity_arc" "biodiversity_biophysical" "biodiversity_biophysical\"

Habitat risk assessment Supporting "hra" "HRA_LaunchGUI_arc" "hra_preprocessor"

Marine water quality Provisioning "marine_water_quality_biophysical" "marine_water_quality_biophysical\"

Nutrient Regulating "nutrient" Pollination Regulating "pollination"

"pollination_biophysical" "pollination_valuation"

Recreation Cultural "recreation_client" "recreation_client_init" "recreation_client_scenario"

Rios Provisioning "rios" "rios_0.3.0" "rios_sediment"

Scenic quality Cultural "aesthetic_quality" "scenic_quality"

Sediment Regulating "sediment" "sediment_biophysical"

Timber Provisioning "timber" "timber\"

Water scarcity Provisioning “water_scarcity" Water yield Provisioning "hydropower_valuation"

138

"hydropower_water_yield" “water_yield" “water_yield\"

Wave energy Provisioning “wave_energy" “wave_energy_biophysical" “wave_energy_biophysical\" “wave_energy_valuation"

Wind energy Provisioning “wind_energy" “wind_energy_biophysical" “wind_energy_uri_handler"

Removed from analysis, either with development activity or separately to filter out non-model use Overlap - "overlap_analysis"

"overlap_analysis_mz" "OverlapAnalysis_arc"

Scenarios - "scenario_generator" Other model types in data log

- "#VALUE!" "adept" "adept_core" "coastal_vulnerability_post_processing" "crop_production" "developme" "development" "fisheries" "GridSeascape_arc" "habitat_suitability" "malaria" "monthly_water_yield" "monthly_water_yield_old" "ntfp" "percent_land" "pollination_10_arc" "recreation_init" "recreation_scenario" "rios_beer" "rios_porter" "rios_rsat" "routedem" "scenario_generator_summary" "sdr" "test" "test_invest_2" "test_string_submission"

139

"test!!! version number?" “viewshed_grass" “wind_energy_valuation"

140

4.8.2. Model Use in Countries with Trainings

Model use over time and average weekly model use (with standard error) in 13-week periods before and after trainings. We analyzed the effect of trainings in detail for the following 9 trainings.

Figure 4.6: Argentina 2-day training 9/12/2013

141

Figure 4.7: Cambodia 3-day training 6/17/2013

142

Figure 4.8: Canada 2-day training 2/04/2013

143

Figure 4.9: Chile 3-day training 9/09/2013

144

Figure 4.10: Korea 1-day training 9/11/2012

145

Figure 4.11: Peru 3-day training 5/27/2013

146

Figure 4.12: Spain 2-day training 11/18/2013

148

Figure 4.13: UK1 3-day training 10/15/2013 and UK2 1-day training 3/07/2013

149

CHAPTER 5: CONCLUSION

Many within the conservation community have embraced ecosystem services as a

strategy for protecting and restoring nature. They aim to inform land use decisions about

the value of nature. Two key assumptions underpin this strategy. The first is that

incorporating information about the value of ecosystem services into decisions leads to

improved environmental and human well-being outcomes. The second assumption,

antecedent to the first, is that scientific knowledge about ecosystem services actually

changes decisions. These assumptions imply that there is a set of different potential future

states that society can choose from, and that some states are better or worse than others in

terms of environmental quality and human well-being. The role of ecosystem services

knowledge is to guide decisions that will result in a future state with better conditions.

In this dissertation, I focused on what difference ecosystem services knowledge

actually makes to decision-making at regional and national levels. I have taken particular

interest in how and why people responsible for making land use decisions change their

minds and begin to consider new knowledge about the benefits provided by nature. I

described only some of the many factors that make scientific knowledge about ecosystem

services used and useful. I found signs that ecosystem services knowledge can

significantly impact how people understand and frame environmental problems, evaluate

options for how to respond to these problems, and decide which option or options to

pursue. My work at the science policy interface fills the gap between scientists doing

research on ecosystem services and the policy decisions they often hope to affect.

150

5.1. Legitimacy

In the chapter on “what explains the impact of ecosystem services knowledge on

decisions,” I found legitimacy to be the most important explanatory variable of all the

factors I analyzed. However, in the chapter on “evaluating the impact of an ecosystem

service assessment,” legitimacy was not as prominent an issue. Legitimacy of knowledge

only arose a few times in the 23 interviews I conducted with land use decision-makers in

California, often in the form of local knowledge about ecosystems held by farmers. This

less significant role for legitimacy could have been because the Healthy Lands, Healthy

Economies Initiative did not focus centrally on incorporating diverse perspectives to

produce unbiased assessment outputs, whereas much of the Natural Capital Project’s

work emphasizes informing and supporting the processes of land use decision-making

across many different decision contexts. Also, the people I interviewed in California were

relatively local in terms of the scale of their conservation decisions, so their pre-existing

connections with other local groups and sources of knowledge could have made

legitimacy less of a prominent issue for them. Or, perhaps the way legitimacy is defined

and cultivated is different enough between California and the international Natural

Capital Project cases that legitimacy was not always seen as a critical factor.

The salience, credibility, and legitimacy of a body of information are ultimately a

matter of perception. They are also claims about and on knowledge, and in this way they

are very much a matter of power and politics. Knowledge is contested and constantly

negotiated. Knowledge producers, holders, distributors, and users are all enmeshed in

political processes of evaluating what is true and deciding on the supposedly best way

forward for society. Credibility and legitimacy, especially, are claims on knowledge that

151

can be used to bolster political positions or to deny the arguments of others. For example,

in the case of climate change in the U.S., questioning the credibility and legitimacy of

climate change knowledge and climate scientists themselves has been used to justify

inaction. Similar claims about the credibility and legitimacy of ecosystem services

knowledge are made in: assessments of pollution damage costs (i.e. who pays damage

costs for oil spills, how much, and to whom); decisions whether to allow development in

ecologically sensitive areas (i.e. should we protect wetlands, or require mitigation for

their disruption); and decisions to grant permits to extract natural resources (i.e. where

can companies extract natural gas or mine rare earth minerals, and what are the

impacts/benefits of these decisions to different groups of people over varying time

scales?). Who defines the attributes of knowledge used in these kinds of decisions? In

terms of legitimacy, who has say about who gets invited to meetings and which points of

view are represented in the process of producing or using knowledge?

These are important questions for ecosystem service proponents to consider. The

basic idea of natural capital does not resonate with some populations. Indigenous

populations may have established and effective ways to understand and interact with

nature, and their livelihoods and cultural traditions can be threatened by the application of

ecosystem services to the land. Applications of ecosystem services need to consider how

to incorporate traditional knowledge into assessments and how to ensure that a

framework designed for conservation is not also systematically providing advantages to

one group over another, which can happen when finance mechanisms such as payments

for ecosystem services are established. The inclusion of diverse perspectives in

ecosystem service projects can take time and effort, but it is essential to consider these

152

issues of power, privilege, equity, and diversity when generating knowledge that may be

used to influence policy.

5.2. Capacity Building

In the course of my research, I considered whether the ecosystem services

projects I assessed were really helping those who most need the help. In California,

conservation is relatively well established and supported, whereas in Colombia, there

may be other more pressing issues that take precedence. The Natural Capital Project

attempts to make ecosystem service science available to a broader set of people, but it is

still biased in the places it works and the partnerships it develops. It is difficult to

accurately assess a place’s level of environmental and human development problems, as

well as a place’s capacity to address environmental and development challenges. But it is

clear from the chapter on “global usage of ecosystem service models” that the majority of

InVEST model usage is in the U.S., followed by the UK, China, Germany, and France.

These countries are not representative of much of the rest of the world, and they are not

the countries with the most need for ecosystem service decision support (though it could

be argued that China does have high need for this kind of decision support because of

population pressures and environmental problems that are significant when compared

with other countries around the world). What can be done about this?

The long game is to build capacity for multiple approaches to understanding and

addressing environmental challenges. An ecosystem services approach, one of many such

options, is being demonstrated and proven in a diversity of contexts including countries

with development assistance needs. Building capacity in places with high development

153

needs could enable a shift in human-nature interactions before significant exploitation of

natural resources occurs. Building capacity in places with less development needs could

reinforce certain inequities between countries. This issue can be mitigated by selecting

locations for trainings that are accessible to diverse participants; offering scholarships to

support participation in trainings or ecosystem service applications; and designing

strategies to provide equal access. Capacity building can happen through trainings,

development and technology assistance, and targeted decision support that engages in

deliberative processes and includes diverse stakeholders. The application of ecosystem

services has serious limitations, but it also has great potential to inspire collective action

for more sustainable forms of development.

5.3. Lessons Learned

In the process of studying how ecosystem services knowledge impacts decisions, I

have learned several important lessons. In general, there is a clear need to more carefully

evaluate the social-ecological impacts of conservation work. We have a limited

understanding of how conservation science becomes used in making policy decisions.

When conservation action does occur, we often have incomplete or anecdotal evidence

about how projects, programs, or policies perform. We need to feed better evidence of

impact into a virtuous cycle in which we test different conservation strategies,

systematically learn about what works, and continuously improve the practice of

conservation.

I have also learned that it is incredibly difficult to track how decision-makers use

particular kinds of knowledge. Many factors beyond scientific knowledge go into land

154

use decisions – potential for profit, voting and elections, cultural traditions, immediate

and pressing priorities such as roads, food, human health, etc. – and it can be difficult if

not impossible to isolate a causal role of each of these factors. It makes sense that this

area of research has been understudied for ecosystem services.

The impact of knowledge is not just about content of the knowledge. It is about

people’s relationships with knowledge, and the processes through which their

relationships develop. The data for studying these relationships are often poor.

Integrating impact evaluation into the design of ecosystem service programs will improve

data availability. Baseline data collection is key and sets the stage for subsequent

evaluation. In situations with existing long-term datasets, statistical matching techniques

between treated and non-treated units can uncover some of the causal pathways through

which ecosystem services projects make a difference.

The science and policy of ecosystem services would benefit from consensus on

clear, measurable ecological and social outcomes. Consensus on what it means for a

program to be successful will enable more consistent, rigorous studies of impact. Clear

ideas of success and how to measure it can also be used to test principles and guidelines

that are important for the success of ecosystem service interventions. In the last several

years, there has been a growing focus on identifying and validating the “enabling

conditions” for success, which can only be done with a clearer vision of the outcomes

that constitute success. Success might look like stabilized populations of species of

concern within some agreed-upon baseline range; decreasing rates of land use change and

resource consumption; positive trends in environmental quality over time as measured by

levels of key air and water pollutants; or maintenance of human activity within a safe

155

operating space as defined by Rockstrom’s planetary boundaries. Not all forms of success

need to involve quantitative metrics, but in order for us to gauge progress toward or away

from a goal we must be able to assess progress consistently across projects. The

“pathways to impact” diagram is a useful conceptual framework in this regard.

An important aspect of exploring what enables program success is to consider

barriers to success and knowledge uptake. Recognizing barriers and admitting where

ecosystem service interventions have been less successful (or not worked at all) could

provide valuable lessons for future efforts. Practitioners and funders are understandably

averse to talking about project failures. But the conservation community can cultivate a

culture of learning and develop capacity for future success by documenting these cases

and honestly sharing about pitfalls and mistakes. Genuine curiosity about the social-

ecological impacts of conservation programs should uncover accounts of failure, not just

stories of success.

Incorporating these lessons into conservation work is a challenge. Any

organization struggles with how evaluation can divert precious, limited resources away

from actually doing projects. And often, it is not easy to evaluate real-world ecosystem

service projects across a variety of cultures and decision contexts. Despite the challenges

in evaluating the impact of ecosystem services knowledge, it is still worth doing for the

conservation community. Conservation organizations need to evaluate their own impact

honestly. They could also invest in reliable third party reviews to provide less biased

appraisals. In order to do this well, the conservation community would benefit from first

defining clearer visions of success (goals), then agreeing on how we would know if we’re

156

making progress toward goals or not (evaluation), and finally deciding what programs

and projects to implement (action).

Other fields of study can offer valuable insights. Quasi-experimental methods

from policy evaluation are applicable to many efforts aimed at developing ecosystem

services policy. The international development world has an established culture of

specifying program outcomes and evaluating the performance of individual interventions

and larger-scale programs. Education scholars regularly evaluate the social impacts of

programs on different populations. Behavioral economics and psychology could

illuminate how and why human decisions about land use are made in various contexts.

Anthropology and ethnographic studies are vital to uncovering the cultural and social

forces at play in a particular place and at the intersection of the science and policy

spheres. And complexity research, with its increasingly relevant findings about the

dynamics of complex social-ecological systems, could help understand governance

networks in PES programs or improve the accuracy of ecosystem service models.

5.4. Concluding Thoughts

At this point in the evolution of the ecosystem services field, there are hundreds

of case studies, sound theory about the production, flow, and distribution of ecosystem

services across landscapes, an academic journal dedicated to the topic, and increasing

demand for ecosystem service assessments from decision-makers. As the field matures, it

is important to undergo an intentional process of self-reflection, in the interest of

consciously evaluating whether hoped-for impacts are being realized. Taking stock

involves answering the fundamental question of how to know whether ecosystem service

157

projects are successful – both in terms of improving decisions and generating positive

environmental and human well-being outcomes. With a clear idea of what success looks

like and how to evaluate it, the ecosystem services community can make measureable

progress toward conservation goals and move forward with confidence that programs are

achieving the intended positive impacts.

As I conclude this dissertation, humans are rapidly changing the planet and

scientists are doing an excellent job documenting declines in global environmental

quality. But while we increasingly crowd the planet and impact the water, air, and land,

we still depend on the diversity of life and nature to support both our immediate well-

being and our long-term survival. Now we need to draw sharper connections between

declines in environmental quality, proposed causes of environmental decline, and

importantly, proven strategies for improving the interactions between people and nature.

In order to move toward ecological sustainability, it is critical that we imagine,

understand, and expand the beneficial relationships between people and the environment.

The concept of ecosystem services contributes to sustainability by providing a

much-needed link between humans and the environment. By explicitly measuring and

mapping the flow of benefits from the environment to people, ecosystem service projects

can illuminate positive interactions between people and the Earth. While the idea of

ecosystem services has an anthropocentric focus on the benefits people receive from

nature, it also suggests there are benefits that nature receives from people.

Our choice of perspective dictates our relationship with the environment. A focus

solely on the one-way flow of benefits to people is incomplete. Reframing ecosystem

services in the context of the whole system underscores people’s reciprocal connections

158

with the natural world. Bringing a deeper sense of interdependence into the ecosystem

services field provides an opportunity to enrich these connections, and an invitation for

people to live and work in service of ecosystems.

159

APPENDIX A: FULL LIST OF REFERENCES

Abson, D. J., Wehrden, H. v., Baumgartner, S., Fischer, J., Hanspach, J., Hardtle, W., et al. (2014). Ecosystem services as a boundary object for sustainability. Ecological Economics, 103, 29-37.

Amara, N., Ouimet, M., & Landry, R. (2004). New evidence on instrumental,

conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75-106.

Andam, K. S., Ferraro, P. J., Sims, K. R. E., Healy, A., & Holland, M. B. (2010).

Protected areas reduced poverty in Costa Rica and Thailand. PNAS, 107(22), 9996-10001.

Anderson, J. E. (1975). Public Policy Making. New York: Praeger.

Andrews, C. J. (2002). Humble analysis : the practice of joint fact-finding. New

York: Praeger. Arkema, K. K., Verutes, G., Wood, S. A., Clarke, C., Rosado, S., Canto, M., et al.

(In Press). Improving the margins: Modeling ecosystem services leads to better coastal plans. PNAS.

Arriagada, R. A., Ferraro, P. J., Sills, E. O., Pattanayak, S. K., & Coredero-

Sancho, S. (2012). Do Payments for Environmental Serices Affect Forest Cover? A Farm-Level Evaluation from Costa Rica. Land Economics, 88(2), 382-399.

Bagstad, K. J., Johnson, G. W., Voigt, B., & Villa, F. (2013). Spatial dynamics of

ecosystem service flows: A comprehensive approach to quantifying actual services. Ecosystem Services, 4, 117-125.

Bagstad, K. J., Semmens, D. J., Waage, S., & Winthrop, R. (2013). A

comparative assessment of decision-support tools for ecosystem services quantification and valuation. Ecosystem Services.

Bagstad, K. J., Villa, F., Batker, D., Harrison-Cox, J., Voigt, B., & Johnson, G.

W. (2014). From theoretical to actual ecosystem services: mapping beneficiaries and spatial flows in ecosystem service assessments. Ecology and Society, 19(2).

Balmford, A., Bruner, A., Cooper, P., Costanza, R., Farber, S., Green, R. E., . . .

Turner, R. K. (2002). Economic Reasons for Conserving Wild Nature. Science, 297, 950-953.

Bamberger, M. (2012). Introduction to Mixed Methods in Impact Evaluation

Impact Evaluation Notes (Vol. 3, pp. 1-42): InterAction and The Rockefeller Foundation.

160


project. Bateman, I. J., Harwood, A. R., Abson, D. J., Andrews, B., Crowe, A., Dugdale,

S., . . . Termansen, M. (2014). Economic Analysis for the UK National Ecosystem Assessment: Synthesis and Scenario Valuation of Changes in Ecosystem Services. Environmental & Resource Economics, 57(2), 273-297.

Batker, D., Schwartz, A., Schmidt, R., Mackenzie, A., Smith, J., & Robins, J.

(2014). Nature's Value in Santa Clara County. Tacoma, WA and San Jose, CA: Earth Economics and the Santa Clara Valley Open Space Authority.

Berkes, F. (2009). Evolution of co-management: Role of knowledge generation,

bridging organizations and social learning. Journal of Environmental Management, 90(5), 1692-1702.

Bernard, H. R. (2011). Research methods in anthropology : qualitative and

quantitative approaches (5th ed.). Lanham, Md.: AltaMira Press. Bhagabati, N. K., Ricketts, T., Sulistyawan, T. B. S., Conte, M., Ennaanay, D.,

Hadian, O., et al. (2014). Ecosystem services reinforce Sumatran tiger conservation in land use plans. Biological Conservation, 169, 147-156.

Bishop, J. (2012). The economics of ecosystems and biodiversity in business and

enterprise. New York: EarthScan. Boumans, R., Costanza, R., Farley, J., Wilson, M. A., Portela, R., Rotmans, J., . . .

Grasso, M. (2002). Modeling the dynamics of the integrated earth system and the value of global ecosystem services using the GUMBO model. Ecological Economics, 41(3), 529-560.

Braat, L. C., & Groot, R. d. (2012). The ecosystem services agenda: bridging the

worlds of natural science and economics, conservation and development, and public and private policy. Ecosystem Services, 1(1), 4-15.

Branas, C. C., Cheney, R. A., MacDonald, J. M., Tam, V. W., Jackson, T. D., &

Have, T. R. T. (2011). A Difference-in-Differences Analysis of Health, Safety, and Greening Vacant Urban Space. American Journal of Epidemiology, 174(11), 1296-1306.

Burkhard, B., Crossman, N., Nedkov, S., Petz, K., & Alkemade, R. (2013).

Mapping and modelling ecosystem services for science, policy, and practice. Ecosystem Services, 4, 1-3.

161

Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: a practical information-theoretic approach (2nd ed.). New York: Springer-Verlag.



Cerbu, G. A., Swallow, B. M., & Thompson, D. Y. (2011). Locating REDD: A

global survey and analysis of REDD readiness and demonstration activities. Environmental Science & Policy, 14(2), 168-180.

Chan, K. M. A., Shaw, M. R., Cameron, D. R., Underwood, E. C., & Daily, G. C.

(2006). Conservation Planning for Ecosystem Services. PLoS Biology, 4(11), 1-15. Cook, C. N., Mascia, M. B., Schwartz, M. W., Possingham, H. P., & Fuller, R. A.


Corbera, E., Kosoy, N., Tuna, M. (2007). Equity implications of marketing

ecosystem services in protected areas and rural communities: Case studies from Meso-America. Global Environmental Change, 17: 365-380.

Costanza, R., & Daly, H. E. (1992). Natural Capital and Sustainable

Development. Conservation Biology, 6(1), 37-46. Costanza, R., & Kubiszewski, I. (2011). Valuing Ecological Systems and Services

F1000 Biology Reports, 3(14). Costanza, R., d'Arge, R., Groot, R. d., Farber, S., Grasso, M., Hannon, B., . . .

Belt, M. v. d. (1997). The value of the world's ecosystem services and natural capital. Nature, 387, 253-260.

Cox, B., & Searle, B. (2009). The State of Ecosystem Services. Boston: The

Bridgespan Group. Crawley, M. J. (2007). The R Book. West Sussex, England: John Wiley & Sons

Ltd.

Creswell, J. W. (2009). Research design : qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, Calif.: Sage Publications.

Crossman, N. D., Burkhard, B., & Nedkov, S. (2012). Quantifying and mapping

ecosystem services. International Journal of Biodiversity Science, Ecosystem Services and Management, 8(1-2), 1-4.

162

Cutts, B. B., White, D. D., & Kinzig, A. P. (2011). Participatory geographic

information systems for the co-production of science and policy in an emerging boundary organization. Environmental Science & Policy, 14, 977-985.

Daily, G. C. (1997). Nature's services : societal dependence on natural ecosystems. Washington, DC: Island Press.

Daily, G. C., & Ellison, K. (2002). The new economy of nature : the quest to

make conservation profitable. Washington, DC: Island Press : Shearwater Books. Daily, G. C., & Matson, P. A. (2008). Ecosystem services: From theory to


Daily, G. C., Kareiva, P. M., & Polasky, S. (2011). Mainstreaming natural capital

into decisions. In P. M. Kareiva, H. Tallis, T. Ricketts, G. C. Daily & S. Polasky (Eds.), Natural Capital: Theory and Practice of Mapping Ecosystem Services. Oxford: Oxford University Press.


L., et al. (2009). Ecosystem services in decision making: time to deliver. Frontiers in Ecology and the Environment, 7(1), 21-28.

Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carre, G., et al. (2013). Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 27-46.

Driscoll, C. T., Lambert, K. F., & Weathers, K. C. (2011). Integrating Science and

Policy: A Case Study of the Hubbard Brook Research Foundation Science Links Program. Bioscience, 61(10), 791-801.

Ehrlich, P. R., & Mooney, H. A. (1983). Extinction, Substitution, and Ecosystem

Services. Bioscience, 33(4), 248-254. Elliott, E. W., & Kiel, L. D. (1999). Nonlinear dynamics, complexity and public

policy. Commack, N.Y.: Nova Science Publishers. Eppink, F. V., Werntze, A., Mas, S., Popp, A., & Seppelt, R. (2012). Land

Management and Ecosystem Services: How Collaborative Research Programmes Can Support Better Policies. Gaia-Ecological Perspectives for Science and Society, 21(1), 55-63.

Ernstson, H. & Sorlin, S. (2013). Ecosystem services as technology of

globalization: On articulating values in urban nature. Ecological Economics, 86: 274-284.

163

Farley, J., Filho, A. S., Burke, M., & Farr, M. (2014). Extending market allocation

to ecosystem services: Moral and practical implications on a full and unequal planet. Ecological Economics, in press and available online.

Ferraro, P. J. (2009). Counterfactual thinking and impact evaluation in

environmental policy. In M. Birnbaum & P. Mickwitz (Eds.), Environmental program and policy evaluation (Vol. 122, pp. 75–84): New Directions for Evaluation

Ferraro, P. J., & Hanauer, M. M. (2014a). Advances in Measuring the

Environmental and Social Impacts of Environmental Programs. Annual Review of Environment and Resources, 39, 495-517.

Ferraro, P. J., & Hanauer, M. M. (2014b). Quantifying causal mechanism to

determine how protected areas affect poverty through changes in ecosystem serices and infrastructure. PNAS, 111(11), 4332-4337.

Ferraro, P. J., & Miranda, J. J. (2014). The performance of non-experimental

designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark. Journal of Economic Behavior & Organization, 107, 344-365.


Evaluation of Biodiversity Conservation Investments. PLoS Biology, 4(4), 0001-0007. Ferraro, P. J., Lawlor, K., Mullan, K. L., & Pattanayak, S. K. (2012). Forest


Fisher, B., Balmford, A., Ferraro, P. J., Glew, L., Mascia, M., Naidoo, R., et al.

(2013). Moving Rio Forward and Avoiding 10 More Years with Little Evidence for Effective Conservation Policy. Conservation Biology, 00(0), 1-3.

Fisher, B., Turner, K., Zylstra, M., Brouwer, R., Groot, R. d., Farber, S., et al.

(2008). Ecosystem Services and Economic Theory: Integration for policy-relevant research. Ecological Applications, 18(8), 2050-2067.

Fisher, B., Turner, R. K., & Morling, P. (2009). Defining and classifying ecosystem services for decision making. Ecological Economics, 68, 643-653.


(2010). Impact Evaluation in Practice. Washington, DC: The World Bank. Geyer, R., & Rihani, S. (2010). Complexity and public policy : a new approach to

twenty-first century politics, policy and society. London ; New York: Routledge.

164

Ghazoul, J. (2007). Challenges to the uptake of the ecosystem service rationale

for conservation. Conservation Biology, 21(6), 1651-1652. Goldman, R. L., & Tallis, H. (2009). A Critical Analysis of Ecosystem Services


Goldstein, J. H., Caldarone, G., Duarte, T. K., Ennaanay, D., Hannahs, N.,

Mendoza, G., et al. (2012). Integrating ecosytem-service tradeoffs into land-use decision. PNAS, 109(19), 7565-7570.

Grindle, M. S., & Thomas, J. W. (1991). Public Choices and Policy Change: The

Political Economy of Reform in Developing Countries. Baltimore: The John Hopkins University Press.

Groot, R. S. d., Alkemade, R., BRaat, L., Hein, L., & Willemen, L. (2010).

Challenges in integrating the concept of ecosystem services and values in landscape planning, management, and decision making. Ecological Complexity, 7, 260-272.

Guston, D. H. (2001). Boundary organizations in environmental policy and

science: An introduction. Science Technology & Human Values, 26(4), 399-408. Haas, P. M. (1992). Epistemic Communities and International-Policy

Coordination - Introduction. International Organization, 46(1), 1-35. Haas, P. M., Keohane, R. O., & Levy, M. A. (1993). Institutions for the earth :

sources of effective international environmental protection. Cambridge, Mass.: MIT Press.

Haklay, M., & Tobon, C. (2003). Usability evaluation and PPGIS: towards a user-

centered design approach. International Journal of Geographic Information Science, 17(6), 577-592.

Hardy, D. J. & Patterson, M. G. (2012). Cross-cultural environmental research in

New Zealand: Insights for ecological economics research practice. Ecological Economics, 73: 75-85.

Haythornthwaite, C. (1996). Social network analysis: An approach and technique

for the study of information exchange. Library & Information Science Research, 18(4), 323-342.

Hinkel, J. (2011). "Indicators of vulnerability and adaptive capacity": Towards a

clarification of the science-policy interface. Global Environmental Change, 21, 198-208.

165

Hirsch, P. D., & Luzadis, V. A. (2013). Scientific Concepts and Their Policy

Affordances: How a focus on compatibility can improve science-policy interaction and outcomes. Nature and Culture, 8(1), 97-118.

Hogan, D., Arthaud, G., Brookshire, D., Gunther, T., Pincetl, S., Shapiro, C., et al.

(2011). Developing an Institutional Framework to Incorporate Ecosystem Services into Decision Making - Proceedings of a workshop (Vol. 1221). Reston, Virginia: USGS.

Holzman, D. C. (2012). Accounting for nature's benefits: the dollar value of

ecosystem services. Environmental Health Perspectives, 120(4), 152-157. Hsu, A., Johnson, L., & Lloyd, A. (2013). Measuring Progress: A Practical Guide

From the Developers of the Environmental Performance Index (EPI). New Haven: Yale Center for Environmental Law & Policy.

Huitema, D., & Turnhout, E. (2009). Working at the science-policy interface: a

discursive analysis of boundary work at the Netherlands Environmental Assessment Agency. Environmental Politics, 18(4), 576-594.

Jones, S. A., Fischhoff, B., & Lach, D. (1999). Evaluating the Science-Policy

Interface for Climate Change Research. Climactic Change, 43, 581-599. Kareiva, P. M., Tallis, H., Ricketts, T. H., Daily, G. C., & Polasky, S. (2011).

Natural Capital: Theory and practice of mapping ecosystem services. Oxford, UK: Oxford University Press.

Karl, H. A., Susskind, L. E., & Wallace, K. H. (2007). A dialogue not a diatribe -

Effective integration of science and policy through joint fact finding. Environment, 49(1), 20.

Kates, R. W., Clark, W. C., Corell, R., Hall, J. M., Jaeger, C. C., Lowe, I., . . .

Svedin, U. (2001). Environment and development - Sustainability science. Science, 292(5517), 641-642.

Kaufmann, D., Kraay, A., & Mastruzzi, M. (2011). The Worldwide Governance

Indicators: Methodology and Analytical Issues. Hague Journal on the Rule of Law, 3(2), 220-246.

Keller, A. C. (2010). Credibility and Relevance in Environmental Policy:

Measuring Strategies and Performance among Science Assessment Organizations. Journal of Public Administration Research and Theory, 20(2), 357-386.

166

Kenward, R. E., Whittingham, M. J., Arampatzis, S., Manos, B. D., Hahn, T., Terry, A., et al. (2011). Identifying governance strategies that effecively support ecosystem serivces, resource sustainability, and biodiversity. PNAS, 108(13), 5308-5312.

Kerkhoff, L. v., & Lebel, L. (2006). Linking Knowledge and Action for

Sustainable Development. Annual Review of Environment and Reosurces, 31, 445-477. Khandker, S. R., Koolwal, G. B., & Samad, H. A. (2010). Handbook on impact


ed.). Boston: Longman. Kinzig, A. P., Perrings, C., Chapin, F. S., Polasky, S., Smith, V. K., Tilman, D., &

Turner, B. L. (2011). Paying for Ecosystem Services-Promise and Peril. Science, 334(6056), 603-604.

Knight, A. T., Cowling, R. M., Rouget, M., Balmford, A., Lombard, A. T., &

Campbell, B. M. (2008). Knowing but not doing: Selecting priority conservation areas and the research-implementation gap. Conservation Biology, 22(3), 610-617.

Kolinjivadi, V., Grant, A., Adamowski, J., Kosoy, N. (2015). Juggling multiple

dimensions in a complex socio-ecosystem: The issue of targeting in payments for ecosystem services. Geoforum, 58: 1-13.

Kosoy, N., Corbera, E. (2010). Payments for ecosystem services as commodity

fetishism. Ecological Economics, 69: 1228-1236. Kremen, C. (2005). Managing ecosystem services: what do we need to know

about their ecology? Ecology Letters, 8, 468-479. Krippendorff, K. (2004). Content Analysis: an introduction to its methodology.

(Second ed.). Thousand Oaks, CA: Sage. Kumar, P., et al. (2010). The economics of ecosystems and biodiversity:

ecological and economic foundations. New York: Earthscan. Landry, R., Amara, N., & Lamari, M. (2001). Utilization of social science

research knowledge in Canada. Research Policy, 30(2), 333-349. Laurans, Y., & Mermet, L. (2014). Ecosystem services economic valuation,

decision-support system or advocacy? Ecosystem Services, 7, 98-105.

167

Laurans, Y., Rankovic, A., Bille, R., Pirard, R., & Mermet, L. (2013). Use of ecosystem services economic valuation for decision making: Questioning a literature blindspot. Journal of Environmental Management, 119, 208-219.

Lavis, J. N., Robertson, D., Woodside, J. M., McLeod, C. B., Abelson, J., & Grp,

K. T. S. (2003). How can research organizations more effectively transfer research knowledge to decision makers? Milbank Quarterly, 81(2), 221.

Lawler, J. J., Lewis, D. J., Nelson, E., Plantinga, A. J., Polasky, S., Withey, J. C.,

et al. (2014). Projected land-use change impacts on ecosystem services in the United States. PNAS, 111(20), 7492-7497.

Lee, K. N. (1993). Compass and Gyroscope: Integrating Science and Politics for

the Environment. Washington, DC: Island Press. Leiss, W. (2000). Between expertise and bureaucracy: Risk management trapped

at the science-policy interface Risky Business: Canada’s Changing Science-Based Policy and Regulatory Regime (pp. 49-74). Toronto: University of Toronto Press.

Lemos, M. C., & Morehouse, B. J. (2005). The co-production of science and

policy in integrated climate assessments. Global Environmental Change-Human and Policy Dimensions, 15(1), 57-68.

Lester, J. P. (1993). The Utilization of Policy Analysis by State Agency Officials.

Knowledge-Creation Diffusion Utilization, 14(3), 267-290. Lin, L., Pattanayak, S. K., Sills, E. O., & Sunderlin, W. D. (2012). Site selection

for forest carbon projects. In A. Angelson (Ed.), Analysing REDD. Indonesia: CIFOR. Lin, L., Sills, E., & Cheshire, H. (2014). Targeting areas for Reducing Emissions

from Deforestation and forest Degradation (REDD+) projects in Tanzania. Global Environmental Change-Human and Policy Dimensions, 24, 277-286.

Liu, S., Costanza, R., Farber, S., & Troy, A. (2010). Valuing ecosystem services


Lovins, A. B., Lovins, L. H., & Hawken, P. (1999). A road map for natural

capitalism. Harvard Business Review, 77(3), 145. Luck, G. W., Chan, K. M. A., Eser, U., Gomez-Baggethun, E.., Matzdorf, B.,

Norton, B., Potschin, M. B. (2012). Ethical considerations in on-ground applications of the ecosystem services concept. Bioscience, 62(12): 1020-1029.

168

MacDonald, D. H., Bark, R. H., & Coggan, A. (2014). Is ecosystem service research used by decision-makers? A case study of the Murray-Darling Basin, Australia. Landscape Ecology.

Manos, B., & Papathanasiou, J. (2008). GEM-CON-BIO Governance and

Ecosystem Management for the Conservation of Biology. Greece: Aristotle University of Thessaloniki.



Martinez-Harms, M. J., & Balvanera, P. (2012). Methods for mapping ecosystem

service supply: a review. International Journal of Biodiversity Science, Ecosystem Services and Management, 8(1-2), 17-25.

McCauley, D. J. (2006). Selling out on nature. Nature, 443, 27-28. McCreless, E., Visconti, P., Carwardine, J., Wilcox, C., & Smith, R. J. (2013).

Cheap and Nasty? The Potential Perils of Using Management Costs to Identify Global Conservation Priorities. PloS ONE, 8(11).

McKenzie, E., Irwin, F., Ranganathan, J., Hanson, C., Kousky, C., Bennett, K., et

al. (2011). Incorporating ecosystem service in decisions. In P. M. Kareiva, H. Tallis, T. Ricketts, G. C. Daily & S. Polasky (Eds.), Natural Capital: Theory and Practice of Mapping Ecosystem Services. Oxford: Oxford University Press.



Meier, G. (Ed.). (1991). Politics and Policy Making in Developing Countries:

Perspectives on the New Political Economy. San Francisco: ICU Press. Mermet, L., Laurans, Y., & Lemenager, T. (2014). Tools for What Trade?


Meyer, B. D. (1995). Natural and Quasi-Experiments in Economics. Journal of

Business & Economic Statistics, 13(2), 151-161. Millennium Ecosystem Assessment (Program). (2005). Ecosystems and human

well-being : synthesis. Washington, DC: Island Press.

169

Miller, C. A., Jasanoff, S., & Long, M. (1997). Shaping knowledge, defining uncertainty: the dynamic role of assessments. In W. C. Clark, J. J. McCarthy & E. Shea (Eds.), A Critical Evaluation of Global Environmental Assessments. Cambridge, MA: Kennedy School of Government.

Mitchell, R. B., Clark, W. C., Cash, D. W., & Alcock, F. (2004). Science,

scientists, and the policy process: lessons from global environmental assessments for the northwest forest In K. Arabas & J. Bowersox (Eds.), Forest Futures: Science, Politics, and Policy for the Next Century. Lanham, MD: Rowman and Littlefield.

Miteva, D. A., Pattanayak, S. K., & Ferraro, P. J. (2012). Evaluation of

biodiversity policy instruments: what works and what doesn't? Oxford Review of Economic Policy, 28(1), 69-92.

Muradian, R. Corbera, E., Pascual, U., Kosoy, N., May, P. H. (2010). Reconciling

theory and practice: An alternative conceptual framework for understanding payments for environmental services. Ecological Economics, 69: 1202-1208.

Muradian, R., Arsel, M., Pellegrini, L., Adaman, F. … (2013). Payments for

ecosystem services and the fatal attraction of win-win solutions. Conservation Letters, 6(4): 274-279.

Naidoo, R., & Johnson, K. (2013). Community-based conservation reduces sexual

risk factors for HIV among men. Globalization and Health, 9(27), 1-5. Nelson, E., Mendoza, G., Regetz, J., Polasky, S., Tallis, H., Cameron, D. R., et al.


Nesshover, C., Timaeus, J., Wittmer, H., Krieg, A., Geamana, N., van den Hove,

S., et al. (2013). Improving the Science-Policy Interface of Biodiversity Research Projects. Gaia-Ecological Perspectives for Science and Society, 22(2), 99-103.

Oh, C. (1996). Linking Social Science Information to Policy Making. Greenwich,

CT: Jai Press. Pandey, K. D., Buys, P., Chomitz, K., & Wheeler, D. (2006). Biodiversity

Conservation Indicators: New Tools for Priority Setting at the Global Environmental Facility. Washington, DC: World Bank, Development Economics Research Group and Environment Department.

Pannell, D. J. (2004). Effectively communicating economics to policy makers.

Australian Journal of Agricultural and Resource Economics, 48(3), 535-555.

170

Peh, K. S. H., Balmford, A., Bradbury, R. B., Brown, C., Butchart, S. H. M., Hughes, F. M. R., et al. (2013). TESSA: A toolkit for rapid assessment of ecosystem services at sites of biodiversity conservation importance. Ecosystem Services, 5, 51-57.

Perrings, C., Duraiappah, A., Larigauderie, A., & Mooney, H. (2011). The

Biodiversity and Ecosystem Services Science-Policy Interface. Science, 331(4 March ), 1139-1140.

Pfaff, A., Robalino, J. A., & Sanchez-Azofeifa, G. A. (2008). Payment for

Environmental Services: Empirical analysis for Costa Rica Working Papers Series SAN08-05. Durham, NC: Terry Sanford Institute of Public Policy at Duke University.

Pielke, R. A. (2007). The Honest Broker. Cambridge: Cambridge University

Press. Polasky, S., Nelson, E., Pennington, D., & Johnson, K. A. (2011). The Impact of

Land-Use Change on Ecosystem Services, Biodiversity and Returns to Landowners: A Case Study in the State of Minnesota. Environmental & Resource Economics, 48(2), 219-242.

Posner, S. M., McKenzie, E., & Ricketts, T. H. (in review). What explains the

impact of ecosystem services knowledge on decisions? Pretty, J., & Smith, D. (2004). Social capital in biodiversity conservation and

management. Conservation Biology, 18(3), 631-638. Pretty, J., & Ward, H. (2001). Social capital and the environment. World

Development, 29(2), 209-227. R. (2011). R: A language and environment for statistical computing: R

Development Core Team. Redford, K. H., & Adams, W. M. (2009). Payment for Ecosystem Services and

the Challenge of Saving Nature. Conservation Biology, 23(4), 785-787. Reed, M. S. (2008). Stakeholder participation for environmental management: A

literature review. Biological Conservation, 141(10), 2417-2431. Reid, R. S., Nkedianye, D., Said, M. Y., Kaelo, D., Neselle, M., Makui, O., et al.

(2009). Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa. PNAS, Early Edition, 1-6.

171

Reyers, B., Polasky, S., Tallis, H., Mooney, H. A., & Larigauderie, A. (2012). Finding Common Ground for Biodiversity and Ecosystem Services. Bioscience, 62(5), 503-507.

Reynolds, K. M. (2007). Sustainable forestry : from monitoring and modelling to

knowledge management and policy science. Wallingford, UK ; Cambridge, MA: CABI. Rosenthal, A., Verutes, G., McKenzie, E., Arkema, K. K., Bhagabati, N., Bremer,


Roux, D. J., Rogers, K. H., Biggs, H. C., Ashton, P. J., & Sergeant, A. (2006).






Ruhl, J. B., Kraft, S. E., & Lant, C. L. (2007). The law and policy of ecosystem

services. Washington: Island Press. Sabatier, P. A., & Jenkins-Smith, H. C. (1993). Policy change and learning : an

advocacy coalition approach. Boulder, Colo.: Westview Press. Sabatier, P. A., & Weible, C. M. (Eds.). (2014). Theories of the Policy Process.

Boulder, CO: Westview Press. Sarkki, S., Niemela, J., Tinch, R., Hove, S. v. d., Watt, A., & Young, J. (2013).


Schmidt, R., Lozano, S., Robins, J., Schwartz, A., & Batker, D. (2014). Nature's

Value in Santa Cruz County. Tacoma, WA and Capitola, CA: Earth Economics and the Resource Conservation District of Santa Cruz County.

Schumacher, E. F. (1973). Small is beautiful; economics as if people mattered.

New York,: Harper & Row.

172

Sherrouse, B. C., Clement, J. M., & Semmens, D. J. (2011). A GIS application for assessing, mapping, and quantifying the social values of ecosystem services. Applied Geography, 31(2), 748-760.

Smith, L. C., Khan, F., Frankenberger, T. R., & Wadud, A. K. M. A. (2012).

Admissable Evidence in the Court of Development Evaluation? The Impact of CARE's SHOUHARDO Project on Child Stunting in Bangladesh. World Development, 41, 196-216.

Smith, R. J., Muir, R. D. J., Walpole, M. J., Balmford, A., & Leader-Williams, N.

(2003). Governance and the loss of biodiversity. Nature, 426(6962), 67-70. Smith, R. J., Muir, R. D. J., Walpole, M. J., Balmford, A., & Leader-Williams, N.

(2007). Revisiting the links between governance and biodiversity loss. Conservation Biology, 21(4), 899-900.

Spilsbury, M. J., & Nasi, R. (2006). The interface of policy research and the

policy development process: challenges posed to the forestry community. Forest Policy and Economics, 8(2), 193-205.

Sutherland, W. J. (2003). Predicting the ecological consequences of

environmental change: a review of the methods. Journal of Applied Ecology, 43, 599-616.

Sutherland, W. J., Goulson, D., Potts, S. G., & Dicks, L. V. (2011). Quantifying

the Impact and Relevance of Scientific Research. PloS ONE, 6(11), 1-10. Tallis, H., & Kareiva, P. (2009). Mapping and Valuing Ecosystem Services as an

Approach for Conservation and Natural-Resource Management. Annals of the New York Academy of Sciences, 1162, 265–283.

Taylor-Powell, E., & Steele, S. (1996). Collecting Evaluation Data: Direct

Observation. Madison, WI: University of Wisconsin Extension. Tomich, T. P., Chomitz, K., Francisco, H., Izac, A. M. N., Murdiyarso, D.,

Ratner, B. D., . . . van Noordwijk, M. (2004). Policy analysis and environmental problems at different scales: asking the right questions. Agriculture Ecosystems & Environment, 104(1), 5-18.

van den Hove, S. (2007). A rationale for science-policy interfaces. Futures, 39(7),

807-826. Vatn, A. & Bromley, D. W. (1994). Choices without prices without apologies.

Journal of Environmental Economics and Management, 26: 129-148.

173

Verutes, G. M., & Rosenthal, A. (2014). Using Simulation Games to Teach Ecosystem Service Synergies and Trade-offs. Environmental Practice, 00(0), 1-11.

Villa, F., Bagstad, K. J., Voigt, B., Johnson, G. W., Portela, R., Honzak, M.,

Baker, D. (2014). A Methodology for Adaptable and Robust Ecosystem Services Assessment. PloS ONE, 9(3).

Voinov, A., & Gaddis, E. J. B. (2008). Lessons for successful participatory

watershed modeling: A perspective from modeling practitioners. Ecological Modelling, 216(2), 197-207.

Waite, R., Kushner, B., Jungwiwwattanaporn, M., Gray, E., & Burke, L. (2014).


Waldron, A., Mooers, A. O., Miller, D. C., Nibbelink, N., Redding, D., Kuhn, T.

S., et al. (2013). Targeting global conservation funding to limit immediate biodiversity declines. PNAS, 110(29), 12144-12148.

Watson, R. (2005). Turning science into policy: challenges and experiences from

the science-policy interface. Phil. Trans. R. Soc. B, 360, 471-477. Watson, R. T. (2012). The science-policy interface: the role of scientific

assesments -- UK National Ecosystem Assessment. Proceedings of the Royal Society A, 468, 3265-3281.

Weible, C. M., Pattison, A., & Sabatier, P. A. (2010). Harnessing expert-based

information for learning and the sustainable management of complex socio-ecological systems. Environmental Science & Policy, 13, 522-534.

Weible, C. M., Sabatier, P. A., Jenkins-Smith, H. C., Nohrstedt, D., Henry, A. D.,

& deLeon, P. (2011). A Quarter Century of the Advocacy Coalition Framework: An Introduction to the Special Issue. Policy Studies Journal, 39(3), 349-360.

Weichselgartner, J., & Kasperson, R. (2010). Barriers in the science-policy-practice interface: Toward a knowledge-action-system in global environmental change research. Global Environmental Change-Human and Policy Dimensions, 20(2), 266-277.

Weiss, C. H. (1977). Research for Policys Sake - Enlightenment Function of

Social-Research. Policy Analysis, 3(4), 531-545. White, D. D., Corley, E. A., & White, M. S. (2013). Water Managers' Perceptions

of the Science-Policy Interface in Phoenix, Arizona: Implications for an Emerging Boundary Organization. Society & Natural Resources: An International Journal, 21(3), 230-243.

174

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (2010). Handbook of

Practical Program Evaluation (3rd ed.). Hoboken: Wiley. Wunder, S., Engel, S., & Pagiola, S. (2008). Taking stock: A comparative

analysis of payments for environmental services programs in developed and developing countries. Ecological Economics, 65(4), 834-852.


Angeles, Calif.: Sage Publications. Young, J. C., Jordan, A., Searle, K. R., Butler, A., Chapman, D. S., Simmons, P.,

et al. (2013). Does stakeholder involvement really benefit biodiversity conservation? Biological Conservation, 158, 359-370.

The impact of ecosystem services knowledge on decisions

Documents

Transcript of The impact of ecosystem services knowledge on decisions