1

Thesis Research Proposal

Anne van Bruggen

MSc. Industrial Ecology

Transformative Modeling Methodology for the Participatory Aspects of Models of Large Scale Transitions

First Supervisor:

Dr. Ir. I. Nikolic

Second supervisor:

Dr. Ir. J. Kwakkel

External Supervisor:

S. Mansour

2

Contents Background ................................................................................................................................................... 3

The need for new levels of organization and global management of resources...................................... 3

The Rise of Complexity Science............................................................................................................. 3

The Role of Computer-Based Models ....................................................................................................... 4

Participatory Modeling ............................................................................................................................. 4

Lack of Insight ............................................................................................................................................... 6

Transformative Modelling......................................................................................................................... 7

Transformative Agent Based Modeling .................................................................................................... 8

Research Goal ............................................................................................................................................... 9

Theoretical Lenses ...................................................................................................................................... 10

Post-Normal Science ............................................................................................................................... 10

Evolutionary Complex Adaptive Systems Theory ................................................................................... 11

Institutional Analysis ............................................................................................................................... 11

Institutional Analysis and Development Framework (IAD) ................................................................. 11

Joint Sense Making through Boundary Objects ...................................................................................... 12

Boundary Institutions .......................................................................................................................... 12

Theory of Transformative Learning ........................................................................................................ 12

Methodology ............................................................................................................................................... 13

Data sources ............................................................................................................................................ 13

Reviewing participatory methodologies using MAIA .............................................................................. 13

MAIA Meta Model............................................................................................................................... 14

Synthesis: Design of a Transformative Agent Based Modeling Process ................................................. 14

Expected Outcomes & Implications of Research ........................................................................................ 15

Towards a model of participatory modeling ....................................................................................... 15

Relevance to Industrial Ecology .................................................................................................................. 15

Planning ...................................................................................................................................................... 16

References .................................................................................................................................................. 17

3

Background

The need for new levels of organization and global management of resources As the earth’s population growths and their standards of welfare increase, a key challenge is to manage

our natural resources in such a way that ecosystem function is maintained, while keeping the sources

renewable (Allen, Tainter, & Hoekstra, 1999; Greer, 2005). The 1970 limits to growth study showed the

signs of a world in overshoot, of growth of such a rapid rate that resource recovery becomes unsustainable

while the earth can no longer uptake the pollutants, will eventually lead to environmental and economic

collapse (Meadows, Randers, & Meadows, 2004). The root cause of this problem, argued in the seminal

paper “Supply Side Sustainability”, lies in the manner of problem solving (Allen et al., 1999; Tainter, 1995,

2000). Currently, the management of resources occurs on a local level through complicated structures

that reduce efficiency (Allen et al., 1999). While these policies are guided by (short-term) rational decision-

making on the local level, in the long run and on the larger scale these “diminishing returns in efforts to

solve problems” shall lead to collapse as it did for the Western Roman Empire and the southern lowland

Classic Maya (Allen et al., 1999). The only type of society that can avert collapse is one that manages to

redefine their relationship to resources and achieve a new level of organization, that is discontinuous,

system level transition towards global management of resources (Allen et al., 1999).

The complexity approach shows how sustainability, or the capacity of a system to keep within certain

limits, can be approached on various organization levels that are dependent on each other; the

subsystems (Holling, 1973; Voinov, 2008). Ultimately, every level of organization or subsystem that is not

sustainable affects the larger whole or the global ecosystem, and thus we must care for every subsystem

as contributing of the sustainability of the whole system or biosphere that serves humanity (Voinov, 2008).

The Rise of Complexity Science Ideas from complexity science are increasingly shaping the way academics and practitioners alike

approach economics, policy, organizational change, and sustainability problems (Bechtold, 1997; Burnes,

2005; Choi, Dooley, & Rungtusanatham, 2001; Colander & Kupers, 2014; Gilchrist, 2000; Macbeth, 2002;

Morgan, Gregory, & Roach, 1997; Stacey, Griffin, & Shaw, 2000; Tetenbaum, 1998). While classical

economics operates on the assumption that people are hyper rational, that system dynamics are linear,

and can be controlled by the government, complexity science offers an alternative view of adaptive and

smart individuals that define welfare more broadly than simply material riches and can take active

ownership over problems (Colander & Kupers, 2014). Organizations can similarly be seen as complex

nonlinear systems or ecologies with interactions that are characterized as both ordered and chaotic from

which new solutions can arise (Morgan et al., 1997).

This new global level of organization that takes a holistic approach, enables long-term planning and goal-

setting, and solves problems on a different level, is new to human kind (Allen et al., 1999). However, for

complexity science to change organizations and lift our problem solving approach to a new level of

organization, complexity science needs to go beyond theoretical and metaphorical applications (Burnes,

2005). Instead, it needs to offer concrete approaches on how nature and organizations alike act as

dynamic non-linear systems that can be transformed (Burnes, 2005).

4

The Role of Computer-Based Models Models are one way to help organizations translate the complexity approach into concrete approaches

for transitions, deal with deep uncertainty (J.H. Kwakkel, W.E. Walker, & Marchau, 2010) and give insights

into the interdependencies of various types of knowledge from various subsystems and disciplines that

are involved in the transition (Holtz et al., 2015). Models are simplified representations of reality and can

exist in our brains as mental models, conceptually as qualitative relationships drawn on maps which point

out relationships, and computationally in computers (Bollinger, Nikolić, Davis, & Dijkema, 2015). As

human cognition is faulty in many ways, models can assist in a variety of processes that are essential to

achieving this new level of organization including, forecasting, improve understanding, guide human

behavior, increasing and spreading knowledge on transitions (Voinov, Kolagani, McCall, et al., 2016).

Computer-based models for socio-technical systems have the ability to develop mental models, providing

deeper insight into the problem and develop new directions of thought as well as intuitions about the

system (Nowak, Rychwalska, & Borkowski, 2013). Mental models have been widely studied by system

dynamics researchers as they were developing techniques to elicit, represent and map mental models on

the basis of which computer models are constructed to enhance decision-making. A shared definition of

mental models is however difficult to state, but Doyle and Ford (1998) argue to define mental models in

dynamic systems as a “relatively enduring and accessible, but limited, internal conceptual representation

of an external system whose structure maintains the perceived structure of that system.”

Computer models can overcome the limitations of human cognition and explore interdependencies of

social, economic, and ecological systems in systematic ways (Carpenter & Gunderson, 2001). They are

used to give further insight into socio-technical-biochemical transitions and enable stakeholders to give

direction to its pathways (Holtz et al., 2015).

In the face of multiscale, -stakeholder, -issue, -perspective, -resolution, and -aspect issues of high

complexity, such as transitions in large scale socio technical issues (LSTS) models need to be able to

encompass a wide variety of knowledge from different disciplines and participatory modeling exercises

are increasingly being undertaken (Holtz et al., 2015).

Participatory Modeling While modeling used to be an exercise of scientists with the occasional involvement of experts to analyze

a system, over the years involvement of stakeholders in several aspects of the modeling exercise from

model conceptualization to validation has become “almost a ‘must’” (Voinov & Bousquet, 2010). The need

to involve stakeholders originates with environmental decision making assessments by the US Army Corps

of Engineers and has since gained traction in a variety of common modeling approaches for complex

systems including system dynamics (SDs), agent based modeling (ABMs), Fuzzy Cognitive Mapping

(FCMs), Bayesian Networks (BNs), Couple Component Models (CCMs), and Knowledge-Based Models

(KBMs), (Kelly et al., 2013; Voinov & Bousquet, 2010; Wagner & Ortolano, 1975).

The reasons to engage participants in the modeling exercise are generally accepted and often work

synergistically, but are still in need of (more) empirical proof for their validity (Voinov & Bousquet, 2010).

These reasons include:

5

1. Improve stakeholder’s knowledge and enable a deeper understanding of the dynamics of the

complex systems in which they are embedded, both by learning from each other and the modeling

outcomes in a process of collaborative learning (Campo, Bousquet, & Villanueva, 2010; Voinov &

Bousquet, 2010)

2. Enhanced support from stakeholders for policies, regulations, or management solutions that are

the outcome of modeling exercises and increased likelihood that the decisions will be

implemented successfully, because those that are responsible for implementation were part of

the exercise and thus have a high degree of ownership motivating them to make a change (Chu,

Drogoul, Boucher, & Jucker, 2012; Gilbert, 2004; Voinov & Bousquet, 2010). While this benefit of

participatory modeling is often assumed, it has not been empirically validated and some of these

studies are perhaps only done out of a “ideological commitment” to participatory modeling

practices (Voinov & Bousquet, 2010; Voinov, Kolagani, McCall, et al., 2016). There does seem to

be a correlation between acceptability and the use of the model, but this is not a prerequisite

(Wassen, Runhaar, Barendregt, & Okruszko, 2011). However, applicability is a perquisite for

acceptability. Overall, it can be said that participatory models can be used to “identify and clarify

the impacts of solutions to a given problem” (Voinov & Bousquet, 2010).

3. Invigoration of the modeling process with original input from stakeholders in the form of data,

ideas and needs (Bousquet & Voinov, 2010; Reed, 2008).

4. Create a level playing field for decision making and enable stakeholders from different parts of a

system to negotiate in a context that differs from formal negotiation (Campo et al., 2010)

5. Mobilize and justify funding (Voinov & Bousquet, 2010).

Computer-based models can leverage these participatory benefits by learning to increase stakeholders in

increasingly large modeling exercises. Furthermore, both the parts of the modeling exercise in which

stakeholders are participating as well as the manner in which they are involved differ from study to study.

Lynam et al. (2007) distinguish between three different types of stakeholder involvement:

1. Extractive use: knowledge and values are extracted from stakeholders and used by a group of

experts and modelers to develop a model from which decisions are derived at

2. Co-learning: understanding of the system through synthesis is developed in a collaboration

between stakeholders and modelers, which is then passed on to a system for decision making

3. Co-management: stakeholders develop the knowledge syntheses and are included in a joint

decision making process

6

There are several components in which the stakeholders can be involved as summarized in Figure 1 below:

Figure 1 Components of Participatory Modeling which can be adapted to particular needs as synthesized by Voinov et al. (2016)

The way in which stakeholders are involved and the parts of the modeling exercises in which they take

part, is informed by various factors including the modeling aims and paradigm. Over the past 40 years that

participatory modeling has been developed, much experience has been gathered with certain ways of

involving stakeholders in various manners, in various components of the modelling exercise and in various

modeling disciplines. However, as the problems that computer-based modelling aims to gain insight

concern the whole globe and transcend tradition geographical, disciplinary, and institutional boundaries

(Costanza et al., 2007) and simultaneously the way people interact with and access computer based

models and data is evolving through widespread availability on the internet for example wiki pages

(Voinov, Kolagani, & McCall, 2016), participatory modeling has to charter unknown territories.

Lack of Insight As stated above, participatory modelling approaches have been better developed for some type of

modelling paradigms and components than for others. Various studies and literature reviews are available

for environmental modelling with stakeholders. Over 400 papers were published in Environmental

Modeling and Software (EMS) with reference to participatory modeling (Bousquet & Voinov, 2010;

Voinov, Kolagani, & McCall, 2016). These studies primarily concern the modeling of watersheds, dairy

farms, forest management approaches, bio-energy and other social-ecological systems (SES) bounded by

physical territory. Those papers address various components of the modeling process, and while the

authors acknowledge that they a stakeholder could be involved in all components, this is often not the

case (Voinov & Bousquet, 2010, p. 198).

Generally, participatory modeling exercises get more challenging as the model aims to tackle on an

increasingly large scale where the ultimate goals and interests of the actors involved are more likely to

conflict (Voinov & Bousquet, 2010). Furthermore, participatory modeling is easier if the system

boundaries are easily defined. While there is now experience with modeling on smaller scales, in

territories that can be defined, there is little experience with participatory model building that aims to

tackle global resource management and the emergence of a new level of organization that is required to

prevent global collapse (Allen et al., 1999). Modeling aimed at a transition on a global scale, involving a

7

wide range of actors with conflicting interest as well as unclear system boundaries, poses a new challenge

to participatory modeling.

Transformative Modelling The need for participatory modeling that studies multiscale, -stakeholder, -issue, -perspective, -resolution,

and -aspect issues of high complexity, can be understood as a need for modelling that is not just

participatory involving stakeholders in some aspects of the modelling, but transformative.

Transformative processes aim not only at improved decision making for a limited group of stakeholders,

but utilize the participatory process to engender active and effective interaction as well as collaborate

decision-making amongst a wide range of stakeholders to bring about discontinuous, large scale, systemic

change. While transformative processes that can occur in modeling have not yet been studied, the

conceptualization of transformative processes in this thesis shall rely on the learning theory for adults

described by sociologist Mezirow named transformative learning and Adam Kahane’s description of

transformative scenario planning (Kahane & Van Der Heijden, 2012; Jack Mezirow, 1997).

Transformative learning occurs as adults are making sense of their experiences in the world and using old

understanding as a frame of reference to evaluate new interpretations of meaning that can guide future

action (J. Mezirow, 1996; Taylor, 2008). Mezirow defines transformative learning as follows:

“Transformative learning is learning that transforms problematic frames of reference—

sets of fixed assumptions and expectations (habits of mind, meaning perspectives,

mindsets)—to make them more inclusive, discriminating, open, reflective, and

emotionally able to change. Such frames of reference are better than others because they

are more likely to generate beliefs and opinions that will prove more true or justified to

guide action” (Jack Mezirow, 2003, p. 59).

Essential to the transformative learning process is critical reflections upon assumptions both through

group interaction or independently (Jack Mezirow, 2003). Transformative learning can also occur in

organizations through critical reflection. However, how such critical reflection could be facilitated is

unclear, changing work flows or climate does not seem to be enough (Henderson, 2002). This research

explores the potential of participatory modelling in this process of critical reflection as a potential aid in

bringing about transformative change in individuals and organizations.

Furthermore, transformation is often triggered through a personal or social crises that pose questions to

the core of individuals or pose a “disorienting dilemma” (Jack Mezirow, 1990). The resource crises as

described in the introductory paragraph could thus provide as a befitting context to explore

transformative modelling.

Transformative modelling has furthermore been undertaking for critical reflection on mental models in

the qualitative scenario planning scenario studies undertaken by Kahane. He argues that a transformative

scenario process is effective for situations with the following characteristics (2013):

1. Stakeholders have identified their situation as “unacceptable, unstable, or unsustainable” and see

a change in the status-quo as the only way out

2. Transformation can only be achieved by working together with a variety of stakeholders in the

system in which they are embedded and cannot be brought merely through collaborations with

colleagues and friends

8

3. Transformation cannot be achieved directly because there is not a common understanding of the

solution or the problem. All they agree on is that there is a problem that must be solved.

The establishment of a new level of organization that enables global resource management, requires

transitions that fulfill the three characteristics of a situation in which a transformational approach can be

helpful. As can be seen from these characteristics, transformative processes require a variety of

stakeholders to work together. Participatory computer-based modeling could enhance this process by

further assisting individuals to engage in critical reflection and enhance the process by allowing for

systematic reflection on a set of assumptions and expectations which human cognition can only conceive

of in faulty ways. Computer-based models can assist transformative processes of transition that occur in

LTSTs, especially through enhancing understanding of the structure of complex systems and how dynamic

and emergent occurrences are a result of this underlying structure as well as have specific policy outcomes

that guide action (Holtz et al., 2015).

Overall, a picture of transformative modeling emerges that requires co-management and thus the deep

involvement of stakeholders through co-management across all components of the modeling process as

have been visualized in Figure 1. Modeling of such a large scale have not been attempted much, notable

exceptions include the Club of Rome’s Limits to Growth study based on system dynamics. However, this

study did not aim to incorporate a wide variety of stakeholders in a process of co-management of the

model. Now that various modeling tools have matured and the challenges that it can solve increase, these

large scale participatory, transformative efforts should be better understood as a crucial instrument in

avoiding global collapse.

Transformative Agent Based Modeling Transformative modelling can be done using a variety of modelling paradigms. Overall, there is more

experience with participatory and largescale models in disciplines that have been long in existence, most

primarily various forms of environmental modeling and system dynamics or differential equation models

including Group Model Building, Mediated Modeling and Companion Modeling. In addition there are

various non-modelling tools that involve stakeholders that can be used in modeling exercises such as

Social Science Experiment, Participatory Action Research, and Participatory Decision Analysis (Voinov &

Bousquet, 2010). Participatory research has however not yet been systematically reviewed and developed

for the newer modeling paradigm of agent based model building and simulation (ABMS). This type of

modeling is particularly useful for modeling questions that are more difficult to model in the Differential

Equations (DE) paradigm including, capturing of “heterogeneity across individuals and in the network of

interactions among them” at higher computational and cognitive costs that could limit both the scope of

the model and sensitivity analysis (Rahmandad & Sterman, 2008). Agent Based (AB) modelling can model

certain behavior that DE cannot, such as show how system level behavior emerges from interactions

between agents and simulating random changes in removing nodes and links of a network that can occur

in an attack or system failure, (Rahmandad & Sterman, 2008).

As the complexity paradigm increases in importance and it now has to prove its worth beyond the

metaphorical use to guide decision making in organizations and in large scale societal transitions and

macroeconomic analysis, agent based modeling becomes a focal point of this research (Doyne Farmer et

al., 2012). While the implementation of complex adaptive systems and agent based model theory is well-

understood and structured in proven methodologies which include roughly 10 steps from problem

9

formulation to model use, the social and participatory processes that are a fundamental part of agent

based models of LTST are not systematically understood (Van Dam, Nikolic, & Lukszo, 2013).

While various participatory AMBs are known and methodology for its execution has been proposed, its

practice is not yet widespread (Berland & Rand, 2009; Chu et al., 2012; Drogoul, 2015; Gilbert, 2004).

Existing participatory ABM studies aim to combine computer based simulation models with participatory

aspects that include stakeholders for example through scenario planning, design workshops, prototyping,

user panels, and more (Gilbert, 2004). Such exercises aim to involve a wide range of stakeholders, so that

the end-product is usable to guide decision making (Chu et al., 2012). While the technical aspects of

building the ABM can still be improved to make it more participatory friendly, especially regarding the

interface and usability for users in participatory processes, another aspect concerns identifying “ways for

linking technological advances with corresponding advances in participatory modelling” (Drogoul, 2015).

This process will need structuring as the problems the models address grow in complexity, such as the

modelling of the economy or the worlds industrial systems and supply chains (Baptista, Roque Martinho,

Lima, A. Santos, & Prendinger, 2014).

Research Goal This research will take on this challenge and aim to identify such linkages by designing a methodology for

participatory, transformative agent based model building that is particularly suited for large-scale

problems that involve a wide range of stakeholders and require action on multiple levels such as global

resource management.

The main goal of this research is to identify ways of linking technological advances in agent based

modeling with corresponding advances in participatory modelling (Drogoul, 2015). The process aims to

establish co-management by stakeholders of the entire modelling process, thus involving the stakeholders

in all components of the modelling as outlined by Voinov & Bousquet (2010).

Previous reviews of participatory modeling have concluded that there can be no “unique guidance for

participatory modeling” due to the human and social complexities and the uniqueness of each group of

stakeholders (Voinov & Bousquet, 2010; Voinov, Kolagani, McCall, et al., 2016). Instead a “toolkit”

approach is taken from which relevant tools can be selected and an underlying philosophy of building

“empowerment, equity, trust, and learning” (Reed, 2008).

This research aims to make guidance more specific by designing a methodology for a modeling process

that:

Is based on Agent Based Modeling

Involves its stakeholders through co-management with the aim to benefit from the fruits of

participatory modeling including enhanced understanding of the system and its dynamics, as well

as create enhanced support from stakeholders for policies

Aims to bring about large scale, discontinuous, system wide, “transformative change”

By setting the specifics of the modeling process, a methodology can be designed using experience from

other modeling paradigms and participatory tools that fits the demands. This methodology could still be

adaptable to different modeling exercise, but just like the Agent Based Modeling process itself is guided

by a process of model building from problem identification to conceptualization, formalization,

10

experimentation, verification and validation (Van Dam et al., 2013). The goal of this research is to

supplement every step in the process with a methodology to involve stakeholders.

To reach this goal, the following sub goals are identified:

1. Explore a definition of transformative change in organizations and across LSTS and how it can be

enhanced by computer based modeling

2. Systematically compare the way existing participatory model building experiences involve their

stakeholders and with what result

The research conducted is primarily exploratory and qualitative in nature, meaning that it tackles a new

problem on which little data is available or possible to collect. The investigation into participatory

methodologies relies primarily on literature review and experiences of modelers and stakeholders in other

cases and other types of mental and computer modeling such as scenario planning and system dynamics.

Theoretical Lenses Participatory and transformative modelling can be viewed through several theoretical lenses, some of

which were already introduced in the preceding sections. This thesis will use the following lenses that

together build the theoretical framework:

Post-Normal Science While traditional science emphasizes the uncovering of truth in the hard and objective facts, studies that

operate in the disciplines where science meets policy, economics and other social processes are not about

uncovering simple and universal truths (S. Funtowicz & Ravetz, 2003). In such social processes

uncertainties occur, a multiplicity of values and perspectives has value, science is no longer about getting

at the truth, but about the quality of the study which can be assessed by making assumptions clear (S.

Funtowicz & Ravetz, 2003). Post-normal science (PNS) aims to account for the complexity and uncertainty

that characterizes the natural systems, and how human society with its values and commitments, is

embedded in the natural world and has influence upon it (S. Funtowicz & Ravetz, 2003).

PNS is particularly useful in those situations in which the system uncertainties and the decision stakes are

high (S. O. Funtowicz & Ravetz, 1993). In the study of complex systems, the uncertainties are irreducible

and there are multiple legitimate perspectives on an issue informed by the discipline or background of

that stakeholder. While science from the traditional perspective, requires only the input of experts to

constitute a successful modeling exercise, the PNS paradigm requires the involvement of stakeholders in

decision-making in an “extended peer community” of people that want to be part of a resolution (S.

Funtowicz & Ravetz, 2003). This community, also characteristic of collaborative modeling exercises,

consists not only of an increasingly larger group of people, but more importantly of stakeholders from

various disciplines, each with their own methods to assess quality for example through peer review or the

market (S. Funtowicz & Ravetz, 2003). This research shall take the PNS paradigm as a useful theory to

“provide a coherent framework for an extended participation in decision-making, based on the new tasks

of quality assurance.” (S. Funtowicz & Ravetz, 2003, p. 1)

11

Evolutionary Complex Adaptive Systems Theory Complex adaptive systems (CAS) are defined by Waldrop (1992) as an ever-changing network of agents

(i.e. individuals, firms, governments) acting in parallel, and constantly reacting to one another. CAS are

adaptive in the sense that their basic components respond to impulses from their surroundings and from

each other, changing, evolving and eventually resulting on a different system macro-structure.

CAS can be conceptualized and studied on the three essential levels of the agent (micro) and its individual

behavior, the network (meso) describing the interaction between agents, and the system (macro) which

shows emergent behavior (Nikolić, 2009). Furthermore, complex systems have a number of properties

that can generally be observed. The most important characteristics include path dependency, emergency,

intractability, system nestedness, instability or chaos due to sensitivity to original parameters, observer

dependence, evolution and diversity as well as self-similarity (Nikolić, 2009).

As outlined above, these characteristics of CAS are increasingly used to understand phenomenon such as

economic, environmental, and social processes. CAS thus forms a fundamental part of the theoretical lens

to study participatory, transformative modeling, particularly for Agent Based Modeling which provides a

simulation tool to study the emergent, system-level behavior of CAS. In this study the process of

transformative modeling is itself characterized as a CAS of which its properties can be studied.

Institutional Analysis To get insight into the actions and interactions of participants in the participatory modeling exercises,

institutional analysis is used. This perspective allows for a description of socio-technical systems such as

the participatory modeling environment, to be structured. Because eliciting individual patters of action is

nearly impossible given the nature of individual behavior, institutional analysis instead aims to elicit sets

of rules that structure social behavior and interaction among social entities (Ostrom, Gardner, & Walker,

1994; Scharpf, 1997).

This research shall use institutional analysis to structure and capture participatory modeling approaches,

enabling the development of a comparative framework for the different approaches based on this

systematic mapping of social processes.

This research takes the approach of Crawford and Ostrom (1995) using the term “institutional statements”

to encompass rules, norms and shared strategies, as the main concepts to define rules that guide human

action.

Rules: Have a clear deontic and established consequence for non-compliance.

Norms: Have a clear deontic with no established consequence.

Social Strategies: concept that has no deontic or consequence for non-compliance, and constitutes an

indicator of usual behavior.

Institutional Analysis and Development Framework (IAD) To enable a higher level of understanding of participatory modeling processes, the IAD framework is used.

This is a tool to (1) understand the structures that conform the social system (Physical structures,

community and rules) (2) capture the environment in which actors operate (action arena, action situations

and participants) and (3) observing patterns of interaction and that resonate in the community given

certain evaluation criteria and derive in institutional change (Ghorbani & Weijnen, 2013).

12

To enable the highest level of organization and unambiguous structuring of the insights into the

participatory modeling process through institutional analysis, the MAIA (Modeling Agent Systems using

Institutional Analysis) meta-model is employed. MAIA formalizes and extends the IAD framework to

Even if the actual model is not executed, MAIA methodology provides a powerful tool with which we can

structure the information gathered on participatory modeling processes.

Joint Sense Making through Boundary Objects In addition to institutional analysis, cognitive behavioral theory shall be used to explore the transformative

aspects of modeling and understand collaborative critical reflection and decision making. To explain how

decision making in a diverse group of stakeholders can occur without reaching prior consensus, the theory

of boundary entities is employed (S. L. Star & Griesemer, 1989). A boundary object is something that brings

a diverse group of stakeholders that can inhabit “intersection social worlds” together without the need

for consensus and could be as simple as a visual representation or causal loop diagram (Black & Andersen,

2012; S. L. Star & Griesemer, 1989). The objects must be able to express the elements and dependencies

in a system as minimalistic ally as possible while allowing for modification by participants. These objects

then allow participants to translate their tacit knowledge into explicit knowledge from which other

participants can then again learn (Rose et al., 2015). Rather the participants agree to disagree and the

boundary object must be sufficiently flexible to allow for a common process while adapting to the local

realities of the participants.

The three essential features of boundary objects are according to Star (2010): (1) flexibility or plasticity

that allows for understanding and action in various social groups, (2) physical and organizational

structures of norms, categorizations, and standards, and (3) a suitable scale of analysis that takes the

whole system under study into account. Overall, boundary entities must be “both adaptable to multiple

viewpoints and robust enough to maintain identity across them” (Leigh Star & Griesemer, 1989, p. 387).

Collaborative models have been studied as boundary objects or boundary organizations in several studies

(Kum et al., 2015; Rose et al., 2015; Waas, 2015).

Boundary Institutions For the study of participatory modeling process, the emphasis is not on boundary objects such as causal

loop diagrams, but also on boundary institutions grounded in Ostrom’s theory introduced above. A

boundary institution has the same function as an object, but in the form of institutions or rules that

govern the collaborative modelling space in which stakeholders meet for the modeling exercise

(Barreteau et al., 2012). Overall, boundary institutions inform the facilitation of interaction between

stakeholders, allowing them to build consensus and leverage the knowledge of the team.

Theory of Transformative Learning The theory of learning described in the section of transformative modeling as described by Mezirov

(1997) is used as a theory of how transformational change can be brought about. The theory is

supplemented with an integrated or planetary perspective on transformational learning that aims at

transformational change in social, political, economic, and educational systems through a holistic

perspective (O’Sullivan, 1999). This theory of transformative learning also acknowledges that learning

occurs not only through interaction with other people, but also through interaction with natural systems

and the physical environment (O’Sullivan, 1999).

13

Above it is already explained how transformational learning can be triggered by “disorienting

experiences” and occurs through critical reflection. Furthermore, it must be noted that transformational

learning is not a mere epistemological change in a perspective, but also brings about an ontological shift

that shows itself in action based upon the newly acquired perspective (Lange, 2004).

Methodology

Data sources The data for this thesis will come primarily from literature and experiences of modelers, stakeholders,

and experts that have been part of participatory modeling exercises. To better understand the exercises,

the computer models themselves may be reviewed as well. Interviews will furthermore be aimed at

eliciting the social institutions governing participatory modeling exercises.

The literature review will be qualitative in nature. The interviews will be semi-structured aimed at

exploring success factors and limitations experienced in participatory processes as well as eliciting the

social institutions governing participatory modeling exercises.

Two case studies will be used. Firstly, the proprietary development of a large-scale ABM that aims to

make the consequences and interdependencies of industry and value chains across the globe insightful.

Secondly, the case of the Rotterdam Harbor, which has been working with several ABMs as well as model

ecologies that include the maintenance of a wiki with data.

Reviewing participatory methodologies using MAIA The following participatory methods will be reviewed:

The institutions will be elicited and coded into ADICO. Per type of participatory methodology, the parts of

MAIA that are completed are to be determined. While for some approaches, the physical attributes should

be mapped, for others they can be left out. Analysis will be made on a case by case basis.

1. Scenario Planning & Backcasting

2. Co-creation (Kuenkel & Schaefer, 2013; Wood, Stillman, & Goss-Custard, 2015)

3. Companion Modelling (Barreteau, 2003; Campo et al., 2010; Daré et al., 2014; Etienne, 2014;

Gurung, Bousquet, & Trébuil, 2006)

4. Group Model Building (GMB) (Richardson, Andersen, Rohrbaugh, & Steinhurst, 1992; Richardson

& Andersen, n.d.)

5. Participatory Modelling in System Dynamics & Agent based models

a. Mediated Modeling (Committee, Systems, & Report, 2006)

b. Open Collaboration for Policy Modelling (OCOPOMO) (Scherer, Wimmer, Lotzmann,

Moss, & Pinotti, 2015)

c. Community Based Modelling (Hovmand, 2015; Janssen, Alessa, Barton, Bergin, & Lee,

2008; Voinov, Hood, & Daues, 2006; Voinov, Zaslavskiy, Arctur, Duffy, & Seppelt, 2008)

6. Participatory Integrated (Environmental) Assessments (PIAs)

7. Knowledge Elicitation Tools (KnETs) such as causal mapping

8. Serious and Role-Playing Games (Barreteau, Bousquet, & Attonaty, 2001; Gourmelon, Chlous-

Ducharme, Kerbiriou, Rouan, & Bioret, 2013; Vieira Pak & Castillo Brieva, 2010)

14

The list of relevant participatory methods will be expanded through literature review. Next, a framework

will be created that allows for a comparison of the different approaches.

MAIA Meta Model The MAIA meta-model provides a tool to organize in a structured and unambiguous manner, qualitative

information gathered for the different participatory processes. MAIA incorporates the determinants that

shape individual decisions: physical world, community and rules that ultimately affect human action

(Ostrom et al., 1994) into a language that helps model social systems and agents ruled by social institutions

(rules norms and shared strategies). Therefore, MAIA is used as a way to “translate” the knowledge

gathered and systematically arrange it as to be able to understand the behavior of agents based on

institutional analysis as well as observe emerging patterns in the macro-structure of the system within

our defined boundaries.

MAIA is organized into 5 structures that group related concepts, with which we can arrive to a

comprehensive overview of a social system. This structures are defined by (Ghorbani, Bots, Dignum, &

Dijkema, 2013) as follows:

1. Collective structure: actors are defined as agents by capturing their characteristics and decision criteria based on their perceptions and goals.

2. Constitutional structure: defines roles and institutions. It refers to the social context. 3. Physical structure: all non-social aspects of the environment agents are embedded in. 4. Operational structure: It encompasses the dynamics of the system. An “action arena” where

agents interact and react to each other and are influenced by the environment. 5. Evaluative structure: provides concepts used to validate and measure the outcomes of the

system. The user should identify variables that serve as indicators for model validity.

These structures are filled out with the relevant information collected on participatory modeling through

literature research and interviews. This information enables the conceptualization of action arenas in

which models are collaboratively constructed and simultaneously influenced by the systems in which they

are embedded. The evaluative structure can be filled in with criteria for model quality such as the checklist

to ensure model credibility, salience, and legitimacy (van Voorn, Verburg, Kunseler, Vader, & Janssen,

2016).

Overall, the formal and systematic description of the participatory process using the MAIA meta-model

will allow for structured comparison to take place

Synthesis: Design of a Transformative Agent Based Modeling Process After comparison, the best of each approach is extracted and synthesizes into a new participatory ABM

methodology, specifically suitable for multi-stakeholder transitions in LTST of a transformative nature.

15

Expected Outcomes & Implications of Research From the literature review an overview of participatory modelling practices from a wide range of

modelling fields, including system dynamics, integrated assessment, scenario planning, and

environmental modeling will result.

A comparative framework will result through which the different participatory methodologies can be

compared. Through this framework essential aspects of participatory methodologies will be identified and

evaluated in the context of other methodologies.

Development of a systematic participatory methodology for the purpose of designing multi-model

ecologies of LSTS that function as strategic foresight tools for multiscale, -stakeholder, -issue, -

perspective, -resolution, and -aspect matters, especially those related to transitions to more sustainable

systems, (Yilmaz, Lim, Bowen, & Ören, 2007). Such a methodology will supplement existing

methodologies for the design of agent based models with systematic overview of the social or

participatory processes that lead to the formulation of an agent based model (Van Dam et al., 2013). While

the steps will remain the same, especially those that precede model formalization, including problem

formulation, they will be extended with theory on how to facilitate the participatory and social aspects of

those steps.

Preliminary exploration of indicators for how the effectiveness of such a methodology can be evaluated

in practice. The thesis lays the foundation for future research to systematically evaluate participatory

modeling for example by formulation of indicators for high quality inputs, processes, outputs, methods to

measure the indicators, and comparing the participatory modeling with relevant alternatives.

Towards a model of participatory modeling While the final results will not be coded into a model, the utilization of MAIA will allow for the coding of

the participatory process into an agent based model. Such a model could serve as an additional evaluation

of the participatory process for agent based models.

Relevance to Industrial Ecology Transitions in LTST such as the circular economy and the energy transition go to the heart of the field of

industrial ecology, which studies the biosphere-technosphere matrix, aiming to bring about a sustainable

co-existence of the technosphere and the biosphere while learning from the biosphere to organize the

physical economy (Korevaar, 2004).

Furthermore, modelling and multi-model ecologies of various forms play an increasingly important role in

industrial ecology, which continually relies on models such as LCA, MFA, E-IOA, system dynamics and ABM

to design more sustainable systems (Bollinger et al., 2015).

By focusing on transformative processes this study also becomes inherently normative, much like the

Industrial Ecology biosphere-technosphere analogy, which implicitly holds that we should transition to

more sustainable systems (Boons & Roome, 2000). Recognizing that science is rarely free of normative

intent, we can use scientific investigation to improve our knowledge on sustainable solutions (Boons &

Roome, 2000).

Lastly, systematically reviewing participatory methods that aims to motivate stakeholders to take an

active role, ownership over the problem and come up with solutions, is also central to IE. Various

16

important fields in IE recognize the importance of stakeholder collaboration, most prominently in the

establishment of industrial symbiosis which aims at engaging “traditionally separate industries in a

collective approach to competitive advantage involving physical exchange of materials, energy, water, and

by-products” (Chertow, 2000). Participatory agent based models aim at a similar symbiotic or

collaborative approach, not based on physical proximity as in eco-industrial parks, but based on modeling

expertise.

Planning Below a preliminary planning is offered for the successful completion of the thesis. All indications are

rough estimates and will be updated as the thesis progresses. Holiday periods and additional time for

unexpected delays is included in the planning.

Table 1 Planning of thesis and major components

June July Aug Sept Oct Nov Dec Jan Feb

TRP

Proposal

Kickoff

Literature review

Comparative framework

Interviews Round 1

Synthesis & Methodology development

Interviews Round 2

Midterm

Green Light (24-28 nov)

Process Expert Input & Final editing

Final draft (jan 9-14)

Defense

Unforeseen Delays

Orange = work in progress

Green = holiday

Red = milestone / deadline

17

References Allen, T. F. H., Tainter, J. A., & Hoekstra, T. W. (1999). Supply-side sustainability. Systems Research and

Behavioral Science, 16(5), 403–427. http://doi.org/10.1002/(SICI)1099-1743(199909/10)16:5<403::AID-SRES335>3.0.CO;2-R

Baptista, M., Roque Martinho, C., Lima, F., A. Santos, P., & Prendinger, H. (2014). Improving Learning in Business Simulations with an Agent-Based Approach. Journal of Artificial Societies and Social Simulation, 17(3), 1–9. http://doi.org/10.18564/jasss.2516

Barreteau, O. (2003). Our Companion Modelling Approach. Journal of Artificial Societies and Social Simulation, 6(1).

Barreteau, O., Abrami, G., Daré, W., Du Toit, D., Ferrand, N., Garin, P., … Werey, C. (2012). Collaborative Modelling as a Boundary Institution to Handle Institutional Complexities in Water Management. In Restoring Lands - Coordinating Science, Politics and Action (Vol. 9789400725, pp. 109–127). Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-94-007-2549-2_6

Barreteau, O., Bousquet, F., & Attonaty, J.-M. (2001). Role-Playing games for opening the black box of MAS: method and teachings of its application to Senegal River Valley irrigated systems. Journal of Artificial Societies and Social Simulation, 4(2), Online: http://jasss.soc.surrey.ac.uk/4/2/5.html. http://doi.org/10.1016/j.foreco.2011.01.001

Bechtold, B. (1997). Chaos theory as a model for strategy development. Empowerment in Organizations. Retrieved from http://www.emeraldinsight.com/doi/abs/10.1108/14634449710195462

Berland, M., & Rand, W. (2009). Participatory Simulation As A Tool For Agent-Based Simulation. International Conference on Agents and Artificial Intelligence - ICAART, (January 2009), 553–557. http://doi.org/10.5220/0001786905530557

Black, L. J., & Andersen, D. F. (2012). Using Visual Representations as Boundary Objects to Resolve Conflict in Collaborative Model-Building Approaches. Systems Research and Behavioral Science, 29(2), 194–208. http://doi.org/10.1002/sres.2106

Bollinger, L. A., Nikolić, I., Davis, C. B., & Dijkema, G. P. J. (2015). Multimodel Ecologies: Cultivating Model Ecosystems in Industrial Ecology. Journal of Industrial Ecology, 19(2), 252–263. http://doi.org/10.1111/jiec.12253

Boons, F., & Roome, N. (2000). Industrial Ecology as a Cultural Phenomenon: On Objectivity as a Normative Position. Journal of Industrial Ecology, 4(2), 49–54. http://doi.org/10.1162/108819800569799

Bousquet, F., & Voinov, A. (2010). Preface to this thematic issue. Environmental Modelling & Software, 25(11), 1267–1267. http://doi.org/10.1016/j.envsoft.2010.03.020

Burnes, B. (2005). Complexity theories and organizational change. International Journal of Management Reviews, 7(2), 73–90.

Campo, P. C., Bousquet, F., & Villanueva, T. R. (2010). Modelling with stakeholders within a development project. Environmental Modelling and Software, 25(11), 1302–1321. http://doi.org/10.1016/j.envsoft.2010.01.005

Carpenter, S. R., & Gunderson, L. H. (2001). Coping with Collapse: Ecological and Social Dynamics in

18

Ecosystem Management. BioScience, 51(6), 451. http://doi.org/10.1641/0006-3568(2001)051[0451:CWCEAS]2.0.CO;2

Chertow, M. R. (2000). Industrial symbiosis: literature and taxonomy. Annual Review of Energy and the Environment, 25(1), 313–337.

Choi, T., Dooley, K., & Rungtusanatham, M. (2001). Supply networks and complex adaptive systems: control versus emergence. Journal of Operations. Retrieved from http://www.sciencedirect.com/science/article/pii/S0272696300000681

Chu, T.-Q., Drogoul, A., Boucher, A., & Jucker, J.-D. (2012). Towards a Methodology for the Participatory Design of Agent-Based Models. In N. Desai, A. Liu, & M. Winikoff (Eds.), Principles and Practice of Multi-Agent Systems (Vol. 7057, pp. 428–442). Berlin, Heidelberg: Springer Berlin Heidelberg.

Colander, D., & Kupers, R. (2014). Complexity and the Art of Public Policy: Solving Society’s Problems from the Bottom Up. Princeton University Press.

Committee, S. A. B., Systems, E., & Report, S. A. B. (2006). Mediated Modeling, (1982), 1–9.

Costanza, R., Graumlich, L., Steffen, W., Crumley, C., Dearing, J., Hibbard, K., … Schimel, D. (2007). Sustainability or collapse: what can we learn from integrating the history of humans and the rest of nature? Ambio, 36(7), 522–527. http://doi.org/10.1579/0044-7447(2007)36[522:SOCWCW]2.0.CO;2

Crawford, S. E. S., & Ostrom, E. (1995). A Grammar of Institutions. The American Political Science Review, 89(3), 582. http://doi.org/10.2307/2082975

Daré, W., Paassen, A. Van, Ducrot, R., Queste, J., Trébuil, G., & Barnaud, C. (2014). Companion Modelling, 233–262. http://doi.org/10.1007/978-94-017-8557-0

Doyle, J. K., & Ford, D. N. (1998). Mental models concepts for system dynamics research. System Dynamics Review. http://doi.org/10.1002/(SICI)1099-1727(199821)14:1<3::AID-SDR140>3.0.CO;2-K

Doyne Farmer, J., Gallegati, M., Hommes, C., Kirman, A., Ormerod, P., Cincotti, S., … Helbing, D. (2012). A complex systems approach to constructing better models for managing financial markets and the economy. European Physical Journal: Special Topics, 214(1), 295–324. http://doi.org/10.1140/epjst/e2012-01696-9

Drogoul, A. (2015). Agent-based modeling for multidisciplinary and participatory approaches to climate change adaptation planning approaches to climate change adaptation planning. Regional Forum on Climate Change, (December). Retrieved from https://www.researchgate.net/publication/276267709_Agent-based_modeling_for_multidisciplinary_and_participatory_approaches_to_climate_change_adaptation_planning

Etienne, M. (2014). Companion Modelling - A participatory Approach to Support Sustainable Development. http://doi.org/10.1007/978-94-017-8557-0

Funtowicz, S. O., & Ravetz, J. R. (1993). Science for the post-normal age. Futures, 25(7), 739–755. http://doi.org/10.1016/0016-3287(93)90022-L

Funtowicz, S., & Ravetz, J. (2003). Post-Normal Science. International Society for Ecological Economics, (2013), 1–10. http://doi.org/10.1007/978-3-8350-9053-8_13

19

Ghorbani, A., Bots, P., Dignum, V., & Dijkema, G. (2013). MAIA: a Framework for Developing Agent-Based Social Simulations. Journal of Artificial Societies and Social Simulation, 16(2), 9. Retrieved from http://econpapers.repec.org/article/jasjasssj/2012-61-3.htm

Ghorbani, A., & Weijnen, M. P. C. (2013). Structuring socio-technical complexity: modelling agent systems using institutional analysis : Proefschrift. Delft.

Gilbert, A. M. R. and N. (2004, October 31). The Design of Participatory Agent-Based Social Simulations. JASSS. Retrieved from http://jasss.soc.surrey.ac.uk/7/4/1.html

Gilchrist, A. (2000). The well-connected community: networking to the edge of chaos. Community Development Journal. Retrieved from http://cdj.oxfordjournals.org/content/35/3/264.short

Gourmelon, F., Chlous-Ducharme, F., Kerbiriou, C., Rouan, M., & Bioret, F. (2013). Role-playing game developed from a modelling process: A relevant participatory tool for sustainable development? A co-construction experiment in an insular biosphere reserve. Land Use Policy, 32, 96–107. http://doi.org/10.1016/j.landusepol.2012.10.015

Greer, J. M. (2005). How Civilizations Fall: A Theory of Catabolic Collapse, 1–14. Retrieved from http://www.ecoshock.org/transcripts/greer_on_collapse.pdf

Gurung, T. R., Bousquet, F., & Trébuil, G. (2006). Companion Modeling, Conflict Resolution, and Institution Building. Sharing Irrigation Water in the Lingmuteychu Watershed, Bhutan, 11 (2)(36).

Henderson, G. (2002). Transformative Learning as a Condition for Transformational Change in Organizations. Human Resource Development Review, 1(2), 186–214. http://doi.org/10.1177/15384302001002004

Holling, C. S. (1973). Resilience and stability of ecological systems. Annual Review of Ecology and Systematics, 1–23.

Holtz, G., Alkemade, F., De Haan, F., K??hler, J., Trutnevyte, E., Luthe, T., … Ruutu, S. (2015). Prospects of modelling societal transitions: Position paper of an emerging community. Environmental Innovation and Societal Transitions, 17, 41–58. http://doi.org/10.1016/j.eist.2015.05.006

Hovmand, P. S. (2015). Community Based System Dynamics : Lessons from The Field Trends in Maternal and Neonatal Health in Honduras. Retrieved from https://brownschool.wustl.edu/Faculty/ResearchCenters/SocialSystemDesignLab/Documents/MSU 2015 Hovmand-Lessons from the field v4.pdf

J.H. Kwakkel, W.E. Walker, & Marchau, V. A. W. J. (2010). From Predictive Modeling to Exploratory Modeling: How to use Non- Predictive Models for Decisionmaking under Deep Uncertainty. In Encyclopedia of Operations Research and Management Science.

Janssen, M. A., Alessa, L. N. ia, Barton, M., Bergin, S., & Lee, A. (2008). Towards a community framework for agent-based modelling. Jasss, 11(2). http://doi.org/6

Kahane, A. (2013, November). Transformative Scenario Planning: Working Together to Change the Future. Standford Social Innovation Review. Retrieved from http://ssir.org/articles/entry/transformative_scenario_planning_working_together_to_change_the_future

Kahane, A., & Van Der Heijden, K. (2012). Transformative scenario planning: working together to change

20

the future. Standford Social Innovation Review (1st ed). San Francisco: Berrett-Koehler Publishers. Retrieved from http://ssir.org/articles/entry/transformative_scenario_planning_working_together_to_change_the_future

Kelly, R. A. . B., Jakeman, A. J. ., Barreteau, O., Borsuk, M. E. ., ElSawah, S. ., Hamilton, S. H. ., … Voinov, A. (2013). Selecting among five common modelling approaches for integrated environmental assessment and management. Environmental Modelling and Software, 47(November 2013), 159–181. http://doi.org/10.1016/j.envsoft.2013.05.005

Korevaar, G. (2004). Sustainable chemical processes and products. Eburon Uitgeverij BV.

Kuenkel, P., & Schaefer, K. (2013). Shifting the way we co-create - How we can turn the challenges of sustainability into opportunities, 1.

Kum, S., Wang, H., Jin, Z., Xu, W., Mark, J., Northridge, M. E., … Metcalf, S. S. (2015). Boundary Objects for Group Model Building to Explore Oral Health Equity. In System Dynamics Society 2015 International Conference Proceeding (pp. 1–16).

Lange, E. a. (2004). Transformative and Restorative Learning: A Vital Dialectic for Sustainable Societies. Adult Education Quarterly, 54(2), 121–139. http://doi.org/10.1177/0741713603260276

Lynam, T., de Jong, W., Sheil, D., Kusumanto, T., & Evans, K. (2007). A review of tools for incorporating community knowledge, preferences, and values into decision making in natural resources management. Ecology and Society, 12(1). http://doi.org/5

Macbeth, D. (2002). Emergent strategy in managing cooperative supply chain change. International Journal of Operations & Production. Retrieved from http://www.emeraldinsight.com/doi/pdf/10.1108/01443570210433517

Meadows, D., Randers, J., & Meadows, D. (2004). Limits to growth: the 30-year update. Retrieved from https://books.google.nl/books?hl=nl&lr=&id=QRyQiINGW6oC&oi=fnd&pg=PR9&dq=limits+to+growth&ots=GpaQbE7aj-&sig=PFx7VuvQ61bZE-Fd2in7iCxfMPQ

Mezirow, J. (1990). How Critical Reflection Triggers Transformative Learning. Fostering Critical Reflection in Adulthood, 1–18. http://doi.org/10.1002/ace.7401

Mezirow, J. (1996). Contemporary Paradigms of Learning. Adult Education Quarterly, 46(3), 158–172. http://doi.org/10.1177/074171369604600303

Mezirow, J. (1997). Transformative Learning: Theory to Practice Transformative Learning Theory. Transformative Learning (Mezirow, (74), 5–12. http://doi.org/10.1002/ace.7401

Mezirow, J. (2003). Transformative Learning as Discourse. Journal of Transformative Education, 1(1), 58–63. http://doi.org/10.1177/1541344603252172

Morgan, G., Gregory, F., & Roach, C. (1997). Images of organization.

Nikolić, I. (2009). Co-evolutionary method for modelling large scale socio-technical systems evolution. Delft.

Nowak, A., Rychwalska, A., & Borkowski, W. (2013). Why Simulate? To Develop a Mental Model. Journal of Artificial Societies and Social Simulation, 16(3), 1–8. http://doi.org/10.18564/jasss.2235

21

O’Sullivan, E. (1999). Transformative learning: Educational vision for the 21st century. Journal of Sustainable Tourism, 19(6), 1–9, 235–258. http://doi.org/10.1080/09669582.2010.542246

Ostrom, E., Gardner, R., & Walker, J. (1994). Rules, games, and common-pool resources. Ann Arbor: University of Michigan Press.

Rahmandad, H., & Sterman, J. (2008). Heterogeneity and Network Structure in the Dynamics of Diffusion: Comparing Agent-Based and Differential Equation Models. Management Science, 54(5), 998–1014. http://doi.org/10.1287/mnsc.1070.0787

Reed, M. S. (2008). Stakeholder participation for environmental management: A literature review. Biological Conservation, 141(10), 2417–2431. http://doi.org/10.1016/j.biocon.2008.07.014

Richardson, G. P., & Andersen, D. F. (n.d.). Teamwork in Group Model Building.

Richardson, G. P., Andersen, D. F., Rohrbaugh, J. W., & Steinhurst, W. (1992). Group Model Building. Proceedings of the 1992 International System Dynamics Conference. Retrieved from http://public.cranfield.ac.uk/c082621/psi connect/documents/psiconnect_factsheet_group model building_dec2009.pdf

Rose, J., Kraus, A., Homa, L., Burgess, K., Cherng, S., Stange, K. C., … Riolo, R. (2015). Boundary Objects for Participatory Group Model Building of Agent-based Models. 48th Hawaii International Conference on System Sciences. http://doi.org/10.1109/HICSS.2015.357

Scharpf, F. W. (1997). Games Real Actors Play Actor-centered Institutionalism In Policy Research. New York: Westview Press. Retrieved from http://public.eblib.com/choice/publicfullrecord.aspx?p=746903

Scherer, S., Wimmer, M., Lotzmann, U., Moss, S., & Pinotti, D. (2015). Evidence Based and Conceptual Model Driven Approach for Agent-Based Policy Modelling. Journal of Artificial Societies and Social Simulation, 18(3), 14. http://doi.org/10.18564/jasss.2834

Stacey, R., Griffin, D., & Shaw, P. (2000). Complexity and management: fad or radical challenge to systems thinking? Retrieved from https://books.google.nl/books?hl=nl&lr=&id=5iaxESQiZKgC&oi=fnd&pg=PR9&dq=Complexity+and+Management:+Fad+or+Radical+Challenge+to+Systems+Thinking&ots=8aKCJ0UL8w&sig=wJIa61ta0PfYLMsk877loIJd194

Star, S. (2010). Ceci n’est pas un objet-frontière ! Revue D’anthropologie Des Connaissances, Vol 4, 1(1), 18. http://doi.org/10.3917/rac.009.0018

Star, S. L., & Griesemer, J. R. (1989). Institutional Ecology, `Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. Social Studies of Science, 19(3), 387–420. http://doi.org/10.1177/030631289019003001

Tainter, J. (1995). Sustainability of complex societies. Futures. Retrieved from http://www.sciencedirect.com/science/article/pii/001632879500016P

Tainter, J. (2000). Problem solving: Complexity, history, sustainability. Population and Environment. Retrieved from http://link.springer.com/article/10.1023/A:1006632214612

Taylor, E. W. (2008). Transformative learning theory. New Directions for Adult and Continuing Education, 2008(119), 5–15. http://doi.org/10.1002/ace.301

22

Tetenbaum, T. (1998). Shifting paradigms: from Newton to chaos. Organizational Dynamics. Retrieved from http://www.sciencedirect.com/science/article/pii/S0090261698900031

Van Dam, K. H., Nikolic, I., & Lukszo, Z. (Eds.). (2013). Agent-based modelling of socio-technical systems. Dordrecht ; New York: Springer.

van Voorn, G. A. K., Verburg, R. W., Kunseler, E.-M., Vader, J., & Janssen, P. H. M. (2016). A checklist for model credibility, salience, and legitimacy to improve information transfer in environmental policy assessments. Environmental Modelling & Software, 83, 224–236. http://doi.org/10.1016/j.envsoft.2016.06.003

Vieira Pak, M., & Castillo Brieva, D. (2010). Designing and implementing a Role-Playing Game: A tool to explain factors, decision making and landscape transformation. Environmental Modelling and Software, 25(11), 1322–1333. http://doi.org/10.1016/j.envsoft.2010.03.015

Voinov, A. (2008). Understanding and communicating sustainability: Global versus regional perspectives. Environment, Development and Sustainability, 10(4), 487–501. http://doi.org/10.1007/s10668-006-9076-x

Voinov, A., & Bousquet, F. (2010). Modelling with stakeholders. Environmental Modelling and Software, 25(11), 1268–1281. http://doi.org/10.1016/j.envsoft.2010.03.007

Voinov, A., Hood, R. R., & Daues, J. D. (2006). Building a Community Modeling and Information Sharing Culture. Proceedings of the 3rd Biennial Meeting of the International Environmental Modelling and Software Society.

Voinov, A., Kolagani, N., & McCall, M. K. (2016). Preface to this Virtual Thematic Issue: Modelling with Stakeholders II. Environmental Modelling & Software, 79, 153–155. http://doi.org/10.1016/j.envsoft.2016.01.006

Voinov, A., Kolagani, N., McCall, M. K., Glynn, P. D., Kragt, M. E., Ostermann, F. O., … Ramu, P. (2016). Modelling with stakeholders - Next generation. Environmental Modelling and Software, 77, 196–220. http://doi.org/10.1016/j.envsoft.2015.11.016

Voinov, A., Zaslavskiy, I., Arctur, D., Duffy, C., & Seppelt, R. (2008). Community modelling, and data-model interoperability. iEMSs 2008: International Congress on Environmental Modelling and Software, (January), 2035–2050.

Waas, R. P. M. Van. (2015). A System Dynamics Model Used as a Boundary Object in an Integrative Approach to Regional Water Schemes in South Africa.

Wagner, T. P., & Ortolano, L. (1975). Analysis of New Techniuqes for Public Involvement in Water Planning. Journal of the American Water Resources Association, 11(2), 329–344. http://doi.org/10.1111/j.1752-1688.1975.tb00684.x

Waldrop, M. M. (1992). Complexity: The Emerging Science at the Edge of Order and Chaos. Simon and Schuster. Retrieved from https://books.google.nl/books?id=VP9TWZtVvq8C

Wassen, M. J., Runhaar, H., Barendregt, A., & Okruszko, T. (2011). Evaluating the role of participation in modeling studies for environmental planning. Environment and Planning B: Planning and Design, 38(2), 338–358. http://doi.org/10.1068/b35114

Wood, K. A., Stillman, R. A., & Goss-Custard, J. D. (2015). Co-creation of individual-based models by

23

practitioners and modellers to inform environmental decision-making. Journal of Applied Ecology, 52(4), 810–815. http://doi.org/10.1111/1365-2664.12419

Yilmaz, L., Lim, A., Bowen, S., & Ören, T. (2007). Requirements and design principles for multisimulation with multiresolution, multistage multimodels. Proceedings - Winter Simulation Conference, 823–832. http://doi.org/10.1109/WSC.2007.4419678