Towards a Context- and Scope-Sensitive Analysis for Specifying Agent Behaviour
Model Replication in the Context of Agent-based Simulation
33
Model Replication in the Context of Agent- based Simulation Richárd O. Legéndi , László Gulyás, Yuri Mansury Eötvös Loránd University, AITIA International, Inc., Cornell University [email protected] , [email protected] , [email protected] 1st European Conference on Political Attitude and Mentality ECPAM 2012, Bucharest, September 3-5, 2012 This work was partially supported by the Hungarian Government (KMOP-1.1.2-08/1- 2008-0002), the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement CRISIS-ICT-2011-288501 (CRISIS – Complexity Research Initiative for Systemic InstabilitieS) and mOSAIC 2011-256910 (Open-Source API and Platform for Multiple Clouds). These supports are gratefully acknowledged.
-
Upload
richard-oliver-legendi -
Category
Technology
-
view
1.649 -
download
3
description
My presentation held at the 1st European Conference on Political Attitudes and Mentalities (ECPAM 2012) conference, Bucharest, Romania, September 3-5, 2012. Electronic paper link: http://mass.aitia.ai/images/publikaciok/2012-ecpam-replication_case_studies-camera_ready.pdf Abstract: This paper examines model replication in the context of agent-based simulation through two case studies. Replication of a computational model and validation of its results is an essential tool for scientific researchers, but it is rarely used by modelers. In our work we address the question of validating and verifying simulations in general, and summarize our experience in approaching different models through replication with different motivations. Two models are discussed in details. The first one is an agent-based spatial adaptation of a numerical model, while the second experiment addresses the exact replication of an existing economic model.
Transcript of Model Replication in the Context of Agent-based Simulation
Model Replication in the Context of Agent-based Simulation
Richárd O. Legéndi, László Gulyás, Yuri MansuryEötvös Loránd University, AITIA International, Inc., Cornell
[email protected], [email protected], [email protected]
1st European Conference on Political Attitude and Mentality
ECPAM 2012, Bucharest, September 3-5, 2012
This work was partially supported by the Hungarian Government (KMOP-1.1.2-08/1-2008-0002), the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement CRISIS-ICT-2011-288501 (CRISIS – Complexity Research Initiative for Systemic InstabilitieS) and mOSAIC 2011-256910 (Open-Source API and Platform for Multiple Clouds). These supports are gratefully acknowledged.
ECPAM 2012 - Replication case studies 2
LayoutMotivation and Background
Replication? Why care?Case Studies and Results
ABM approach for the New Economic Geography
Replication of the Bottom-up Adaptive Macroeconomics
Summary
2012.09.03.
ECPAM 2012 - Replication case studies 3
Motivation and Background
2012.09.03.
ECPAM 2012 - Replication case studies 4
Replication? Why care?Replication of experiments, validation of
results are essential„Simulations as experiments”
If cannot be reproduced, its scientific value is in question
Models never replicated - except a few classical ones
Helps us get a deeper understandingOf relevant properties, key issuesDeploy simulation as a research tool
2012.09.03.
ECPAM 2012 - Replication case studies 5
Validation?Docking – alingment of different models
Different computational models for the same phenomenon
ReplicationW/o being able to replicate results of an
artificial model, how to target real-world systems?
Several problems, e.g. ambiguity Different approaches exist (AgentUML, ODD, etc.) But there’s no consensus on using them...
2012.09.03.
ECPAM 2012 - Replication case studies 6
Case Study 1An Agent-Based Adaptation of the New Economic Geography
2012.09.03.
ECPAM 2012 - Replication case studies 7
New Economic GeographyPaul Krugman’s city
formation modelOriginally a
numerical modelApplied agent-based
approach
Masahisa Fujita, Paul Krugman, Anthony J. Venables: „The Spatial Economy.” MIT Press, Cambridge, MA, 1999.
2012.09.03.
ECPAM 2012 - Replication case studies 8
Model Structure
2012.09.03.
ECPAM 2012 - Replication case studies 9
Zipf’s Law in City Formation
2012.09.03.
City Population(2010)
Rank
New York 8,175,133 1
Los Angeles
3,792,621 2
Chicago 2,695,598 3
Houston 2,099,451 4
Philadelphia
1,526,006 5
Phoenix 1,445,632 6
San Antonio
1,327,407 7
San Diego 1,307,402 8
Dallas 1,197,816 9
San Jose, CA
945,942 10
ECPAM 2012 - Replication case studies 10
MotivationPrevious works explains Zipf’s law
successfullyBut lacks micro-foundations
We extended th FKV modelGeneral-equilibrium modelExcellent micro-foundationsBut cannot generate a hierarchical system of
cities
2012.09.03.
ECPAM 2012 - Replication case studies 11
Why the Agent-Based approach?Introduce heterogeneity
NoiseAgent-specific migration thresholds
Enables migration to proceed in a non ad-hoc way
Extensibility
2012.09.03.
ECPAM 2012 - Replication case studies 12
ResultsWe proposed a spatial AB version of FKVApplied an inherently different approachRetains the key features of the original model
Including consumers’ love for varietiesIncreasing returns in productionTension between centripetal (agglomeration)
and centrifugal (dispersion) forces
2012.09.03.
ECPAM 2012 - Replication case studies 13
Tomahawk-diagramPopulation migration
(λ) vs. „freeness” of trade (φ)
Break and sustain pointφB and φS
Closed-form solution and implicit function to evaluate
2012.09.03.
ECPAM 2012 - Replication case studies 14
Replication ResultsSimulations
replicates expected resultst = 2000 / 5000 time
stepsφB and φS verified
2012.09.03.
ECPAM 2012 - Replication case studies 15
Case Study 2Replication of the Macroeconomics from the Bottom-up
2012.09.03.
ECPAM 2012 - Replication case studies 16
Macroeconomics from the Bottom-upAgent-based macro
modelEmpirical external
validationUsing real-world data
Replication of the same modelIn a different
environment
Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:Macroeconomics from the Bottom-up. 1st ed. Springer, 2011.
2012.09.03.
ECPAM 2012 - Replication case studies 17
Model Structure
2012.09.03.
Source: Domenico Delli Gatti, personal communications
ECPAM 2012 - Replication case studies 18
AgentsHouseholds
Supply laborBuy consumption goodsHold deposits
FirmsDemand laborProduce and sell consumption goods
BankReceive deposits from householdsExtend loans to firms
2012.09.03.
ECPAM 2012 - Replication case studies 19
Market Processes I1. Fims compute net worth, production/price
and labour demand2. Credit market:
1. Bank decides credit conditions2. Firms decide to whether take loan or not
3. Job market:1. Firms redefine labour demand, publish
vacancies:1. Excess workforce: fire workers2. Insufficient workforce: hire if possible
2012.09.03.
ECPAM 2012 - Replication case studies 20
Market Processes II4. Consumption goods market:
1. Workers get wages and compute consumption budget
2. Firms post their price3. Consumers contact z firms randomly
Ordered by price
4. Unspent money Involuntary savings5. Unsold goods Sold at zero cost (non-durable)
5. Accounting1. Firms calculate profits2. Earnings are retained profits
Used to update net worth.2012.09.03.
ECPAM 2012 - Replication case studies 21
Why to replicate? Parameter sweeps„[...] suppose that in a model there are just 10 relevant parameters, and that each parameter can assume 10 different values (a rather simplifying assumption). As a result, one obtains that the constellation of the parameter space is given by 10^10 vectors. If we perform 20 different runs for each one of them to take into account the possible effects of changing the random seeds, the total number of simulations would amount to 2*10^11!”
2012.09.03.
Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)
ECPAM 2012 - Replication case studies 22
Why to replicate?In a different environment?
Matlab Java/Mason
EfficiencyReduce required time for a single simulation run
Tool support: MEMEParameter sweep explorationBeing Strong
Exploiting Grid/Cloud systemsBeing Smart
Design of Experiments
2012.09.03.
ECPAM 2012 - Replication case studies 23
Background“The CRISIS project addresses building a next generation macroeconomic and financial system policymaking model: a bottom-up agent-based simulation that fully accounts for the heterogeneity of households, firms, and government actors. The model will incorporate the latest evidence from behavioral economics in portraying agent behavior, and the CRISIS team will also collect new data on agent decision making using experimental economics techniques. While any model must make simplifying assumptions about human behavior, the CRISIS model will be significantly more realistic in its portrayal of relevant agent behavior than the current generation of policymaking models.”
Crisis project description: https://www.crisis-economics.eu/
2012.09.03.
ECPAM 2012 - Replication case studies 242012.09.03.
Web-based Game
(Participatory Experiments)
Economic Simulator
(Cloud-Based Parameter Sweep
Execution)
Replicated Model
ModellingFramework
Models
ECPAM 2012 - Replication case studies 25
Results I - Benchmarking
2012.09.03.
ECPAM 2012 - Replication case studies 26
Result II – VerificationScaled agents (w/o changing overall ratio)
Up to 7500 agentsAvg’d 40 runst = 1000 time stepsIncluded initial state
High oscillationsUntil spontaneous
order emerges(„equilibrium”)
2012.09.03.
ECPAM 2012 - Replication case studies 272012.09.03.
ECPAM 2012 - Replication case studies 28
Summary
2012.09.03.
ECPAM 2012 - Replication case studies 29
Summary: Case Study 1We created a replication of the FKV by using
a different approachRetains hallmark of the original model
Introduced heterogeneity at several levelsAllows further studies
With different activation regimesN-cities model
2012.09.03.
ECPAM 2012 - Replication case studies 30
Project Info
2012.09.03.
http://emergingcities.aitia.ai
ECPAM 2012 - Replication case studies 31
Summary: Case Study 2We created a replication of the MacroABM model in a
different environmentIdentic outputResults are platform, environment-independentOpens up the window of standardized simulation tools
Extensive parameter space explorations (MEME)
Performance speedupBy the factor 5x-10xOn the other hand, code length is increased similarly:
Matlab: ~300 LoC Java: 1500 + 1000 LoC
2012.09.03.
ECPAM 2012 - Replication case studies 32
Download!
2012.09.03.
http://www.crisis-economics.eu/jmark-i-build-report
Thank you!Richard O. LegendiMail: [email protected]
Twitter: @legendi_ELTEBlog: http://xcafebabe.blogspot.com
Web: http://people.inf.elte.hu/legendi/
September 3., 2012.
Emerging Cities Website: http://emergingcities.aitia.ai/ Crsisis Website: http://www.crisis-economics.eu/
