TOOLS FOR STATISTICAL AND BEHAVIORAL ANALYSIS OF MASON MODELS
László Gulyás, Richard O. Legendi, Tamás Máhr
Rajmund Bocsi, Vilmos Kozma, Peter Rieger{lgulyas, rlegendi, tmahr, rbocsi, vkozma, prieger}@aitia.ai
AITIA International, Inc. – Intelligent Applications and Web Services
NSF MASON WorkshopGeorge Mason University
15-16 June, 2013, Fairfax, USAhttp://cs.gmu.edu/~eclab/projects/mason/workshop/pmwiki.php
The research leading to these results has received funding from the European Union, Seventh Framework Programme FP7/2007-2013 under grant agreement no. CRISIS-ICT-2011-288501.
NSF MASON Workshop 2
DEMO
2013.06.15.
NSF MASON Workshop 3
The El Farol Bar Problem• Agents are researchers (N=100)• They visit a popular but small bar in Santa Fé
• If attendance > 60 (overcrowded)• Who hasn’t come
• If attendance <= 60 • Who hasn’t come
• Each day agents decide individually and in the same time
W. Brian Arthur, “Inductive Reasoning and Bounded Rationality”, American Economic Review (Papers and Proceedings), 84,406–411, 1994.
2013.06.15.
NSF MASON Workshop 4
The El Farol Bar Problem• MASON implementation of a NetLogo variant• Artifical agents:
• ARMA-based prediction with history
• Players have two actions: No go / Go
• If +1 Score!• Goal: get max score
Rand, W. and Wilensky, U. (2007). NetLogo El Farol model. Center for Connected Learning and Computer-Based Modeling, Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL. http://ccl.northwestern.edu/netlogo/models/ElFarol
2013.06.15.
NSF MASON Workshop 5
Let’s play!
http://demo1.aitia.ai/
Simulations
2013.06.15.
NSF MASON Workshop 62013.06.15.
Model List
Create Experiments
Additional Info
NSF MASON Workshop 7
Admin Page (Admins are subjects too)
2013.06.15.
Subject list(joined)
Admin tools
NSF MASON Workshop 8
In-experiment Admin Page
2013.06.15.
Player status (moved/waiting)
NSF MASON Workshop 9
In-experiment Subject Page
2013.06.15.
Status messages
NSF MASON Workshop 10
Post-experiment Scores
2013.06.15.
NSF MASON Workshop 11
Outline• Background
• The CRISIS project
• CRISIS Simulation Library• CRISIS Game & PET v2.0• MEME• Future works
2013.06.15.
NSF MASON Workshop 12
Background: The CRISIS Project“The CRISIS project addresses building a next generation macroeconomic and financial system policymaking model: a bottom-up agent-based simulation that fully accounts for the heterogeneity of households, firms, and government actors. The model will incorporate the latest evidence from behavioral economics in portraying agent behavior, and the CRISIS team will also collect new data on agent decision making using experimental economics techniques. While any model must make simplifying assumptions about human behavior, the CRISIS model will be significantly more realistic in its portrayal of relevant agent behavior than the current generation of policymaking models.”
Crisis project description: https://www.crisis-economics.eu/
2013.06.15.
NSF MASON Workshop 13
CRISIS Simulation Library• Olaf covered that
2013.06.15.
NSF MASON Workshop 142013.06.15.
Web-based Game(Participatory Experiments)
Economic Simulator(Cloud-Based Parameter
Sweep Execution)
ModellingFramework
Models
NSF MASON Workshop 152013.06.15.
Web-based Game(Participatory Experiments)
Economic Simulator(Cloud-Based Parameter
Sweep Execution)
NSF MASON Workshop 16
CRISIS Game
2013.06.15.
(Planned design)
http://game.crisis-economics.eu
NSF MASON Workshop 17
The Model Under the CRISIS Game
2013.06.15.
Source: Domenico Delli Gatti, personal communications
(Single player mode)
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati. Macroeconomics from the Bottom-up. 1st ed. Springer, 2011.
NSF MASON Workshop 18
Architecture
2013.06.15.
NSF MASON Workshop 19
PET v2.0• Idea came from AITIA's PET Software
• Participatory Extension
• Generalized components behind the CRISIS Game• Modern, scalable infrastructure
• Previous version is 5+ years old • Components may run on (several) different machines
• Software already available for any experiment
2013.06.15.
NSF MASON Workshop 20
PET v2.0• Can be used:
• Locally• Laboratory experiments• Policy makers (scenario analysis with a proper model)
• On any webserver to run constantly• Gather data (scores from model and all user actions replay)• Dissemination
• Questionnaire module• Verify if subjects understood the rules
• Software is already in use by Universiteit van Amsterdam
2013.06.15.
NSF MASON Workshop 21
What is required to use?
1. An ABM model• PET is a generic framework• Currently Java and MASON models are supported• (But! No serialization is required )
2013.06.15.
NSF MASON Workshop 22
What is required to use?
1. An ABM model• PET is a generic framework• Currently Java and MASON models are supported• (But! No serialization is required )
2. Development of the web interface• No restriction on tools (HTML5, GWT, ...)• Communication to server by standardized tools (AJAX)
• Config files to map incoming messages to model actions
2013.06.15.
NSF MASON Workshop 23
MEME
2013.06.15.
WEHIA 2012 - Replication of MacroABM 24
Parameter sweeps?
„[...] suppose that in a model there are just 10 relevant parameters, and that each parameter can assume 10 different values (a rather simplifying assumption). As a result, one obtains that the constellation of the parameter space is given by 10^10 vectors. If we perform 20 different runs for each one of them to take into account the possible effects of changing the random seeds, the total number of simulations would amount to 2*10^11!”
2012.06.22.
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)
WEHIA 2012 - Replication of MacroABM 25
Parameter sweeps?
„[...] suppose that in a model there are just 10 relevant parameters, and that each parameter can assume 10 different values (a rather simplifying assumption). As a result, one obtains that the constellation of the parameter space is given by 10^10 vectors. If we perform 20 different runs for each one of them to take into account the possible effects of changing the random seeds, the total number of simulations would amount to 2*10^11!”
2012.06.22.
Domenico Delli Gatti, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)
Even if 1 run / second would take ~6 585 000 years
NSF MASON Workshop 26
MEME – Model Exploration Module• MEME is the tool for the job
• Standalone application• Parameter sweep exploration
• Being Strong• Exploiting Grid/Cloud systems
(LAN, Amazon, etc.)
• Being Smart• Design of Experiments
2013.06.15.
NSF MASON Workshop 27
MEME – Model Exploration Module• Generic tool
• Supports various platforms (NetLogo/Repast J/MASON/...)
• Easy-to-use graphical interface• Setting the parameter tree• Recording• External tool support (R/Matlab/Octave/...)
• Recently declared an open source project
2013.06.15.
NSF MASON Workshop 28
Design of Experiments
• Reduce # of runs• Ignore unimportant cases
„smartly”• Factorial Designs (2/3 lev.)• Latin Hypercube,• Box-Behnken,• Box-Wilson Central
Composite • Genetic Algorithm Designs,• Iterative Uniform
Interpolation• Adaptive Nonlinear Tests
2013.06.15.
NSF MASON Workshop 29
Demo
2013.06.15.
NSF MASON Workshop 30
FUTURE WORKS
2013.06.15.
NSF MASON Workshop 31
Future Works• Pet v2.0
• Tools to support front-end creation• JavaScript library (WIP)• User-friendly Eclipse editor plug-ins (flexible, easy to use)
• Both for the UI, questionnaire
• Tutorials, docs to be developed• Additional platforms
• MEME• Publish open source project, documentation...
• Cloud-based economic simulator
• +Your ideas?
2013.06.15.
NSF MASON Workshop 32
Interested?
• „A New Set of Tools Supporting Agent-Based Economic Modeling”, EPIA 2013 (under publication, ~Sept.)
• Statistical and Behavioral Model Analysis Tools.• Hands-on workshop @ ESSA 2013
• CRISIS Website: http://www.crisis-economics.eu/
2013.06.15.
NSF MASON Workshop 33
THANK YOU!• Questions?
• Will be available soon from http://iaws.aitia.ai/
• Download the CRISIS Simulation Library:• http://www.crisis-economics.eu/software
• Try the CRISIS Game• http://game.crisis-economics.eu
• Take a look on the El Farol model:• http://github.com/rlegendi/mason-examples
• Download MEME:• http://meme.aitia.ai
Richard O. [email protected]
http://people.inf.elte.hu/legendi/@legendi_ELTE
June 15, 2013.
2013.06.15.
The research leading to these results has received funding from the European Union, Seventh Framework Programme FP7/2007-2013 under grant agreement no. CRISIS-ICT-2011-288501.
NSF MASON Workshop 34
„Visionary things”• Usability:
• API level: making it easier to use (annotations, generics)• step(Object) vs. step(Model)
• DevTools:• Maven (Graddle, Ivy, Sbt)• Recording and parameterizing simulations
• Distribution• Networks• MASON Simulation Library for the community
2013.06.15.
Top Related