ICPSR - Complex Systems Computing - Session 7 - 2012

Introduction to Computing for Complex Systems (Session XIII)!

Eric Provins Daniel Martin Katz!

Goals For Today!

Simple Birth Rates Revisited !

Automation III --- !" " "The Behavior Space!

Multiple Variable Models!


• We’ve seen that Netlogo provides several ways to output data and run the model so that we can get statistical information.!


• What if there are several variables in a model? Will this increase the time it would take to test the parameter space?!

• Open the Simple Birth rate model!

• There are three variables, carrying capacity, blue fertility rate, and red fertility rate.!

!

Simple Birth Rates!

• The Blues and Reds reproduce according to their fertility rate and the entire population is limited by the carrying capacity.!

Simple Birth Rates!

• There is already an output window on the bottom of the screen that will show how long it took for one of the colors to go extinct.!

• Use the “run experiment” button to start the model. Run the model a few times, varying the fertility rates.!

Simple Birth Rates!

Simple Birth Rates!

• The output is relatively simple, but it would probably take a long time to move through each combination of blue and red fertility rates, especially if you wanted to run the model more than once for each combination. !

• We could also vary the carrying capacity, which would also increase the amount of time it would take to run these models by hand.!

Simple Birth Rates!

Automated Analysis !of Model Runs!

Part III!

T-X!

The Behavior Space!

What is BehaviorSpace?!

•  “BehaviorSpace is a software tool integrated with NetLogo that allows you to perform experiments with models.” !

!!• http://ccl.northwestern.edu/

netlogo/docs/behaviorspace.html !


•  “Behavior Space runs a model many times, systematically varying the model's settings and recording the results of each model run. This process is sometimes called "parameter sweeping” !


• It lets you explore the model's "space" of possible behaviors and determine which combinations of settings cause the behaviors of interest.” !

Why Behavior Space is Useful!

• BehaviorSpace automates the movement of the model runs through each combination of variables, allowing us to move quickly across the parameters we wish to test.!


• It also outputs the information into a .csv file so that we can use statistical software to analyze the data.!

• So if you have a lot of model runs, multiple variables, or want to analyze model data with statistical software, then BehaviorSpace may be of use to you!!


• Let’s see how BehaviorSpace works.!

!

• Go to the Tools scroll down menu and click BehaviorSpace.!

Why Behavior Space !is Useful!

Working with the !Behavior Space!

• Now you should have the experiment page.!

!

• You will see the options New, Edit, Duplicate, Delete, and Run.!

• The New button creates a new experiment, and brings you to the page where you will be able to specify the parameters of the model you wish to test and how many times you wish the model to run per combination.!


• The Edit button allows you to edit any experiment that is saved in the BehaviorSpace experiment list. !

!


• If you save your Netlogo program, it will also save your experiments, so you do not have to write out the parameters you wish to test each time you turn off Netlogo.!


• The Duplicate button allows you to create another copy of an experiment that will be saved in the experiments box of BehaviorSpace.!


• The Delete button permanently deletes an experiment from the experiments box.!


• The Run button begins the process of running your experiment and outputting the data into a file.!


• Let’s go to the experimental setup!

• Click the ‘New’ button.!

• You should now see a screen that will allow you to detail the variables you wish to test and how the program should run the experiment.!

Experiment Setup!

• The Experimental setup screen has a title box, a variables box, an area that allows you to control how many times the model should run per combination, and boxes that specify any commands you wish the program to run when the setup or go button is pressed.!

!

Setting up an Experiment!

!

• There are also boxes that allow you to detail any conditions that should stop the running of the program and any commands that should go into effect at the end of the model run.!

• It also includes the ability to determine a limit to how many steps the model will run for.!


• Let’s take an in depth look at each of these features.!

• I will give the title “Experiment #1: Blue & Red Fertility [0 1 10]” to my experiment and move on to the variables box.!


Variables Box!

• The variables box includes variables that come from sliders, switches, and choosers (draw down menus) on the interface. It can also include variables found within the program code.!

Variables Box!

• The user is be able to specify the boundaries of the parameter space they wish BehaviorSpace to “sweep” through. This is done by writing in the variables and the values the user wishes to test.!

Variables Box!

• BehaviorSpace keeps variables higher up in the box constant as it cycles through the lower variables’ value settings, only moving the higher variables to their next setting after finishing a complete cycle through the possible alternatives in the variables below it.!

Variables Box!

• If the runs are taking place in parallel, the output may not exactly mirror this process.!

• Regardless of the case, no matter where you locate a variable, the entire parameter space for all the variables you define in the box will be tested.!

How to’s!

• Assigning a value to a variable:!

• Ex: ["blue-fertility" 10] This will give the variable blue-fertility the value of 10 in all of the model runs.!

How to’s!• Assigning two values to a

variable (listing values)!

!

• Ex: ["blue-fertility" 1 2] gives the variable blue-fertility the value of 1 and runs through all other combinations (if there are other variables), and then moves on to 2 and runs through all the other possible combinations with red-fertility and carrying capacity.!

How to’s!

• Assigning multiple values to a variable (listing values)!

• Ex: ["blue-fertility" 1 2 4 7] Runs through 1 and 2 and also does 4 and then 7. !

How to’s!

• Using an interval to assign multiple values!

• The interval must be inside a set of brackets!

!

How to’s!

• Ex: ["blue-fertility" [1 1 3]] This example runs through the values 1 through 3, moving at an increment of 1. Therefore, it will assign the value 1, 2, and 3 to the variable as it moves through the different combinations.!

!

How to’s!

• When assigning values to a variable with an interval, be careful to only include the numbers you wish to test. The interval is inclusive and will test all the numbers you specify.!

•  Ex: ["blue-fertility" [0 1 4]] This will give the variable the values 0, 1, 2, 3, and 4.!

How to’s!

• Behavior Space is entirely inclusive of the specified range!

• In other words, Inclusive will execute all Parameter Combos in the Range!

!

• Repetitions: The user can select how many times Netlogo should run a given combination of variables!

How to’s!

• Measure runs using these reporters: This is what the model will measure and output in the data.!

• Measure runs at every step: Checking this means the data will include output for each step in the model!

How to’s!

• Setup Commands: The user can include commands that will be executed in addition to calling the ‘to setup’ procedure when the setup button is hit.!

How to’s!

• Go Commands: The user can include commands that will be executed in addition to calling the ‘to go’ procedure when the go button is hit.!

How to’s!

• Stop Conditions: This box allows the user to specify conditions that would end the model run if they were met. !

!

How to’s!

• While there may be many reasons a modeler would want to do this, one reason could be to prevent illogical or undesirable combinations of variables from occurring while the program sweeps through parameter space. !

How to’s!

• Final Commands: The user can specify commands to run at the end of a model run.!

How to’s!

• Time Limit: The user can specify how many ticks a given combination of variables will be allowed to run before stopping.!

• In certain instances model run will go on forever … need to have a time limit in that case!

How to’s!

Simple Birth Rates in the Behavior Space!

!

• OK, now let’s run the model. We will put a time limit of 100 ticks for each model run.!

Enter!

these !

Values !

As !

shown!

!

Running BehaviorSpace!

• Select the experiment we just created and hit “Run”.!

• You will now be given the option to select if your data will be put in a spreadsheet or in a table.!

• The data file will be saved as a comma separated values (.csv) file.!

Parallelization!

• You will also be given the option to choose how many experiments should run in parallel.!

• By default, Netlogo will run one experiment for each core processor in your computer.!

Parallelization!!

• Only one experiment will be shown on screen, the others will be run in the background.!

• You should individually test what your computer can handle, since many parallel runs can slow down your computer.!

•  After making those choices the model will begin to run.!

• You will have the option to turn off viewing the plot and updating the visualization, plots, and monitors. This will help increase the speed of the runs.!

Increasing the Speed!

• You can also use the speed slider, which determines how !

many times the image should be updated.!

Increasing the Speed!

Output!• Spreadsheet data!

!

• At the top, the data sheet will include the model and experiment name, and the size of the world.!

• The commas will allow you to demarcate where the columns should be in the program you will use to analyze the data.!

Output!• Tables (might be easier to work with in

outside statistical software)!

!

• At the top, the data sheet will include the model and experiment name, and the size of the world.!

• The commas will allow you to demarcate where the columns should be in the program you will use to analyze the data.!

• You will probably want to edit the file before sorting it. The information at the top can be copied and pasted elsewhere if you need it. !

• The actual output from the model runs will be below.!

Output!

• Once you have edited your file, you will be able to sort the data as you wish. !

"Example in Excel Data --> TexttoColumns!

"Then select delimited and by comma!

!

• As we saw before, the output may not be in the order, since many runs may have been running in parallel.!

Output!

• Once your data is in a useful format, you can begin to analyze it! !

Output!

• We have seen that BehaviorSpace can run through the parameter space much faster than having a human go by hand through all the different variable combinations.!

In Summary!

!

• The previous data file took a little more than 5 minutes to create and output.!

In Summary!

• But it moved in increments of 1.0 and the carrying capacity remained constant across the different combinations. Most importantly, the model was limited to 100 ticks, which may not be enough time to witness the dynamics the model is supposed to display.!

There are Limitations!

• By adding these other features we will see how quickly the length of time required to run the models would grow if we added smaller increments of movement across the parameter space and the extra carrying capacity variable.!

There are Limitations!

Just 40,814,201 Runs to Go

Limitations!

• 40,814,201 runs is a lot!

• Let’s assume (probably incorrectly) that it takes 1 sec / run (Including all the steps in one run of the model)!

• 40814201 / 60 secs / 60 mins / 24 hours / 7 days = 67.484 weeks, 1.298 years!

!

• That’s not including parallel runs, but even if you could run 100 in parallel, it would still take a considerable amount of time!

• Thus, full parameter sweep Probably Not feasible in Netlogo!

Limitations!

Limitations!

• One option would be to limit the parameter space to an area of specific interest. !

• For example, the dynamics of two competing groups with varying fertility rates is not particularly interesting when the carrying capacity is zero from the outset. !

Limitations!

• Netlogo was mainly designed for visualizations and as an introductory language.!

• If you write good modular code, you will be able to convert your model into another language that can perform massively parallel runs more efficiently than Netlogo (e.g. Python or C++)!

ICPSR - Complex Systems Computing - Session 7 - 2012

Technology

Transcript of ICPSR - Complex Systems Computing - Session 7 - 2012