Post on 21-Dec-2015
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
PAK346 Framework workshop
Marco Huigen (P8)
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Content
Introduction (20 min.)Expert N: needs and requirements
Putting this workshop in PAK-perspective (30 min.)P8: OverviewThe world according to me
Framework thinking: Living between worlds (30 min. - 1 hr)
DANUBIA theoretical (1 hr) DANUBIA practical (45 min.)
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
The PAK346 P8 project
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Objectives and responsibilities of P8
1. Facilitate project communication on integrationIntegration-on-Demand approach
2. Facilitate technical implementation of the ‘overall land system model’
Danubia framework
3. Facilitate ‘Integrative simulation experiments’Experimental frame
4. Data Management Simulation input and output
Model coupling and data management - Project P8
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
2 Facilitate technical implementation P8 will support the model integration by using
the Danubia framework*„Framelets“ (knowledge manager, model manager,
experiment manager)
Every project group is responsible for programming its model, its model wrapper and its model interfacesP8 assists in finding and designing the required
components (in the IoD approach)
Model coupling and data management - Project P8
* Open source, clearance will be requested from developer group
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
3 Facilitate ‘Integrative simulation experiments’ P8 will set up and maintain the integration
environment for running simulations (without dynamic P1).Framework control (e.g. model-sequencer, time-
controller)Maintain calculation nodes hardwareData storage capacity
Model coupling and data management - Project P8
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
4 Data Management of simulation input and output P8 will set up and maintain a Data Management
System (DMS) for simulation input and outputAs much as possible and useful simulation data will be
stored in universal database format (approachable with SQL)
Allows for analyzing scenariosMaintain DMS software (MySQL) and hardware
Model coupling and data management - Project P8
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Framework Thinking
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DANUBIA framework
If your model-coupling project:Requires multiple model-coupling configurationsHas long time horizonRequires re-usabilityRequires (scientific and technical) adaptability
• Scientific functionality model could (easily) change• Technical functionality might change
…
Then: Use a computational architecture
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DANUBIA Theoretical
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DANUBIA: System Structure and Framework Technology
Research group “Computer Science” Ludwig-Maximilians-Universität München
Rolf Hennicker, Stephan JanischAndreas Kraus, Matthias Ludwig
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
The DANUBIA system is designed as an integrativeplatform for
• coupled simulations of various models of natural-science and socio-economic disciplines
• analysis of transdisciplinary effects of mutually dependent processes (including "acting" entities like households, farmers, ...)
• integrates 17 simulation models developed by the Danube groups
• runs on a computer cluster with more than 32 processors
• the simulation models run in parallel and exchange data at run-time
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Common graphical modeling language (UML) used by all project groups for the documentation of interfaces, concepts and designs
Framework technology to facilitate the integration of simulation models
Object-oriented approach in all development phases (system analysis, design and programming)
Formal software engineering methods to verify critical parts of the integrated DANUBIA system
System Development Principles
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DANUBIA – Architecture
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
The Framework Idea
Extract common properties and rules which hold for all DANUBIA models and implement them in a general abstract pattern (template).
The model developer must only implement the open pieces of the template by model specific realizations.
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
The DANUBIA Developer FrameworkCommon properties of all DANUBIA models
<<extends>> <<extends>> <<extends>> <<extends>>
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Common Behaviour of all DANUBIA Models
run()getData()compute()commit()
getData
computecommit
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DeepActor Models
Deep actor models integrate into their simulations decision-making entities, called actors (e.g. households, water suppliers, farmers).
Any actor has a repository of potential plans that the actor can execute (initial plans) .
In each computation step an actor decides which of the initial plans should actually be executed (active plans).
To support decisions each actor has sensors through which he can observe the "environment" and a history to remember previous decisions
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
options
rating
execute filter
querymodelGetData
modelPreCompute
modelPostCompute
run()getData()compute()commit()
getData()compute()commit()modelGetData()…
modelGetData()…
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
• A model is blocked (waitForGetData) if the coordination condition for getData is not satisfied (similarly, waitForCommit).
• The correctness of the coordination can be formally verified with model checking techniques.
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Conclusion: DANUBIA framework
Common modelling language (UML) has been proved to provide a valuable tool for integrative system modelling.
Framework technology
supports model developers to integrate (conceptually and technically) their simulation models into the overall system structure of DANUBIA
provides general rules (templates) which support the trustability and quality of the system
Additional provisions for software quality:Formal verification of the correct coordination of concurrently running simulation models according to their local time scales.
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Conclusion: DANUBIA framework Uses Object Oriented technology
Gives and keeps a very clear, transparent overview of complex models/components
Completely written in JAVAEach model/component is packed into a compiled jar-file
Component based Each component can be plugged on and off in a simulation
Uses JAVA R(emote)M(ethod)I(invocation) technology Individual model components may run on different computers
and platforms Models and Components (=client) log on to the framework server.
The server controls the data exchanges and does the time-management
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Conclusion: DANUBIA framework Badly chosen name
Framework is generically applicable, not only for Danube area Framework is spatially explicit
By using configuration files users may set up any ‘simulation environment’ based upon ASCII grid-files.
The extent and resolution (= proxel size) are user-defined in the configuration. Framework allows for partial area testing.
Framework is temporally explicit In each model configuration file users indicate the model time step. The time scale is flexible (can be expressed in seconds, min, hours, days, months,
years). Framework does not support itself dynamic coupling within a time step, but due to
time scale flexibility this can be overcome. Framework allows for distributed and parallel processing (task based)
Uses JAVA RMI (OS independent)
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Conclusion: DANUBIA framework Framework has a high learning curve
Due to its many functionalities and advanced level of programming, users will find it difficult to understand the ‘inner-working’.
Framework is transparent The architecture, although evolved drastically over the years, has a very logical
built-up Framework requires models to have ‘plug-points’
In a simulation run the framework controls all flows and needs therefore control over the components.
Framework supports model development and ‘testing’ very well. Has extensive logging capacities and j-unit support
Framework is open source Although open source, it is unlikely that our project will be able to structurally
adapt the code due to lack of required skills
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
DANUBIA Practical
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Our DeepFarming model consists of the following components: 1. ACRE: An agricultural optimization model
Programmed in GAMS, Calculates on district level, Calculation annual interval, Calculation provides input for agents to create a new land use plan
2. ACRE wrapper: a java program controlling the GAMS model Responsible for file maintenance; zips, deletes and moves files, Responsible for data maintenance; reads and writes GAMS from/to MySQL.
3. DeepFarming Core model: a deepactors model implementation Responsible for data maintenance; reads and writes our input and output
from and to a MySQL database. Responsible for all data interactions with other models in the simulation
environment. 4. DeepFarming District models: a deepactors model implementation
Responsible for actual calculations of our agents. Every model represents a set of districts and has around 6000 farm actors. Because of this sub-model architecture, a higher level of distributed
calculation is achieved. 5. MySQL database: a Relational Data Management System 6. mydfm connector: a controller for unified access to the mysql RDMS 7. Several tools for data modification and analysis
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Deep Farming
In the Danubia framework our DeepFarming model directly interacts with the models
- Biological (crop growth)- Soil (nutrient flows)- WaterSupply (animal drinking water, irrigation)- Meteorology (weather)
DeepFarming actors:- Total of more than 50.000 actors ‘in sim’- Each actor ‘lives’ on a proxel- 28 farm system types (e.g. CashSugMaize, MeatBreedSum)- Each actor has statistically based size of land (consisting of arable
and/or grassland)- Each farm system type has unique crop rotation system (17 land use
types are represented)- Each farm system type has unique combination of husbandry (9
animal types are represented)
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Deep Farming
Per simulated day our actors may perform various actions. Whether these actions take place depend on:
1. Actors’ plan2. Environmental sensors (weather
conditions)3. Crop conditions
Yr2Yr1
Daily decisions:Daily decisions:• Planting (weather forecast, vegetation time demands)Planting (weather forecast, vegetation time demands)• Fertilizing (Crop stage, weather conditions)Fertilizing (Crop stage, weather conditions)• Harvesting (Crop stage, weather conditions)Harvesting (Crop stage, weather conditions)• Cutting (biomass increase, weather conditions)Cutting (biomass increase, weather conditions)• IrrigateIrrigate
•Run ACRERun ACRE•Make district plan per year (after crop silage is planted)Make district plan per year (after crop silage is planted)•Disaggregate plan of ACRE to AgentsDisaggregate plan of ACRE to Agents
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Deep Farming
The farm actors derive their crop rotation plan based on their interpretation of the ACRE output
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
Deep Farming
Per crop advanced farm management decision rules have been programmed, taking into account weather conditions, environmental conditions, crop requirements and conditions.
e.g.
Planting depends on:• vegetation period in days, • vegetative rest, • temperature sum, • minimum germination temperature,• „Temperaturgrenzwert“ for germination
and• “Befahrbarkeit” of land
Do not plant today
Requirements:· the crop is in the crop rotation of the actor· the crop was not planted yet
Does the actor want to plant winter wheat?
No
Yes
Requirements:· no heavy rain predicted· land is accessible by machinery
Crop requirements:In the development stage BBCH 10-30 (Leaf development to stem elongation) the temperature should be below 5°C
Crop requirements:During the BBCH stage 39-49 (end of stem elongation until the beginning of heading) enough rainfall is required
The actual weather situation is acceptable?
Vernalization
Winter Wheat has to get into the stage of tillering before winter
dormant.
In the period of heading rainfall is enough?
Plant today
No
No
No
No
Yes
Yes
Yes
District: “climate history“
DANUBIA: Current state
District: “climate history“
Crop requirements:Planting date should be 0-30 days before the appearance of the first frost
District: “climate history“
Decision tree: Planting of Winter Wheat
Yes
Framework workshop PAK346 – 26nd November 2008, Hohenheim – Huigen
-150
-100
-50
0
50
100
150
200
250
3001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
1 2 3 4 5 6 7 8 9 10 11 12
-200
-100
0
100
200
300
4001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
1 2 3 4 5 6 7 8 9 10 11 12
Weather forecast
Is it time to drill winter wheat?
Requirements of winter wheat:- Germination temperature- Vernalisation- Enough rainfall during heading
heading germination temperature
Vernalisation
Deep Farming
An actor dynamically adjusts its plan based on experiences regarding weather in combination with crop requirements.