Alessandro Fabris on behalf of the Elettra RF Group Sincrotrone Trieste, Trieste, Italy.
SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas...
-
Upload
gerard-lynch -
Category
Documents
-
view
216 -
download
4
Transcript of SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas...
![Page 1: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/1.jpg)
SWAN 2006, ARRI_
Control ofDistributed-Information
Nonlinear Stochastic Systems
Prof. Thomas Parisini
University of Trieste
![Page 2: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/2.jpg)
SWAN 2006, ARRI
Summary
Examples
Problem Formulation
Approximate NN Solution
Two Significant Applications
![Page 3: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/3.jpg)
SWAN 2006, ARRI
Example:Water Distribution Networks
Objective: control the spatio-temporal distribution of drinking water disinfectant throughout the network by the injection of appropriate amount of disinfectant at appropriately chosen actuator locations
![Page 4: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/4.jpg)
SWAN 2006, ARRI
Example:Robot Soccer
Objective: By the year 2050, develop a team of fully autonomous humanoid robots that can win against the human world soccer champions.
Robot soccer incorporates various technologies including: design principles of autonomous agents, multi-agent cooperation, strategy acquisition, real-time reasoning, robotics, and sensor-fusion.
![Page 5: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/5.jpg)
SWAN 2006, ARRI
Typical Applications
Traffic control systems
Coordination of teams of UAVs
Geographically distributed systems
Dynamic routing in communications networks
Large-scale process-control systems
Coordination of teams of UAVs
etc.
![Page 6: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/6.jpg)
SWAN 2006, ARRI
Distributed Decision and Control
How did this happen?
Outer-loop control: as we move from low-level control to higher-level control, we face the need to take into consideration other feedback systems that influence the environment.
Complexity: due to the presence of more complex systems, we had to break down the controller into smaller controllers.
Communications and Networks: data networks and wireless communications have facilitated the design of feedback systems that require distributed decision and control techniques.
![Page 7: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/7.jpg)
SWAN 2006, ARRI
Distributed Decision and Control
What does it imply?
Need for different type of control problem formulations.Need to handle competition from other controllers (agents).
Need to handle cooperation with other controllersNeed to handle inter-controller communication issues.Need for suitable individual and team evaluation methods
![Page 8: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/8.jpg)
SWAN 2006, ARRI
Distributed Decision and Control
How does Learning come in?
Learning the environment
Learning the strategy of adversarial agents
Learning is crucial because it is a highly time-varying (evolving) environment.
Predicting the actions of collaborating agents
![Page 9: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/9.jpg)
SWAN 2006, ARRI
Structure of a “Team” of Agents
![Page 10: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/10.jpg)
SWAN 2006, ARRI
Problem formulation
Definitions
Unpredictable variables:
Random vector representing all uncertainties in the external world with known p.d.f.
![Page 11: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/11.jpg)
SWAN 2006, ARRI
Problem formulation
Definitions
Information function:
![Page 12: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/12.jpg)
SWAN 2006, ARRI
Problem formulation
Definitions
Decision function:
![Page 13: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/13.jpg)
SWAN 2006, ARRI
Problem formulation
Definitions
Cost functional:
![Page 14: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/14.jpg)
SWAN 2006, ARRI
Problem formulation
Problem T
Given
and
Find the optimal
strategies
that minimize
Solving analytically Problem T is in general impossible
![Page 15: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/15.jpg)
SWAN 2006, ARRI
About Problem T
Further Definitions and Concepts
Predecessor:
The control actions generated by affect the information set of for any possible
![Page 16: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/16.jpg)
SWAN 2006, ARRI
About Problem T
Further Definitions and Concepts
Information set inclusion:
The information set is a function of the information set ( is “nested” in )
![Page 17: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/17.jpg)
SWAN 2006, ARRI
About Problem T
Further Definitions and Concepts
Information network:
![Page 18: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/18.jpg)
SWAN 2006, ARRI
About Problem T
Further Definitions and Concepts
Information Structures
Static: information of each is not influenced by decisions of other
Dynamic: otherwise
![Page 19: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/19.jpg)
SWAN 2006, ARRI
About Problem T
Sufficient conditions for analytic solution
LQG Static Teams
Linear optimal control strategy
![Page 20: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/20.jpg)
SWAN 2006, ARRI
About Problem T
Sufficient conditions for analytic solution
“Partially Nested” LQG Dynamic Teams
Any can reconstruct the information of the influencing its own information
Linear optimal control strategy
![Page 21: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/21.jpg)
SWAN 2006, ARRI
About Problem T
Sufficient conditions for analytic solution
Existence of a sequential partition
Optimal control strategy by DP
![Page 22: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/22.jpg)
SWAN 2006, ARRI
Approximate Solution of Problem T
A Simple Methodology
Assumption: no loops in the information network
Given parametric structure
Vector of “free” parameters to be
optimized
![Page 23: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/23.jpg)
SWAN 2006, ARRI
Formulation of the Approximate Problem
Problem T’
Substitute and into the cost function
![Page 24: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/24.jpg)
SWAN 2006, ARRI
Formulation of the Approximate Problem
Problem T’
Given
and
Find the optimal vector
that minimizesGiven NN structures
Functional Optimization Problem T
Nonlinear Programming Problem T’
![Page 25: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/25.jpg)
SWAN 2006, ARRI
NN Learning Algorithm
Gradient Method
cannot be written in explicit form
However:
![Page 26: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/26.jpg)
SWAN 2006, ARRI
NN Learning Algorithm
Stochastic Approximation
Compute the “realization”
![Page 27: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/27.jpg)
SWAN 2006, ARRI
NN Learning Algorithm
Stochastic Approximation
randomly generated
according to the (known) p.d.f. of
The step is chosen so as
Example:
![Page 28: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/28.jpg)
SWAN 2006, ARRI
NN Learning Algorithm
Important Remark
Gradient method + Stochastic Approximation
Distributed Learning: each DM is able to compute “locally” its control function by “exchanging messages” with cooperating DMs according to the Information Structure
![Page 29: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/29.jpg)
SWAN 2006, ARRI
Methodology: Conceptual Steps
Problem T: minimize Exact optimal solutions
Replace with the NN structure
Problem T’: minimize
Stoc. Appr. to solve Problem T’
Approximate optimal solutions
![Page 30: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/30.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Problem W
Given:
and independentFind the optimal
strategies
that minimize information functions:
cost function:
![Page 31: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/31.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Problem W
![Page 32: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/32.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Remarks on Problem W
LQG hypotheses hold Information structure not partially nested
An optimal solution does exist
But:
are not affine functions of
![Page 33: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/33.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Remarks on Problem W
Best affine solution:
Wit. solution:
For and
![Page 34: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/34.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Remarks on Problem W
Optimized Wit. solution:
For and
given the structures
![Page 35: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/35.jpg)
SWAN 2006, ARRI
The Witsenhausen Counterexample
Remarks on Problem W
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40
5
10
15
20
25
30
35
40
45
Opt. Wit. solution outperforms the best
linear solutions:
![Page 36: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/36.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
Given parametric structures
Vector of “free” parameters to be
optimized
Choice of the parametric structures
![Page 37: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/37.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
Problem W’
Substitute into the cost functional
![Page 38: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/38.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
Problem W’
Given
and independentFind the
optimal NN weights
that minimize
information functions:
cost function:
![Page 39: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/39.jpg)
SWAN 2006, ARRI
Conceptual Steps to Solve Approximately Problem W
Problem W: minimize Exact optimal solutions
Replace with the NNs
Problem W’: minimize
Stoc. Appr. to solve Problem W’
Approximate optimal solutions
![Page 40: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/40.jpg)
SWAN 2006, ARRI
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Approximate NN Solution of Problem W
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Results and Comparisons
Best Linear
NN
Opt.
W.
Best Linear
NN
Opt.
W.
![Page 41: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/41.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Results and Comparisons
Best Linear
NN
Opt.
W.
Best Linear
NN
Opt.
W.
![Page 42: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/42.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Results and Comparisons
Best Linear
NN
Opt.
W.
Best Linear
NN
Opt.
W.
![Page 43: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/43.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Results and Comparisons
Best Linear
NN
Opt.
W.
Best Linear
NN
Opt.
W.
![Page 44: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/44.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
-15 10 -5 0 5 10 15
-15
10
-5
0
5
10
15
Results and Comparisons
Best Linear
NN Opt.
W.
Best Linear
NN Opt.
W.
![Page 45: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/45.jpg)
SWAN 2006, ARRI
3- 3- 5- 5- 5- 5- L+ L+
3- 3- 5- 5- 5- 5- L+ L+
3- 3- 5- 5- 5- 5- L+ L+
3- 3- 5- 5- 5- 5- 5+ L+
3- 3- 3- 5- 5- 5- 5- 5+ L+
3- 3- 3- 5- 5- 5- 5+ L+
3- 3- 3- 5- 5- 5- L+ L+3+ 3- 3- 3- 3- 5- L+ L+
3+ 3- 3- 3- 3- 5- L+ L+3+ 3- 3- 3- 3- 5- L+ L+ L+
3+ 3- 3- 3- 5- L+ L+3+ 3- 3- 3- 3- L+ L+
0 0.1 0.2 0.3 0.4 0.5 0.6 0.70
5
10
15
20
25
30
35
40
45
Approximate NN Solution of Problem W
Results and Comparisons
The “3-step” area The “linear” area The “5-step” area
![Page 46: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/46.jpg)
SWAN 2006, ARRI
Approximate NN Solution of Problem W
Results and Comparisons
Costs of the Neural, Optimized Witsenhausen, and Best Linear Solutions
![Page 47: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/47.jpg)
SWAN 2006, ARRI
Concluding Remarks
Decision makers act as cooperating members of a team
General approximate methodology for the solution of distributed-information control problems:
Team functional optimization problem reduced to a nonlinear programming one
Distributed learning scheme: each DM can compute (or adapt) its “personal” control function “locally” Straightforward extension to the infinite-horizon case (receding-horizon paradigm)
![Page 48: SWAN 2006, ARRI_ Control of Distributed-Information Nonlinear Stochastic Systems Prof. Thomas Parisini University of Trieste.](https://reader035.fdocuments.in/reader035/viewer/2022070410/56649ea45503460f94ba8943/html5/thumbnails/48.jpg)
SWAN 2006, ARRI
Acknowledgments
Riccardo Zoppoli, Marios Polycarpou, Marco Baglietto, Angelo Alessandri, Alessandro Astolfi, Daniele Casagrande, Riccardo Ferrari, Elisa Franco, Frank Lewis, R. Selmic, Jason Speyer, Marcel Staroswiecki, Jakob Stoustrup, …