1992Thresholding of Digital Images Using Two-dimensional Entropies
EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS
-
Upload
querida-carvalho -
Category
Documents
-
view
14 -
download
1
description
Transcript of EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS
![Page 1: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/1.jpg)
This research has been supported in part by European Commission FP6 IYTE-Wireless
Project (Contract No: 017442)
![Page 2: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/2.jpg)
EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS
EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS
Ferit Acar SAVACIIzmir Institute of Technology
Dept. of Electrical Electronics Engineering
Urla 35430, Izmir
Serkan GÜNELDokuz Eylül University
Dept. of Electrical Electronics Engineering
Buca, 35160, Izmir
![Page 3: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/3.jpg)
Contents Deterministic and indeterministic systems under influence of uncertainty...
Evolution of state probability densities Transformations on probability densities
Markov Operators & Frobenius—Perron Operators
Estimating state probability densities using kernel density estimators Parzen’s density estimator Density estimates for Logistic Map and Chua’s Circuit
The 2nd Law of Thermodynamics and Entropy Estimating Entropy of the system using kernel density estimations
Entropy Estimates for Logistic Map and Chua’s Circuit Entropy in terms of Frobenius—Perron Operators
Entropy and Control Maximum Entropy Principle Effects of external disturbance and observation on the system entropy Controller as a entropy changing device Equivalence of Maximum Entropy minimization to Optimal Control
![Page 4: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/4.jpg)
Motivation
Thermal noise effects all dynamical systems,
Exciting the systems by noise can alter the dynamics radically causing interesting behavior such as stochastic resonances,
Problems in chaos control with bifurcation parameter perturbations,
Possibility of designing noise immune control systems Densities arise whenever there is uncertainty in system
parameters, initial conditions etc. even if the systems under study are deterministic.
![Page 5: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/5.jpg)
Frobenius—Perron Operators
Definition
![Page 6: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/6.jpg)
Evolution of The State Densities of The Stochastic Dynamical Systems
• i’s are 1D Wiener Processes
Fokker—Planck—Kolmogorov Equ.
• p0(x) : Initial probability density of the states
![Page 7: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/7.jpg)
Infinitesimal Operator of Frobenius—Perron Operator
AFP : D(X)D(X)
D(X): Space of state probability densities
FPK equation in noiseless case
![Page 8: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/8.jpg)
Stationary Solutions of FPK Eq.
Reduced Fokker—Planck—Kolmogorov Equ.
![Page 9: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/9.jpg)
Frobenius—Perron Operator
x0
x1 xn-1
xnS
S(n-2)S
X
D(X)
f0
f1 fn-1
fn
n-2
![Page 10: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/10.jpg)
Calcutating FPO
S differentiable & invertible
![Page 11: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/11.jpg)
Logistic Map
α=4
![Page 12: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/12.jpg)
Estimating Densities from Observed Data Parzen’s Estimator
Observation vector : d i=1,...,n
} = 1
![Page 13: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/13.jpg)
Logistic Map — Parzen’s Estimation
![Page 14: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/14.jpg)
Logistic Map =4
![Page 15: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/15.jpg)
Chua’s Circuit
-E
E
![Page 16: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/16.jpg)
Chua’s Circuit — Dynamics
![Page 17: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/17.jpg)
Chua’s Circuit — The state densities
x
p(x)
Limit Cycles
Period-2 Cycles
Scrolls
Double Scroll
Details
![Page 18: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/18.jpg)
The 2nd Law of Thermodynamics & Information
0
T
QH
Q : Energy transfered to the systemT : Temprature (Average Kinetic Energy)
Classius Boltzman
Thermodynamics
Shannon
n: number of events pi: probability of event “i”
Information Theory
Entropy = Disorder of the system = Information gained by observing the systemEntropy = Disorder of the system = Information gained by observing the system
![Page 19: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/19.jpg)
Entropy
![Page 20: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/20.jpg)
Estimated Entropy – Logistic Map
![Page 21: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/21.jpg)
Estimated Entropy — Chua’s Circuit
![Page 22: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/22.jpg)
Estimated Entropy — Chua’s Circuit II
![Page 23: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/23.jpg)
Entropy in Control Systems I External Effects
x(t)p(x)
e(t)p(e)
If State transition transformation is measure preserving, then
Change in entropy :
Observer Entropy
x(t)p(x)
y(t)p(y)
![Page 24: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/24.jpg)
Entropy of Control Systems II Mutual Information
Theorem
![Page 25: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/25.jpg)
Uncertain v.s. Certain Controller Theorem
Theorem
![Page 26: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/26.jpg)
Principle of Maximum Entropy
Theorem
![Page 27: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/27.jpg)
Optimal Control with Uncertain Controller II
Select p(u) to maximize
subject to
![Page 28: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/28.jpg)
Optimal Control with Uncertain Controller III
![Page 29: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/29.jpg)
Optimal Control with Uncertain Controller IV
![Page 30: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/30.jpg)
Optimal Control with Uncertain Controller V
Theorem
![Page 31: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/31.jpg)
Summary I The state densities of nonlinear dynamical systems can be estimated
using kernel density estimators using the observed data which can be used to determine the evolution of the entropy.
Important observation : Topologically more complex the dynamics results in higher stationary entropy
The evolution of uncertainty is a trackable problem in terms of Fokker—Planck—Kolmogorov formalism.
The dynamics in the state space are converted to an infinite dimensional system given by a linear parabolic partial diff. equation (The FPK Equation),
The solution of the FPK can be reduced to finding solution of a set of nonlinear algebraic equations by means of weighted residual schemes,
The worst case entropy can be used as a performance criteria to be minimized(maximized) in order to force the system to a topologically simpler dynamics.
![Page 32: EVOLUTION OF THE STATE DENSITIES AND THE ENTROPIES OF DYNAMICAL SYSTEMS](https://reader030.fdocuments.in/reader030/viewer/2022032606/56812dd9550346895d93250e/html5/thumbnails/32.jpg)
Summary II
The (possibly stochastic) controller performance is determined by the information gather by the controller about the actual system state.
A controller that reduces the entropy of a dynamical system must increase its entropy at least by the reduction to be achieved.