Information Theory for Control Systems: Causality and … Information Theory for Causal Systems...
Transcript of Information Theory for Control Systems: Causality and … Information Theory for Causal Systems...
![Page 1: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/1.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Information Theory for Control Systems: Causalityand Feedback
Charalambos D. Charalambous
Department of Electrical and Computer EngineeringUniversity of Cyprus
E-mail: [email protected]
Workshop on Communication Networks and ComplexityAthens, Greece, 2006
![Page 2: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/2.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Outline
1 Introduction
2 Information Theory for Causal SystemsDefinition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
3 Control of Dynamic Systems over Finite Capacity ChannelsAsymptotic Stability and Observability
4 Example: Necessary and Sufficient ConditionsAsymptotic Stability and Observability
5 Conclusions
6 References
![Page 3: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/3.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 4: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/4.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Introduction I
Control/CommunicationSystem
Information Theory forCausal Control Systems
Channel Capacity withFeedback (Causal)Rate Distortion (Causal)
Feedback Control
ObservabilityStability
Plant(Information
Source)Encoder
Discrete Time Channel
Decoder
tY~Controller
tY
tZ
tU
tZ~
![Page 5: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/5.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Introduction II
Objective
Design Encoders, Decoders, Controllers to Achieve Control andCommunication Objectives
Trade Offs
What are the Trade Offs Between Control and CommunicationDesign Objectives?
Separation Principle
Does Separation Hold Between Communication and ControlSystem Design?
![Page 6: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/6.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Introduction III
Structure of Presentation
Information Theory for Causal Systems
Control of Stochastic Systems over Limited Capacity Channels
Partially Observed Linear Stochastic Control Systems
![Page 7: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/7.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 8: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/8.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Definition of Subsystems I
Information Source
The information source is identified by the joint probability
distribution P(dY T ), Y T 4= (Y0, . . . ,Y
T−1)
Communication Channel
The communication channel is modeled by a feedback channelwith memory via the family of stochastic kernels{P(dZt ; z
t , z t−1)}Tt=0, t,T ∈ N+, where Z t = z t is the specificrealization of the channel input, and Z t−1 = z t−1 is a specificrealization of the previous channel outputs.
![Page 9: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/9.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Definition of Subsystems II
Causality Y T C→ ZN .
Given any two sequences Y T and ZN ,T ,N ∈ N+ we shall say thatthe stochastic kernel connecting Y T to ZN is causal if and only ifP(dZt ; y
n, z t−1) = P(dZt ; yt , z t−1),∀n > t, n, t ∈ N+.
![Page 10: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/10.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Definition of Subsystems III
Causal Channel, Feedback, Compression
ZT C→ ZT is equivalent toP(dZt ; z
n, z t−1) = P(dZt ; zt , z t−1), ∀n > t, where
t, n ∈ N+, which means the communication channel isnon-anticipative or causal.
ZT C← ZT is equivalent to P(dZt ; zn, z t−1) = P(Zt ; z
t , z t−1),∀n > t, where t, n ∈ N+, which means that thecommunication channel is used with non-anticipative or causalfeedback.
Y T C→ Y T is equivalent toP(dYt ; y
t−1, yn) = P(dYt ; yt−1, y t), ∀n > t, where
t, n ∈ N+, which means that the source reproductionstochastic kernel is non-anticipative or causal.
![Page 11: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/11.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Restricted Mutual Information for Causal Systems I
Shannon’s Self-Mutual Information
Given a communication channel with input ZN−1 and outputZN−1, the self-mutual information is defined by
i(ZN−1; ZN−1)4= log
P(dZN−1;ZN−1)
P(dZN−1)
= logP(dZN−1, dZN−1)
P(dZN−1)× P(dZN−1),
P(dZN−1;ZN−1)
P(dZN−1)is the RND between the stochastic kernel
P(dZN−1;ZN−1) and distribution P(dZn−1).
Symmetry: i(ZN−1; ZN−1) = i(ZN−1;ZN−1)
![Page 12: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/12.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Restricted Mutual Information for Causal Systems II
Mutual Information
I (ZN−1; ZN−1) = EP(dZN−1,dZN−1)i(ZN−1; ZN−1)
= I (ZN−1;ZN−1).
The Shannon information capacity for the time horizon N is
CN4= sup
P(dZN−1)∈DN−1CI
I (ZN−1; ZN−1)
![Page 13: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/13.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Restricted Mutual Information for Causal Systems III
Restricted Self-Mutual Information
Given a channel with input ZN−1 and output ZN−1, the restrictedself-mutual information is defined by
iR(ZN−1; ZN−1)4= log
P(dZN−1;ZN−1)
P(dZN−1)|R
= logP(dZN−1, dZN−1)
P(dZN−1)× P(dZN−1)|R
which denotes the logarithm of the restricted RND associated with
ZN C→ ZN .
![Page 14: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/14.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Restricted Mutual Information for Causal Systems IV
The Restricted Mutual Information
I (ZN−1; ZN−1)|R4= EP(dZN−1,dZN−1)iR(ZN−1; ZN−1)
=N−1∑i=0
I (Z i ; Zi |Z i−1) =N−1∑i=0
logP(dZi ;Z
i , Z i−1)
P(dZi ; Z i−1)× P(dZ i , dZ i )
Symmetry Fails: I (ZN−1; ZN−1)|R 6= I (ZN−1;ZN−1)|RTighter Bound: I (ZN−1; ZN−1)|R ≤ I (ZN−1; ZN−1)
![Page 15: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/15.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Restricted Mutual Information for Causal Systems V
Relation to Previous Work: Directed Information
H. Marko, 1973 (IEEE Communication Systems)
J. Massey, 1990 (ISIT)
G. Kramer, 1998 (Ph.D. Thesis)
S. Tatikonda, 2000 (Ph.D. Thesis)
![Page 16: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/16.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Causal Rate Distortion I
Shannon’s Rate Distortion
Let Y T−1 and Y T−1 denote the source and reproduction outputs,respectively, and denote the distortion measure
DTDC
4=
{P(dY T−1; yT−1);EρT (Y T−1, Y T−1) ≤ Dv
}The information rate distortion function for the time horizon T isdefined by
RDT (Dv )
4= inf
P(dY T−1;yT−1)∈DTDC
I (Y T−1; Y T−1)
![Page 17: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/17.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Causal Rate Distortion II
The Minimizing Kernel is Non-Causal
The infimizing reproduction stochastic Kernel can be factoredas P(dY T−1; yT−1) =
∏T−1i=0 P(dYt ; y
t−1, yT−1), which is anon-causal operation on the source data.
![Page 18: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/18.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Causal Rate Distortion III
Rate Distortion for Causal Systems
Let Y T−1 and Y T−1 denote the source output and thereproduction of the source output, respectively, and
DTDC
4=
{P(dY T−1; yT−1);
T−1∑t=0
Eρt(Yt , Y t) ≤ Dv
}The restricted information rate distortion function is defined by
RDT (Dv )|R
4= inf
P(dY T−1;yT−1)∈DTDC
I (Y T−1; Y T−1)|R
![Page 19: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/19.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Causal Rate Distortion IV
The Minimizing Kernel is Causal
The infimizing reproduction stochastic Kernel is given by
P∗(dYt ; yt , y t−1) =
esρt(y t ,y t−1,Yt)P∗(dYt ; yt−1)∫
Ytesρt(y t ,y t−1,Yt)P∗(dYt ; y t−1)
![Page 20: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/20.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Data Processing Inequalities I
Data Processing Inequalities for Causal Channels
Suppose Y T−1 → ZN−1 → ZN−1 → Y T−1 → UT−1 form aMarkov chain.For Zn C→ Zn,Y t C→ Y t , t, n ∈ N+ then
I (Zn; Zn) ≥ I (Zn; Zn)|R ≥ I (Y t ; Zn) ≥ I (Y t ; Y t) ≥ I (Y t ; Y t)|R
![Page 21: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/21.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Definition of Control/Communication SubsystemsMutual Information for Causal SystemsCausal Rate DistortionData Processing InequalitiesInformation Transmission Theorem
Information Transmission Theorem I
Information Transmission Theorem
Suppose Y T−1 → ZN−1 → ZN−1 → Y T−1 → UT−1 form aMarkov chain.For Zn C→ Zn,Y t C→ Y t , t, n ∈ N+
Then
CN |R ≥ RDT (Dv )|R
![Page 22: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/22.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 23: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/23.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability I
Asymptotic Observability
Consider the general control/communication system.The system is asymptotically observable in r -mean if there exist acontrol sequence and an encoder and a decoder such thatlimT→∞
1T
∑T−1t=0 Eρ(Yt , Yt) ≤ Dv , where ρ(Y , Y ) = ||Y − Y ||,
r > 0 and Dv ≥ 0 is finite.
![Page 24: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/24.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability II
Asymptotic Stabilizability
Consider the general control/communication system in which,Yt = Ht + Υt , where Υt , t ∈ N+ is a function of the measurementnoise.This system is asymptotically stabilizable in r -mean if there exist acontroller, encoder, and decoder such thatlimT→∞
1T
∑T−1t=0 Eρ(Ht , 0) ≤ Dv , where ρ(H, 0) = ||H − 0||r ,
r > 0 and Dv ≥ 0 is finite.
![Page 25: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/25.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability III
Necessary Conditions for Asymptotic Observability andStabilizability
Suppose Y T−1 → ZN−1 → ZN−1 → Y T−1 → UT−1 form aMarkov chain.
For Zn C→ Zn,Y t C→ Y t , t, n ∈ N+ theNecessary Conditions are
C|R ≥ RD(Dv )|R ≥ HS(Y)− log edr + log(
r
dVdΓ(dr )
(d
rDv)
dr )
where HS(Y) is Shannon Entropy Rate of Output Process Y T .
![Page 26: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/26.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 27: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/27.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability I
Stochastic Control System
Xt+1 = AXt + NUt + BWt , X0 = X ,
Yt = Ht + DVt , Ht = CXt
AGN Communication Channel
Zt = Zt + Wt , 0 ≤ t ≤ T
where with power constraint 1T
∑T−1i=0 E ||Zt ||2 ≤ P.
![Page 28: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/28.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability II
Channel Does Not Assume Feedback
The encoder is an innovations encoder without channel feedback
Zt = Yt − E [Yt |Y t−1,Ut−1]
Necessary and Sufficient Condition for Asymptotic Observability
C|R ≥ RD(Dv )|R =1
2log Λ∞ −
1
2log Dv
where Λ∞ is the steady state value of the innovations errorcovariance
![Page 29: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/29.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability III
The Sufficiency is obtained by matching the source to thechannel
HS(Y) ≥ 12 log(2πeD2) + max{0, log |A|}
This encoder only works for stable control systems (thedecoder cannot stabilize the control system
Next, we present one that works!
![Page 30: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/30.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Asymptotic Stability and Observability
Asymptotic Stability and Observability IV
AGN Communication Channel With Feedback
Encoder (uses channel feedback)
Zt = A0(Zt−1,Ut−1) + A1(Z
t−1,Ut−1)Yt
Decoder (Least-Squares Decoder)
Yt|t−1 = E(Yt |Z t−1,Ut−1
)= CE
(Xt |Z t−1,Ut−1
)Controller (LQG)
Ut = KtYt|t−1
![Page 31: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/31.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 32: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/32.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Conclusions I
Causality of Rate Distortion
Separation Principle Holds for Gaussian Control andCommunication Channels
Uncertain Control Systems and Channels (done some work)
Nonlinear Stochastic Control Systems (future work)
![Page 33: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/33.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
Outline
1 Introduction
2 Information Theory for Causal Systems
3 Control of Dynamic Systems over Finite Capacity Channels
4 Example: Necessary and Sufficient Conditions
5 Conclusions
6 References
![Page 34: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/34.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
References I
C. E. Shannon, The Mathematical Theory of Communication,Bell Systems Technical Journal, vol. 27, pp. 379–423,pp. 623–656, 1948.
R. L. Dubrushin, Information Transmission in Channel withFeedback, Theory of Probability and its Applications, vol. III,No. 4, pp. 367–383, 1958.
H. Marko, The Bidirectional Communication Theory-AGeneralization of Information Theory, IEEE Transactions onCommunication Systems, vol. 21, pp. 1345–1351, 1973.
![Page 35: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/35.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
References II
J. Massey, “Causality, Feedback and Directed Information”, inthe in Proceedings of the 1990 IEEE International Symposiumon Information Theory and its Applications, pp. 303–305,Nov.27–30, Hawaii, U.S.A.1990.
S. Tatikonda, Control Under Communication Constraint,Ph.D. Thesis, Department of Electrical Enginnering andComputer Science, MIT., September 2000.
G. Kramer, Directed Information for Channels with Feedback,Ph.D. Thesis, Swiss Federal Institute of Technology, Diss. ETHNo. 12656, 1998.
![Page 36: Information Theory for Control Systems: Causality and … Information Theory for Causal Systems Control of Dynamic Systems over Finite Capacity Channels Example: Necessary and Sufficient](https://reader034.fdocuments.in/reader034/viewer/2022051601/5ae178cc7f8b9a595d8bf7e9/html5/thumbnails/36.jpg)
IntroductionInformation Theory for Causal Systems
Control of Dynamic Systems over Finite Capacity ChannelsExample: Necessary and Sufficient Conditions
ConclusionsReferences
References III
S. Tatikonda and S. Mitter, Control Over Noisy Channels,IEEE Transactions on Automatic Control, vol. 49, No. 7,pp. 1196–1201, 2004.
S. Tatikonda and S. Mitter, Control Under CommunicationConstraints, IEEE Transactions on Automatic Control, vol. 49,No. 7, pp. 1056–1068, 2004.
S. Yang and A. Kavcic and S. Tatikonda, Feedback Capacityof Finite-State Machine Channels, IEEE Transactions onInformation Theory, vol. 51, No. 3, pp. 799–810, 2005.