AdaptiveLab Talk1
-
Upload
noha-elprince -
Category
Documents
-
view
250 -
download
0
Transcript of AdaptiveLab Talk1
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
A Model-Based Feedback-Control Approach toBehaviour Modification ThroughReward-Induced Attitude Change
J.Ni, D. Kulic, and D. Davison
presented by: Noha El-Prince
April 16, 2013
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
1 Outline
2 Problem Definition
3 System ModelOverall ModelTheory of Planned BehaviorCognitive DissonanceTheory of Overjustification
4 Controller DesignAssumptions and Initial ConditionsController Design: Stage1Controller Design: Stage2
5 Simulation Results
6 Conclusion
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Problem Definition
Trying to change the behavior of a person to a desiredbehavior.
The person may have either a negative/positive attitudetowards the desired behavior.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Methodology
Model the internal cognitive psychological state of aperson.Design a controller based on the cognitive model.Goal: Tracking desired behavior via a sequence ofrewards.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overall System Model
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Theory of Planned Behavior
Aout[k] = Aout[k − 1] + ∆Aout[k − 1], (1)
∆Aout[k] = ∆ACDout [k] + ∆AOJ
out[k], (2)
Arew[k] = r1Arew[k − 1] + µ1(1− r1)R[k − 1], (3)
BI[k] = Aout[k] +Arew[k], (4)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Theory of Planned Behavior
B[k] =
Bd[k] if BI[k] ≥ Bd[k] and Aout[k] ≤ Bd[k]
Aout[k] if (BI[k] < Bd[k] and Aout[k] ≥ 0)
or Aout[k] > Bd[k]
0 otherwise.(5)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying Dissonance Pressure
Dissonance = ”%” of inconsistent cognitive pairs
PCDraw [k] =
Bsgn[k] Mincon[k]
Mincon[k]+Mcon[k]if Mincon[k] +Mcon[k] > 0
0 otherwise.
(6)
Bsgn[k] =
{+1 if B[k] ≥ Bd[k] or Aout[k] ≥ 0−1 otherwise.
(7)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying Dissonance Pressure - cont.
M1incon[k] =
{|Arew[k]| if sgn(Arew [k]) 6= Brel[k]
0 otherwise,(8)
M2incon[k] =
{|Aout[k]| if sgn(Aout[k]) 6= Bsgn[k]
0 otherwise,(9)
M1con[k] =
{|Arew[k]| if sgn(Arew [k]) = Brel[k]
0 otherwise,(10)
M2con[k] =
{|Aout[k]| if sgn(Aout[k]) = Bsgn[k]
0 otherwise,(11)
Mincon[k] =2∑
i=1
Miincon[k], Mcon[k] =
2∑i=1
Micon[k], (12)
Brel[k] =
{+1 if B[k] ≥ Bd[k]−1 otherwise.
(13)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Special Case: Attitude Reversal
Aout[k] small, R[k] small, Bd[k] is high ⇒ Child declinesthe reward
To reduce Diss. pressure: increase Aout OR “give up”jogging ⇒ Aout[k] <<<
r[k] =
+1 if Bd[k]−BI[k] > αrevAout[k], Aout[k] ≥ 0,
K1PCD[k] > 2Aout[k], and Arew[k] > 0,
−1 otherwise.
(14)
PCD[k] =
{(1− r2)PCD
raw [k] if r[k − 1] = 1
r2PCD[k − 1] + (1− r2)PCD
raw [k] otherwise.(15)
PrawCD
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying ∆Aout
Assume the change in Aout[k] is proportional to dissonancepressure, with proportionality constant K1 > 0:
∆ACDout [k] =
{−K1P
CD[k] if r[k] = 1
+K1PCD[k] otherwise.
(16)
PCD
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory (Block B)
Overjustification Theory
when a reward is given to a person to do something thatshe/he already enjoys doing, such rewards arecounter-productive in that they reduce the intrinsic desire ofthe person towards that behavior.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory - cont.
Let Bt[k] = minimal attitude level to which theoverjustification effect can drive Aout[k].Assume Bt[k] is a constant fraction of Bd[k], i.e.,
Bt[k] = αBd·Bd[k], (17)
for some constant 0 < αBd< 1.
If Bt[k] > Aout[k] ⇒ overjustification pressure does notdecrease Aout, and the reverse is true i.e.
Arelout[k] = max{0, Aout[k]−Bt[k]}. (18)
where Arelout[k]: a relative attitude with respect to Bt[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory - cont.
Then the raw and filtered overjustification pressures, and the resulting change in intrinsic attitude, arecomputed just as in our previous work, but using Arel
out instead of Aout, as follows:
POJraw [k] =
Arelout[k]Arew[k] if Arel
out[k] > 0 and Arew[k] > 0and B[k] ≥ Bd[k]
0 otherwise,
(19)
POJ
[k] = r3POJ
[k − 1] + (1− r3)POJraw[k], (20)
∆AOJout[k] =
{−K2P
OJ [k] if K2POJ [k] ≤ Arel
out[k]
−Arelout[k] otherwise.
(21)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage1
BI[k + 1] ≥ Bd[k + 1].
⇓R[k] >>> enough to force B[k + 1] > 0. >>
⇓Bsgn[k + 1] = +1.
⇓PCDraw [k + 1] > 0. >>
⇓Goal: increase Aout from −ve to +ve.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage1- cont.
BI[k] = Aout[k] +Arew[k]
= A0 + µ1R[k] ≥ Bd[k + 1]
R[k] =Bd[k + 1] + |A0|
µ1(22)
The associated dissonance pressure is:
PCDraw [k + 1] =
Bsgn[k + 1] · |Aout[k + 1]
|Aout[k + 1]|+Arew[k + 1]=
|A0||A0|+ µ1R[k]
.
(23)
Maximizing (23) subject to (22) results in Bd[k + 1] = 0 andR[k] = |A0|/µ1. For improved robustness:
Bd[k + 1] = 2ε (24)
R[k] =2Bd[k + 1] + |Aout[k]|
µ1=
2ε+ |Aout[k]|µ1
(25)
(26)Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .
Avoid exciting the OVJ dynamics that makes Aout ⇓ .Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .Avoid exciting the OVJ dynamics that makes Aout ⇓ .
Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .Avoid exciting the OVJ dynamics that makes Aout ⇓ .Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
To enforce the child to reject the reward R[k] force:
BI[k + 1] < Bd[k + 1]
Aout[k] +Arew[k] < Bd[k + 1]
Aout[k] + r1Arew[k − 1] + µ1(1− r1)R[k − 1] < Bd[k + 1]
R[k] <Bd[k + 1]− r1Arew[k]−Aout[k]
µ1(1− r1)
R[k] <Bd[k + 1]−Aout[k]
µ1(27)
Equation(27) gurantees child reject reward and OJ = 0.
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
Attitude reversal is avoided on day k+1 if R[k] is chosen s.t :
Bd[k]−BI[k] ≤ αrevAout[k], Aout[k] ≥ 0
Bd[k] +Aout[k]−Arew[k] ≤ αrevAout[k]
Aout[k] + r1Arew[k − 1] + µ1(1− r1)R[k − 1] ≤ Bd[k + 1]
R[k] ≥ Bd[k + 1]− (αrev + 1)Aout[k]
µ1(28)
Equation(28) gurantees avoidance of attitude reversal.Q. How to keep R[k] at a reasonable level ?
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
By introducing a controller tuning parameter β ∈ (0, 1), theaggressiveness of attitude increase can be adjusted:
Ad = βAout[k] + (1− β)(Aout[k] +K1(1− rN−12 )).
R[k] =Aout[k]
µ1
(K1(1− rN−1
2 )
Aout[k] +K1(1− rN−12 )−Ad
− 1
). (29)
To avoid driving the attitude higher than needed (i.e., beyondB∗
d), we add a saturator as follows:
Ad = min{B∗d , βAout[k] + (1− β)(Aout[k] +K1(1− rN−1
2 ))}.(30)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
Get the value of Bd[k + 1] from the formulas of R[k] :
Bdmin[k] = Aout[k](K1(1− r2)N−1
Aout[k] +K1(1− r2)N−1 −Ad(31)
Bdmax[k] = Aout[k](K1(1− r2)N−1
Aout[k] +K1(1− r2)N−1 −Ad+ αrev
(32)
Bdmin[k] < Bd[k + 1] ≤ Bdmax[k]. (33)
Bd[k + 1] = γBdmin[k] + (1− γ)Bdmax[k]. (34)
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES NO
NO
NO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES
NONO
NO
NO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES
NO NO NO NO NO NONO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.
Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.
In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.
Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).
Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.
Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series