AnActiveLearningApproachForInferringDiscreteEventAutomataMohammadMahdiKarimiPhD.Candidate,ECESupervisor: Dr AliKarimoddiniSummer2015 1

Content1. DiscreteEventSystems• Definitions• Applications

2. AutomataTheory• Language• RegularLanguage• AutomataRepresentation• Modeling inAutomata

3. Lstar Learning• Definitions• Algorithm• UAVExample

4. Lstar Features5. Lstar Matlab Toolbox6. Conclusion 2

SECTION1:DISCRETEEVENTSYSTEMS

DefinitionsandApplications

3

WhatisDES?

4

Discrete Event Systems:Definition: A Discrete Event System (DES) is a discrete-state,event-driven system, that is, its state evolution dependsentirely on the occurrence of asynchronous discrete eventsover time.

Startmission

ReturntoHangar

L,R SMissionMode

HangarMode

EmergencyMode

L,S,R

Lowfuel

Motivation,WhyDES?

• To treat large-scale systems we need an abstract andsimple tool:• DES can effectively model large scale systems in anabstract mathematical fashion.

• DES can capture system’s logics and rules and used forconstructing the decision making units of systems.

• Many systems are inherently DES (Ex. Queuing system)• Many others can be abstracted to DES (Ex. Robotic)

5

SomeDESExamples• UnmannedArialVehicles

• Roboticapplications

6

-Karimoddini,Ali,etal."Hybridformationcontroloftheunmannedaerialvehicles."Mechatronics 21.5(2011):886-898.-Tzionas,Panagiotis G.,Adonios Thanailakis,andPhilipposG.Tsalides."Collision-freepathplanningforadiamond-shapedrobotusingtwo-dimensionalcellularautomata."RoboticsandAutomation,IEEETransactionson13.2(1997):237-250.

SomeDESApplications• Power-trainSpecification

• InternetOfThings(IOT):

7Balluchi,Andrea,etal."Automotiveenginecontrolandhybridsystems:Challengesandopportunities." ProceedingsoftheIEEE 88.7 (2000):888-912.Lohstroh,Marten,Lee,EdwardA.,“AnInterfaceTheory fortheInternetofThings.”13thInternationalConferenceonSoftwareEngineeringandFormalMethods (SEFM),2015.

AUTOMATATHEORYLanguageandAutomata

8

1. DiscreteEventSystemsDefinitionsApplications

2. AutomataTheoryLanguageRegularLanguageAutomataRepresentationModelinginAutomata

3. Lstar LearningDefinitionsAlgorithmUAVExample

4. Lstar Features5. Lstar Matlab Toolbox6. Conclusion

WhatisaLanguage?• BehaviorofadeterministicDEScanbeequivalentlydescribedbythesetofallpossiblesequencesofevents:

σ1σ2…• andtheinitialstateX0.Eachsucheventsequenceiscalledatrace orstring ofthesystem,andacollectionoftracesstartingatX0 andendingatfinalstatesiscalleda(marked)language.

L={L,R,SR,SLR,…}

9

Kumar,Ratnesh,andVijayK.Garg.Modelingandcontroloflogicaldiscreteeventsystems.Vol.300.SpringerScience&BusinessMedia,2012.

Problem:Todescribeasystemwithitslanguage,wemaycomeupwithasetofinfinitestrings.

RegularLanguages• Regularlanguageisaclassoflanguage.Inregularlanguages,alanguageisaninfiniteset,butitcanberepresentedusingfinitestatemachines(Automata).

• Theorem:• Givenaregularlanguagemodel(Km,K),thereexistsaDFSM,G=(X,∑,α,x0,Xm),suchthat(Lm(G),L(G))=(Km,K).

• ItisprovedthatforanygivenregularlanguagethereisaDFSM,

• Question isnowhowtorepresentmodelinmoregraphicalway? 10

Kumar,Ratnesh,andVijayK.Garg.Modelingandcontroloflogicaldiscreteeventsystems.Vol.300.SpringerScience&BusinessMedia,2012.

AutomataRepresentation• Adeterministicfiniteautomatonisrepresentedformallybya5-tuple(Q,Σ,δ,q0,F)• Qisafinitesetofstates.• Σisafinitesetofsymbols,calledthealphabet (A)oftheautomaton.

• δisthetransitionfunction,thatis,δ: Q × Σ → Q.• 𝐫𝐨𝐰 𝒔𝒂 = δ(row(s),a)

• q0 isthestart state,thatis,thestateoftheautomatonbeforeanyinputhasbeenprocessed,whereq0∈Q.

• FisasetofstatesofQ(i.e.F⊆Q)calledaccepted statesormarkedstates.• F={𝒓𝒐𝒘(𝒔)|𝒔 ∈ 𝑺 ∧ 𝑻 𝒔 = 𝟏}

11

Problem:HowtomodelasystembyanAutomaton.

HowtomodelsystemwithAutomata• Analyticalmodeling• AllinformationaboutthesystemisavailableintheformofalanguageandwetrytobuildaDFSMoutofgivenstrings.(e.g.Myhill-Nerode constructionmethod)

• Question:• Howaboutthecasewedonothaveinformationaboutthesystem?Weneedtolearn thesystem.

• Learningtechniques

12PassiveLearning: ActiveLearning:

PassiveLearningApproach:• Teacherprovidesaboundingnumberofexamples

• Learnerfitamodelfortheprovidedexamples.

• Thelearnerpassivelylearnsthetrainedinformationandonlycanworkonthetrainingrange.

13Question:Howaboutthecasethatanewsituationhappensandthelearnerisnottrainedforit?

ActiveLearningApproach:• Imaginewedon’thaveinformationaboutthesystemandwedonothaveaccesstothesystem.

• Inthismethod,thelearnertriestoactivelybuildthemodelbyaskingminimalquestionsfromateacher.

• Theteacherisanexpertpersonwhocananswerthelearner’squestionsaboutthesystem.

• Lstar Learningisandexampleofactivelearning

14

LSTAR LEARNINGAlgorithmandUAVExample

15

1. DiscreteEventSystemsDefinitionsApplications

2. AutomataTheoryLanguageRegularLanguageAutomataRepresentationModelinginAutomata

3. Lstar LearningDefinitionsAlgorithmUAVExample

4. Lstar Features5. Lstar Matlab Toolbox6. Conclusion

Lstar Learning• TheLstar algorithmhasintroducesbyAngluin in1987.

• Thistechniqueactivelyidentifiesanunknownregularlanguage,andgeneratesaminimumstateDeterministicFiniteAutomata(DFA).

• ItisassumedthattheregularsetisavailabletoaminimallyadequateTeacherwhichcananswerthelearnersquestionsandvalidatethemodel.

• ThelearneractivelyasksqueriesduringthelearningprocesstobuildanobservationtableandeventuallycomesupwithconjecturesetandDFSM. 16

Lstar learning• Queries:

Twotypesofqueriesareaskedbylearner:1. Membershipqueries

• whetheracertaininput sequenceiscontainedinU2. Equivalencequeries

• whethertheconstructedDFAMiscorrect,ifnot,learnerasksforcounterexample.

• Counterexample:• Thosestringswhereareeitherareinsystembutnotinmarkedlanguageorviseversa,areprovidedtothelearnerascounterexampleoftheAutomata:

𝐶 = {∀𝑡 ∈ 𝑄|𝑡𝜖𝑈/𝐿< ∧ 𝑡𝜖𝐿</𝑈} 17

Cont.• Observationtableconsistofthreesections:• anonemptyfiniteprefix-closedsetSofstrings• anonemptyfinitesuffix-closedsetEofstrings• afinitefunctionTmapping((S⋃S.A).E)to{0,1}

• TheinterpretationofTisthatT(u)is1ifandonlyifuisamemberoftheunknownregularset,U.

18

S.A

S

W

ConsistencyandCloseness• Consistent• Anobservationtableiscalledconsistentprovidedthatwhenevers1 ands2areelementsofSsuchthatrow(s1)=row(s2),forallainA.

• Closed• AnobservationtableisconsideredclosedifforeachtinS.AthereexistsansinSsuchthat:

191

2

3A

A

?

S.A

S

W

1

𝒔𝟏, 𝒔𝟐 ∈ 𝑺, 𝒂 ∈ 𝑨, 𝒘 ∈𝑾𝐫𝐨𝐰 𝐬𝟏 = 𝐫𝐨𝐰 𝐬𝟐 ∨ 𝑻 𝒔𝟏𝒂𝒘 = 𝑻(𝒔𝟐𝒂𝒘)}

𝒔 ∈ 𝑺,𝒂 ∈ 𝑨, ∀𝒕 ∈ 𝑺𝐫𝐨𝐰 𝒔𝒂 = 𝐫𝐨𝐰 𝐭 }

Solution• NotConsistent• Iftheobservationtableisnotconsistent:

𝒂𝒘 = 𝒔𝟏, 𝒔𝟐 ∈ 𝑺, 𝒂 ∈ 𝑨,𝒘 ∈ 𝑾 𝐫𝐨𝐰 𝐬𝟏 = 𝐫𝐨𝐰 𝐬𝟐 ∨ 𝑻(𝒔𝟏𝒂𝒘) ≠ 𝑻(𝒔𝟐𝒂𝒘)}

• aw isaddedtoW andupdatetheobservationtable.

• NotClosed• Iftheobservationtableisnotclosed:• 𝒔𝒂 = 𝒔 ∈ 𝑺, 𝒂 ∈ 𝑨, ∀𝒕 ∈ 𝑺 𝐫𝐨𝐰 𝒔𝒂 ≠ 𝐫𝐨𝐰 𝐭 }

• Addsa toS andaddall‘sa.w’,∀w∊W toS.A.20

CounterExample• If anobservationtableisconsistentandclosed,correspondingDFAisdefined.

• M(S,E,T)=(Q,Σ,δ,q0,F)• Q ={row(s):s∊S},• q0 =row(ℇ),• F ={row(s):s∊S andT(s)=1},• δ(row(s),a)=row(s.a).

• TheteacherreplieseitherwithmatchorreplieswithacounterexampleC.

𝐶 = {∀𝑡 ∈ 𝑄|𝑡𝜖𝑈/𝐿< ∧ 𝑡𝜖𝐿</𝑈}

• ThenC andallitsprefixesareaddedtoS. 21

Lstar Algorithm

22

Initializingobservation table

*Checktheconsistency:ifnotconsistent𝒂𝒘 = 𝒔𝟏, 𝒔𝟐 ∈ 𝑺, 𝒂 ∈ 𝑨, 𝒘 ∈𝑾 𝐫𝐨𝐰 𝐬𝟏= 𝐫𝐨𝐰 𝐬𝟐 ∨ 𝑻(𝒔𝟏𝒂𝒘) ≠ 𝑻(𝒔𝟐𝒂𝒘)}

*Addaw toW.

*Checkthecloseness:ifnotclosed𝒔𝒂 = 𝒔 ∈ 𝑺, 𝒂 ∈ 𝑨,∀𝒕 ∈ 𝑺 𝐫𝐨𝐰 𝒔𝒂 ≠ 𝐫𝐨𝐰 𝐭 }*Addsa toS.

*ConstructDFSMandcheckforcounterexample.𝐶 = {∀𝑡 ∈ 𝑄|𝑡𝜖𝑈/𝐿< ∧ 𝑡𝜖𝐿</𝑈}

*AddC anditsprefixtoS.

DFSM

Ifcountere

xampleisprovided

UntiltheOTisconsistentandclosed

UAV- Example• ConsiderthefollowingexampleofUAV,Quadratormissionoperationmodel.UAVreceivescommandstostartitsmissionandreturntohangar.Butduringthemissionitmayfacewithemergencyconditionandneedtogetbacktohangar.

• Assumingwedon’tknowthesystem,wewilltrytouseLstarlearningtoactivelygeneratecorrespondingAutomata.

23

S

R

L,R SMissionMode

HangarMode

EmergencyMode

L,S,R

L

S:StartmissionR:ReturntohangarL:Lowfuelalarm

UAV,Example:• Step1:• S:

• Startingwiththeinitialobservation tablewithemptystring.• S={λ}

• S.A:• Considering A={S,L,R}theS.A is:• S.A={S,L,R}

• Analyze:• Tableisnotclosedduetotherow(S) fromS.A whichdoesnothavesimilarrowinS.

• Action:• Adding row(S)toS

24

L*DFSM λλ 1

S 0L 1R 1

UAV,Example:• Step2:• S:

• StringSisadded• S={λ,S}

• S.A:• S.A={L,R,SL,SS,SR}

• Analyze:• Tableisnowclosedandconsistent.

• Action:• GeneratingtheDFSMandaskingforanyexistingcounterexample.

• Counterexample:• C={SLL}

25

L*DFSM λλ 1S 0

L 1R 1SL 0SS 0SR 1

S

R

L,R L,SState2State1

Ex.Cont.• Step3:• S:

• StringSLLanditsprefixstringsareadded• S={λ,S,SL,SLL}

• S.A:• S.A={L,R,SL,SS,SR,SLS,SLR,SLLL,SLLS,SLLR}

• Analyze:• Tableisnotconsistent:• row(S)=row(SL)butT(SL)≠T(SLL).

• Action:• AddingLtoeventcolumnandupdatethetable.

26

L*DFSM λλ 1S 0SL 0SLL 1

L 1R 1SS 0SR 1SLS 1SLR 1SLLL 1SLLS 0SLLR 1

Ex.Cont.• Step2:• S:

• Nochange.

• S.A:• Nochange.

• Analyze:• Tableisnowclosedandconsistent.

• Action:• GeneratingtheDFSMandteacherapprovesthemodel, repliesmatch.

27

L*DFSM λ Lλ 1 1S 0 0SL 0 1SLL 1 1

L 1 1R 1 1SS 0 0SR 1 1SLS 1 1SLR 1 1SLLL 1 1SLLS 0 0SLLR 1 1

S

R

L,R SState2State1 State3

L,S,R

L

SECTION4:LSTAR LEARNINGFEATURES

TheoremsandProofs

28

1. DiscreteEventSystemsDefinitionsApplications

2. AutomataTheoryLanguageRegularLanguageAutomataRepresentationModelinginAutomata

3. Lstar LearningDefinitionsAlgorithmUAVExample

4. Lstar Features5. Lstar Matlab Toolbox6. Conclusion

Lstar LearningFeatures• ConstructedDFAwithLstar learningisminimal.AnyotherDFAconsistentwithT,wouldhavemorethanorequalnumberofstates.

• AnyDFAconsistentwithT,withthesamenumberofstatesisisomorphic withM.

• Totalnumberofoperation isatmostn-1• sincethereisinitiallyatleastonevalueofrow(s)andtherecannotbemorethann.(n isnumberofstates)

• L* terminates aftermakingatmostn conjecturesandexecutingitsmainloopatotalofatmostn- 1times. 29

Angluin,Dana."Learningregularsetsfromqueriesandcounterexamples."Informationandcomputation 75.2(1987):87-106.

• Proof:

• Proofisbycontradiction,weassumethatthereexistsaclosedandconsistentacceptorM’haslessthanorequalnumberofstates

• WewillshowinorderforM’tobeconsistentwithTitmusthaveatleastn states

30

AnyotherDFAconsistentwithT,wouldhavemorethanorequalnumberofstates.

ProofCont.• AsM’isconsistentwithT,

{∀𝑠 ∊ 𝑆. 𝐴⋀𝑒 ∊ 𝐸|𝛿’(𝑞0’, 𝑆. 𝑒) ∈ 𝐹’ ⟺ 𝑇(𝑠. 𝑒) = 1}

Where:

δ’ 𝑞0’, 𝑠. 𝑒 = 𝛿’(𝛿’(𝑞0’, 𝑠), 𝑒)

• AlsofromM,itisconsistentwithT:

{∀𝑠 ∊ 𝑆. 𝐴𝑎𝑛𝑑𝑒 ∊ 𝐸|𝑇(𝑠. 𝑒) = 1 ⟺ 𝛿(𝑞0, 𝑆. 𝑒) ∈ 𝐹}

• Wecanalsosay:

δ 𝑞0, 𝑠. 𝑒 = 𝛿(𝛿(𝑞0, 𝑠), 𝑒)31

ProofCont.• Therefore:

𝑅𝑜𝑤(𝑠) = 𝑟𝑜𝑤(𝛿’(𝑞0’, 𝑠))

• Ass rangesovern differentstates,therearen differentrow(s)andasresultn differentrow(δ’(q0’,s))inQ’.WhichcontradictswiththeearlierassumptionthatM’haslessthann states.

• Conclusion:• anyM’consistentwithThasatleastn states.

32

SECTION4:LSTAR LEARNINGTOOLBOX

TheoremsandProofs

33

Matlab Toolbox

34

• WedevelopedaLstar learningMatlab toolbox.• UAVexample• WehaveusedthedevelopedtoolboxandcameupwithDFSMoftheUAVsystem.

SECTION5:CONCLUSION

SummaryandFutureWork

35

SummaryüWeintroduceddiscreteeventsystemsandtheirvastareaofapplications.

ü WeprovideddifferentaspectofmodelingtechniquesusedforidentifyingDESsystems.

üWeprovidedinformationaboutAutomatarepresentation

üWediscussedanapplicationexampleofAutomataforanUAVsystem.

üWeintroducedLstarLearningasaneffectiveactivelearningapproachforDESsystems

üLstarMatlab toolboxisprovidedandtestedwithUAVexample. 36

FutureWorkØDevelopingalearningtechniquefordiagnosisofDES.

ØAlloftheexistingtechniquesforfaultdiagnosisofDESassumeperfectknowledgeaboutthesystem.

ØWewilltrytodevelopalearning-basedtechniquefordiagnosisofunknownDES.

ØDevelopinglearningtechniqueforNFSMandcorrespondingfaultdiagnosing.

37

References• Angluin,Dana."Learningregularsetsfromqueriesandcounterexamples."Informationandcomputation 75.2(1987):87-106.

• Kumar,Ratnesh,andVijayK.Garg.Modelingandcontroloflogicaldiscreteeventsystems.Vol.300.SpringerScience&BusinessMedia,2012.

• Karimoddini,Ali,etal."Hybridformationcontroloftheunmannedaerialvehicles."Mechatronics 21.5(2011):886-898.

• Lohstroh,Marten,Lee,EdwardA.,“AnInterfaceTheoryfortheInternetofThings.”13thInternationalConferenceonSoftwareEngineeringandFormalMethods(SEFM),2015. 38

OtherReferences• Cassandras,ChristosG. Introductiontodiscreteeventsystems.SpringerScience&BusinessMedia,2008.

• Lee,EdwardAshford,andSanjit Arunkumar Seshia. ”Introductiontoembeddedsystems:Acyber-physicalsystemsapproach.”Lee&Seshia,2011.

• Nam,Wonhong,P.Madhusudan,andRajeevAlur."Automaticsymboliccompositionalverificationbylearningassumptions." FormalMethodsinSystemDesign 32.3(2008):207-234.

• Ipate,Florentin."Learningfinitecoverautomatafromqueries." JournalofComputerandSystemSciences 78.1(2012):221-244. 39

40

ThankyouMohammadMahdiKarimiPhD.Candidate,ECESupervisor: Dr AliKarimoddinimmkarimi@aggies.ncat.edu