Lecture12:KnowledgeRepresentationandReasoningPart1:Logic

SanjeevArora EladHazan

COS402– MachineLearningand

ArtificialIntelligenceFall2016

(BorrowsfromslidesofPercyLiang,StanfordU.)

Thusfarinthecourse

• Formalizationoflearningfromdata(statisticallearningtheory)

• Languagemodelsandlanguagesemantics.(examplesofunsupervisedlearning)

• Recommendersystems.

Reminder:In-classmidtermthisThurs.75min;closedbook(arriveontime!)(Studyguidepostedonpiazza.)

Today:Knowledgerepresentationandreasoningusinglogic

AlsobasisofdigitalcircuitsincomputerchipsEE206/COS306

RoleoflogicinAI

• For2000years,peopletriedtocodify“humanreasoning”andcameupwithlogic.

• MostAIworkuntil1980s:Buildmachinesthatrepresentknowledgeanddoreasoningvialogic.“Rulebasedreasoning.”

• ”Learningfromdata”ispopulartoday,butlacksaspectsthatweretrivialinthepre-1980ssystems(e.g.allowhumanprogrammertoeasilycommunicatehis/herknowledgetothesystem).“Howdoyouteachadeepnettomultiplytwonumbers?”

• Logicalreasoningnowseemspoisedforacomeback.

Goalsoflogic

• Represent knowledgeabouttheworld.

• Reason withthatknowledge.

Naturallanguage?

• Adime isbetterthananickel.

• Anickel isbetterthanapenny.

• Therefore,adime isbetterthanapenny.

• Apennyisbetterthananothing.

• Nothing isbetterthanworldpeace.

• Therefore,apenny isbetterthanworldpeace.

Knowledge

Reasoning

Naturallanguageistricky!

Useoflogicremovesambiguity(similartocomputerlanguages);butalsomakessystemlessflexible.(Willstudymoreflexibleversionslater.)

Componentsofanylogicalsystem

• Syntax

• Semantics.

• Reasoning

Differentsyntax,samesemantics

2+3⇔ 3+2

Samesyntax,differentsemantics

3/2inPython2.7vs3/2inPython3.

PropositionalLogic(akaBooleanLogic;rememberCOS126!)

Syntax:

(A _ ¬B) ^ (¬A _B)

⋁ and⋀ and⟷ aresymmetric,like“+”and“times”inarithmetic.

Syntaxprovidessymbols.

No“meaning”yet(semantics)!

Semanticsprovidedbya“Model”(unrelatedto“model”usedinmachinelearning!)

Forpropositionallogic,amodelissimplyanassignmenttoallvariables.(eachvariableassigned0(false)or1(true),not both)

Sanitycheck:Whatis#ofpossiblemodels ifthereare3variables?Howaboutnvariables?

Interpretationfunction

I(f,w):Givenformulafandmodelw,assignsexactlyoneof1(True)or0(False)tof.

• Trueiff fisfalse• Trueiff bothfandgaretrue• Trueiff atleastoneoff,gistrue• Falseiff fistrueandgisfalse.• Trueiff fandghavethesamevalue(trueorfalse)

Definition:M(f)=Setofmodelswforwhich I(f,w)=True

• Example:Forf=A⋀ B

M(f)={A=1,B=1}.

• Forf=A⟷ B

M(f)={A=1,B=1};{A=0;B=0}

Tautology: FormulafsuchthatM(f)=Allpossiblemodels.(“Trueinallpossibleworlds”)

Example:A⋁ ¬A.(TruewhetherA=0orA=1!)

Contradiction: FormulafsuchthatM(f)=Emptyset.(”Falseinallpossibleworlds.”)

Example:A⋀ ¬A.(FalsewhetherA=0orA=1!)

Formulafcompactly representsM(f)(“Setofpossibleworldswhere

fistrue.”)

Knowledgerepresentationvialogic

Knowledgebase:Setofformulae{f1,f2,…, fn}M(KB)=Allpossiblemodelsforf1⋀ f2⋀… ⋀ fn

Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)

KB:R⋁ S⋁ C;R→C⋀ ¬ S;C⟷ ¬S

(”ItiseitherRainyorSunnyorCloudy.”)(“IfitisRainythenitisCloudyandnotSunny.”)(”IfitisCloudythenitisnotSunny,andviceversa”)

ModelsforKB:{R=1,S=0,C=1};{R=0,C=1,S=0};{R=0,C=0,S=1}.

Formulae=“knownfacts”Models=allpossible“worlds”whereallthesefactshold(AddingmorefactstoKBcanonlyshrink setofpossibleworlds.)

SatisfiabilityDefn:Knowledge-baseKBissatisfiable if

M(KB)≠∅(i.e.thereissomeassignmenttovariablesthatmakesallformulaeinKBevaluatetoTrue)

Defn:KBentails formulaf(denotedKB⊨ f)ifM(KB∪{f})=M(KB).

(ineveryworldwhereKBistrue,fisalsotrue)

Defn:KBcontradicts formulafifKB∪{f}isnotsatisfiable

Sanitycheck:KBentailsfiff itcontradicts¬f.

Defn:KBisconsistentwithformulafifM(KB∪{f})isnon-empty(thereisaworldinwhichKBistrueandfisalsotrue)

Anexample

Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)

KB: R⋁ S⋁ C;R→C⋀ ¬S;C⟷ ¬S

DoesKB⊨ S⋁ C?

ModelsforKB:{R=1,S=0,C=1};{R=0,C=1,S=0};{R=0,C=0,S=1}.

S⋁Cistrueinallthesemodels✔

Anexample

Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)

KB: R⋁ S⋁ C;R→C⋀ ¬S;C⟷ ¬S

Addavariable:U(“Carryanumbrella”).Whatcommon-sense“facts”canweaddaboutUtotheaboveKB?

Examples:R →U;S→¬U;AIsystemstill1980susedsuchreasoning;”facts”wereaddedbyprogrammersbaseduponintrospection.

Decision-makingatrun-time=whichformulaeareentailed/contradicted/consistent

Recapoflogicsofar

• Defn offormulae.

• KB=Listofformulae.(“Factsabouttheworld”)

• KBcanentail orcontradict anotherformula,orbeconsistentwithit.

• Todecidewhichofthethreepossibilitiesofprev.lineholds,drawuplistofallpossiblemodels.(“Truthtablemethod.”)

Truthtablemethod(tocheckifKBhasanymodel)

• Ifnvariables,cantake2n time.(infeasibleforevenn=100)Anyfasteralgorithm?

• Polynomialtimealgorithmè P=NP(Famousopenproblem)

• Inpracticetherearereasonablealgorithmsthatuseresolution andotherrelatedreasoningmethods.

ResolutionproceduretodecidesatisfiabilityofaKB(simplestversion;[Davis-Putnam,1950s])

KBconsistsonlyofformulaethatareclauses (ie ⋁ ofvariablesornegatedvariables).

(Withsomework,canconverteveryKBtothisform.)

Warmup:Whatcanweconcludeunderfoll.conditions?

KBhassingletonclauses(A),(¬A).

KBcontainsclausepairsofform(A ⋁ B1 ⋁ … ⋁ Bn)and(¬A⋁ C1 ⋁ … ⋁ Cm)

HASNOMODEL(UNSATISFIABLE)!

EverymodelforKBmustmake(B1 ⋁ … ⋁ Bn⋁ C1 ⋁ … ⋁ Cm)TRUE

ResolutionproceduretodecidesatisfiabilityofaKB(simplestversion;[Davis-Putnam,1950s])

KBconsistsonlyofformulaethatareclauses (ie ⋁ ofvariablesornegatedvariables).

(Withsomework,canconverteveryKBtothisform.)

While KBnonemptydo{If KBcontainsclausepairsofform(A),(¬A)

Print (“NoModel.”)andSTOP. /*WHY?*/If KBcontainsclausepairsofform(A ⋁ B1 ⋁ … ⋁ Bn)and(¬A⋁ C1 ⋁ … ⋁ Cm)

Add(B1 ⋁ … ⋁ Bn⋁ C1 ⋁ … ⋁ Cm)toKB./*WHY?*/else

Print(“Modelexists”)andSTOP. /*WHY??*/}

Claim(won’tprove):FinishesinfinitetimeforeveryKBandprintscorrectanswer.

Goodluckwithmidterm,Andhaveagoodfallbreak!