Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs...
Transcript of Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs...
![Page 1: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/1.jpg)
Probabilistic AI Srihari
1
Local Probabilistic Models: Context-Specific CPDs
Sargur [email protected]
![Page 2: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/2.jpg)
Probabilistic AI Srihari
Topics• Context-Specific CPDs
1. Regularity in parameters for different values of parents
2. Tree CPDs3. Rule CPDs4. Multinets 5. Similarity Networks
2
![Page 3: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/3.jpg)
Probabilistic AI Srihari
Context-Specific CPDs• Deterministic dependency is one example of
structure in CPDs• A very common type of regularity arises when
we have the same effect in several contexts– Several different distributions are the same
• Example is given next
3
![Page 4: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/4.jpg)
Probabilistic AI Srihari
Augmented student network
4
Original student network with CPDs Augmented student network
A more augmented networkUsed for later analysis
J: Student is offered Job at Acme Consultingj1: offered job, j0: otherwiseJob depends on SAT & LetterStudent may Apply: a1, or not a0
Next we look at how to specify the CPD P(J|PaJ)=P(J|L,A,S)which has 8 parameters
![Page 5: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/5.jpg)
Probabilistic AI Srihari
CPD P(J|A,S,L) has regularities1. Recruiter offers job even without applying
2. Recruiter feels SAT more important than letter– High SAT generates offer without letter:
• i.e., two probabilities are equal– Low SAT requires letter
• Several values of PaJ={A,S,L} specify same conditional probability over J. – We need 8 parameters here
• But many probabilities are the same, depending on the context
P(J|a1,s1,l1)=P(J|a1,s1,l0)
If A=a0, no access to L and S. Thus, among 8 values of parents A,S,L, four with A=a0 induces identical distributions over variable J
![Page 6: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/6.jpg)
Probabilistic AI Srihari
Representing regularity in CPDs
• We have seen several values of PaJ specify the same conditional probability over J
• How to capture this regularity in our CPD representation
• Many approaches for capturing functions over a scope X that are constant over subsets of instantiations to X– Trees– Rules
6
![Page 7: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/7.jpg)
Probabilistic AI Srihari
Tree-CPD for P(J|A,S,L)• Internal nodes represent tests
– on parent variables• Leaves are annotated with distribution over J1. Job offer without applying: Parent context <a0>
P(j1|a0)=0.2, i.e., no Letter or SAT2. Good SAT: P(j1|a1,s1): Parent context <a1,s1>
Letter immaterial• choose path A=a1 and S=s1
• P(j1|a1,s1)=0.9
• Need 4 parameters instead of 8
7
![Page 8: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/8.jpg)
Probabilistic AI Srihari
Definition of Tree CPD• A tree-CPD for a variable X is a rooted tree
– Nodes are called t-nodes as distinct from BN nodes• Each t-node in the tree is either a leaf t-node or
an interior t-node• Each leaf is labeled with a distribution P(X )
• Each interior node is labeled with some variable Z ε PaX
• Each interior node has a set of arcs to its children each one associated with a unique assignment Z=zi for zi εVal(Z)
8
![Page 9: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/9.jpg)
Probabilistic AI Srihari
Another example of regularity
• Some events can occur only in certain situations
• Parent Context: Outside (o0,o1)
– Variable Wet (W) depends on variable Raining (R)
P(W|R,o1)
9
![Page 10: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/10.jpg)
Probabilistic AI Srihari
Multiplexer CPD• George has to decide whether to give the
recruiter the letter from the Professor of CSE 674or from the Professor of CSE 601
• Depending on which choice George makes the dependence will only be on one of the two
10
![Page 11: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/11.jpg)
Probabilistic AI Srihari
Multiplexer CPD• A CPD P(Y|A,Z1,..Zk) is a multiplexer CPD if
Val(A)={1,..,k} and P(Y|a,Z1,..,Zk)=1{Y=Za}– Where a is the value of A– The variable A is the selector variable of the CPD
• In other words, the value of the selector is a copy of the value of one of its parents – The role of A is to select the parent who is being
copied
11
![Page 12: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/12.jpg)
Probabilistic AI Srihari
Multiplexer: Tree and BN
(a) network fragment(b) tree CPD for P(J|C,L1,L2)
(c) Modified network with new variable L that has a multiplexer CPD
12
(a) (b) (c)
![Page 13: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/13.jpg)
Probabilistic AI Srihari
Advantage of Trees
• Provide natural framework for representing context-specificity in a CPD
• People find it convenient• Lends itself well to automated learning
algorithms– To construct a tree automatically from a data set
13
![Page 14: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/14.jpg)
Probabilistic AI Srihari
Tree Application: Diagnostic Networks
• Trouble-shooting of physical systems• Context specificity is due to presence of
alternative configurations• Diagnosis of faults in a printer
– Part of trouble-shooting network for MS Windows 95
– Printer can be hooked up to either network via• Ethernet cable (Network transport medium)
– Affects printer output only if printer is hooked to network
• Or to local computer via cable (Local Transport medium)14
![Page 15: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/15.jpg)
Probabilistic AI Srihari
Context-Specific Dependencies
(a) Real-world BN for Microsoft Online
Trouble-shooting system
(b) Structure of Tree-CPD for Printer Output variable
Reduces no. of parameters required from 145 to 55 15
![Page 16: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/16.jpg)
Probabilistic AI Srihari
Rule CPD• Trees capture entire CPD in a single data
structure• A finer-grained specification is via rules
– Each rule corresponds to a single entry in the CPD of the variable
– A rule ρ is a pair (c ; p)• where c is an assignment to some subset of variables C
and p ε [0,1].• C is the scope of ρ denoted Scope[ρ]
• This representation decomposes a tree-CPD into its most basic elements 16
![Page 17: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/17.jpg)
Probabilistic AI Srihari
Ex: Tree CPD for p(J|A,S,L)
• There are 8 entries in the CPD tree• Such that each one corresponds to a branch in
the tree and an assignment to the variable Jitself
• Thus the CPD defines eight rules 17
![Page 18: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/18.jpg)
Probabilistic AI Srihari
Ex: Rule CPD for p(J|A,S,L)• ρ1:<a0, j0; 0.8>• ρ2:<a0, j1; 0.2>• ρ3:<a1, s0, l0, j0; 0.9>• ρ4:<a1, s0, l0, j1; 0.1>• ρ5:<a1, s0, l0, j1; 0.4>• ρ6:<a1, s0, l1, j1; 0.6>• ρ7:<a1, s1, j0; 0.1>• ρ8:<a1, s1, j1; 0.9>
• There are 8 entries in the CPD tree• Such that each one corresponds to a branch in the tree and an
assignment to the variable J itself• Thus the CPD P(J|A,S,L) is defined by eight rules
• A formal definition of rule-based CPDs follows
![Page 19: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/19.jpg)
Probabilistic AI Srihari
Definition of Rule-based CPD• A rule-based CPD p(X|PaX) is a set of rules R
such that– For each rule ! ∈ R we have that Scope[!]⊆{X}∪PaX
– For each assignment (x,u) to {X}∪PaX we have precisely one rule (c;p) ∈ R such that c is compatible with (x,u).
– In this case we say that P(X=x|PaX=u)=p
• The resulting CPD P(X|U) is a legal CPD in that Σx P(x|u)=1
19
![Page 20: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/20.jpg)
Probabilistic AI Srihari
Other Representations
• Tree and rule representations are useful for representation, inference and learning
• However other representations are possible• They both induce partitions of {X}∪PaX defined
by branches of the tree or rule contexts– Each partition is associated with a different entry in X’s CPD
• Other such methods are decision diagrams, multinets and similarity networks
20
![Page 21: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/21.jpg)
Probabilistic AI Srihari
Multinets
• A more global approach to specifying context-specific independence
• A simple multinet– A network centered on a single class variable C,
which is the root of the network– The multinet defines a separate network Bc, for
each value of C– The structure and parameters can differ for these
different networks21
![Page 22: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/22.jpg)
Probabilistic AI Srihari
Common form of multinet
22
![Page 23: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/23.jpg)
Probabilistic AI Srihari
A subtlety in multinet
23
![Page 24: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/24.jpg)
Probabilistic AI Srihari
Usefulness of multinet
• Although a multinet can be represented as a standard BN with context-specific CPDs, it is nevertheless useful
• Since it explicitly shows the independencies in a graphical form, making them easier to understand and elicit
24
![Page 25: Local Probabilistic Models: Context-Specific CPDssrihari/CSE674/Chap5/5... · Context-Specific CPDs •Deterministic dependency is one example of structure in CPDs •A very common](https://reader033.fdocuments.in/reader033/viewer/2022050213/5f5ef083f4b02d69646ecae3/html5/thumbnails/25.jpg)
Probabilistic AI Srihari
Similarity Network• Related to the multinet representation• In a similarity network we define a network BS
for certain subsets of values S⊊Val(C) which contain only those attributes relevant for distinguishing between values in S– The underlying assumption is that if a variable X
does not appear in network BS then P(X|C=c) is the same for all c∈S
– Moreover if X does not appear in the network BSthen X is contextually independent of Y given C ∈Sand X�s other parents in this network 25