1
Outline
• Criticism to support/confidence• Loglinear modeling • Casual modeling
Background — Interaction analysis
• Association rule, Creighton & Hanash 03 Associations instead of interaction Undirected Need to discretize data
• Loglinear Modeling, Wu et al. 03 multi-way non-linear interactions Undirected Need to discretize data
• Graphical gaussian model, Kishino & Waddell 00 Pairwise interactions Undirected Efficient
• Causal Network Pairwise Directed High complexity
3
Background on Association Rule
• An association rule X Y satisfies with minimum confidence and support
support, s = P(XUY), probability that a transaction contains {X U Y}
confidence, c = P(Y|X), conditional probability that a transaction having X also contains Y
• Efficient algorithms Apriori by Agrawal & Srikant, VLDB94 FP-tree by Han, Pei & Yin, SIGMOD 2000 etc.
Customerbuys Y
Customerbuys both
Customerbuys X
4
Criticism to Support and Confidence
• Example 1: (Aggarwal & Yu, PODS98) Among 5000 students
3000 play basketball 3750 eat cereal 2000 both play basket ball and eat cereal
play basketball eat cereal [40%, 66.7%] is misleading because the overall percentage of students eating cereal is 75% which is higher than 66.7%.
play basketball not eat cereal [20%, 33.3%] is far more accurate, although with lower support and confidence
basketball not basketball sum(row)cereal 2000 1750 3750not cereal 1000 250 1250sum(col.) 3000 2000 5000
5
• We need a measure of dependent or correlated events
• P(Y|X)/P(Y) is also called the lift of rule X => Y
Criticism to Support and Confidence
)(
)|(
)()(
)(, YP
XYP
YPXP
YXPcorr YX
6
Criticism to lift
• Suppose a triple ABC is unusually frequent because Case 1: AB and/or AC and/or BC are unusually frequent Case 2: there is something special about the triple that all three
occur frequently.
• Example 2: (DuMouchel & Pregibon, KDD 01) Suppose in a db of patient adverse drug reactions, A and B are
two drugs, and C is the occurrence of kidney failure Case 1: A and B may act independently upon the kidney, many
occurrences of ABC is because A and B are sometimes prescribed together
Case 2: A and B may have no effect on the kidney if taken alone, but when taken together a drug interaction occurs that often leads to kidney failure
Case 3: A and B may have small effect on the kidney if taken alone, but when taken together, there is a strong effect.
7
Criticism to lift• EXCESS2
FAlleeEXCESS 22
Predicted count of all-two-factor model based on two-way distributions
Shrinkage estimates, (or we can use raw count)
an estimate of the number of transactions containing the item set over and above those that can be explained by the pairwise associations of the items
8
Motivation• EXCESS2
can separate case 2 and 3 from case 1, but can not separate between case 2 and 3.
need to build many all-two-factor models. For itemset ABCDE, they need to build 15 all-two-factor models, one for each multi-item set (ABC, ABD, …ABCD,… ABCDE)
can not fully analyze the interestingness of multi-item associations E.g., even we know the EXCESS2 for ABCD is large, is it due to ABC, ABD, or
ABCD?
• Fit to get one optimal loglinear model to describe all the possible associations instead of building many all-two-factor models
The -terms can precisely describe the interactions of items By analyzing residues, we can pick up the multi-item associations that can
not be explained by all the associations included in the fitted model.
9
Saturated log-linear model
ABCDijkl
BCDjkl
ACDikl
ABDijl
ABCijk
CDkl
BDjl
BCjk
ADil
ACik
ABij
Dl
Ck
Bj
Aiijkly
ˆlog
main effect 1-factor effect
2-factor effect which shows the dependency within the distributions of A,B.
10
Computing -term
0
...
0...
0
....
....
....
ABCDijk
ABCDkli
ABCDlij
ABCDijk
CDl
ACi
ABj
ABi
DCBA
• Linear constraints of coefficients
• UpDown method (Sarawagi et al, EDBT98)
Loglinear parameters sum to 0 over all indices
Ck
Bj
Ai
BCjk
ACik
ABijijk
ABCijk
Bj
Aiij
ABij
iAi
l
l
l
l
.
..
...
...
....
KDD’03 Washington, D.C. 11
Interpreting associations• Comparison with lift, EXCESS2
• Derive association patterns by examining -terms E.g. we can derive
positive interaction between AC, negative interaction between AC, no significant interaction between BC, and positive three-factor interaction among ABC
ABDABCCDBDBC
ACDCBAfitted
CDBDBCADAC
ABDCBApairwise
DCBAlift
y
y
y
ˆlog
ˆlog
ˆlog
Independence model
pairwise model
fitted model
223.0,765.0,681.0 ABCBCAC
12
Decomposition
• Decomposition is necessary as The contingency table from market basket data is too sparse The complexity is exponential in the number of dimensions
• Step 1.1, build one independence graph• Step 1.2, apply graph-theoretical results to decompose
the graph into non-decomposable irreducible components
13
Independence graph
F I
E H
G
B
C D
A
J
• Every vertex of the graph corresponds to an variable.
• Each edge denotes the dependency of the two variables linked
• A missing edge represents the conditional independence of two variables associated with the two edges
• Test conditional independence for every pair of variables, controlling for the other variables.
Cochran-Mantel-Hasenzel test etc.
14
Independence graph decomposition
• Graph-theoretical result:
If a graph corresponding to a graphical model for one contingency table is decomposable into subgraphs by a clique separator, the MLEs for the parameters of the model can easily be derived by combining the estimates of the models on the lower dimensional tables represented by the simpler subgraphs.
• Divide and conquer
F I
E H
G
B
C D
A
J
F I
E H
G
B
C D
A
J
C
A
G
A
15
Data generator• http://www.cs.loyola.edu/~cgiannel/assoc_gen.html
parameter value meaning
ntrans 10k-1M Number of transactions
nitems 50,100 Number of different items
tlen 10 Average items per transaction
npats 10000 Number of patterns(large item sets)
patlen 4 Average length of maximal pattern
corr 0.25 Correlation between patterns
conf 0.75 Average confidence in a rule
Other measures
2 x 2 contingency table
Objective measures for A=>B
17
Outline
• Criticism to support/confidence• Loglinear modeling • Casual modeling
Partial correlation
• Partial correlation The correlation between two variables after the common effects of the
third variables are removed
)1)(1( 22.
yzxz
yzxzxyzxy
rr
rrrpr
Causal Interaction Learning• Bayesian approaches (search and score), Friedman et al. 00
Apply heuristic searching methods to construct a model and then evaluate it using some scoring measure (e.g., bayesian scoring, entropy, MDL etc.)
Averaging over the space of structures is computationally intractable as the number of DAGs is super-exponential in the number of genes
Sensitive to the choice of local model
• Constraint-based conditional independence approaches, PC by Spirtes et al. 93
Instead of searching the space of models, it starts from the complete undirected graph, then thins this graph by removing edges with zero order conditional independence relations, thins again with first order conditional independence relations and so on so force.
Slow when dealing with large amount of variables
PC Algorithm
D-Separation
• X is d-separated from Y, given Z, if all paths from a node in X to a node in Y are blocked, given Z.
let p be any path between a vertex in X and a vertex in Y, Z is said to block p if there is a vertex w on p satisfying one of the following:
w has converging arrows along p, and neither w nor any of its descendants are on Z or
w does not have converging arrows along p, and w is in Z.
• Equally we can say that X and Y are independent conditional on Z.
Path BlockageA path is active, given evidence Z, if
• Whenever we have the configuration
•B or one of its descendents are in Z
• No other nodes in the path are in Z
A path is blocked, given evidence Z, if it is not active.
•A •C
•B
Top Related