Collective Classification A brief overview and possible connections to email-acts classification...
-
Upload
joel-grant -
Category
Documents
-
view
216 -
download
0
Transcript of Collective Classification A brief overview and possible connections to email-acts classification...
Collective Classification A brief overview and possible connections to
email-acts classification
Vitor R. Carvalho
Text Learning Group Meetings,
Carnegie Mellon University
November 10th 2004
Data Representation
• “Flat” Data – Object: email msgs– Attributes: words, sender, etc– Class: spam/not spam– Usually assumed IID
• Sequential Data– Object: words in text– Attr: capitalized, number, dict– Class: POS (or name/not)
• Relational Data– class+attributes– +links(relations)– Example: webpages
pron namedetnameverb
spamspam
spam Not spam
Not spam
J. Neville et al., 2003
Relational Data and Collective Classification
•Different objects interact
•Different types of relations (links)
•Attributes may be correlated
•Examples: – actors, directors, movies, companies– papers, authors, conferences, citations– company, employee, customer,
Classify objects collectively
Use prediction on some objects to improve prediction on related objects
Collective Classification Methods
• Relational Probability Trees (RPT)
• Iterative methods (Relaxation-based Methods)
• Relational Dependency Networks (RDN)
• Relational Bayesian Networks (RBN/PRM)
• Relational Markov Networks (RMN)
• Other models (ILP based, Vector Space based, etc)
•Overall:
– Lack of direct comparison among methods
– Results are usually compared to “flat” model
– Splitting data into train/test sets can be an issue
Relational Probability Trees
• Decision Trees applied to Relational data
• Predicts the target class label based on:– same object attributes– attributes + links in “relational neighborhood” (one link away)– counts of attributes and links in the “neighborhood”
• Enhanced feature selection (Chi-square, pruning, randomization tests)
• Results were not exciting
•Neville et al. KDD2003, related work from Blockeel et al. (Artificial Intelligence, 1998), Kramer AAAI-96
Iterative Methods
• Predicts the target class label based on:– Same object attributes– Attributes and links of relational
neighborhood– CLASS LABEL of neighborhood– Features derived from CLASS LABELS
• Different update strategies:– By threshold in prediction confidence
– By top-N most confident predictions
– Heuristic-based
• Slattery & Mitchell, ICML-2000;Neville & Jensen, AAAI-2000; Chakrabarti et al. ACM-SIGMOD-98
• Some results with Email-acts
Relational Bayesian Networks (RBN/PRM)
• Bayes Net extended to Relational domain
• Given an “instantiation”, it induces a bayes-net that specifies a joint probability distribution over all attributes of all entities
• Directed graphical model, with acyclicity constraint.
• Exact model - Closed form for parameter estimation – Products of conditional probabilities
• Was applied to simple domains, since the acyclicity constraints is very restrictive to most relational applications
• Friedman et al, IJCAI-99; Getoor et al., ICML-2001; Taskar et al. IJCAI-2001
Relational Markov Networks (RMN)
• Extension of CRF idea to Relational Domain
• Given an instantiation, it induces a Markov Network that specifies a probability distribution of labels, given links and attributes
• Undirected, Discriminative model
• Parameter estimation is expensive, requires approximate probabilistic inference (belief propagation)
•Taskar et al., UAI2002
Relational Dependency Networks (RDN)
• Dependency Networks extended to Relational domain
• P(X) = π [ Prob (Xi | Neighbor(Xi)) ]
• Given an “instantiation”, it induces a DN that specifies an “approximate” joint probability distribution over all attributes of all objects
• Undirected graphical model, no acyclicity constraint.
• Approximate model - Simple parameter estimation – approximate inference (Gibbs sampling)
• Neville & Jensen, KDD-MRDM-2003
Other Models
From Neville et al., 2003
Comparing Some Results
• Comparing PRM, RMN, SVM and M^3N
• Diff: PRM and RMN• Diff: mSVM and RMN
• RN* (Relational Neighbor) is a very simple Relational Classifier
• RN* (Macskassy et al., 2003)• M^3N(Taskar et al., 2003)
PRM
RMN
End of overview…now, the email-act problem
DeliveryRequest
CommitProposalRequest
Commit
Commit
Delivery
Request
Proposal
Delivery
Acknowled
Request
Time
• Strong correlation with previous and next message
• Flat data?
• Sequential data?
• A “verb” has little or no correlation with other “verbs” of same message