INFORMATION NETWORKS DIVISION COMPUTER FORENSICS UNCLASSIFIED 1 DFRWS2002 Language and Gender...
22
INFORMATION NETWORKS DIVISION COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au 1 DFRWS2002 Language and Gender Author Cohort Analysis of E-mail for Computer Forensics Olivier de Vel Computer Forensics DSTO, Australia
-
Upload
avice-jackson -
Category
Documents
-
view
215 -
download
2
Transcript of INFORMATION NETWORKS DIVISION COMPUTER FORENSICS UNCLASSIFIED 1 DFRWS2002 Language and Gender...
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
1DFRWS2002
Language and Gender Author Cohort Analysis of E-mail for
Computer Forensics
Olivier de VelComputer Forensics
DSTO, Australia
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
2DFRWS2002
World MapDSTO (Adelaide)
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
3DFRWS2002
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
4DFRWS2002
Computer forensics and e-mail authorship analysis
Previous work in authorship attribution
Experimental methodology
Results and Conclusion
Presentation:
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
5DFRWS2002
Authorship Analysis: Sub-Areas
Authorship Analysis
Authorship Profiling
Authorship AttributionSimilarity Detection
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
6DFRWS2002
Objective of E-mail Authorship Analysis
Develop algorithms for analysing the style and
content of an e-mail message for the purpose of
categorising
its author, or
its author’s cohort type
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
7DFRWS2002
E-mail authorship analysis is NOT about:
e-mail document filtering
e-mail document filing
e-mail text categorisation
e-mail topic detection/tracking
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
8DFRWS2002
Computer forensics and e-mail authorship analysis
Previous work in authorship attribution
Experimental methodology
Results and Conclusion
Presentation:
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
9DFRWS2002
Previous Work in Authorship Attribution
Shakespeare’s works & Federalist papers
Forensic linguistics
Code authorship
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
10DFRWS2002
Previous Work in Authorship Attribution: Issues
Conclusions (so far!):
a large number of stylometric features (>1000)
no definite subset of discriminatory
stylometric features
no consensus on methodology
Questionable analysis
Hard problem!
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
11DFRWS2002
Previous Work in Authorship Attribution
Previous work limited to (c.f. e-mails):
Large sections of text
Formal text and non-interactive
Relatively large number of training
examples
Relatively homogeneous style
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
12DFRWS2002
Previous Work in Authorship analysis: E-mails
E-mail authorship attribution:
We have investigated the effect of several
parameters (eg., text size, number of
documents per author) [2000, 2001].
Thomson et al have investigated gender-
preferential language styles [2001].
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
13DFRWS2002
Computer forensics and e-mail authorship analysis
Previous work in authorship attribution
Experimental methodology
Results and Conclusion
Presentation:
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
14DFRWS2002
Difficulty in obtaining e-mail corpus:
privacy and ethical concerns,
large % of noise (eg, cross-postings, off-the-topic
spam, empty body with attachments),
some difficulty in verifying author cohort class,
non-orthogonality of topics and cohorts.
Experimental Methodology: E-mail Corpus
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
15DFRWS2002
Two author cohort experiments (gender and language)
Experimental Methodology: E-mail Corpus
Two sub-corpuses derived from an E-mail corpus:
M/F: 325 authors, 4369 e-mails
EFL/ESL: 522 authors, 4932 e-mails
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
16DFRWS2002
Attributes/features used:
183 style markers that are known to have
reduced content bias (incl. function words
and word freq. distribution),
28 e-mail structural attributes,
11 gender-preferential language attributes
Experimental Methodology: E-mail Corpus
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
17DFRWS2002
Experimental Methodology: Classifier
SVMlight as the (two-way) classifier,
Obtain two-way categorisation matrix for each
author category, using 10-fold cross-validation
sampling,
Calculate per-author category performance
statistics – precision, recall and F1 statistics.
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
18DFRWS2002
Computer forensics and e-mail authorship analysis
Previous work in authorship attribution
Experimental methodology
Results and Conclusion
Presentation:
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
19DFRWS2002
Results: Classification performance
M/F Author Cohort Attribution:
50 100150 200
50
100200
300400
1000
0
10
20
30
40
50
60
70
80
Performance (F1), %
Minimum Word Count
Number of Emails per Cohort
Gender Cohort Attribution
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
20DFRWS2002
Results: Classification performance
EFL/ESL Author Cohort Attribution:
50 100 150 200
50100
200300
400500
600700
0
10
20
30
40
50
60
70
80
90
Performance (F1), %
Minimum Word Count
Number of Emails per Cohort
Language Cohort Attribution
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
21DFRWS2002
Conclusions:
Promising author cohort categorisation results.
Further experiments:
with extended and specific set of cohort-
preferential attributes,
more within-cohort diversity,
subset feature selection (particularly
function words).
Extend to other forms of computer-mediated
communications (chat rooms etc.)
INFORMATION NETWORKS DIVISION
COMPUTER FORENSICS UNCLASSIFIED www.dsto.defence.gov.au
22DFRWS2002
Questions ?
