Proteomic Mass Spectrometry. Outline Previous Research Project Goals Data and Algorithms...
-
date post
20-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Proteomic Mass Spectrometry. Outline Previous Research Project Goals Data and Algorithms...
Proteomic Mass Spectrometry
Outline
• Previous Research
• Project Goals
• Data and Algorithms
• Experimental Results
• Conclusions
• To Do List
Motivation
• MS spectra has high dimension– Most ML algorithms are incapable of handling
such high dimensional data
• Dimensionality Reduction (DR)– Preserve as much information as possible,
while reducing the dimensionality.
• Feature Extraction (FE)– Removal of irrelevant and/or redundant features
(information)
Previous research
• Usually applies DR then FE• Does Order matter ?
DR: Down Sampling, PCA, WaveletsFE: T-Test, Random Forests, Manual Peak Extraction
• In [conrads03] show that high resolution MS spectra produces better classification accuracy.– Most previous research down samples spectraCONJECTURE: Down Sampling detrimental to
performance.
Project Goals
• Test Down Sampling Conjecture
• Compare FE algorithms (NOTE: Optimal FE is NP-hard !)– Use a simple but fast classifier to test a number
of FE approaches
• Test across different data sets– Are there any clearly superior FE algorithms ?
Three Data Sets
Heart/Kidney (100/100)– 164,168 features, 2 classes
Ovarian Cancer (91/162)– 15,154 features, 2 classes
Prostate Cancer (63/190/26/43)– 15,154 features, 4 classes
• Normal, Benign, Stage 1, Stage 2 Cancer• Transformed into Normal/Benign Vs Cancer (1&2)
Algorithms
Centroid Classifier– given class means P, Q and sample point s
C = argmin (d(P,s), d(Q,s))C = argmin (d(P,s), d(Q,s))
P Q
d(P,s) d(Q,s)
Algorithms
• T-testT-test – do the means of 2 distr. Differ ?
• KS-testKS-test – do the cdf differ ?
• CompositeComposite – (T-test)*(KS-test)
• IFEIFE - Individual Feature Evaluation using the centroid classifier
• DPCADPCA – discriminative principle component analysis
Preliminary Experiments
• Compare normalization approaches
• Compare similarity metrics– Cross correlation– (-L1)– Angular
• Across 3 data sets => 27 configurations
L1à norm;1 à norm;no normalization
Preliminary Experiments (cont)
• No single norm/metric clearly superior on all data sets
• 2-5% increase in performance if suitable normalization and similarity metric chosen (can be up to 10% increase)
• L1-norm with angle similarity metric worked well on Heart/Kidney and Ovarian Cancer sets (easy sets)
• L1-norm and L1-metric best on Prostate 2-class problem (hard set).
Down Sampling
Statistical Tests
• T-test, KS-test, Composite– Ranks features in terms of relevance
• SFS – Sequential Forward Selection– Selects ever increasing feature sets
• I.e., {1}; {1,2}; {1,2,3}; {1,2,3,4}
Heart/Kidney
Ovarian Cancer
Prostate Cancer
Single Feature Classification
• Use each feature to classify test samples
• Rank features in terms of performance
• SFS
Performance Comparison
Performance of Statistical Feature Extraction on Heart/Kidney Data Set
0.75
0.8
0.85
0.9
0.95
1
None(164168)
KS-test (2601)
T-test (887)
Composite(1810)
Best SingleFeature
SFS-L1(3) SFS-Angle(70)
Feature Extraction M ethod (# of Features)
Ac
cu
rac
y
Performance Comparison
Performance of Statistical Feature Extraction on Ovarian Cancer Set
0.75
0.8
0.85
0.9
0.95
1
None(15154)
KS-test(12)
T-test (91) Composite(20)
BestSingle
Feature
SFS-L1(48)
SFS-Angle(14)
Feature Extraction Method (# of Features)
Acc
ura
cy
Performance Comparison
Performance of Statistical Feature Extraction on Prostate Cancer Set
0.50.55
0.60.65
0.70.75
0.80.85
0.90.95
1
Feature Extraction Method (# of Features)
Ac
cu
rac
y
Summary
• For each data set, for each FE algorithm ran 15,000 3-fold cross validation experiments.
• Total of 810,000 FE experiments ran
• DE experiments ~ 100,000 experiments
• Additional 50,000 experiments using DPCA classifier
• did not produce significantly different results than the centroid classifier
Conclusions• HK and Ovarian Data sets considerably easier to classify than
Prostate Cancer• Feature Extraction (in general) significantly improves
performance on all data sets• No single technique superior on all data sets.
– Best Performance using SFS with feature weighting– Smallest feature set with T-test of KS-Test– Composite test inferior to all others.
• Down Sampling appears to be detrimental – What about other Dim. Red. Techniques ?
• E.g. PCA and Wavelets
Conclusions
• Down Sampling appears to be detrimental – What about other Dim. Red. Techniques ?
• E.g. PCA and Wavelets
• What about FE after Down Sampling ?– On Prostate data performance appears to drop
w.r.t. to best single feature.
To Do List
• Check PCA, Wavelets and other DR techniques• Use other (better) classifiers• General Hypothesis
– Use a simple fast classifier together with FE techniques to extract a good feature set
– Replace classifier with a more effective one.• Need to verify that other classifiers respond well to the extracted
features.