09/05/2005 סמינריון במתמטיקה ביולוגית Dimension Reduction - PCA Principle...

09/05/2005סמינריון במתמטיקה ביולוגית

Dimension Reduction - PCA

Principle Component Analysis

סמינריון במתמטיקה ביולוגית

The Goals

Reduce the number of dimensions of a data set. Capture the maximum information present in

the initial data set. Minimize the error between the original data

set and the reduced dimensional data set.

Simpler visualization of complex data.

The Algorithm Step 1: Calculate the Covariance Matrix of the

observation matrix.

Step 2: Calculate the eigenvalues and the corresponding eigenvectors.

Step 3: Sort eigenvectors by the magnitude of their eigenvalues.

Step 4: Project the data points on those vectors.

The Algorithm Step 1: Calculate the Covariance Matrix of

the observation matrix.

PCA – Step 1: Covariance Matrix C

- Data Matrix

nn XXXX

NC )()(

)(..),(

),(..)(

XVarXXCov

XXCovXVar

Covariance Matrix - Example Tn

nn XXXX

NC )()(

05.45.7

242418

observation matrix.

Linear Algebra Review – Eigenvalue and Eigenvector

C - a square nn matrix

Example

eigenvector

eigenvalue

Singular Value Decomposition

jinn VUC

SVD ExampleLet us find SVD for the matrix

1) First, compute XTX:

2) Second, find the eigenvalues of XTX and the corresponding eigenvectors:

( use the following formula - )

12XXC T

0)( xCI

0161092510

)3()3()5()5(53

35)det(

SVD Example - Continue3) Now, we obtain the U and Σ :

4) And the decomposition C=UΣVT:

22111 uXv ;2,

22222 uXv ;22,

observation matrix.

PCA – Step 3 Sort eigenvectors by the

magnitude of their eigenvalues

observation matrix.

PCA – Step 4 Project the input data onto

the principal components.

The new data values are generated for each observation, which are a linear combination as follows:

kikpcipciipcpci XbXbXbSc ,,2,2,1,,, .. score observation principal component loading (-1 to 1) variablek

PCA - Fundamentals

1st PC

2nd PC

Projections

The first PC is the eigenvector with the greatest eigenvalue for the covariance matrix of the dataset. The Eigenvalues are also the variances of the observations in each of the new coordinate axes

Var(PC1)Var(PC2)

PCA: Scores

Obs. i 1st PC

2nd PC

The scores are the places along the component lines where the observations are projected.

pciSc ,

PCA: Loadings

The loadings bpc,k (dimension a, variable k) indicate the importance of the variable k to the given dimension. bpc,k is the direction cosine (cos a) of the given component line vs. the xk

coordinate axis.

1st PC

Cos(X/PC

PCA - Summary Multivariate projection technique. Reduce dimensionality of data by transforming

correlated variables into a smaller number of uncorrelated components.

Graphical overview. Plot data in K-Dimensional space. Directions of maximum variation. Best preserves the variance as measured in the high-

dimensional input space. Projection of data onto lower dimensional planes.

Biological Background

Reverse

scriptase

Areas Being Studied With Microarrays

To compare the expression of a protein (gene) between two or more tissues.

To check whether a protein appears in a specific tissue.

To find the difference in gene expression between a normal and a cancerous tissue.

cDNA Microarray Experiments

Different tissues, same organism (brain v. liver).

Same tissue, different organisms. Same tissue, same organism (tumour v.

non-tumour). Time course experiments.

Microarray Technology

Method for measuring levels of expression of thousands of genes simultaneously.

There are two types of arrays: cDNA and long oligonucleotide arrays. Short oligonucleotide arrays.

• Each probe is ~25 nucleotide long.• 16-20 probes for each gene.

The Idea

Target: cDNA (variables to be detected)

Probe: oligos/cDNA(gene templates) +

Hybridization

Brief Outline of Steps for Producing a Microarray Produce mRNA Hybridise

Complimentary sequence will bind

Fluorescence shows binding

Scan array (Extraction of intensities with picture analysis software)

Hybridization

RNA is cloned to cDNA with reverse transcriptase.

The cDNA is labeled. Fluorescent labeling is most common, but

radioactive labeling is also used. Labeling may be incorporated in hybridization,

or applied afterwards. Then the labeled samples are hybridized to

the microarrays.

Gene Expression Database – a Conceptual View

Gene expression levels

Gene expression matrix

Genes Gene annotations

Samples annotations

The Article

The Biological Problem

The very high dimensional space of gene expression measurements obtained by DNA micro arrays impedes the detection of underlying patterns in gene expression data and the identification of discriminatory genes.

Why to Use PCA?

To obtain a direct link between patterns in gene and patterns in samples.

Sample annotations

Gene annotations

The Paper Shows:

Distinct patterns are obtained when the genes are projected an a two-dimensional plane.

After the removal of irrelevant genes, the

scores on the new space showed distinct tissue patterns.

The Data Used in Experiment

Oligonucleotide microarray measurements of 7070 genes made in 40 normal human tissue samples.

The tissues they used were from brain, kidney, liver, lung, esophagus, skeletal muscle, breast, stomach, colon, blood, spleen, prostate, testes, vulva, proliferative endometrium, myometrium, placenta, cervix, and ovary.

Results PCA Loadings Can Be Used to Filter

Irrelevant Genes The data from 40 human tissues were first

projected using PCA. The first and second PCs account for 70% of ∼

the information present in the entire data set.

Gene Selection Based on the Loadings on the Principal Components

Graph A shows the score plot of the samples before any filtering is implemented.

Score Plot of the Tissue Samples

Scores on Principle Component 1

Graphs B shows the loading plot of the genes before any filtering is implemented.

Loadings on Principle Component 1

Loading Plot of the Genes

The Filter on Loadings

Graph E displays quantitatively the decisions that went into the choice of the filtering threshold. It displays the distortion in the observed patterns, as measured through the squared difference, and the number of genes retained for analysis as the threshold is varied.

Threshold

2,,,, )(

s pcopcsfpcs yySqDif

The Filter on the Loadings - Continue

The chosen filter threshold was 0.001.

Filtering reduced the number of genes from 7070 to 425. S

Threshold

Graphs C show the score plot after the filtering.

Graphs D show the loading plot after the filtering.

Compare ..

Dramatic reduction from the initial 7070 genes to the 425, finally retained, resulted in a minimal information loss relevant to the description of the samples in the reduced space.

Compare ..

Three linear structures can be identified in the loadingplot of the 425 genes selected by the above analysis.

Each structure comprising a set of genes.

PCA – Discussion PCA has strong, yet flexible, mathematical

structure. PCA simplifies the “views” of the data. Reduces dimensionality of gene expression

space. The correspondence between the score plot and

the loading plot enables the elimination of redundant variables.

PCA allowed the classification of new samples belonging to the used types of tissues.

PCA – Discussion (Cont.)

In the article this method facilitated the identification of strong underlying structures in the data. The identification of such structures is uniquely dependent on the data and is not generally guaranteed.

No “correct” way of classification, “biological understanding” is the ultimate guide.

My Critics

Positives Can deal with large data sets. There weren’t done any assumptions on the

data. This method is general and may be applied to any data set.

Negatives Nonlinear structure is invisible to PCA The meaning of features is lost when linear

combinations are formed

True covariance matrices are usually not known, estimated from data.

The Graph : First component will be

chosen along the largest variance line => both clusters will strongly overlap.

Projection to orthogonal axis to the first PCA component will give much more discriminating power.

Thank you !!!Thank you !!!

09/05/2005 סמינריון במתמטיקה ביולוגית Dimension Reduction - PCA Principle...

Documents

Transcript of 09/05/2005 סמינריון במתמטיקה ביולוגית Dimension Reduction - PCA Principle...

1 Principle of 2-D Electrophoresis 1. First dimension: denaturing isoelectric focusing separation according to the isoelectric point 2. Second dimension:

Dimension reduction : PCA and Clustering

PCA and Autoencoderspeople.cs.pitt.edu/~milos/courses/cs3750/lectures/class7.pdf•Provides a roadmap for dimension reduction •Used for: •Visualization •Preprocessing •Compression

BINF6201/8201 Principle components analysis (PCA) -- Visualization of amino acids using their physico-chemical properties 08-31-2010.

Visualization Slides by Damon WischikGeron(2017). Hands-on Machine Learning with Scikit-Learn & TensorFlow. 7. Dimension reduction / PCA PCA is a tool that takes in high-dimensional

Effective PCA for high-dimension, low-sample-size data ...

Principle Component Analysis (PCA) A mechanism used to make the analysis of remote sensing data simpler….

Metabolomics - unc.edu · Metabolomics Nature Reviews: Drug ... Principal Components Analysis (PCA): This is a data dimension reduction method that involves a mathematical procedure

Noise & Data Reduction. Paired Sample t Test Data Transformation - Overview From Covariance Matrix to PCA and Dimension Reduction Fourier Analysis - Spectrum.

Dimension reduction : PCA and Clustering - CBS · Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU

Dimension Reduction using PCA and SVD - UCSD DSE MAS · Dimension Reduction using PCA and SVD. Plan of Class Starting the machine Learning part of the course. Based on Linear Algebra.

The Price of Fair PCA: One Extra Dimension › pdf › 1811.00103.pdf · with running PCA, and PCA does a better job representing men than women, or white people over minorities,

SpectralMethodsforLearning LatentVariableModels: … · 2015. 3. 27. · Learning through Spectral Clustering Dimension reduction through PCA (on data matrix) Clustering on projected

Latent Semantic Analysisberlin.csie.ntnu.edu.tw/Courses/Information Retrieval and...Latent Semantic Analysis: Schematic • Dimension Reduction and Feature Extraction – PCA – SVD

Scalable Probabilistic PCA - ww2.amstat.org › meetings › sdss › 2018 › online... · Scalable Probabilistic PCA PCA Probabilistic PCA Results Probabilistic PCA EM algorithm:

Supplemental Information Supplemental Figures...Fig. S7 Principle Component Analysis (PCA) and ontology (GO) enrichment analyses of transcriptomes in developing seeds. A, PCA analysis

Dimensionality Reduction and Principle Components Analysiswcohen/10-601/601-pca.pdf · • Principle Components Analysis (PCA) – Example (Bishop, ch 12) – PCA vs linear regression

DIMENSION REDUCTION 1 Principal Component Analysis (PCA)larry/=sml/dimension_reduction.pdf · DIMENSION REDUCTION 1 Principal Component Analysis (PCA) Principal components analysis

Dimension reduction : PCA and Clustering Agnieszka S. Juncker Slides: Christopher Workman and Agnieszka S. Juncker Center for Biological Sequence Analysis.

Dimension Reduction using PCA and SVD - UCSD DSE MAS · 2020. 8. 12. · Dimension Reduction using PCA and SVD. Plan of Class Starting the machine Learning part of the course. Based