Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen]...

99
3/1/05 1 Bioinformatics (Lec 15) Gene Expression Process of transcription and/or translation of a gene is called gene expression. Every cell of an organism has the same genetic material, but different genes are expressed at different times. Patterns of gene expression in a cell is indicative of its state.

Transcript of Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen]...

Page 1: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 1Bioinformatics (Lec 15)

Gene Expression• Process of transcription and/or translation

of a gene is called gene expression.• Every cell of an organism has the same

genetic material, but different genes are expressed at different times.

• Patterns of gene expression in a cell is indicative of its state.

Page 2: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 2Bioinformatics (Lec 15)

Hybridization• If two complementary strands of DNA or

mRNA are brought together under the right experimental conditions they will hybridize.

• A hybridizes to B ⇒– A is reverse complementary to B, or – A is reverse complementary to a subsequence of

B.• It is possible to experimentally verify

whether A hybridizes to B, by labeling A or Bwith a radioactive or fluorescent tag, followed by excitation by laser.

Page 3: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 3Bioinformatics (Lec 15)

Measuring gene expression• Gene expression for a single gene can be

measured by extracting mRNA from the cell and doing a simple hybridization experiment.

• Given a sample of cells, gene expression for every gene can be measured using a single microarray experiment.

Page 4: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 4Bioinformatics (Lec 15)

Microarray/DNA chip technology• High-throughput method to study gene expression

of thousands of genes simultaneously.• Many applications:

– Genetic disorders & Mutation/polymorphism detection– Study of disease subtypes– Drug discovery & toxicology studies– Pathogen analysis– Differing expressions over time, between tissues,

between drugs, across disease states

Page 5: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 5Bioinformatics (Lec 15)

Microarray DataGene Expression Level

Gene1

Gene2

Gene3

Page 6: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 6Bioinformatics (Lec 15)

Page 7: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 7Bioinformatics (Lec 15)

Microarray/DNA chips (Simplified)• Construct probes corresponding to reverse

complements of genes of interest.• Microscopic quantities of probes placed on solid

surfaces at defined spots on the chip.• Extract mRNA from sample cells and label them.• Apply labeled sample (mRNA extracted from cells)

to every spot, and allow hybridization.• Wash off unhybridized material.• Use optical detector to measure amount of

fluorescence from each spot.

Page 8: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 8Bioinformatics (Lec 15)

Gene Chips

Page 9: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 9Bioinformatics (Lec 15)

Affymetrix DNA chip schematic

www.affymetrix.com

Page 10: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 10Bioinformatics (Lec 15)

What’s on the slide?

Page 11: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 11Bioinformatics (Lec 15)

DNA Chips & Images

Page 12: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 12Bioinformatics (Lec 15)

Page 13: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 13Bioinformatics (Lec 15)

Microarrays: competing technologies• Affymetrix & Synteni/Stanford• Differ in:

– method to place DNA: Spotting vs. photolithography

– Length of probe– Complete sequence vs. series of fragments

Page 14: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 14Bioinformatics (Lec 15)

How to compare 2 cell samples with Two-Color Microarrays?

• mRNA from sample 1 is extracted and labeled with a red fluorescent dye.

• mRNA from sample 2 is extracted and labeled with a green fluorescent dye.

• Mix the samples and apply it to every spot on the microarray. Hybridize sample mixture to probes.

• Use optical detector to measure the amount of green and red fluorescence at each spot.

Page 15: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 15Bioinformatics (Lec 15)

http://www.arabidopsis.org/info/2010_projects/comp_proj/AFGC/RevisedAFGC/Friday/

Page 16: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 16Bioinformatics (Lec 15)

Sample

Treated Sample(t1) Expt 1 Treated Sample(t2) Expt 2Treated Sample(t3) Expt 3…Treated Sample(tn) Expt n

Study effect of treatment over time

Page 17: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 17Bioinformatics (Lec 15)

• Variations in cells/individuals.• Variations in mRNA extraction, isolation, introduction of

dye, variation in dye incorporation, dye interference.• Variations in probe concentration, probe amounts, substrate

surface characteristics• Variations in hybridization conditions and kinetics• Variations in optical measurements, spot misalignments,

discretization effects, noise due to scanner lens and laser irregularities

• Cross-hybridization of sequences with high sequence identity.

• Limit of factor 2 in precision of results.

Sources of Variations & Errors

Need to Normalize data

Page 18: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 18Bioinformatics (Lec 15)

Types of bias/variation• Intensity & Range

– Variation changes with intensity. Larger variation at lower end.

• Spatial – Spot location changes expression

• Plate– Printing plate changes expression

http://www.arabidopsis.org/info/2010_projects/comp_proj/AFGC/RevisedAFGC/Friday/index.htm

Page 19: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 19Bioinformatics (Lec 15)

Clustering• Clustering is a general method to study

patterns in gene expressions. • Several known methods:

– Hierarchical Clustering (Bottom-Up Approach)– K-means Clustering (Top-Down Approach)– Self-Organizing Maps (SOM)

Page 20: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 20Bioinformatics (Lec 15)

Hierarchical Clustering: Example

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8 9 10

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8 9 10

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8 9 10

Page 21: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 21Bioinformatics (Lec 15)

A Dendrogram

Page 22: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 22Bioinformatics (Lec 15)

Hierarchical Clustering [Johnson, SC, 1967]• Given n points in Rd, compute the distance

between every pair of points• While (not done)

– Pick closest pair of points si and sj and make them part of the same cluster.

– Replace the pair by an average of the two sij

Try the applet at:http://www.cs.mcgill.ca/~papou/#applet

Page 23: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 23Bioinformatics (Lec 15)

Distance Metrics• For clustering, define a distance function:

– Euclidean distance metrics

– Pearson correlation coefficient

k=2: Euclidean Distancekd

i

kiik YXYXD

/1

1)(),( ⎥⎦

⎤⎢⎣

⎡−= ∑

=

⎟⎟⎠

⎞⎜⎜⎝

⎛ −⎟⎟⎠

⎞⎜⎜⎝

⎛ −= ∑

= y

i

x

id

ixy

YYXXd σσ

ρ1

1-1 ≤ ρxy ≥ 1

Page 24: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 24Bioinformatics (Lec 15)

Page 25: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 25Bioinformatics (Lec 15)

Clustering of gene expressions• Represent each gene as a vector or a point in

d-space where d is the number of arrays or experiments being analyzed.

Page 26: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 26Bioinformatics (Lec 15)

From Eisen MB, et al, PNAS 1998 95(25):14863-8

Clustering Random vs. Biological Data

Page 27: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 27Bioinformatics (Lec 15)

Page 28: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 28Bioinformatics (Lec 15)

Page 29: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 29Bioinformatics (Lec 15)

Page 30: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 30Bioinformatics (Lec 15)

K-Means Clustering: Example

Example from Andrew Moore’s tutorial on Clustering.

Page 31: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 31Bioinformatics (Lec 15)

Start

Page 32: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 32Bioinformatics (Lec 15)

Page 33: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 33Bioinformatics (Lec 15)

Page 34: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 34Bioinformatics (Lec 15)

Start

End

Page 35: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 35Bioinformatics (Lec 15)

K-Means Clustering [McQueen ’67]Repeat– Start with randomly chosen cluster centers– Assign points to give greatest increase in score– Recompute cluster centers– Reassign pointsuntil (no changes)

Try the applet at: http://www.cs.mcgill.ca/~bonnef/project.html

Page 36: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 36Bioinformatics (Lec 15)

Comparisons• Hierarchical clustering

– Number of clusters not preset.– Complete hierarchy of clusters– Not very robust, not very efficient.

• K-Means– Need definition of a mean. Categorical data?– More efficient and often finds optimum

clustering.

Page 37: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 37Bioinformatics (Lec 15)

Functionally related genes behave similarly across experiments

Page 38: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 38Bioinformatics (Lec 15)

Self-Organizing Maps [Kohonen]• Kind of neural network.• Clusters data and find complex relationships

between clusters.• Helps reduce the dimensionality of the data.• Map of 1 or 2 dimensions produced.• Unsupervised Clustering• Like K-Means, except for visualization

Page 39: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 39Bioinformatics (Lec 15)

SOM Architectures• 2-D Grid• 3-D Grid• Hexagonal Grid

Page 40: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 40Bioinformatics (Lec 15)

SOM Algorithm• Select SOM architecture, and initialize

weight vectors and other parameters.• While (stopping condition not satisfied) do

for each input point x– winning node q has weight vector closest to x.– Update weight vector of q and its neighbors.– Reduce neighborhood size and learning rate.

Page 41: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 41Bioinformatics (Lec 15)

SOM Algorithm Details

• Distance between x and weight vector:• Winning node: • Weight update function (for neighbors):

• Learning rate:

iwx −

)]()()[,,()()1( kwkxixkkwkw iii −+=+ µ

ii

wxxq −= min)(

⎟⎟

⎜⎜

⎛ −−= 2

2)(

exp)(),,( 0

σηµ

xqi rrkixk

Page 42: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 42Bioinformatics (Lec 15)

World Bank Statistics• Data: World Bank statistics of countries in

1992. • 39 indicators considered e.g., health,

nutrition, educational services, etc. • The complex joint effect of these factors

can can be visualized by organizing the countries using the self-organizing map.

Page 43: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 43Bioinformatics (Lec 15)

World Poverty PCA

Page 44: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 44Bioinformatics (Lec 15)

World Poverty SOM

Page 45: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 45Bioinformatics (Lec 15)

World Poverty Map

Page 46: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 46Bioinformatics (Lec 15)

Page 47: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 47Bioinformatics (Lec 15)

Page 48: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 48Bioinformatics (Lec 15)

Viewing SOM Clusters on PCA axes

Page 49: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 49Bioinformatics (Lec 15)

1

5

4

3

2

1

2 3 4 5 6

SOM Example [Xiao-rui He]

Page 50: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 50Bioinformatics (Lec 15)

Neural Networks

ΣInput X

SynapticWeights W

ƒ(•)

Bias θ

Output y

Page 51: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 51Bioinformatics (Lec 15)

Learning NN

×

×

× Σ

Σ

Adaptive Algorithm

Input X

1

Weights W

−+

Desired Response

Error

Page 52: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 52Bioinformatics (Lec 15)

Types of NNs• Recurrent NN• Feed-forward NN• Layered

Other issues• Hidden layers possible• Different activation functions possible

Page 53: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 53Bioinformatics (Lec 15)

Application: Secondary Structure Prediction

Page 54: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 54Bioinformatics (Lec 15)

Support Vector Machines• Supervised Statistical Learning Method for:

– Classification– Regression

• Simplest Version:– Training: Present series of labeled examples

(e.g., gene expressions of tumor vs. normal cells)– Prediction: Predict labels of new examples.

Page 55: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 55Bioinformatics (Lec 15)

A AB A

AB B

B

Learning Problems

Page 56: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 56Bioinformatics (Lec 15)

SVM – Binary Classification• Partition feature space with a surface.• Surface is implied by a subset of the

training points (vectors) near it. These vectors are referred to as Support Vectors.

• Efficient with high-dimensional data. • Solid statistical theory• Subsume several other methods.

Page 57: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 57Bioinformatics (Lec 15)

Learning Problems• Binary Classification• Multi-class classification• Regression

Page 58: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 58Bioinformatics (Lec 15)

Page 59: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 59Bioinformatics (Lec 15)

Page 60: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 60Bioinformatics (Lec 15)

Page 61: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 61Bioinformatics (Lec 15)

SVM – General Principles• SVMs perform binary classification by

partitioning the feature space with a surface implied by a subset of the training points (vectors) near the separating surface. These vectors are referred to as Support Vectors.

• Efficient with high-dimensional data. • Solid statistical theory• Subsume several other methods.

Page 62: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 62Bioinformatics (Lec 15)

SVM Example (Radial Basis Function)

Page 63: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 63Bioinformatics (Lec 15)

SVM Ingredients• Support Vectors• Mapping from Input Space to Feature Space• Dot Product – Kernel function• Weights

Page 64: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 64Bioinformatics (Lec 15)

Classification of 2-D (Separable) data

Page 65: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 65Bioinformatics (Lec 15)

Classification of(Separable) 2-D data

Page 66: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 66Bioinformatics (Lec 15)

Classification of (Separable) 2-D data

•Margin of a point•Margin of a point set

+1

-1

Page 67: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 67Bioinformatics (Lec 15)

xi

Separatorw•x + b = 0

w•xi + b > 0

w•xj + b < 0

xj

Classification using the Separator

Page 68: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 68Bioinformatics (Lec 15)

Perceptron Algorithm (Primal)

Given separable training set S and learning rate η>0 w0 = 0; // Weightb0 = 0; // Biask = 0; R = max 7xi7repeat

for i = 1 to N if yi (wk•xi + bk) ≤ 0 then

wk+1 = wk + ηyixibk+1 = bk + ηyiR2

k = k + 1Until no mistakes made within loopReturn k, and (wk, bk) where k = # of mistakes

Rosenblatt, 1956

w = Σ aiyixi

Page 69: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 69Bioinformatics (Lec 15)

Theorem: If margin m of S is positive, then

i.e., the algorithm will always converge, and will converge quickly.

Performance for Separable Data

k ≤ (2R/m)2

Page 70: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 70Bioinformatics (Lec 15)

Perceptron Algorithm (Dual)Given a separable training set S a = 0; b0 = 0; R = max 7xi7repeat

for i = 1 to N if yi (Σaj yj xi•xj + b) ≤ 0 then

ai = ai + 1b = b + yiR2

endifUntil no mistakes made within loopReturn (a, b)

Page 71: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 71Bioinformatics (Lec 15)

Non-linear Separators

Page 72: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 72Bioinformatics (Lec 15)

Main idea: Map into feature space

Page 73: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 73Bioinformatics (Lec 15)

Non-linear Separators

X F

Page 74: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 74Bioinformatics (Lec 15)

Useful URLs• http://www.support-vector.net

Page 75: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 75Bioinformatics (Lec 15)

Perceptron Algorithm (Dual)Given a separable training set S a = 0; b0 = 0; R = max 7xi7repeat

for i = 1 to N if yi (Σaj yj k(xi ,xj) + b) ≤ 0 then

ai = ai + 1b = b + yiR2

Until no mistakes made within loopReturn (a, b)

k(xi ,xj) = Φ(xi)• Φ(xj)

Page 76: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 76Bioinformatics (Lec 15)

Different Kernel Functions• Polynomial kernel

• Radial Basis Kernel

• Sigmoid Kernel

dYXYX )(),( •=κ

⎟⎟

⎜⎜

⎛ −−= 2

2

2exp),(

σκ

YXYX

))(tanh(),( θωκ +•= YXYX

Page 77: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 77Bioinformatics (Lec 15)

SVM Ingredients• Support Vectors• Mapping from Input Space to Feature Space• Dot Product – Kernel function

Page 78: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 78Bioinformatics (Lec 15)

Generalizations• How to deal with more than 2 classes?

Idea: Associate weight and bias for each class.• How to deal with non-linear separator?

Idea: Support Vector Machines.• How to deal with linear regression?• How to deal with non-separable data?

Page 79: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 79Bioinformatics (Lec 15)

Applications• Text Categorization & Information Filtering

– 12,902 Reuters Stories, 118 categories (91% !!)• Image Recognition

– Face Detection, tumor anomalies, defective parts in assembly line, etc.

• Gene Expression Analysis• Protein Homology Detection

Page 80: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 80Bioinformatics (Lec 15)

Page 81: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 81Bioinformatics (Lec 15)

Page 82: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 82Bioinformatics (Lec 15)

Page 83: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 83Bioinformatics (Lec 15)

SVM Example (Radial Basis Function)

Page 84: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 84Bioinformatics (Lec 15)

Sources of Variations & Errors in Microarray Data

• Variations in cells/individuals.• Variations in mRNA extraction, isolation,

introduction of dye, variation in dye incorporation, dye interference.

• Variations in probe concentration, probe amounts, substrate surface characteristics

• Variations in hybridization conditions and kinetics• Variations in optical measurements, spot

misalignments, discretization effects, noise due to scanner lens and laser irregularities

• Cross-hybridization of sequences with high sequence identity.

• Limit of factor 2 in precision of results.Need to Normalize data

Page 85: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 85Bioinformatics (Lec 15)

Significance Analysis of Microarrays (SAM) [Tusher, Tibshirani, Chu, PNAS’01]

• Fold change is a typical measure to decide genes of interest.

• However, variations in gene expression are also gene dependent. If repeats are available, then such variations can be measured for each gene. This helps to give a better analysis of significant genes of interest.

Page 86: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 86Bioinformatics (Lec 15)

Genomics• Study of all genes in a genome, or comparison of

whole genomes.– Whole genome sequencing– Whole genome annotation & Functional genomics– Whole genome comparison

• PipMaker: uses BLASTZ to compare very long sequences (> 2Mb); http://www.cse.psu.edu/pipmaker/

• Mummer: used for comparing long microbial sequences (uses Suffix trees!)

Page 87: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 87Bioinformatics (Lec 15)

Genomics (Cont’d)– Gene Expression

• Microarray experiments & analysis– Probe design (CODEHOP)– Array image analysis (CrazyQuant)– Identifying genes with significant changes (SAM)– Clustering

Page 88: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 88Bioinformatics (Lec 15)

Proteomics• Study of all proteins in a genome, or

comparison of whole genomes.– Whole genome annotation & Functional

proteomics– Whole genome comparison– Protein Expression: 2D Gel Electrophoresis

Page 89: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 89Bioinformatics (Lec 15)

2D Gel Electrophoresis

Page 90: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 90Bioinformatics (Lec 15)

Other Proteomics ToolsFrom ExPASy/SWISS-PROT:• AACompIdent identify proteins from aa composition[Input: aa composition, isoelectric point, mol wt., etc. Output: proteins from DB]• AACompSim compares proteins aa composition with other proteins• MultIdent uses mol wt., mass fingerprints, etc. to identify proteins• PeptIdent compares experimentally determined mass fingerprints with

theoretically determined ones for all proteins• FindMod predicts post-translational modifications based on mass difference

between experimental and theoretical mass fingerprints.• PeptideMass theoretical mass fingerprint for a given protein.• GlycoMod predicts oligosaccharide modifications from mass difference• TGREASE calculates hydrophobicity of protein along its length

Page 91: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 91Bioinformatics (Lec 15)

Databases for Comparative Genomics• PEDANT useful resource for standard questions in

comparative genomics. For e.g., how many known proteins in XXX have known 3-d structures, how many proteins from family YYY are in ZZZ, etc.

• COGs Clusters of orthologous groups of proteins.• MBGD Microbial genome database searches for

homologs in all microbial genomes

Page 92: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 92Bioinformatics (Lec 15)

Gene Networks & Pathways• Genes & Proteins act in concert and

therefore form a complex network of dependencies.

Page 93: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 93Bioinformatics (Lec 15)

Pathway Example from KEGG

Staphylococcus aureus

Page 94: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 94Bioinformatics (Lec 15)

Pseudomonas aeruginosa

Page 95: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 95Bioinformatics (Lec 15)

STSs and ESTs• Sequence-Tagged Site: short, unique

sequence• Expressed Sequence Tag: short, unique

sequence from a coding region– 1991: 609 ESTs [Adams et al.] – June 2000: 4.6 million in dbEST– Genome sequencing center at St. Louis produce

20,000 ESTs per week.

Page 96: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 96Bioinformatics (Lec 15)

What Are ESTs and How Are They Made?

• Small pieces of DNA sequence (usually 200 - 500 nucleotides) of low quality.

• Extract mRNA from cells, tissues, or organs and sequence either end. Reverse transcribe to get cDNA(5’ EST and 3’EST) and deposit in EST library.

• Used as "tags" or markers for that gene. • Can be used to identify similar genes from other

organisms (Complications: variations among organisms, variations in genome size, presence or absence of introns).

• 5’ ESTs tend to be more useful (cross-species conservation), 3’ EST often in UTR.

Page 97: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 97Bioinformatics (Lec 15)

DNA Markers• Uniquely identifiable DNA segments.• Short, <500 nucleotides.• Layout of these markers give a map of

genome.• Markers may be polymorphic (variations

among individuals). Polymorphism gives rise to alleles.

• Found by PCR assays.

Page 98: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 98Bioinformatics (Lec 15)

Polymorphisms• Length polymorphisms

– Variable # of tandem repeats (VNTR)– Microsatellites or short tandem repeats– Restriction fragment length polymorphism (RFLP) caused

by changes in restriction sites.• Single nucleotide polymorphism (SNP)

– Average once every ~100 bases in humans– Usually biallelic– dbSNP database of SNPs (over 100,000 SNPs)– ESTs are a good source of SNPs

Page 99: Gene Expressionusers.cis.fiu.edu/~giri/teach/Bioinf/S05/Lecx5.pdf · Self-Organizing Maps [Kohonen] • Kind of neural network. • Clusters data and find complex relationships between

3/1/05 99Bioinformatics (Lec 15)

SNPs• SNPs often act as “disease markers”, and

provide “genetic predisposition”.• SNPs may explain differences in drug

response of individuals.• Association study: study SNP patterns in

diseased individuals and compare against SNP patterns in normal individuals.

• Many diseases associated with SNP profile.