Computer Vision in Cell BiologyWhy bioimage informatics matters

Paper Talk

Chen Yang

1 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

2 / 27

Author: Introduction

GaudenzDanuser

Before moving to Harvard Medical School in fall 2009,Gaudenz directed research groups at ETH Zurich andat The Scripps Research Institute in La Jolla. Heentered the field of cell biology as a postdoctoral fellowin the Program for Architectural Dynamics of LivingCells at the MBL in Woods Hole. Since then, he hasbeen fascinated by the complex interface betweenchemical and mechanical signaling and structuraladaptation that dictates the regulation of cell function.His teaching activities focus on computational cellbiology, molecular biomechanics and measurementtheory applied to cell biological analyses.

3 / 27

Author: Lab

4 / 27

Author: Publications

• Reis GF, Yang G, Szpankowski L, Weaver C, Shah SB, Robinson JT, Hays TS,Danuser G, Goldstein LS. Molecular motor function in axonal transport in vivoprobed by genetic and computational analysis in Drosophila. Mol Biol Cell. 2012May;23(9):1700-14.

• Tkachenko E, Sabouri-Ghomi M, Pertz O, Kim C, Gutierrez E, Machacek M,Groisman A, Danuser G, Ginsberg MH. Protein kinase A governs aRhoA-RhoGDI protrusion-retraction pacemaker in migrating cells. Nat Cell Biol.2011 Jun;13(6):660-7.

• Welch CM, Elliott H, Danuser G, Hahn KM. Imaging the coordination of multiplesignalling activities in living cells. Nat Rev Mol Cell Biol. 2011 Oct21;12(11):749-56.

•• Applegate KT, Besson S, Matov A, Bagonis MH, Jaqaman K, Danuser G.

plusTipTracker: Quantitative image analysis software for the measurement ofmicrotubule dynamics. J Struct Biol. 2011 Nov;176(2):168-84.

• Matov A, Edvall MM, Yang G, Danuser G. Optimal-Flow Minimum-CostCorrespondence Assignment in Particle Flow Tracking. Comput Vis ImageUnderst. 2011 Apr;115(4):531-540.

•• Danuser G. Computer vision for systems biology. Biomedical Imaging: Nano to

Macro, 2006. 3rd IEEE International Symposium on. 6-9 April 2006.

5 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

6 / 27

Difference between image processing and image analysis(computer vision)

• Image processing transform the signal to emphasize a particular aspect of animage. (e.g. contrast and color enhancement, deconvolution, image registration,filtering, and even segmentation.)

• Image analysis interpret the image content, assigning meaning to image events.•• The goal of image processing is to manipulate the images in order to better see

what needs to be seen.• The goal of Image analysis is to replace the human observer, at least in part.•• The output of image processing is an image.• The output of a computer vision program is an abstract representation, a model

of a particular aspect of the image or of the entire image content. subsequentdecision procedures can be made through the model.

•• image-processing programs run through a fixed sequence of algorithms

independent of the input image.• computer vision programs adjust the strategy of visual information processing to

the properties of the images and to the prior knowledge of the images.

7 / 27

Typical applications of Computer Vision

• control of robots and autonomous vehicles,

• video surveillance and inspection,

• organization of image databases,

• object modeling,

• and human-machine interfaces.

8 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

9 / 27

Early and Recent Uses of Computer Vision in Cell BiologicalStudies

• karyotyping 1

• counting or tracking cells 2 (e.g. neutrophil tracking)

• identify phenotypes in cell morphology and subcellular proteindistribution in large-scale pharmacological or genomic screens 3

1Gilbert, 1966; Castleman et al., 19762Farnoush, 1977; Lewandow- ska et al., 19793Bakal et al., 2007; Carpenter et al., 2006; Colli- net et al., 2010;

Neumann et al., 2010; Perlman et al., 2004; Vitorino and Meyer, 200810 / 27

What CV brings

• Automation

• Completeness

• Access Invisible Information

11 / 27

Completeness: example

12 / 27

Access Invisible Information

Model Convolution

• images

• model that can explain the images

• estimate the parameters of the model by the images

13 / 27

Access Invisible Information: example

investigate metaphase kinetochore dynamics in thebudding yeast when individual kinetochores are notresolvable.

4

4Sprague et al., 200314 / 27

Access Invisible Information: example

force fluctuations during epithelial cell migration

5

5Ji et al., 200815 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

16 / 27

Human vision and Computer vision

17 / 27

Human vision and Computer vision

Why Human Vision Often Outperforms Computer Vision

Human vision system identifies relevant events associates theobservable image signals with our memory of previous visualexperiences and the size of this memory is enormous.

Human vision

• interpolate missing information due to low signals

• combine associations with different visual experiences and tointegrate information from different spatial scales

• generalization power (e.g. we can recognize the overallorganization of cells although we have never seen that particularspecimen, for instance, multinucleated, mitotic, and apoptoticcells among the many cell types present in the scene.)

18 / 27

Human vision and Computer vision

Computer vision

• objective

I when analyzing images by eye our focus of attention isdirected to events that match our expectation

I every individual may have a slightly different focus, leadingto nonreproducible data.

• quantitative

I similarly looking yet distinct image events is hard to beidentified by human vision.

• completeness

I exceedingly rare but mechanistically significant imageevents.

19 / 27

Where Computer Vision Can Outperform Human Vision

• subcellular location proteomics

I image texture produced by the labeled proteins can only becharacterized by a large number of features, human canhardly integrate all this at once.

• particle tracking

I Human vision: greedy assignment.I Computer vision (Multi-Hypothesis- Tracking (MHT)

algorithm): global optimum (evaluate all possible pathes).

The key scenario wherein current computer vision systems bearsignificant advantages over human vision: When a decision requiresintegrated analysis of a large number of possible partial solutions in awell-defined but multidimensional space, the computer will fare better.

20 / 27

particle tracking

21 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

22 / 27

What can CV do in cell biology

• object recognition / classification

• existing pattern recognition / classification (e.g. distribution;density)

• object tracking

• moving pattern recognition / classification

• model convolution

•

• What do you think that’s can be done ???

23 / 27

Computer Vision in Cell BiologyAuthorDifference between image processing and computer visionComputer Vision in Cell BiologyHuman vision and Computer visionSummarization

Why bioimage informatics mattersAuthor

24 / 27

Author: Introduction

EugeneMyers

• Eugene "Gene" Wimberly Myers, Jr. (born 1953) isan American computer scientist andbioinformatician, who is best known forcontributing to the early development of the NCBI’sBLAST tool for sequence analysis.

• Gene was voted the most influential inbioinformatics in 2001 by Genome TechnologyMagazine and was elected to the NationalAcademy of Engineering in 2003. In 2004 he wonthe International Max-Planck Research Prize.

• His current research interests includecomputational reconstructions of neuroanatomicaldata, as well as algorithms for analysis offunctional neuroscience data.

• Eugene Myers will lead new Systems BiologyCenter in Dresden of Max Planck Institute.

25 / 27

main point

information of spatial and physical factors in the control andfunction of cellular functions must be detected by bioimageinformatics

26 / 27

Nature Methods focus

http://www.nature.com/nmeth/journal/v9/n7/index.html#fo

27 / 27

Computer vision resources

• Nature Methods focus (new and biology related):http://www.nature.com/nmeth/journal/v9/n7/index.html#fo

• coferrence papers (best CV papers, CV frontier):http://www.cvpapers.com/

• citeseerx (CS papers with times cited):http://citeseerx.ist.psu.edu/index

• visionbib (Annotated/classified Computer Vision Bibliography):http://www.visionbib.com/bibliography/contents.html

• dblp (computer science bibliography) :http://www.informatik.uni-trier.de/ ley/db/indices/a-tree/index.html

28 / 27