Object Orie’d Data Analysis, Last Time Statistical Smoothing –Histograms – Density Estimation...

Object Orie’d Data Analysis, Last Time

• Statistical Smoothing– Histograms – Density Estimation

– Scatterplot Smoothing – Nonpar. Regression

• SiZer Analysis– Replaces bandwidth selection

– Scale Space

– Statistical Inference:

Which bumps are “really there”?

– Visualization

Kernel Density EstimationChoice of bandwidth (window width)?• Very important to performance

Fundamental Issue:Which modes are “really there”?

SiZer BackgroundFun Scale Spaces Views (Incomes

Data)Surface View

SiZer BackgroundSiZer analysis of British Incomes data:

SiZer BackgroundFinance "tick data":

(time, price) of single stock transactions

Idea: "on line" version of SiZerfor viewing and understanding trends

SiZer BackgroundFinance "tick data":

(time, price) of single stock transactions

Idea: "on line" version of SiZerfor viewing and understanding trends

Notes: • trends depend heavily on scale • double points and more • background color transition

(flop over at top)

SiZer BackgroundInternet traffic data analysis:SiZer analysis oftime series of packet timesat internet hub (UNC)

Hannig, Marron,

and Riedi (2001)

SiZer BackgroundInternet traffic data analysis:

SiZer analysis oftime series of packet times

at internet hub (UNC)• across very wide range of scales • needs more pixels than screen allows • thus do zooming view

(zoom in over time) – zoom in to yellow bd’ry in next frame – readjust vertical axis

SiZer BackgroundInternet traffic data analysis (cont.)

Insights from SiZer analysis:

• Coarse scales:

amazing amount of significant structure

• Evidence of self-similar fractal type process?

• Fewer significant features at small scales

• But they exist, so not Poisson process

• Poisson approximation OK at small scale???

• Smooths (top part) stable at large scales?

Dependent SiZer

Rondonotti, Marron, and Park (2007)

• SiZer compares data with white noise

• Inappropriate in time series

• Dependent SiZer compares data with

an assumed model

• Visual Goodness of Fit test

Dep’ent SiZer : 2002 Apr 13 Sat 1 pm – 3

pm

Zoomed view (to red region, i.e. “flat top”)

Further Zoom: finds very periodic behavior!

Possible Physical Explanation

IP “Port Scan”• Common device of hackers• Searching for “break in points”• Send query to every possible

(within UNC domain):– IP address– Port Number

• Replies can indicate system weaknesses

Internet Traffic is hard to model

SiZer OverviewWould you like to try a SiZer analysis? • Matlab software: http://www.unc.edu/depts/statistics/postscript/papers/marron/Matlab6Software/Smoothing/

• JAVA version (demo, beta): Follow the SiZer link from the Wagner Associates home page:

http://www.wagner.com/www.wagner.com/SiZer/

• More details, examples and discussions:

http://www.stat.unc.edu/faculty/marron/DataAnalyses/

SiZer_Intro.html

PCA to find clustersReturn to PCA of Mass Flux Data:

PCA to find clustersSiZer analysis of Mass Flux, PC1

PCA to find clustersSiZer analysis of Mass Flux, PC1

Conclusion:

• Found 3 significant clusters!

• Correspond to 3 known “cloud types”

• Worth deeper investigation

Recall Yeast Cell Cycle Data

• “Gene Expression” – Micro-array data

• Data (after major preprocessing): Expression “level” of:

• thousands of genes (d ~ 1,000s)

• but only dozens of “cases” (n ~ 10s)

• Interesting statistical issue:

High Dimension Low Sample Size data

(HDLSS)

Yeast Cell Cycle Data, FDA View

Central question:Which genes are “periodic” over 2 cell cycles?


Periodic genes?

Naïve approach:Simple

PCA


• Central question: which genes are “periodic” over 2 cell cycles?

• Naïve approach: Simple PCA• No apparent (2 cycle) periodic structure?• Eigenvalues suggest large amount of

“variation”• PCA finds “directions of maximal

variation”• Often, but not always, same as

“interesting directions”• Here need better approach to study

periodicities

Yeast Cell Cycles, Freq. 2 Proj.

PCA on

Freq. 2

Periodic

Component

Of Data

Frequency 2 Analysis

Frequency 2 Analysis• Project data onto 2-dim space of sin and

cos (freq. 2)

• Useful view: scatterplot

• Angle (in polar coordinates) shows phase

• Colors: Spellman’s cell cycle phase classification

• Black was labeled “not periodic”

• Within class phases approx’ly same, but notable differences

• Now try to improve “phase classification”

Yeast Cell CycleRevisit “phase classification”, approach:• Use outer 200 genes

(other numbers tried, less resolution)• Study distribution of angles• Use SiZer analysis

(finds significant bumps, etc., in histogram)

• Carefully redrew boundaries• Check by studying k.d.e. angles

SiZer Study of Dist’n of Angles

Reclassification of Major Genes

Compare to Previous Classif’n

New Subpopulation View

OODA in Image Analysis

First Generation Problems:

• Denoising

• Segmentation (find object

boundaries)

• Registration (align objects)

(all about single images)

OODA in Image Analysis

Second Generation Problems:

• Populations of Images

– Understanding Population Variation

– Discrimination (a.k.a.

Classification)

• Complex Data Structures (& Spaces)

• HDLSS Statistics

HDLSS Data in Image Analysis

Why HDLSS (High Dim, Low Sample Size)?

• Complex 3-d Objects Hard to Represent– Often need d = 100’s of parameters

• Complex 3-d Objects Costly to Segment– Often have n = 10’s of cases

Image Object Representation

Major Approaches for Images:

• Landmark Representations

• Boundary Representations

• Medial Representations

Landmark Representations

Main Idea:

• On each object find important points

• Treat point locations as features

• I.e. represent objects by vectors of point locations (in 2-d or 3-d)

(Fits in OODA framework)

Landmark RepresentationsBasis of Field of Statistical Shape

Analysis:

(important precursor of FDA & OODA)

Main References:

• Kendall (1981, 1984)

• Bookstein (1984)

• Dryden and Mardia (1998)

(most readable and comprehnsive)

Landmark RepresentationsNice Example:

• Fly Wing Data (Drosophila fruit flies)

• From George Gilchrist, W. & M. U.

http://gwgilc.people.wm.edu/

• Graphic Illustrating Landmarks (next page)– Same veins appear in all flies

– And always have same relationship

– I.e. all landmarks always identifiable

http://gwgilc.people.wm.edu/

Landmark RepresentationsLandmarks for fly wing data:

Landmark RepresentationsImportant issue for landmark approaches:

Location, i. e. Registration

Illustration with Fly Wing Data (next slide)

Problem:

• coordinates are “locations in photo”

• & unclear where wing is positioned…


Illustration of Registration, with Fly Wing Data

Landmark RepresentationsStandard Approach to Registration

Problem:

Procrustes Analysis

Idea: mod out location

• Can also mod out rotation

• Can also mod out size

Recommended reference:

Dryden and Mardia (1988)


Procustes Results for Fly Wing Data

Landmark RepresentationsEffect of Procrustes Analysis:

Study Difference Between Continents• Flies from Europe & South America• Look for important differences• Project onto mean difference

direction• Visualize with movie

– Equal time spacing– Through range of data

Landmark RepresentationsNo Procrustes Adjustment:

Movies on Difference Between Continents



• Raw Data– Driven by location effects– Strongly feels size– Hard to understand shape

Landmark RepresentationsLocation, Rotation, Scale Procrustes:




• Raw Data– Driven by location effects– Strongly feels size– Hard to understand shape

• Full Procrustes– Mods out location, size, rotation– Allows clear focus on shape


Major Drawback of Landmarks:

• Need to always find each landmark

• Need same relationship

• I.e. Landmarks need to correspond

• Often fails for medical images

• E.g. How many corresponding landmarks on a set of kidneys, livers or brains???


Landmarks for brains???

(thanks to

Liz Bullit)

Very hard to

identify

Landmark RepresentationsLook across people:

Some structurein common

But “folds” are different

ConsistentLandmarks???

Boundary Representations

Major sets of ideas:

• Triangular Meshes– Survey: Owen (1998)

• Active Shape Models– Cootes, et al (1993)

• Fourier Boundary Representations– Keleman, et al (1997 & 1999)

Boundary Representations

Example of triangular mesh rep’n:

From:www.geometry.caltech.edu/pubs.html

http://www.geometry.caltech.edu/pubs.html

Boundary RepresentationsExample of triangular mesh rep’n for a

brain:

From: meshlab.sourceforge.net/SnapMeshLab.brain.jpg

http://meshlab.sourceforge.net/SnapMeshLab.brain.jpg

Boundary RepresentationsMain Drawback:

Correspondence

• For OODA (on vectors of parameters):

Need to “match up points”

• Easy to find triangular mesh

– Lots of research on this driven by gamers

• Challenge match mesh across objects

– There are some interesting ideas…

Medial RepresentationsMain Idea: Represent Objects as:• Discretized skeletons (medial atoms)• Plus spokes from center to edge• Which imply a boundary

Very accessible early reference:• Yushkevich, et al (2001)

Medial Representations2-d M-Rep Example: Corpus Callosum(Yushkevich)

Medial Representations2-d M-Rep Example: Corpus Callosum(Yushkevich)

AtomsSpokesImpliedBoundary

Medial Representations3-d M-Rep Example: From Ja-Yeon Jeong

Bladder – Prostate - Rectum

Atoms - Spokes - Implied Boundary

Medial Representations3-d M-reps: there are several variations

Two choices:From Fletcher(2004)

Medial RepresentationsStatistical Challenge

• M-rep parameters are:– Locations– Radii– Angles (not comparable)

• Stuffed into a long vector• I.e. many direct products of

these

32 , 0

Medial RepresentationsStatistical Challenge:• How to analyze angles as data?• E.g. what is the average of:

– ??? (average of the numbers)– (of course!)

• Correct View of angular data:Consider as points on the unit circle

1811

359,358,4,3

Medial RepresentationsWhat is the average (181o?) or (1o?) of:

359

,358

,4

,3

Medial RepresentationsStatistical Analysis of Directional Data:• Common Examples:

– Wind Directions (0-360)– Magnetic Fields (0-360)– Cracks (0-180)

• There is a literature (monographs):– Mardia (1972, 2000)– Fisher, et al (1987, 1993)

Medial RepresentationsStatistical Challenge• Many direct products of:

– Locations– Radii– Angles (not comparable)

• Appropriate View:Data Lie on Curved Manifold

Embedded in higher dim’al Eucl’n Space

32 , 0

Medial RepresentationsData on Curved Manifold Toy Example:

Medial RepresentationsData on Curved Manifold Viewpoint:• Very Simple Toy Example (last movie)• Data on a Cylinder = • Notes:

– Simplest non-Euclidean Example– 2-d data, embedded on manifold in – Can flatten the cylinder, to a plane– Have periodic representation– Movie by: Suman Sen

• Same idea for more complex direct prod’s

11 S

3R

Object Orie’d Data Analysis, Last Time Statistical Smoothing –Histograms – Density Estimation...

Documents

Transcript of Object Orie’d Data Analysis, Last Time Statistical Smoothing –Histograms – Density Estimation...