Application of light fields in computer visionAMARI LEWIS – REU STUDENT

AIDEAN SHARGHI- PH.D STUENT

Main objective increase object recognition through using the EPI of light field images

Using the light field camera

Using the Lytro light field camera

conventional methods- involve using 2D information Light field images- captures all 3D information in a single shot. Using the Lytro light field camera to collect dataset camera captures light field direction, intensity and color

Datasets- 1. Collected own dataset using the Lytro light field camera

◦ Bikes◦ Buildings◦ Trees◦ Vehicles

- Studying the 7 different image perspectives

2. Dataset from Switzerland using the iphone video◦ Buildings

– 50 categories- Ranging from 4-30 videos- Extracted 300 frames from each video

Epipolar planar images- EPI It is a 2D representation or slice of an image

Taking the same line from each image and putting it on top of each other

Using the multiple shots taken from the camera and the extracted frames

Light field

7 lines from each of the images concatenated- total of 1080

Concatenated the 300 lines – total 720

Implementing DCT Steps: Separate the RGB into 3 channels

Calculate the row-wise mean- calculates the mean of each row to create a vector

Calculate the DCT for each channels

Concatenate some coefficients, using as a feature vector (smaller)

For classification Apply Principal component analysis (PCA) gmm- Gaussian mixture model Linear SVM

Best Results Using this method on EPIs

◦ Lytro Light field camera dataset 77% accuracy

• Switzerland dataset 96% accuracy

Thank you