Virtual Image PeepholeBy

Kyle PatienceSupervisor: Reg Dodds

Co Supervisor: Mehrdad Ghaziasgar

Quick Recap

• Development of an interface that does the following: • A very large virtual screen is

imagined to exist, • The mobile phone screen is used

as a "peephole" into that interface.

Overview

•Design Decisions and System Changes• Implementation •Finding and Tracking Features•User Interface •Tools

Design Decisions and System Changes

PREVIOSLY CURRENTLY

• No NDK • Uses NDK

• Use Android camera API • Uses device’s camera natively

• Didn’t use sensor readings • Uses both accelerometer and gyroscope

• Display was modifies camera frames • 3D interface

NDK

Implementation

Get frames Process frames

Get sensor readings

Output

Finding and Tracking Features• The Shi-Tomasi Corner Detector

• Based on the Harris Corner Detector. • Tries to find little patches of image that generate a large variation when moved around.• Ultimately finds small corners in a frame.

• Lucas & Kanade Method• Technique which can provide an estimate of the movement of certain features in successive images of a

scene.• Algorithm makes a "best guess" of the displacement of a neighbourhood by looking at changes in pixel

intensity.• These intensities are known through intensity gradients of the image in that neighbourhood.

User Interface using camera framesMoving the device without tilt will move the interface

User Interface using gyroscope

Tilting phone to the left Tilting phone to the right

User Interface using accelerometer

Tilting phone upwards

Tilting phone downwards

ToolsPlatform Windows 8.1 x64

Applications Android Studio

SDK Android SDKOpenCV Android SDKAndroid NDK

Libraries OpenCVjPCT

Languages JavaC++

Project Plan

Term 1 Learn OpenCV Learn Android

Term 2Learn NDKBuild Android PrototypeCapture Footage

Term 3Process FramesProcess Accelerometer and Gyroscope ReadingsCreate 3D InterfaceIntergrade all components

Term 4ImplementingTuningTesting

References

• Ali, S. I., Jain, S., Lal, B., & Sharma, N. (2012). A framework for modelling and designing of intelligent and adaptive interfaces for human computer interaction. International Journal of Applied Information Systems (IJAIS) Volume.

• Anuar, A., Saipullah, K. M., Ismail, N. A., & Soo, Y. (2011). OpenCV based real-time video processing using android smartphone. International Journal of Computer Technology and Electronics Engineering (IJCTEE), 1(3).

• Shi, J., & Tomasi, C. (1994, June). Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp. 593-600). IEEE.

• Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. Pittsburgh: School of Computer Science, Carnegie Mellon Univ..

Questions?