Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.
-
date post
19-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.
![Page 1: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/1.jpg)
Interactive Visual System
By
Arthur Evans, John Sikorski, and Patricia Thomas
![Page 2: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/2.jpg)
The Client
Graduate student and professor at Rensselaer Polytechnic Institute
Implemented CAVIAR on a desktop PC
Interested in a mobile version
![Page 3: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/3.jpg)
CAVIAR
CAVIAR George Nagy, DocLab RPI, Troy, NY 12180
13
![Page 4: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/4.jpg)
The Idea
Image recognition is difficult Humans excel at segmentation Computers excel at large scale memory and
calculations Combine the two for optimal performance
Faster than a layperson and more accurate than a fully automatic system
![Page 5: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/5.jpg)
Requirements
Demonstrate user interaction with an image on a handheld computer.
Allow for three methods of user interaction Pointing to image Outline part of image User response to queries
Top three matches must be displayed with target image.
Create a log file to log user actions and time.
![Page 6: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/6.jpg)
Hardware
![Page 7: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/7.jpg)
Our Handheld Criteria
Must run java
Must take a camera attachment
Must take a wireless LAN card
![Page 8: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/8.jpg)
Personal Digital Assistants (PDA’s) PDA’s run Personal Information Management
(PIM) software. Address books Calendars
Limited computational power. Examples:
Palm Pilot Handspring Visor
![Page 9: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/9.jpg)
Handheld Computers
Extensive computational power. Designed as portable computers. Fill the gap between desktop/laptop PC’s and
PDA’s. Examples:
Compaq Ipaq series Sharp Zaurus
![Page 10: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/10.jpg)
Hardware Selection
Sharp Zaurus SL-5500 200 MHz Intel StrongArm processor 64 MB memory 1 Compact Flash port and 1 SD port
Sharp CE-AG06 camera attachment Plugs into Compact Flash port
Linksys wireless LAN card (WCF11) 802.11b Plugs into Compact Flash port
![Page 11: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/11.jpg)
Design
![Page 12: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/12.jpg)
Data Flow Diagram
HandheldCamera
PDA Stylus Pad
User commands and data
Photo of flower
PDA screen display
Server/Pattern Recognition Software
FlowerDataobject
Encapsulate Flower Image & results of user interaction
into flowerData object
Transmit flowerData Object to Remote
Server
Load user selected photo
Store image as local file
Transmit receive confirmation back to
user
Transmit classification decision to user
![Page 13: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/13.jpg)
Package Structure
GUI – implemented with AWT MainGUI class provides layout for panels Each ‘action’ implemented as a class that extends java.awt.Panel Additional supporting classes
Data FlowerData class encapsulates image and data to send to
server. FlowerMatch class encapsulates image matches from recognition
engine. Additional supporting classes
Remote - access recognition engine via RMI Well defined protocol Modular design Consists of several server and client side classes
![Page 14: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/14.jpg)
Java 2 Micro Edition (J2ME)
Recent specification
Provides Java functionality to a broad range of devices
Uses concept of Configurations and Profiles
![Page 15: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/15.jpg)
Basic J2ME Architecture
Operating System
Java Virtual Machine
Configuration
Profile
![Page 16: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/16.jpg)
Configurations
Tied to JVM on a low level Consider memory, processor, display and
connectivity Currently 2 are defined
Connected, Limited Device Protocol (CLDC) Connected Device Protocol (CDC)
![Page 17: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/17.jpg)
Profiles
Sits on top of a Configuration Provides implementation API Currently 2 are defined:
Mobile Information Device Protocol (MIDP) Typically implemented on PDA’s and cellular phones
Personal Profile Targets high end handheld Reference implementation available for the Zaurus
![Page 18: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/18.jpg)
Personal Java
Implementation by Insignia Systems – Jeode
Not a J2ME specification
Based on JDK 1.1.8
Implements most Personal Profile specifications
![Page 19: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/19.jpg)
Summary
Successfully demonstrated user interaction on a handheld
Wireless connection to server Created structure for IVS application
Easily extendible Identify different objects by changing database
and data points Exceeds client expectations
![Page 20: Interactive Visual System By Arthur Evans, John Sikorski, and Patricia Thomas.](https://reader031.fdocuments.in/reader031/viewer/2022032704/56649d3e5503460f94a17068/html5/thumbnails/20.jpg)
Next Semester
Add pattern recognition Call C++ code with JNI? Rewrite in java?
Add segmentation Automatic User interaction