1Department of Electrical and Computer Engineering

Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz

Ian McAlister

Colin Smith

Chris Neyland

Erick Drummond

TEAM GANZ Comprehensive Design

Review

2Department of Electrical and Computer Engineering

OUTLINE Project Update Summary CDR Deliverables Review Final Product Design

Beagle Board Hardware

Enclosure, Battery, Interface, Lighting Image Processing

Experimental Design Extensive testing of chosen algorithms on PC Implement Chosen Algorithm on Beagle Board Demonstration Proposed Demonstration for FPR and SDP Day Conclusion and Pathway to FPR

3Department of Electrical and Computer Engineering

Project Update Summary Product Requirements

Low Cost Intuitive Interface Design – 2 to 3

buttons max Small Form Factor - <50in3

Battery Life – One Hour Continuous Use Performance - <10 sec, >90% Accuracy Upgradable Bill Library

4Department of Electrical and Computer Engineering

Project Update Summary

System Specifications: Bill Reading Device The Bill Reader Device Estimated Specifications:

1 Processing Unit: Beagle Board Commercial Off-The-Shelf (COTS) Camera: PS3

EYE Speaker LED Lights SD Card Microcontroller and a few Buttons for User

Interface Low Power Usage System standby when not in use

5Department of Electrical and Computer Engineering

Project Update Summary MDR Milestones

Beagle Board - Developed Image to support Webcam Drivers, USB Keyboard, Audio

Playback, Implementation of compilers, Open CV Support

Image Processing Algorithms (Description on Page One of Handout)

Autorotation Template Matching

6Department of Electrical and Computer Engineering

CDR Deliverables

Final Product Design Beagle Board Hardware

Enclosure, Battery, Interface, Lighting Image Processing Algorithm

Experimental Design Extensive testing of chosen algorithms on PC Implement Chosen Algorithm on BeagleBoard

Conduct time measurements Demonstration

Current Status and Demonstration

7Department of Electrical and Computer Engineering

Final Product Design – Beagle Board Beagle Board Integration Challenges:

Difficult to Implement Code on Beagle Board Subtle Differences in OS Caused Issues with Coding Subtle Differences in Platform Caused Similar Issues

Needed to set up a Fast, Stable, and Intuitive Development Environment SSH Used to Speed up Testing Cycle Caused Audio Playback Issues

Significant Time Delay in Template Matching

8Department of Electrical and Computer Engineering

Final Product Design – User Interface Status of Intuitive Interface

Breadboard Setup Using Atmega168 to Simulate Keyboard Coding/Debugging in Progress

Need to Finalize and Back-up the Drive Image Ensure all Drivers for Auxiliary Devices are Loaded at

Startup

9Department of Electrical and Computer Engineering

Final Product Design – Hardware Hardware

Lithium Ion Battery Pack 2 x 3.4V Re-chargeable Batteries 2200mAh

Need to Disassemble Hub Finish Intuitive Interface Complete Breadboard Wiring of auxiliary devices Research Ideal Speaker Fabricate Enclosure

10Department of Electrical and Computer Engineering

Final Product Design – Hardware

Development Model: Production Model:

Major Differences:

Production model removes Keyboard, Monitor, and Ethernet

USB Hub, Microcontroller, and Voltage Regulator will be condensed into a smaller PCB

11Department of Electrical and Computer Engineering

Final Product Design – Image Processing

Find Best Match by Using Normalized Cross-Correlation Coefficients Function (NCC)

C = Correlation Matrix I = Captured Image, = Sample Image Mean at location: u,v T = Template Image, = Template Mean u,v = Indices at Point of Correlation

Best Match Retrieved by OpenCV Function: cvMinMaxLoc( image, &minval, &maxval, &minloc, &maxloc, 0 );

Where maxval is a number from -1 to +1 corresponding to best match.

This Value Can then Be Used to Determine if There is a Match

yx yxvu

vuyx

TyxTIyvxuI

TyxTIyvxuIvuC

, ,

22,

,,

]),([]),([

]),(][),([),(

12Department of Electrical and Computer Engineering

Experimental Design

Metrics Time False Positives False Negatives Correct Identification

Variables – Reduced From Change in Design Degraded Glass Surface Lighting Degraded Bill Condition

13Department of Electrical and Computer Engineering

Experimental Design

Metrics

Experiment Timing False Postives Correct ID False Negatives

1. Ideal Conditions       

2. Degraded Glass Surface       

2a) Fingerprints       

2b) Scratches       

2c) Oily type substance       

3. Unlevel Surfaces       

4. Degraded Bill Condition       

4a) Crumpled Bill       

4b) Faded Bill       

4c) Written on Bill       

4d) Missing or Folded Corner

       

5. Varying Conditions - No Fixed conditions

       

14Department of Electrical and Computer Engineering

Results From Testing Initial Lighting Bad

Initial LEDs cause reflection and non-uniform illumination Solution

High Intensity to be located away from Camera Large Degree of Illumination for uniform lighting

When “Scanned” Image Deviates from Center Correct Identification Significantly Reduced Solution

Flip BATBRD Upside Down Guide Bill using “rails” BATBRD Upside Down with Rails to Guide User

15Department of Electrical and Computer Engineering

Results From Testing (cont.) Implementation of Image Processing Algorithm on Beagle Board

> 20 seconds With Only Partial Bill Library Solution

Reducing pixels by a factor of 2 = Reduction of 15 seconds. (Increased reduction may occur).

Remove Autorotation Algorithm Remove Image Normalization from Algorithm

16Department of Electrical and Computer Engineering

Final Prototype Show a picture of all the devices again only highlight the

devices we plan to hack and reduce in size

17Department of Electrical and Computer Engineering

Final Prototype Put the 3D CAD Picture here make sure that we have

either recorded rotating or have the program up to rotate it

18Department of Electrical and Computer Engineering

Miniaturization of Prototype Talk about how we plan to reach an eventual wallet sized

device

19Department of Electrical and Computer Engineering

FPR and SDP Demonstration Proposal FPR Demonstration

Using Final Prototype including Project Enclosure Request Evaluators to close eyes and use BATBRD

Scan Different types of Bills Ideal Faded or Degraded Bill Defaced Bill

SDP Day Demonstration Jenny has volunteered to demonstrate/discuss the BATBRD Include blind fold for others who wish to test the device

20Department of Electrical and Computer Engineering

Conclusion Testing Revealed Many Issues

Lighting – Solved by Specific Lighting Scheme Scanned Image Not Centered reducing correct identification Image Processing on Beagle Board Slower than expected

Reduce Pixel Size Optimize Algorithm Abandon Autorotation Change Design

21Department of Electrical and Computer Engineering

Conclusion and Pathway to FPR Continue optimizing Algorithms to reduce time while ensuring

>90% correct identification. Develop large database of testing results

Disassemble and customize other devices to ensure small form factor

Develop Project Enclosure Speak with Jenny and Product Developers regarding

design and material to use Ensure Device can still be easily troubleshot once

enclosed

22Department of Electrical and Computer Engineering

Gantt Chart

ID Task Name Start Finish DurationJan 2010 Feb 2010 Mar 2010

1/3 1/10 1/17 1/24 1/31 2/7 2/14 2/21 2/28

1 .4w1/5/20101/4/2010Complete

Experimental Design

2 2.2w1/19/20101/5/2010Consider Alternative

Algorithms

4 3.2w1/26/20101/5/2010Extensive Testing of

Algorithms

6 1w2/1/20101/26/2010Implement Algorithms

on BeagleBoard

7 4w2/22/20101/26/2010Determine Battery

Design

8 4w2/22/20101/26/2010Develop User

Interface

10 2w3/5/20102/22/2010Finalize Project

Enclosure Design

13 3w4/15/20103/26/2010Test Using Finalized

Product

Apr 2010

3/7 3/14 3/21 3/28 4/4 4/11

3 2.2w1/19/20101/5/2010Determine Lighting

Setup

11 3.2w3/26/20103/5/2010Construct Enclosure and Integrate Parts

12 .2w3/26/20103/26/2010Product Complete

5 0w1/26/20101/26/2010Algorithm Chosen

9 0w2/22/20102/22/2010All Hardware Choices

Made

1/5/2010

3/5/2010

4/15/2010

3/26/2010

3/26/2010

2/22/2010

23Department of Electrical and Computer Engineering

FPR Deliverables

List FPR Deliverables here

24Department of Electrical and Computer Engineering

Questions ?