Applications of Image Processing to Sign Language Recognition
description
Transcript of Applications of Image Processing to Sign Language Recognition
![Page 1: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/1.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Applications of Image Processing to Sign
Language Recognition
![Page 2: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/2.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Overview Average person typing speed
Composing: 19 wpm Transcribing: 33 wpm
Sign speaker Full sign language: 200—220 wpm Spelling: est. 50 wpm
![Page 3: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/3.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Purpose and Scope Native signers faster “typing”
Benefits: Hearing & speaking disabled Sign interpreters
Just letters & numbers
![Page 4: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/4.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Research Original Related projects
Using mechanical gloves Tracking body parts
Image techniques Edge detection Line detection
![Page 5: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/5.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Program Architecture (1)
Webcam interface Using Video 4 Linux 1 Saves images to disk
Edge detection Robert’s Cross
Line detection Hough transform
![Page 6: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/6.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Program Architecture (2)
Line processing Image data dropped AI to find “best fit”
Letter matching Load hand data from XML files Try to match hand
![Page 7: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/7.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Sample Code From edge_detect.c : int row,col;// loop through the imagefor(row=0; row<rows; row++) { for(col=0; col<cols; col++) { // avoid the problems from trying to calculate on an edge if(row==0 || row==rows-1 || col==0 || col==cols-1) { image2[row][col]=0; // so set value to 0 } else { int left = image1[row][col-1]; // some variables int right = image1[row][col+1]; // to make the final int top = image1[row-1][col]; // equation seem a int bottom = image1[row+1][col]; // little bit neater image2[row][col]=(int)(sqrt(pow(left – right , 2) + pow(top - bottom, 2))); } }}
![Page 8: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/8.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Testing (1) General testing model
bhood@bulusan: ~/syslab-tech $ \> ./main images/hand.pgm in.pgm
[DEBUG] Edge detect time: 486 ms[DEBUG] Wrote image file to `in.pgm’
Errors: 0 Warnings: 0
![Page 9: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/9.jpg)
Byron HoodComputer Systems Lab Project 2007-08
Testing (2) Results:
Of edge detection
Similar for cropping or line finding
Automated testing difficult and impractical
Original image(800 x 703)
Final image(800 x 703)
![Page 10: Applications of Image Processing to Sign Language Recognition](https://reader036.fdocuments.in/reader036/viewer/2022082517/56813aea550346895da35258/html5/thumbnails/10.jpg)
Byron HoodComputer Systems Lab Project 2007-08
The Mysterious Future Finish line detection
Detection and parameterization Build AI to interpret lines
Finish camera-computer interaction
OPTIMIZE!!!