Embedded Optical Braille Recognition on Tamil Braille … · 2014-08-28 · Embedded Optical...

Embedded Optical Braille Recognition on Tamil Braille System

using Raspberry Pi

1Capt.Dr.S Santhosh Baboo, Associate Professor

2V.Ajantha Devi, Ph.D Research Scholar

P.G.& Research Dept of Computer Science

D.G.Vaishnav College,

Chennai Tamilnadu, India. [email protected]

[email protected]

Abstract

Optical Braille recognition is used to digitize and

reproduce texts that have been produced with non-

computerized systems, such as with Braille

typewriters. Digitizing Braille texts also helps reduce

storage space, as Braille texts take up much more

space than their natural language counterparts.

Editing and Reprinting of Braille document that were

embossed on paper are time consuming and labour

intensive. Optical Braille recognition is also useful

for people who cannot read Braille, but need to

access the content of Braille documents. This paper

is on Methodology of a camera based assistive device

that can be used by people to read Braille document.

The framework is on implementing image capturing

technique in an embedded system based on

Raspberry Pi board.

1. Introduction

The Braille system is a method that is used by

visually impaired people to read and write which is

invented in 1825 by Louis Braille. The preferred

means of communication for a visually impaired is

through Tactile Braille system, where Tactile means

sense by touching or rubbing the surface of the

corresponding output device.

With the invention of computers, several

approaches to computerized Braille translation have

been developed and thus this makes computerized

Braille[26] is a recently new phenomenon with the

Braille script. Many applications have been created

for translation from Braille to Speech. However,

there is a shortage of programs designed for Tamil

Braille.

The written communications barrier between sighted

and visually impaired persons is particularly apparent

in the schooling system, where, nowadays, visually

impaired students are taught in mainstream classes.

Many of these students perform assessment, tests and

homework writing using the Braille system.

However, most staff of these students are not Braille

literate. Problems also exist in the workplace where

any information written by a Braille user that is to be

interpreted by other Braille illiterate persons, needs to

be first translated by the Braille user.

To overcome this communications barrier, the

device developed in this project provides a means by

which Braille illiterate persons can read what a

Braille user has written. The device is essentially a

hand held scanner that capture the image of the Tamil

Braille and converts into equivalent decoded Tamil

text which is capable of doing the extraction[24] of

Tamil Braille Character from a Braille document

followed by decoding them into Tamil Characters

and then the decoded characters are normalized to

Tamil text are also then converted into audio output.

2. Tamil Braille System

The Braille system[9] comprises of a “cell” per

character and these themselves consist of six

embossed or raised dots arranged in a rectangle

containing two columns of three dots each as shown

in Figure 1. A dot may be raised at any of the six

positions to form 2 6

(64) combinations (including the

combination in which no dots are raised). For

reference purposes, a particular combination may be

described by naming the positions where dots are

raised, the positions being universally numbered 1

V Ajantha Devi et al, Int.J.Computer Technology & Applications,Vol 5 (4),1566-1574

IJCTA | July-August 2014 Available [email protected]

1566

ISSN:2229-6093

http://f5mail.rediff.com/ajaxprism/writemail?mode=mail_to_individual&[email protected]&els=48cfb7097459b28786bea31609aaa215

mailto:[email protected]

http://en.wikipedia.org/wiki/Digitize

http://en.wikipedia.org/wiki/Braille_typewriter



through 3 from top to bottom on the left, and 4

through 6 from top to bottom on the right.

Figure 1. Braille cell

The Braille code system has become widely used by

several communities because of its simplicity,

comfortable for blinds to use it when read and write.

Braille was applied or translated into several

languages including Tamil language[16].

Figure 2. Tamil Braille cell[1]

Tamil alphabet has descended from the Brahmi

script of ancient India. It is a syllabic language which

contains 12 vowels and 18 consonants in its script.

The vowels and consonants combine to form 216

compound characters, as this combination turns out

be in big number, two Braille cells are needed to

represent these combination. Thus only 38 cells have

been assigned for Tamil Braille as shown in Figure 2.

3. System Hardware Design

The whole system is composed by following parts:

an image capturing camera[27][35], Raspberry Pi

board to run image recognition programs on it and a

Headphone to deliver the output speech. The system

block diagram is shown in Figure 3.

Figure 3. System Hardware Block Diagram

A. Raspberry pi

The Raspberry Pi[20] is a credit card sized

single-board computer developed in the UK by the

Raspberry Pi Foundation with the intention of

stimulating the teaching of basic computer science in

schools. Raspberry Pi contains a comprehensive

amount of features packed in a very reasonable price.

Raspberry Pi is constructed for Broadcom's

BCM2835 system around the circuit, which includes

a 700 MHz ARM11 family processor and include

250 MHz clock -frequency Broadcom's Video

Graphics Core IV. Memory B-model is 512 MB, and

it is divided into the graphics card.

Figure 4. View of Working Hardware system

B. Raspberry pi Camera

The camera module used in this project is

Raspberry PI NOIR (No IR) CAMERA BOARD[19]

as shown in the Figure 5. The camera plugs directly

into the Camera Serial Interface (CSI) connector on

the Raspberry Pi. It‟s able to deliver clear 5MP

resolution image, or 1080p HD video recording at

30fps. The module attaches to Raspberry Pi, by way



1567

ISSN:2229-6093

of a 15 pin Ribbon Cable, to the dedicated 15 pin

MIPI CSI, which was designed especially for

interfacing to cameras. The CSI bus is capable of

extremely high data rates, and it exclusively carries

pixel data to the BCM2835 processor.

C. Enabling the camera

Open the raspi-config tool from the Terminal:

sudo raspi-config

1. Select Enable camera and hit Enter,

2. then go to Finish and you'll be prompted to

reboot.

Figure 5. Raspberry PI NOIR (No IR) CAMERA

BOARD

This camera board which has no infrared filter

making it perfect for taking infrared photographs or

photographing objects in low light (twilight)

conditions. Other features of this camera board are

Automatic image control functions Programmable

controls for frame rate 32 bytes of embedded one

time programmable (OTP) memory and Digital video

port (DVP) parallel output interface Excellent

D. Take a picture

The following code will take a single picture and

save it to 'foo.jpg'.

import time

import picamera

with picamera.PiCamera() as camera:

camera.resolution = (1024, 768)

camera.start_preview()

# Camera warm-up time

time.sleep(2)

camera.capture('foo.jpg

E. Storage (Memory)

The design does not include a built in hard disk or

solid state drive, instead relying on an SD card for

booting and long term storage. This board is intended

to run Linux kernel based operating systems. This

Raspberry Pi module has a Samsung class 4 micro

SD card preloaded with the official Raspberry Pi

NOOBS (New Out of Box Software) package, and a

beautifully screen printed Micro SD card adaptor.

The system designed can be operated in two sessions,

ie one for capturing and creating a data base and the

other session is to capture the image and which can

be used for identifying or comparing the images in

the database.

4. System Software Design

The system that is able to translate a Braille script

into Tamil script and represents the converted script

as text or voice to the user passes through several

stages including image capturing or scanning,

converting the Image to Gray Level, image

Thresholding (converting it to binary), de-skewing

the image, edging the Braille dots, filling Braille dots,

opening the image by removing all micro scale

objects, cropping the cell frames, cropping the dot

frames from cell frames, generating binary equivalent

of activated Braille dots, generating equivalent

decimal Braille code, apply matching algorithm to

Braille frames, get equivalent text and voice file of



1568

ISSN:2229-6093

matched Braille cell[28] stored in database.

Figure 6. System Software Block Diagram

Braille document is a page that written using

Braille alphabets. In this project the design based on

standard specification of Tamil Braille writing, so to

make sure that the Braille page which used in

implementation will meet the standards

specifications.

The test image has captured by using Raspberry

PI NOIR (No IR) CAMERA BOARD and the image

store in JPEG format.

i) Image pre-processing

Gray scale and binary conversion: Gray scale and

binary conversion using rgb2gray and im2bw matlab

functions

Figure 7. scanned test image

ii) Converting the Image to Gray Level

Inside a computer system, colored images

are stored in 3-D arrays while gray level images are

stored in 2-D arrays. Dealing with 2-D arrays is much

easier and faster.

Figure 8. Gray level of Figure 7

Therefore, the first step in the proposed

system converts colour scanned images to gray level

so that any pixel value in the image falls within the

range 0- 255 .

iii) Cropping the Image Frame

The scanned images suffer from having

either black or white frames that would affect the

thresholding step coming next. It is a must that those

frames are cropped before proceeding forward. To do

that, the average gray level is calculated for the

whole image and then for each of the rows and

columns separately. Finding a row or column average

gray level that is above or below 15% of the whole

image average gray level is an indication to delete it

as experiences have shown.

iv) Image Thresholding



1569

ISSN:2229-6093

An algorithm was developed to handle the

thresholding[33]. This algorithm works as follows;

after converting the image to gray level and removing

highest and lowest values to leave out distinct values,

the average gray level for the whole image[25] is

calculated.

Figure 9 Image after adaptive thresholding

v) Image De-Skewing

Thus a binary search algorithm was

developed to correct any de-skewing in tilted scanned

images like Figure 10. The maximum degree of

recognizing a de-skewed image is 4 degrees from

either the left or the right side.

Figure 10. Crop effect on skewed image

Figure 11. Crop effect on skewed Braille image

Figure 12. Finding the deviation degree level

Crop effect on skewed Braille[29] image as

in Figure 11 evaluated. After determining the

deviation degree as in Figure 12, rotate the original

gray image and then Horizontal projection is

performed to count the number of pixels in each row

for the selected part in the previous step. Then the

rows having more than 10 pixel points was

calculated.

Figure 13. Cropping on skewed image

The image is rotated 4 degrees one time to

the left and one to the right for the rotated image as

by Figure 13. This is done because if the image is

skewed then cropping may remove some of the

image parts that constitute Braille cells as could be

seen in Figure 14.

Figure 14. Image after croppe and skewed



1570

ISSN:2229-6093

vi) Dot Parts Detection

Experiences have shown that detection of Braille dot

[32] parts is better than detection of the whole dot. It

was also proven that the average dot height is 8

pixels. Each dot is composed of a bright and a dark

region with a small space between them. Oyama et al.

[36] detected both the verso [34] (depression) and

recto (Protrusions) Braille dots utilizing shadow

patterns The testing image analyzed by using Single

side Braille Document by computing centroids on

dot[15].

vii) Whole Dot Detection

The framing process[30] to determine the

cells, words and lines in the scanned image based on

standard dimensions of Braille documents , the cell

frame size is known and experimentally recorded as

95X80 pixels in resolution of 200 dpi. Also the dot

size recorded as 20X15 pixels in resolution of 200

dpi.

Figure 15 shows the dots at the end of the search.

viii) Braille letter recognition and transcription

In this stage of the project the Braille letter

was recognized using matching algorithm to match

each of the input decimal Braille code from an input

processed image with codes of each Tamil letter by

Figure 16. Then the image is converted to its decimal

code representation:

Decimal-Code=b1+b2*2+b3*4+b4*8+b5*16+ b6*32

After recognition process implemented the

recognized letter transcript into equivalent text file

through addressing process to run matched equivalent

files from stored addressed database.

Figure 16. ASCII Code for each cell

ix) Conversion to text

This is the last stage in implementation, the

word recognition process flow through letter

recognition[31] and fill Braille decimal array of

word, to apply matching process with stored

addressed text files of words, then to run equivalent

files from addressed word database

Each Braille character is coded into a six bit

binary number based on the presence or absence of

dots. The presence of a dot is indicated by 1 and its

absence by 0. This step is a simple one to one

mapping from a database containing the binary code

and the letter corresponding to this representation.

x) Text To Speech scope

The scope of this module is initiated with

the conclusion of the receding module of Braille

Recognition. The module performs the task of

conversion of the transformed Tamil text to audible

form.



1571

ISSN:2229-6093

The Raspberry Pi has an on-board audio

jack, the on-board audio is generated by a PWM

output and is minimally filtered. a USB audio card

can greatly improve the sound quality and volume.

Two options of attaching a microphone into

Raspberry Pi. One is to have USB mic, another to

have an external USB sound card.

To enable USB audio output, load the sound driver:

sudo modprobe snd_bcm2835

Enable USB audio output by default

sudo nano /etc/asound.conf

To playback: aplay test.wav

To adjust some volumes: alsamixer

5. Experimental Analysis

The methodology used in this paper is

analysised by using the Tamil Braille System on

Thirukural Braille Book. Tirukkural is a classic

Tamil sangam literature consisting of 1330 couplets

or Kurals. It was authored by a Jain ascetic

Thiruvalluvar, a poet who is said to have lived

anytime between 2nd century BCE and 5th century

CE. This particularly apparent in the schooling

system. Following are the snapshots on result of the

analysis.

A couplet or Kural consists of seven cirs, with

four cirs on the first line and three on the second. A

cir is a single or a combination of more than one

Tamil word. For example, Thirukkural is a cir

formed by combining the two words Thiru and

Kural, i.e. Thiru + Kural = Thirukkural

Figure 17. Braille image of 1st Kural

Figure 18. Text conversion of the Figure 17

Figure 19. Braille image of a tilted document

Figure 20. Text conversion of the Figure 19

Figure 21. Thirukural in 3 lines

The standard form of thirukural, couplet or

Kural consists of seven cirs, with four cirs on the

first line and three on the second. When embossed in

Braille format if the space in not sufficient for the

forth word in the first line, it is embossed in second

line as shown in the Figure 21 underline in blue and



1572

ISSN:2229-6093

the three cirs in the second line be embossed in the

third line.

The Final process of the above methodology

is complete as the above text is synthesized and

converted in to audio format and played using

microphone or mini speaker connecting to on-board

audio jack of Raspberry Pi.

6. Conclusion

Dealing with images in term of image

processing issue it is not an easy task. Adaptive

thresholding technique that has been used to separate

the Braille dots from the background is an effective

technique and it gives a very good result for more

than 90% from the images. Morphology techniques

can help to enhance the image from a noise. The

captured image always has a skew angle (or the

image has a rotated angle in 3rd

axis).

Braille has been developed as the reading

and writing system for the visually impaired. The

attention was given on this is very difficult to teach a

visually impaired people in the early stage and more

training is needed for teaching them and converting

Braille to text, is costly and cumbersome work.

7. Acknowledgment

The author wish to thank the Christian

Foundation for the Blind- India, Chennai for the

support. The above paper is part of UGC Major

Research Project of the Principal Investigator

Capt.Dr.S Santhosh Baboo

8. Reference

[1] Bharati Braille URL

http://acharya.iitm.ac.in/disabilities/bh_brl.php

[2] J.Bhattacharya, S.Majumder, G.Sanyal,

“Automatic inspection of Braille Character: A

vision based approach.” International Journal of

computer & Organization trends – Volume 1

Issue 3 – 2011.

[3] BRL: Braille through Remote Learning -

http://brl.org/refdesk/brlcont.html

[4] Abdallah M.Abualkishik, and Khairuddin Omar

“Quranic Baille System”, World Academy of

Science, Engineering and Technology,4-6,2008.

[5] Mohammad Abu Yousu and S.M.Sai Shams, “Bangla Braille Inormation System: An

Affordable System for the Sightless Population”, Asian Journal of Information Technology 6 (6):

2007, pp.696-699.

[6] Al-Salman(2003), “Towards a Computerized

Arabic Braille environment,PDF”, King Saudi

University.

[7] Ebnet, Matthew (2001), "Braille Challenge

Gives Young Blind Students a Chance to Shine",

Los Angeles Times,

http://articles.latimes.com/2001/jun/30/local/me-

16960.

[8] State of Queensland (Department of

Education).(2000). Literate futures: Report of the

literacy review for Queensland state Schools,

Queensland, Australia.

[9] Swenson,A.M. (1999). “Beginning with Braille”.

NewYork: AFB Press.

[10] P.Blenkhorn,"A System for Converting Braille

into Print," IEEE Transactionson Rehabilitation

Engineering, vol.3, pp.215-221, 1995.

[11] Trent,S.D. &Truan, M.B (1997). “Speed,

accuracy, and comprehension of adolescent

braille readers in a speialized school”. Journal of

Visual Impairment and Blindness,91, pp.494-

500.

[12] Mangold,S.S.(1999).”Trends in the use of Braille

Contractions in the United States: Implications

for UBC Decisions”. Unpublished manuscript.

[13] Millar,S.(1997). “Reading by Touch”.

London:Routledge.

[14] Slaby W A(1990), “Computerized Braille

Translation”, Microcomputer Applications,

Vol.13, pp.107-113.

[15] Nathan Altshiller-Court (2007), "College

Geometry", Dover Publications, (orig. Barnes &

Noble 1952), pp.70-71.

[16] Capt.Dr.S Santhosh Baboo, V.Ajantha Devi;

"COMPUTER BASED TAMIL BRAILLE

SYSTEM

– A Review";11th International Tamil Internet

Conference, Annamalai University,

Chidambaram, India , Jointly conducted by

International Forum for Information Technology

(INFITT) and Annamalai University Dec 28-30,

2012. Pg.57 – 61

[17] Capt.Dr.S Santhosh Baboo, V.Ajantha Devi;

"TAMIL BRAILLE SYSTEM: A Conversion

methodology of Tamil into Contracted Braille Script (Grade2)"; UACEE International Journal

of Advances in Computer Science and its

Applications – IJCSIA Volume 3: Issue 2 [ISSN

2250 – 3765] Publication Date: 05 June 2013 –

pg.72 -76



1573

ISSN:2229-6093

http://articles.latimes.com/2001/jun/30/local/me-16960

http://articles.latimes.com/2001/jun/30/local/me-16960

[18] Capt.Dr.S Santhosh Baboo, V.Ajantha Devi; "TAMIL BRAILLE SYSTEM: A Conversion

methodology of Tamil into Contracted Braille

Script (Grade2)"; Second Intl. Conf. on

Advances in Computer, Electronics and

Electrical Engineering -CEEE 2013, Institute of

Research Engineers and Doctors. ISBN: 978-

981-07-6260-5 doi: 10.3850/ 978-981-07-6260-

5_36. pg171 – 175.

[19] http://www.linuxcircle.com

[20] www.raspberrypi.org

[21] Oi-Mean Foong and Nurul Safwanah Bt Mohd

Razali, “Signage Recognition Framework for

Visually Impaired People”, 2011 International

Conference on Computer Communication and

Management, Proc .of CSIT vol.5 IACSIT Press, Singapore, pg 488 – 492.

[22] Santanu Halder, Abul Hasnat, Amina Khatun,

Debotosh Bhattacharjee, Mita Nasipuri,

“Development of a Bangla Character

Recognition (BCR) System for Generation of

Bengali Text from Braille Notation”,

International Journal of Innovative Technology

and Exploring Engineering (IJITEE) ISSN:

2278-3075, Volume-3, Issue-1, June 2013, pg 5

- 10.

[23] Amany Al-Saleh, Ali El-Zaart and Abdul Malik

Al-Salman, “Dot Detection of Braille Images

Using A Mixture of Beta Distributions”, Journal

of Computer Science 7 (11): 1749-1759, 2011,

ISSN 1549-3636, pg 1749 – 1759.

[24] Aisha Mousa, Hazem Hiary, Raja Alomari, and

Loai Alnemer, “Smart Braille System

Recognizer”, IJCSI International Journal of

Computer Science Issues, Vol. 10, Issue 6, No 1,

November 2013, pg 52 – 60.

[25] Vidyashankar, G. Hemantha Kumar, and

P.Shivakumara (2004), "Rotation Invariant

Histogram Features For Recognition Of Braille

Symbols", Proceedings of the International

Conference on Cognitive Systems (ICCS „04),

New Delhi, India, pp. 32.

[26] Avinash Chaudhary, Pardeep Garg, Arjun

Agarwal (2012), "Using Rotation Method for

Removal of Misalignment of Scanned Braille Pattern", International Journal of Advancements

in Electronics and Electrical Engineering

(IJAEEE), Vol. 1, No. 2, pp. 145-149.

[27] .Iain Murray, and Andrew Pasquale (2006), "A

Portable Device for the Translation of Braille to

Literary Text", Proceedings of ACM Conference

on Assistive Technologies (ASSETS), pp. 231-

232.

[28] Rawan Ismail Zaghloul and Tomader Jameel Bani-Ata (2011), "Braille Recognition System –

With a Case Study Arabic Braille Documents",

European Journal of Scientific Research, Vol. 62

pp. 116-122.

[29] Saad D. Al-Shamma, and Sami Fathi (2010),

"Arabic Braille Recognition and Transcription

into Text and Voice", Proceedings of Biomedical

Engineering Conference (CIBEC), Cairo, Egypt,

pp. 227-231.

[30] Jan Mennens, Luc Van Tichelen, Guido

Francois, and Jan J. Engelen (1994), "optical

recognition of Braille writing using standard

equipment", IEEE Transactions on Neural

Systems and Rehabilitation Engineering, Vol. 2,

No. 4, pp. 207-212.

[31] P. Blenkhorn (1997), "A system for converting

braille into print", IEEE Transactions on

Rehabilitation Engineering, Vol. 3, No. 2, pp.

215-221.

[32] C. Ng, V. Ng, and Y. Lau (1999), "Regular

Feature Extraction for Recognition of Braille",

Proceedings of the International Conference on

Computational Intelligence and Multimedia

Applications, New Delhi, India, pp. 302-306.

[33] Lisa Wong, Waleed Abdulla, and Stephan

Hussmann (2004), "A Software Algorithm

Prototype for Optical Recognition of Embossed

Braille", Proceedings of the International

Conference on Pattern Recognition (ICPR),

Cambridge, UK, Vol. 2, pp. 586-589.

[34] X. Hermida, A. Corbacho Rodriguez, and F.

Martin Rodriguez (1996), "A Braille O.C.R. for

Blind People", Proceedings of the International

Conference on Signal Processing Applications &

Technology (ICSPAT 96), Vol. 1,. Boston, USA.

[35] Y. Mihara, A. Sugimoto, E. Shibayama, and S.,

Takahashi (2005), "An Interactive Braille-

Recognition System for the Visually Impaired

Based on a Portable Camera", Proceedings of

CHI Extended Abstracts on Human Factors in

Computing Systems, pp. 1653-1656.

[36] Y. Oyama, T. Tajima, and H. Koga (1997),

"Character Recognition of Mixed Convex-

Concave Braille Points and Legibility of Deteriorated Braille Points", System and

Computer in Japan, Vol. 28, No. 2, pp. 44-53.



1574

ISSN:2229-6093

http://www.raspberrypi.org/

Embedded Optical Braille Recognition on Tamil Braille … · 2014-08-28 · Embedded Optical...

Documents

Transcript of Embedded Optical Braille Recognition on Tamil Braille … · 2014-08-28 · Embedded Optical...