vision based intelligence wheel chair

download vision based intelligence wheel chair

of 26

Transcript of vision based intelligence wheel chair

  • 8/11/2019 vision based intelligence wheel chair

    1/26

  • 8/11/2019 vision based intelligence wheel chair

    2/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    The user head may be out of the ima"e view, or only the profile face is in the captured

    ima"eA

    The face color is user-dependent, and may chan"e dramatically in varyin" illumination

    conditionsA

    The user may have different facial appearances, such as mustache and "lassesA and The bac+"round may be cluttered when Ws move in the real world&

    n this report, a novel head "esture based interface (58) is developed for our Ws,

    namely :obo#hair, based on the inte"ration of the daboost face detection al"orithm(6rads+i,

    0119) and the #amshift ob*ect trac+in" al"orithm (.iola and Bones, 2')& n this process, head

    "esture reco"nition is conducted by means of real time face detection and trac+in"&

    This report presents an ntelli"ent Wheelchair(W) control system for the people with various

    disabilities& To facilitate a wide variety of user abilities , his system involves the use of face-

    inclination and mouth-shape information ,where the direction of an W is determined by the

    inclination of the users face, while proceedin" and soppin" are determined by the shapes of the

    users mouth& The system is composed of electric powered wheelchair, data acquisition board,

    ultrasonicCinfrared sensors, a P# camera and vision system& Then the vision system o analy=e

    users "estures is performed by three sta"es > detector, reco"ni=er and convertor&

    n the detector, the facial re"ion of the intended user is first obtained usin" daboost A

    thereafter the mouth re"ion is detected based on ed"e information&

    The e?tracted features are sent to the reco"ni=er, which reco"ni=es the face inclination

    and mouth shape usin" statistical analysis and @-means clusterin" , respectively

    These reco"nition results are then delivered to the convertor to control the wheelchair

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e2

  • 8/11/2019 vision based intelligence wheel chair

    3/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    /. ROLE OF IW0S

    /.1 WHAT IS THE VITTAL ROLE PLAYED BY IW0S

    With the increase of elderly and disabled people, a wide ran"e support devices and care

    equipment has been developed to help improve their quality of life (D3/) & n particular,

    intelli"ent wheelchairs (Ws) have received considerable attention as mobility aids& Essentially,

    Ws are electric powered wheelchairs (EPWs) with an embedded computer and sensors, "ivin"

    them intelli"ence& i"ure 0 shows the various Ws

    Fi'!r" 12 Int"&&i'"nt Wh""&%hair 3IW4 3a4 GRASP La5#rat#r( S6art Chair ) 354

    Wh""&%hair #$ Y!ta7a "t. a&) 3%4 Na8 Chair.

    Two basic techniques have been required to develop Ws> 0) auto navi"ation techniques

    for automatic obstacle detection and avoidance, 2) convenient interfaces that allow handicapped

    users to control the W themselves usin" their limited physical abilities& While it is important to

    develop a system that enables the user to assist in the navi"ation, the system is useless if it

    cannot be adapted to the abilities of the user& or e?ample, in the case a user cannot manipulate astandard *oystic+, other control options need to be provided&

    /./ RELATED RESEARCH

    %o far many access methods for Ws have been developed and then they can be classified

    as intrusive and non-intrusive& They are summari=ed in Table 0& ntrusive methods use "lasses, a

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e3

    http://www.jneuroengrehab.com/content/6/1/33/figure/F1http://www.jneuroengrehab.com/content/6/1/33/figure/F1
  • 8/11/2019 vision based intelligence wheel chair

    4/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    headband, or cap with infraredCultrasound emitters to measure the userFs intention based on

    chan"es in the ultrasound waves or infrared reflect & n contrast, non-intrusive methods do not

    require any additional devices attached to userFs face or head&

    TABLE 12 IW CONTROLS IN LITERATURES

    Intr!i8" Int"r$a%" 2

    INTELLIGENT

    WHEELCHAIR

    FEATURE DEVICE SUPPORTING

    COMMANDS

    &/hen,et, al 5ead

    3rientation

    Tilt %ensors, icroprocessor 8o, 6ac+, /eft, :i"ht

    %3 Pro*ect Eye 8a=e Electrode 8o, 6ac+, /eft, :i"ht

    Wheelsley Eye 8a=e nfrared %ensors,

  • 8/11/2019 vision based intelligence wheel chair

    5/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    WHEELCHAIR COMMANDS

    oshida ace

    8o, /eft, :i"ht, :otate

    i?in" #ommands>

    8o-/eft, 8o-:i"ht

    s shown in Table 0, voice-based and vision-based methods belon" to the nonintrusive

    methods& .oice control is a natural and friendly access method, however, the e?istence of other

    noises in a real environment can lead to command reco"nition failure, resultin" in safety

    problems& ccordin"ly, a lot of research has been focused on vision-based interfaces, where

    control is derived from reco"ni=in" the userFs "estures by processin" ima"es or videos obtained

    via a camera& With such interfaces, face or head movements are most widely used to convey the

    userFs intentions& When a user wishes to move in a certain direction, it is a natural action to loo+

    in that direction, thus movement is initiated based on noddin" the head, while turnin" is

    "enerated by the head direction& 5owever, such systems have a ma*or drawbac+, as they are

    unable to discriminate between intentional behavior and unintentional behavior& or e?ample, it

    is natural for a user to loo+ at an obstacle as it "ets close, however, the system will turn and "o

    towards that obstacle&

    /.9 THE TECHINAL IDEA

    ccordin"ly, we develop a novel W interface usin" face and mouth reco"nition for the

    severely disabled& The main "oal of the present study is to provide a more convenient and

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e5

    http://www.jneuroengrehab.com/content/6/1/33/table/T1http://www.jneuroengrehab.com/content/6/1/33/table/T1
  • 8/11/2019 vision based intelligence wheel chair

    6/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    effective access method for people with various disabilities& or accurate reco"nition of the userFs

    intention, the direction of the W is determined accordin" to the face inclination, while

    proceedin" and stoppin" are determined by the shape of the mouth& This format was inspired

    based on the operation of car, as the userFs face movements correspond to the steerin" wheel,

    while the userFs mouth corresponds to the bra+e and "as pedal& The mechanisms prevent an

    accident in the case the user instinctively turns their head to loo+ at an obstacle, thereby ma+in"

    safer& oreover, the proposed control mechanisms require minimal user motion, ma+in" the

    system more comfortable and more adaptable for the severely disabled when compared to

    conventional methods&

    The W system consists of acial eature !etector (!etector), acial eature :eco"ni=er

    (:eco"ni=er), and #onverter & n our system, the facial re"ion is first obtained usin" daboost

    al"orithm, which is robust to the time-varyin" illumination& Thereafter the mouth re"ions are

    detected based on ed"e information& These detection results are delivered to the :eco"ni=er,

    which reco"ni=es the face inclination and mouth shape& These reco"nition results are then

    delivered to the #onverter, thereby the wheelchair are operated& To assess the effectiveness of the

    proposed interface, it was tested with G' users and the results were compared with those of other

    systems& Then, the results showed that the proposed system has the superior performance to

    others in terms of accuracy and speed, and they also confirmed that the proposed system can

    accurately reco"ni=e userFs "estures in real-time&

    9. IW0S ARCHITECURE AND ITS OVERVIEW

    9.1 SYSTEM3IW0S4 ARCHITECTURE

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e6

  • 8/11/2019 vision based intelligence wheel chair

    7/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    The proposed W is composed of electric powered wheelchair, data acquisition board,

    and a P# camera and vision system& data acquisition board (!D-board) is used to process the

    sensor information and control the wheelchair& The !D-board and a vision system are

    connected via a serial port& n our system, a EPW-!E%E & care :ider

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e7

    http://www.jneuroengrehab.com/content/6/1/33/figure/F2http://www.jneuroengrehab.com/content/6/1/33/figure/F2
  • 8/11/2019 vision based intelligence wheel chair

    8/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    !D 6oard > #ompile Technolo"y %!D-!'EH

    nput device > /o"itech (7' I '9)

  • 8/11/2019 vision based intelligence wheel chair

    9/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    i"ureGdescribes the process to reco"ni=e userFs "estures, where the reco"nition is

    performed by three steps> !etector, :eco"ni=er, and #onverter& irst, the facial re"ion is obtained

    usin" the daboost al"orithm, and the mouth re"ion is detected based on ed"e information&

    These detection results are then delivered to the :eco"ni=er, which reco"ni=es the face

    inclination and mouth shape usin"K-means clusterin" and a statistical analysis, respectively&

    Thereafter, the reco"nition results are delivered to the #onverter, which operates the wheelchair&

    oreover, to fully "uarantee user safety 0 ran"e sensors are used to detect obstacles in

    environment and avoid them& n what follows, the details for the respective components are

    shown&

    FIGURE 92 Th" O8"ra&& Ar%hit"%t!r" #$ th" Pr#:#"d C#ntr#& S(t"6

    ;.1 FACIAL FEATURE DETECTOR2 DETECT USER

  • 8/11/2019 vision based intelligence wheel chair

    10/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    or each frame of an input streamin", this module locali=es the facial re"ion and mouth

    re"ion, and sends them to the :eco"ni=er& The facial re"ion is obtained usin" the daboost

    al"orithm for robust face detection, and the mouth re"ion is obtained usin" ed"e information

    within the facial re"ion&

    or application in a real situation, the face detection should satisfy the followin" two

    requirements>

    t should be robust to time-varyin" illumination and cluttered environments and

    t should be fast enou"h to supply real-time processin"& Thus, the daboost al"orithm is

    used to detect the facial re"ion&

    This al"orithm was ori"inally proposed by .iola and has been used by many researchers& The

    daboost learnin" method is an iterative procedure for selectin" features and combinin"

    classifiers& or each iteration, the features with the minimum misclassification error are selected,

    and wea+ classifiers are trained based on the selected features& The daboost learnin" method

    +eeps combinin" wea+ classifiers into a stron"er one until it achieves a satisfyin" performance&

    To improve the detection speed, a cascade structure is adopted in each of the face detectors, to

    quic+ly discard the easy-to-classify non-faces& This process is illustrated in i"ure '&

    i"ure$shows some face detection results& To demonstrate its robustness, the face

    detection method was tested with several standard !6s such as .@ !6 & oreover, it was

    tested on the data obtained from real environment& i"ures$(a)and $(b) show the results for

    .@ !6s, respectively& nd i"ures $(c)is the results for online streamin" data& s seen in

    i"ure$, the proposed method is robust to the time-varyin" illumination and the cluttered

    environments& To reduce the comple?ity of the mouth detection, it is detected based on the

    position of the facial re"ion usin" the followin" properties>

    The mouth is located in the lower re"ion of the face and The mouth has a hi"h contrast compared to the surroundin"s&

    Thus, the mouth re"ion is locali=ed usin" an ed"e detector within a search re"ion estimated

    usin" several heuristic rules based on the facial re"ion& The details for the search re"ion are "iven

    in our previous wor+ by the current authors&

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e10

    http://www.jneuroengrehab.com/content/6/1/33/figure/F4http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F4http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5http://www.jneuroengrehab.com/content/6/1/33/figure/F5
  • 8/11/2019 vision based intelligence wheel chair

    11/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    FIGURE ;2 O!t&in" #$ Fa%" D"t"%ti#n Uin' Ada5##t A&'#rith6

    FIGURE =2 Fa%" D"t"%ti#n R"!&t 3a4 Th" R"!&t $#r MMI DB)354 Th" R"!&t $#r VA*DB) 3%4 Th" R"!&t $#r On&in" Str"a6in' Data

    i"ure7shows mouth detection results& %ince the detection results include both narrow ed"es

    and noise, the noise is eliminated usin" the post-processin"&

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e11

    http://www.jneuroengrehab.com/content/6/1/33/figure/F6http://www.jneuroengrehab.com/content/6/1/33/figure/F6
  • 8/11/2019 vision based intelligence wheel chair

    12/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    FIGURE >2 Th" M#!th D"t"%ti#n R"!&t 3a4 Ed'" D"t"%ti#n R"!&t 354 N#i" R"6#8"d

    R"!&t

    ;./ FACIAL FEATURE RECOGNI?ER 2 RECOGNI?E FACE

    INCLINATION AND MOUTH SHAPE OF THE INTENDED

    USER

    This module reco"ni=es the userFs face inclination and mouth shape, both of which are

    continuously and accurately reco"ni=ed usin" a statistical analysis and template matchin"& s a

    result, the proposed reco"ni=er enables the user to control the wheelchair directly by chan"in"

    their face inclination and mouth shape& or e?ample, if the user wants the wheelchair to move

    forward, the user *ust says L8o&L #onversely, if the user wants the wheelchair to stop, the user

    *ust says L

  • 8/11/2019 vision based intelligence wheel chair

    13/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    To minimi=e the inertia, the derivative is ta+en with respect to& ccordin"ly, the

    orientationcan then be obtained by equation (2)&

    (2)

    whererr,cc andrc are the second moments, the respective of which are defined

    as and & f the value of M is less than , this

    means that the user nods their head slantin" to the left& 3therwise, it means that the user nods

    their head slantin" to the ri"ht& i"ure Jshows the reco"nition results for the face inclination&

    FIGURE 2 Th" R"%#'niti#n R"!&t F#r Fa%" In%&inati#n. 3A4 Th" C#66and O$ T!rn-

    L"$t) 3B4 Th" C#66and O$ T!rn-Ri'ht.

    To reco"ni=e the mouth shape in the current frame, template matchin" is performed, where the

    current mouth re"ion is compared with mouth-shape templates& These templates are obtained

    byK-means clusterin" from 00' mouth ima"es&K-means clusterin" is a method of classifyin" a

    "iven data set into a certain number of clusters fi?ed a priori& n this e?periment, multiple mouth-

    shape templates were obtained, which consisted of 7 different shapes of L8oL and L

  • 8/11/2019 vision based intelligence wheel chair

    14/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    FIGURE @ 2 Th" 6#!th ha:" t"6:&at". 3a4 G# 6#!th ha:" t"6:&at" and) 354 Uh6

    6#!th ha:" t"6:&at".

    The results of the comparin" the templates with a candidate are represented by matchin" scores&

    The matchin" score between a mouth-shape template and a candidate is calculated usin" the

    5ammin" distance, where the 5ammin" distance between two binary strin"s is defined as the

    number of di"its in which they differ& 5ere the matchin" scores for all the mouth-shape templates

    and a mouth candidate are calculated, and the mouth-shape template with the best matchin"

    score is selected&

    ;.9 CONVERTER2 TRANSLATE USER

  • 8/11/2019 vision based intelligence wheel chair

    15/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    FIGURE > Data A%!iiti#n 5#ard 3SD-DA;E4

    The commands "iven from the user interface are passed to the control pro"ram runnin"

    the wheelchair throu"h the serial port& The board pro"ram then controls the speed and direction

    of wheelchair by modifyin" the volta"e passin" throu"h the wheelchair&

    Table2shows command map between wheelchair movement and output volta"e& The

    proposed system is able to control both the direction and the velocity of the wheelchair, as the

    user can produce a different output volta"e by chan"in" their mouth shape or face orientation& n

    addition to simple commands, such as "o-forward, "o-bac+ward, turn-left, or turn-ri"ht, the

    proposed system can also "ive a mi?ture of two simple commands, similar to *oystic+ control&

    or e?ample, the wheelchair can "o in a '$ de"ree direction by combinin" the "o-forward and

    "o-ri"ht commands&

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e15

    http://www.jneuroengrehab.com/content/6/1/33/table/T3http://www.jneuroengrehab.com/content/6/1/33/table/T3
  • 8/11/2019 vision based intelligence wheel chair

    16/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    TABLE /> OPERATION VOLTS OF INTELLIGENT WHEELCHAIR

    COMMANDS OUTPUT 1 OUTPUT /

    8o 2&$.NG&J. 2&'$.

    6ac+ 0&2.N2&'$. 2&'$.

    /eft 2&'$. 0&2.NG&J.

    :i"ht 2&'$. 2&'$.NG&J.

    %top 2&'$. 2&'$.

    The interface system of W was developed in P# platform> the operation system is

    Windows HP and #P< is Pentium 0&J 85=& The #amera is /o"itech, which was connected to the

    computer usin" the TESTING GROUPS

    STAGE TIME NUMBER

    EPW USAGE

    COMPUTER USAGE3434

    ble-6oiled

  • 8/11/2019 vision based intelligence wheel chair

    17/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    The e?periments were performed by two steps& irst, the performance of the proposed

    system is presented, which was tested in various environments& The effectiveness of the proposed

    system is then discussed in comparison with other methods&

    EPERIMENT I2 TO MEASURE THE ACCURACY OF OUR INTERFACE

    or the proposed system to be practical in the real environments, it should be robust to

    various illuminations and cluttered bac+"rounds& Thus, the proposed method was applied to

    detect and reco"ni=e the userFs facial features in a comple? bac+"round& i"ure 0shows that the

    facial feature detection results& The scenes had a cluttered stationary bac+"round with a varyin"

    illumination& s seen in i"ure0, the results accurately detected the face and mouth, confirmin"

    the robustness to time-varyin" illumination, and low sensitivity to a cluttered environment&

    FIGURE 1 2 Fa%" and 6#!th d"t"%ti#n r"!&t&

    With the proposed system the userFs intention is represented by the inclination of the face

    and shape of the mouth, ma+in" the accurate reco"nition of these features crucial&

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e17

    http://www.jneuroengrehab.com/content/6/1/33/figure/F10http://www.jneuroengrehab.com/content/6/1/33/figure/F10http://www.jneuroengrehab.com/content/6/1/33/figure/F10http://www.jneuroengrehab.com/content/6/1/33/figure/F10
  • 8/11/2019 vision based intelligence wheel chair

    18/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    i"ures 00(a)and00(b)show the reco"nition results for the face inclination and the mouth

    shapes, respectively& s shown in these fi"ures, they are continuously and accurately reco"ni=ed&

    FIGURE 11 2 Fa%" and 6#!th r"%#'niti#n r"!&t& 3a4 $a%" in%&inati#n r"%#'niti#n) 354

    6#!th ha:" r"%#'niti#n.

    To quantitatively evaluate the performance of the proposed system, we as+ed each user to

    perform contain commands, such as "o strai"ht, stop, turn left or turn ri"ht and repeat the action

    $ times& Table 'shows the avera"e time ta+en to detect the face and facial features, then

    reco"ni=e them in an indoor and outdoor environment& s a result, the avera"e time ta+en to

    process a frame was about 72 ms, allowin" the proposed system to process more than 0$

    framesCsec on avera"e (07 framesCsec in indoor and 0' framesCsec in outdoor)& Table $ shows the

    reco"nition rates of the proposed interface for the respective commands& The proposed systemshows the precision of 0O and the recall of 17&$O on avera"e& Thus, this e?periments proved

    that the proposed system can accurately reco"ni=e userFs intentions in real-time&

    TABLE ; 2 PROCESSING TIME 3.MS4

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e18

    http://www.jneuroengrehab.com/content/6/1/33/figure/F11http://www.jneuroengrehab.com/content/6/1/33/figure/F11http://www.jneuroengrehab.com/content/6/1/33/table/T5http://www.jneuroengrehab.com/content/6/1/33/figure/F11http://www.jneuroengrehab.com/content/6/1/33/figure/F11http://www.jneuroengrehab.com/content/6/1/33/table/T5
  • 8/11/2019 vision based intelligence wheel chair

    19/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    STAGE INDOOR OUTDOOR

    ace !etection G G2

    outh !etection 0$ 09

    ace nclination :eco"nition 2 2

    outh %hape :eco"nition 0$ 07

    Total 72 79

    TABLE = 2 PERFORMANCEEVALUATION RESULTS

    COMMANDS RECALL PRECISION

    /eft Turn &19 0

    :i"ht Turn &1' 0

    8o %trai"ht &17 0

    %top &19 0

    i"ure02shows some snapshots for the proposed system to be applied on various

    environments& 3utdoor environments have a time-varyin" illumination and more comple?

    bac+"round, as shown in i"ures 02(b)and02(d)& 5owever, despite those comple?ities, the

    proposed system wor+ed very well in both environments& n particular, in case of someone

    comes to tal+ the user (in i"ure 02(c)), our system can accurately discriminate between

    intentional and unintentional behaviors, thereby preventin" potential accidents, when the user

    instinctively turns their head to loo+ at a person&

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e19

    http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12http://www.jneuroengrehab.com/content/6/1/33/figure/F12
  • 8/11/2019 vision based intelligence wheel chair

    20/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    FIGURE 1/ 2 IW C#ntr#& On R"a& En8ir#n6"nt&

    =./ EPERIMENT II2 TO COMPARE WITH OTHER INTERFACES

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e20

  • 8/11/2019 vision based intelligence wheel chair

    21/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    To prove the efficiency and effectiveness of the proposed W interface, it was also

    compared with other systems& 5ere, two methods were adopted, one is headband-based method

    and the other is method usin" face trac+in" & Then, the former belon"s to the intrusive method

    and the latter belon"s to the vision-based method& n the headband-based method, the "o-and-

    stoppin" is controlled by noddin" userFs head to the front or to the rear, and the direction is

    chan"ed by noddin" userFs head to the left side or to the ri"ht side& n such system, the head

    motions are measured throu"h a headband that includes an accelerometer sensor& 3n the other

    hand, a face-based interface detects userFs face in the first frame and trac+s it continuously, and

    then userFs face are detected usin" s+in-color model&

    i"ure0Gshows the control commands for respective methods& When visually inspected,

    our system requires the smaller motions than others& This tells us our system is more comfortable

    and suitable to the severely disabled&

    FIGURE 19 2Int"&&i'"nt Wh""&%hair in:!t 6"th#d&

    or the practical use by the severely disabled, such systems should be operable on both

    indoor and outdoor environments& Thus, the three systems were evaluated across indoors and

    outdoors, chan"es in time of day and weather conditions& %uch conditions are summari=ed in

    Table7& nd some test maps in indoor and outdoor environments are shown in i"ure 0'&

    TABLE >2 TEST ENVIRONMENTS

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e21

    http://www.jneuroengrehab.com/content/6/1/33/figure/F13http://www.jneuroengrehab.com/content/6/1/33/table/T7http://www.jneuroengrehab.com/content/6/1/33/figure/F14http://www.jneuroengrehab.com/content/6/1/33/figure/F13http://www.jneuroengrehab.com/content/6/1/33/table/T7http://www.jneuroengrehab.com/content/6/1/33/figure/F14
  • 8/11/2019 vision based intelligence wheel chair

    22/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    PLACES TIME AND ILLUMINATION

    Ind##r (!aytime, fi?ed illumination)

    (4i"ht time, fi?ed illumination)

    O!td##r (!aytime, time-varyin" illumination and a shadow)

    (4i"httime, -)Ind##r t# #!td##r) #r 8i%" 8"ra (!aytime, time-varyin" illumination and shadow)

    FIGURE 1; 2 S#6" EJa6:&" #$ T"t Ma:& 3A4 O!td##r T"t Ma:) 3B4 Ind##r T"t Ma:&

    The participants to navi"ate each map 0 times usin" three interfaces& The performances

    for three interfaces were then evaluated in terms of the accuracy and speed& n those e?periments,

    the face-based method was tested in only the indoor environments, due to its sensitivity to the

    time-varyin" illumination& s mentioned above, it used the s+in-color model to e?tract userFs

    face, so it is very sensitive to illumination chan"es&

    Tables J and9show the summari=ed performance comparisons for the three methods&

    Table Jshows the avera"e times ta+en to reach the destination for three interfaces& mon" three

    methods, the face-based method is the slowest, whereas there is no si"nificant difference

    between our method and the head-band method& eanwhile, Table9shows the reco"nition

    accuracies of three methods, where the proposed-based method produced the best performance

    with an avera"e accuracy of 17&$O, while the face-based method had an accuracy of 9J&$O and

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e22

    http://www.jneuroengrehab.com/content/6/1/33/table/T8http://www.jneuroengrehab.com/content/6/1/33/table/T9http://www.jneuroengrehab.com/content/6/1/33/table/T9http://www.jneuroengrehab.com/content/6/1/33/table/T8http://www.jneuroengrehab.com/content/6/1/33/table/T9http://www.jneuroengrehab.com/content/6/1/33/table/T9
  • 8/11/2019 vision based intelligence wheel chair

    23/26

  • 8/11/2019 vision based intelligence wheel chair

    24/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    inimal user motion, ma+in" the proposed system more adaptable to the severely

    disabled, than conventional methods&

    :obustness to a cluttered bac+"round and the time-varyin" illumination and

    ccurate reco"nition of user intention based on discriminatin" between intentional and

    unintentional behavior&

    To prove these advanta"es, the proposed system was tested with G' users on the indoor

    outdoor environments& 5owever, to "uarantee full user safety, the proposed system also needs to

    be able to detect and avoid obstacles automatically thus further research in this area is currently

    underway&

    . CONCLUSION

    nthis report, a few interestin" views were discussed re"ardin" an W system , which

    is adaptable, efficient androbust for disabled people with physical abilities& n this system , the

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e24

  • 8/11/2019 vision based intelligence wheel chair

    25/26

    VISION BASED INTERFACE SYSTEM FOR HANDS FREE CONTROL OF AN INTELLIGENT WHEEL CHAIR

    direction of W is determined by users face inclination, while "oin" and stoppin" is determined

    by users mouth shape& This report describes the desi"n and implementation of a novel hands

    free control system for Ws& This W system provides enhanced mobility for the elderly and

    disabled people who have very restricted limb movements or severe handicaps&

    robust 58 is desi"ned for vision-based head "esture reco"nition of the :obo#hair

    user& The reco"ni=ed "estures are used to "enerate motion control commands so that the

    :obo#hair can be controlled accordin" to the users intention& To avoid unnecessary movements

    caused by user loo+in" around randomly, the 58 is focused on the central position of the

    wheelchair to identify useful head "estures&

    The proposed system was tested with G' users in indoor and outdoor environments

    and the results were compared with those of other systems, then the results showed that the

    proposed system has superior performance to other systems in terms of speed and accuracy&

    Therefore, it is proved that proposed system provided a friendly and convenient interface to the

    severely disabled people&

    The future research will be focused on some e?tensive e?periments and evaluation of

    58 in both indoor and outdoor environments where cluttered bac+"rounds, chan"in" li"htin"

    conditions, sunshine and shadows may brin" complications to head "esture reco"nition&

    @. REFERENCES

    1. Bournal of 4euroEn"ineerin" and :ehabilitation2009-Bin %un Bu, unhee %hin, and Eun

    i @im

    PVP Siddhartha Intit!t" #$ T"%hn#'() *an!r!) Vi+a(a,ada - Pa"e25

  • 8/11/2019 vision based intelligence wheel chair

    26/26