BLUE EYE TECHNOLOGY

Future of Computers

By

Lavanya Rajsekaran

10BM60042

MBA 1st year

VGSoM

Abstract:

Blue Eyes Technology

Artificial intelligence aims in creating intelligence to the “Machines”. Intelligence alone cannot make a

machine perfect; even then it can only perform the work it is designed for. But they never understood

the perceptions or emotions of Human beings. Blue Eye Technology now aims at incorporating these

perceptions and sensory abilities into the computational machines. Machines will be more powerful,

effective and helpful if they even have a little of these perceptions in them. In future, we might see in

reality of a computer playing your favorite movie, songs or photos to cheer you up. You might just

check your mails by asking your computer, or call your friends by telling the computer to make the call

for you. Those which were considered to be a science fiction will be reality soon enough with the help

of this technology. Cars with emotional intelligence can reduce the number of accidents, by judging

the emotional state of the driver and help him driving safely. The technology aims to enable these

perceptions in machines by using natural inputs like facial expressions. This technology has a power

to change the future of the world.

Introduction:

“A computer on every desk” – was the vision of Bill gates when he started Microsoft. The vision has

now become the almost reality of the present world. As technologies evolve, old ones are replaced by

the new ones. Like Floppies were got ridden by Cds and Pen drives, Walkman which almost

dominated the market for a long time is dead now because of mp3 players and iPods. Similarly

prospective future of Computers is Blue Eye Technology. Since nineties, the world has seen a rapid

change because of computers. It has changed the way the world was working before and has brought

us to a scenario where no business can be called big and successful without computers in them as a

part of it.

Animal survival depends on highly developed sensory abilities. Likewise human cognition depends on

highly developed abilities to perceive, integrate and interpret visual, auditory and touch information. If

such perceptions are added to computers even in a small fraction, definitely computers would become

even more powerful than they are today. Also they can help human beings in much more better

manner. Blue Eye aims at creating the perceptual and sensual abilities to computers.

IBM scientists, at Almaden Research Center (ARC) in San Jose, the main IBM research laboratory,

are trying to enable the computers to listen to our speech, sense our gaze and read our body

language since 1997. Affective Computing is computing that relates to, arises from, or deliberately

influences emotion or other affective phenomena. Affective Computing research combines

engineering and computer science with psychology, cognitive science, neuroscience, sociology,

education, psychophysiology, value-centered design, ethics, and more. The various steps involved in

affective computing are

Adding sensory abilities to computers

Detecting the emotions of human being

Respond accordingly

2 | P a g e

Blue Eyes Technology

They are trying to enable these senses to computers with the help of video cameras, speech

recognition technology, gaze tracking systems, facial geometry to detect expressions etc to gather

key information. This information is then analyzed to determine the user's physical, emotional, or

informational state, which in turn can be used to help make the user more productive by performing

expected actions or by providing expected information. The exciting science fiction of being able to tell

your computer what it should do is not far away from being reality with the help of these technologies

in place. Machines then can act according to the commands received from the users by their senses

rather than requiring user to be present physically present before the system and enter the various

commands to execute the desired action. For example, ask the computer for a search in Google, turn

on music or turn off, make a phone call for you etc, or even better computers can sense your

emotional state and try reacting accordingly like trying to play your favourite music when you are

tensed or little low etc. In the future, ordinary household devices -- such as televisions, refrigerators,

and ovens -- may be able to do their jobs when we look at them and speak to them.

Different ways to get inputs from the user can be:

Gestures – Video monitoring

Speech – ex. Artificial intelligence Speech Recognition systems

Gaze Tracking – ex. MAGIC

Facial expressions – ex. Facial geometry

Touch – ex. Emotional mouse

Some of these examples can be discussed briefly to gather idea about implementation of the

technology in the machines.

Emotional Mouse:

The mouse developed to evaluate the emotional state of the user such as anger, fear, sadness,

disgust, happiness, surprise etc when the computer is used.

The information the emotional mouse can detect are of two types

BehavioralMouse movementsButton click frequencyFinger pressure when user presses his/ her button

PhysiologicalHeart rate(Electrocardiogram, Photoplethysmogram)Skin temperature(Thermester)Skin electricity(Galvanic skin response)Electromyographic activity(Electromyogram)

3 | P a g e

Blue Eyes Technology

Figure 1 - Emotional Mouse

The emotional mouse is designed with an objective to measure the physical and physiological data without user’s perception or obstruction as much as possible.

Figure 2 - System configuration for emotional mouse

4 | P a g e

Blue Eyes Technology

Data Acquisition System is used to get the analog data from the sensors of the emotional mouse. The

components of the emotional mouse can also be crafted in the regular usage items like chair, steering

wheel, and phone handle etc.

Emotions are mapped to the sensor outputs using a correlation model. The correlation model is

derived from a calibration process in which a baseline attribute-to-emotion correlation is rendered

based on statistical analysis of calibration signals generated by users having emotions that are

measured or otherwise known at calibration time. From the information obtained, emotional state of

the user and the activity he is currently involved in can be related as information about the user.

A variation of emotional mouse called the sentic mouse has also been tested to identify directional

pressure sensor which would interpret the likings or dislikings.

Manual And Gaze Input Cascaded (MAGIC) Pointing:

This explores a new direction in utilizing eye gaze for computer input. This technology aims at placing

the mouse where the user gazes at. Gaze tracking system was the one which was initially the more

effective technology where it places the mouse in about half an inch radius from the users gaze. Eye

tracking data can be acquired simultaneously with MRI scanning using a system that illuminates the

left eye of a subject with an infrared (IR) source, acquires a video image of that eye, locates the

corneal reflection (CR) of the IR source, and in real time calculates/displays/records the gaze direction

and pupil diameter. IBM researchers have come up with two MAGIC pointing techniques, one liberal

and the other conservative in terms of target identification and cursor placement.

The liberal approach is to warp the cursor to every new object the user looks at. The user may move

the cursor once he decides to acquire the target he is looking at. On the other hand, the user may

also feel that the cursor is over-active when he is merely looking at a target, although he may

gradually adapt to ignore this behavior.

Figure 3 - Magic pointing methods

5 | P a g e

Blue Eyes Technology

The more conservative MAGIC pointing technique explored does not warp a cursor to a target until

the manual input device has been actuated. Once the manual input device has been actuated, the

cursor is warped to the gaze area reported by the eye tracker. This area should be on or in the vicinity

of the target. The user would then steer the cursor manually towards the target to complete the target

acquisition.

Artificial Intelligence Speech Recognition:

Speech recognition includes natural language understanding by the computer. By filtering the voice

received and recognizing the words in the natural language, trying to enable the computer to perform

activities is what this technology aims at. The microphone serves as the medium to get the input from

the user. The sound level, the speed at which the user states the words, the grammar of the user all

counts when the system tries to work. This involves understanding of the input and initiating the

action. Identifying the input received involves finding the best match for it with the database. This

requires high-speed processors and large RAM to encounter only few milliseconds to initiate the

action as commanded. Dynamic time warping process helps to identify the word when pronounced in

different speeds as well as elongate different parts of the word.

POD:

Toyota's "Personalization On Demand [POD]" concept vehicle, was jointly developed by Toyota and

Sony Corp., pod explores the potential for communications between vehicles and their drivers.

‘POD’ was the first device to have blue eye enabled in it. The vehicle was not for production but was

shown as an example of how the feel of future of cars would be like. When the car was exhibited

senior vice president of Toyota commented “The Pod defies the axiom that vehicle control is a one-

way-street.” Also said “POD was developed to explore and strengthen the bond between car and

driver through a give-and-take relationship. Much like forming a friendship, the pod listens to driver,

monitors his driving habits, and records a wide variety of his personal preferences. By expanding its

intelligence, pod can create an enhanced driving experience”.

Happy POD Sleepy POD Sad POD Angry POD

Toyota's Telematic, Telepathic "POD" Smiles, Cries And Wags Its Tail

The vehicle expresses itself to the drivers or others. It has been designed to express itself through

different configurations of its eyes[headlamps], eyebrows, mouth[grille] and ears[side mirrors],

coordinated with different colours of LEDs to express the mood. It has been configured to express

6 | P a g e

Blue Eyes Technology

ten different emotions including happiness, surprise, sadness, anger with the help of the different

colours of the lights. The tail like antenna on its trunk at the rear end is configured to be moveable

and it acts like a tail to wag when it sees its driver approaching.

Pod is happy when its refueled, washed and it displays bright orange light to express its happiness.

After a long distance journey, people are usually tired, and pod expresses the same too. Bad

weather, poor visibility, bad roads brings upon a worried look on pod. The current emotional state of

Pod is displayed to all the people in the car through a main display near the driver or through their

individual displays. Constructive criticism is provided to improve the skills of the driver like clapping

hands are displayed when driver is driving smoothly, reckless driving results in a surprised or fearful

expression on display.

The gathering data about the driver to understand their emotional state also happens apart from

expressing itself. Overtime it records all the data regarding the personality, driving skills and tastes of

the driver. It can gather information from its external surroundings too. Dedicated Short Range

Communication (DSRC) is used by pod to identify or detect pedestrians nearby and caution the driver

for slowing down. Horn Signal Exchange System equipped vehicles can be contacted and one of the

appropriate horn signals– ‘warning’, ‘please give way’ or ‘thank you’ --can be communicated to them.

Driving system has been simplified to be controlled with just one hand using drive-by-write

technology. It is similar to using a joystick. The reaction time of the driver is reduced with this in place.

Comfort is placed as priority with pod. If hurry is being sensed then it tries to relax the driver by

displaying warnings, blowing cool air or playing relaxing music. Suspension of the car is altered

according to the driver’s skills and the road conditions. Air quality is sensed and automatically opens

windows to let in fresh air.

The seats inside pod are 360 degree rotatable, and so as a welcoming gesture it arranges the chairs

in a position so as to provide a comfortable entering. At parking it turns around the chairs to a circle

and creates a cozy environment. It can accommodate requests for a direct drive or scenic drive. It can

send notification ahead if pod expects a late arrival at its destination. It can provide details of the

closest refueling center, if it is low in fuel. It can snap pictures if it senses a happy or nice environment

inside the car and display the images. It can run automatic checks to find any abnormality and provide

advises for remedy of the situation. Pod will happily bid a bye by providing feedback on that drive also

lowers in height as a sign of resting.

7 | P a g e

Blue Eyes Technology

Secure Pad:

The device was basically designed for health-care providers so that they can keep track of various

activities of the doctors and others in a large medical facility. It is a badge, and it can identify, track the

movements and activities of its wearer. It will enhance security and accountability of the medical

facilities available to the patients and thus benefit them. It can keep track of the records of patient and

that can be made confidential enough so as they are displayed on the wall of the patient’s room to the

doctor wearing the badge, if anyone else enters the details would disappear. This would reduce the

burden of carrying around the papers and avoid the risk of misplacing or losing the records of the

patients. The drug store can be tracked by the amount of drug withdrawn by the chemist. Also the

access can be restricted to the authorized persons. The data will be cleaned like in a slate as soon as

badge is removed by the user.

Applications of Blue Eye Technology:

Security and control systems can be controlled with the help of this technology. Human

supervision is always required at such areas. Human beings can get tired easily and

distracted at times and this can cause a huge devastation in such systems. So if blue eyes

are implemented in this area. The surveillance camera’s can capture more than just recording

the video of particular area. They can recognize the emotional state of the user and help

security to focus on the persons whose emotional conditions are out of normal and seemingly

dangerous. Also these cameras can provide data to the shopkeepers to track their customers,

giving them idea which section attracts people, which advertisement displayed attracted more

people, which made people feel boredom etc. It can also used in the other control systems in

banks, airplanes etc. The security concerns of getting a wrong person with no harmful

intentions in the security systems should be avoided while still a proper alarming system is in

place with the help of it. They can prove to be better alarm systems.

Assisting Human Operators by monitoring and recording the operator’s physical condition.

In the complex industries like chemical industry or nuclear sector where exists danger in the

environment of exposure to toxic substances or radiations, monitoring helps the human

operators. The operator has to raise the alarm by himself by announcing the danger he is in

but with the help of these systems, the technology keeps an eye on the operator for normal

conditions. If any parameter is found in abnormal conditions like a raise in pulse rate or low

level of oxygenation then an alarm can be triggered automatically.

Driving systems can be controlled by blue eye technology and this can help in reduction of

the number of accidents happening in the world. When a sensor is attached to the steering

wheel, it can assess the emotional stability of the driver and guide him in traffic conditions.

For an example, when a driver is very angry, not emotionally stable, stressed and the driver is

increasing his speed of the vehicle, then this technology can detect it and reduce the speed,

or use auto-drive mode stating the emotional instability of the driver and help him drive back

8 | P a g e

Blue Eyes Technology

safely. Proper exceptions should be implemented for situations like emergency or in case of

ambulance to not make the technology look like a hindrance to the driver. Also they can be

used to alert the users who are drowsy or sleepy during driving.

Medical supervision is possible with this technology. It can help the doctor in concentrating

more on his work than getting distracted for putting down all the observations he is making.

He can go through data of many patients and then just tell the computer to make note of the

observations he is making through voice instructions. This helps especially in case of x-rays,

scans etc. Monitoring of patients can be done by detecting the physiological conditions of

him/her like blood pressure, pulse rate, oxygenation. The position of the person can be

determined like whether the person is sitting or standing or lying down.

Access to data can be made by just giving a simple command to the computer rather than

executing a physical search for data. Stating the urgency of the situation, the computer can

be made to prioritize the tasks to be done. Like in a situation where the user have to make an

urgent call to some other location which is a known location for both the sides, the technology

can assess the priorities of the call on both the sides and could make the call in a faster pace.

Other use might be where we can access net by just seeing the computer on the link we

would like to open in a search, where the search can be initiated by voice command. With

more ease to access the data the risk of intrusion in the systems increase. Proper biometric

authentication systems should be in place to protect the sensitive data.

Acquiring data through Bluetooth. The data is available to anyone now in wireless mode and

can be collected also in easy manner.

In future, this can be implemented in House hold devices. The technology can make every

day life more comfortable. We can switch on or off Television, music player, a/c, fan, washing

machine, oven, coffee machine etc by just having one look at them or by instructing them

through voice commands. Imagine the computer showing you the list of events for the day

and displaying happy pictures and mails to you every morning to cheer you up.

Economic Potential:

Technology is like the engine of economic growth. Advancement in technology has always been a

major contributor of growth in terms of productivity.

Change in the technology always has affected the number of jobs directly. While skilled labour being

appreciated always, low-skilled labours are replaced by the new technology. Thus it can affect the

total employment opportunities available. But it increases the demand for more scientists and

engineers in the field.

Change in the organizational structure can also be initiated. The organization includes industries,

politics and society too. The changes are reflected in organizations in form of economy at large

9 | P a g e

Blue Eyes Technology

meaning it has to change the way it works and get updated with the new technology involving lots of

new investments to be made.

The countries now compete with each other in terms of their knowledge, networking capability etc.

Change in the computers would then require adjustments to harshness information for economic and

social developments. These adjustments require urgent new policies to be devised, appropriate

regulatory and institutional reforms and investments to be done.

Computers had a drastic effect of rapid diffusion through almost all sectors of the economy. If blue

eye technology comes in future, it will have even bigger effect of combining computers with not

electronics and telecommunication but also in other fields of production, manufacturing etc also.

It can minimize the ecological consequences of financial loss which is a threat to human life.

They are likely to reduce the cost of goods sold as the number of people involved can be decreased.

And more over automated mechanisms can deliver faster results than the manual handling.

It can create both challenge and means to new ways of doing business and put the infrastructure

required in place. The challenge is created in the sense that old methodologies have to be improved

to update them to be compatible with the new technology.

Challenges faced:

This faces both technological and implementation challenges.

It cannot predict or interfere with the thoughts of the user. They cannot instruct or force the

users to work.

Misinterpretation of the emotions can happen as emotions are expressed in different manner

by human beings.

Gaze tracking methodology is not very accurate in the implementation. It might be annoying

factor for users when it is not accurate.

Speech recognition for languages with local dialect variations for the same is very difficult to

be tracked and implemented as they require proper grammatically correct language with a

neutral dialect for recognition of the words to be converted to action.

Multiple interpretations for a single gesture can be misleading information obtained.

Accuracy of the technology with the distance.

Implementation of the technology under various climatic conditions i.e. like implanting the

electrodes for translating under rain or snow.

Assessing the likes and dislikes by the stress or variation in the touch of the user.

10 | P a g e

Blue Eyes Technology

Current Scenario and Future:

IBM, Sony and MIT (Massachussets Institute of Technology) are the important pioneers in the field.

Success achieved in implementation of the technology has been limited as they have a lot of

challenges to encounter on their path. Identifying different emotional states by translating the

neurological activities has been a major challenge. Research is still going on in the field and

commercial availability of the technology is expected to be in sometime very soon.

The technology on its successful implementation has got limitless application in every possible

industry out there in the world. Interactive environment will be built where anything is possible without

making the user feel that the technology is actually complex in nature. Till today the technology

advancement has required the users to adapt themselves to the technology by dedicating time to

learning what it is all about, making them feel they are ignorant of it. This new technology actually will

make the user more comfortable.

Many Business aspects are also involved in the future. From marketing perspective of business, the

targeted customers can be attracted with interactive entertainment and advertisement. Live feedbacks

of the customers can be observed and recorded without much bothering the customers with the old

method of person to person feedback collection mechanism with questionnaire. The reaction to the

advertisements can be taken as a part of the feedback from the customers. The information about

their major competitors and how the consumers feel about them can be obtained.

11 | P a g e

Blue Eyes Technology

Summary:

Human are prone to error. They are not perfect, they fail drastically at times. A small mistake

committed unintentionally can cause a disaster for them. Machines can be helpful in preventing the

mistakes and correcting the errors of human. The opinion about machines as being complicated and

difficult to handle will be changed by blue eyes. User friendliness is increased in computing devices

making the life more convenient and simple possible. The gap between the physical and electronic

world can be minimized to a bare minimum.

12 | P a g e

Blue Eyes Technology

References

http://www.almaden.ibm.com/cs/BlueEyes/index.html

http://domino.research.ibm.com/comm/wwwr_thinkresearch.nsf/pages/interface198.html

http://www.almaden.ibm.com/u/zhai/papers/magic/magic.pdf

http://www.allbusiness.com/technology/955157-1.html

http://www.ecse.rpi.edu/homepages/qji/HCI/mouse.html

http://affect.media.mit.edu/

http://www.toyotaoffroad.com/Articles/Toyota/POD/toyota_pod.htm

13 | P a g e