Blue Eyes Technology

By : Priyanka Datta(Roll No.-23)

Nilanjana Sarkar(Roll No.- 18)Stream-CSE (A)

Batch:2013-2017

The Emotion Sensory World identifying human emotions defining new cognitive approaches to

the vast technological world.

Contents• INTRODUCTION• REASEARCH & INVENTION• SYSTEM OVERVIEWS• EMOTION COMPUTING• RESULTS OF EMOTION COMPUTING• EMOTION SENSORS• APPLICATIONS• THE IBM ALMADEN EYE TRACKER• MEASURED & GAZE INPUT CASCADED MAGIG

POINTING• IMPLEMENTATION of MAGIC POINTING• ARTIFICIAL INTELLIGENT SPEECH RECOGNITION• Conclusion• References

IntroductionHuman cognition depends primarily on the ability to perceive, interpret, and integrate audio-visuals and sensoring information. Adding extraordinary perceptual abilities to computers would enable computers to work together with human beings as intimate partners. Researchers are attempting to add more capabilities to computers that will allow them to interact like humans, recognize human presence, talk, listen, or even guess their feelings. The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings. It uses sensing method, employing most modern video cameras and microphones to identify the user’s actions through the use of imparted sensory abilities. The machine can understand what a user wants, where he is looking at, and even realize his physical or emotional states.

Research & Invention(Historical Background)

Paul Ekman’s facial expression work gave the correlation between a person’s emotional state & psychological state which described the Facial Action Coding System (Ekman and Rosenberg,1997).

Blue Eyes is a technology conducted by the Research Team of IBM at its Almaden Research Institute(ARC) ,Sanjose , California since 1997.

Blue Eyes Technology aims at creating a computer that have the abilities to understand the perceptual powers by recognizing their facial expressions and react accordingly to them.

One of his experiments involved participants attached to devices to record certain measurements including pulse, galvanic skin response (GSR), temperature, somatic movement and blood pressure.

What is FACS?FACS-Facial Action Coding System.Paul Ekman and W.V Friesen developed the original FACS in 1970s.It is a guide to categorize the facial expressions.It is based on the muscular contractions which produce our facial expressions.

THE TERM: Blue Eyes(Why this technology is referred to as Blue Eyes?)

Blue in terms of Bluetooth which enable reliable wireless communication .

Eyes because the eye movement enable us to obtain a lot of interesting and important information.

People spend approximately 1/3 of their total computer time touching input device. Physiological data is obtained and emotional state is determined during the process.

System OverviewBlue Eyes system consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the central unit. ID cards assigned to each of the operators and adequate user profiles on the central unit side provide necessary data personalization's so the system consists of:

Mobile measuring device (DAU)Central System Unit(CSU)

Data Acquisition Unit is a mobile part of the Blue eyes system. Its main task is to fetch the physiological data from the sensor and to send it to the central system to be processed. To accomplish the task the device must manage wireless Bluetooth connections (connection establishment, authentication and termination).Central System Unit hardware is the second peer of the wireless connection. The module is interfaced to a PC using parallel, serial and USB cable.

Emotion Computing

Emotion Computing also known as Affective Computing is the study and development of systems and devices that can recognize, interpret, process and simulate human affects.It is an in disciplinary field spanning computer science, psychology and cognitive science.Rosalind Picard ‘s 1995 paper gave birth to a new technology “Affective Computing” to computer science field .Emotion Computing aims to bridge the gap between emotions and computational technology.There are two aspects of emotion computing: giving the computer the ability to detect emotions and to express emotions. Emotion detection is an important step to an adaptive computer system. Some of the important algorithms which allows a computer system to identify quick and accurate emotion are: Linear Discriminant Classifiers (LDC),k-nearest neighbour(k-NN), Gaussian mixture model (GMM),etc.

TYPES OF EMOTION SENSORS Emotion Sensors can be for hand, eyes and voice.

•For Hand:•Emotion Mouse•Sentic Mouse

•For Eyes:• Expression Glasses•Magic Pointing •Eye Tracking

•For Voice:•Artificial intelligence speech recognition

Emotion Mouse: • One method for gaining user information through touch is via a computer input device, the mouse. • This allows the user to relate the cardiac rhythm, the body temperature, electrical conductivity of the skin and other physiological attributes with the mood. This has led to the creation of the “Emotion Mouse”.

Sentic Mouse:It is a modified computer mouse that includes a directional pressure sensor for aiding in recognition of emotional valence. continued to next slide......

TYPE OF EMOTION SENSORS Expression Glasses :A wearable device which allows any viewer to visualize the confusion and interest levels of the wearer.

The Sentic Mouse is an experiment inspired by the work of Peter J. Lang, Ward Winton, Lois Putnam, Robert Kraus and Dr. Manfred Clynes.

IBM Almaden Eye TrackerEye tracker was first developed at IBM Almaden research institute.Working of Eye Tracker(in brief) :Its first task is to detect the pupil and it is done using infra-red rays.

Once the pupil has been detected, the corneal reflection (the glint reflected from the surface of the cornea due to one of the light sources) isdetermined from the dark pupil image.

The reflection is then used to estimate the user's point of gaze in terms of the screen coordinates where the user is looking at.

The following picture on the next slide would make this above explanation more vivid. continued to next slide..

Applications Speech recognition system ->it allows the users to do multiple works simultaneously.

Radiologist scanning hundreds of X-rays, ultra sonograms, CT scans and simultaneously dictating conclusions to a speech recognition system.

Visual attention monitoring (eye motility analysis)

Physiological condition monitoring (pulse rate, blood oxygenation)9. Operator's position detection (standing, lying).--------------------------------------------------------------------------------------------

MAGIC POINTING(Introduction)

Reduce the cursor movement needed for target selection.Based on Eye tracking.This work explores a new direction in utilizing eye gaze for computer input. Click on the target with a regular eye tracker.

MAGIC POINTING(Types)

LIBERAL CONSERVATIVE

To wrap the cursor to every new object user looks at.

Does not wrap the cursor target until the manual input device has been actuated.

IMPLEMENTATION OF MAGIC POINTING

MANUAL INPUT DEVICE ( such as mouse ).

EYE TRACKING SYSTEM (connected by serial port).

ARTIFICIAL INTELLIGENT SPEECH RECOGNITION

Input words are scanned and matched with the internally stored words.

Speech recognition is done using microphone.

Filtered and fed to ADC and stored in RAM.

APPLICATIONS OF AISR Can be used in military operations ( e.g. Voice control).

Pilots can give command and information to the computers by speaking into microphones.

Voice recognition system can be connected to word processor.

SUITOR( THE SIMPLE USER INTEREST TRACKER )

Helps in fetching more information at his desktop.

Fills a scrolling ticker on a computer screen with information related to user’s current task.

e.g. If I start to read a story, pops up the story in the browser window.

ADVANTAGES

Reduces manual stress and fatigue.

Reduces human effort.

Prevention from dangerous incidents.

DISADVANTAGES Cannot force the operator to do work.

Target not selected if not looked at for a set threshold.

Target selected if stared at without user intension.

FUTURE AND CURRENT DEVELOPMENTS

EMOTIONAL MOUSE.

MAGIC POINTING.

SUITOR

In future may be ordinary house hold devices may do their jobs by looking at them or speaking to them.( e.g. Ovens, televisions etc).

CONCLUSION Provides more delicate and user friendly in computing devices.

Gap between the electronic and physical world is reduced.

The computer can run on implicit commands instead of the explicit commands.

References

http://www.research.ibm.comhttp://www.media.mit.eduhttp://www.thehumansolution.comhttp://www.academia.eduhttps://en.wikipedia.org/wiki/Affective_computinghttp://youtube.com/ https://en.wikipedia.org/wiki/Magic_Pointing

Thank You!