A PAPER PRESENTATION

ON

SIXTH SENSE TECHNOLOGY

PRESENTED BY: J.G.M.Jagagdeesh Kumar

Department of C.S.E. (IV year)

Affiliated to JNTU K

DJR College of Engineering and Technology,

Gudavalli, Vijayawada

Krishna (dt.), Andhra Pradesh, India.

A BS T RAC T

Sixth Sense is a wearable gestural

interface device developed by Pranav

Mistry, a PhD student in the Fluid

Interfaces Group at the MIT Media Lab. It

is similar to Telepointer. The Sixth Sense

prototype comprises a pocket projector, a

mirror and a camera contained in a

pendant like, wearable device. Both the

projector and the camera are connected

to a mobile computing device in the user’s

pocket. The projector projects visual

information enabling surfaces, walls and

physical objects around us to be used as

interfaces; while the camera recognizes

and tracks user's hand gestures and

physical objects using computer- vision

based techniques. The software program

processes the video stream data captured

by the camera and tracks the locations of

the colored markers (visual tracking

fiducially) at the tip of the user’s fingers.

The movements and arrangements of

these fiducially are interpreted into

gestures that act as interaction

instructions for the projected application

interfaces. Sixth Sense supports multi-

touch and multi-user interaction.

IN T R O DUC T I O N

Sixth Sense is a wearable

gestural interface that augments

the physical world around us with digital

information and lets us use natural hand

gestures to interact with that information.

It was developed by Pranav Mistry, a PhD

student in the Fluid Interfaces Group at the

MIT Media Lab.

Sixth Sense bridges the gap by bringing

intangible, digital information out into the

tangible world, and allowing us to

interact with this information via natural

hand Sixth Sense comprises a pocket

projector, a mirror and a camera. The

hardware components are coupled in a

pendant like mobile wearable device Sixth

Sense is a mini-projector coupled with a

camera and a cell phone—which acts as

the computer and your connection to the

Cloud, all the information stored on the

web. Sixth Sense can also obey hand

gestures, like in the infamous Minority

Report. The camera recognizes objects

around you instantly, with the micro-

projector overlaying the information on any

surface, including the object i tself or your

hand. Then, you can access or manipulate

the information using your fingers. Need to

make a call? Extend your hand on front of

the projector and numbers will appear for

you to click. Need to know the time? Draw

a circle on your wrist and a watch will

appear. Want to take a photo? Just make a

square with your fingers, highlighting what

you want to frame and the system will

make the photo—which you can later

organize with the others using your own

hands over the air. The true power of Sixth

Sense lies on its potential to connect the

real world with the Internet, and overlaying

the information on the world itself.

W H Y S I X TH S E N SE?

C O M PO N E N T

The hardware components are coupled in a

pendant like mobile wearable device.

Camera

Projector

Mirror

Mobile Component

Colored Markers

CA MERA

Captures an object in view and

tracks the user’s hand gestures

It sends the data to smart

phone

It acts as a digital eye,

connecting you to the world

of digital information

P R O JE C T O R

The projector projects visual

information enabling surfaces

and physical objects to be

used as interfaces.

The project itself contains a

battery inside, with 3 hours of

battery life.

A tiny LED projector displays

data sent from the smart phone on

any surface in

View–object, wall, or person

M IRR O R

The usage of the mirror is significant

as the projector dangles pointing

downwards from the neck.

SM AR T PHO N E

A Web-enabled smart phone in the

user’s pocket processes the video data Other

software searches the Web and interprets

the hand gestures

C O L O R M AR K E R S

It is at the tip of the user’s fingers. Marking

the user’s fingers with red, yellow, green,

and blue tape helps the webcam recognize

gestures

The movements and arrangements of these

makers are interpreted into gestures that

act as interaction instructions for the

projected application interfaces.

HO W I T W O R K S?

The hardware that makes Sixth Sense work

is a pendant like mobile wearable interface

It has a camera, a mirror and a projector

and is connected wirelessly to a

Bluetooth smart phone that can slip

comfortably into one’s pocket

The camera recognizes individuals,

images, pictures, gestures one makes with

their hands

Information is sent to the Smartphone

for processing The downward-facing

projector projects the output image on

to the mirror

Mirror reflects image on to the desired

surface

Thus, digital information is freed from its

confines and placed in the physical world

R EL A T E D TE C H N O L OG I ES

Augmented reality is a term for a live

direct or indirect view of a physical

real world environment whose

elements are augmented by virtual

computer generated imagery.

Gesture recognition is a topic in

computer science and

language technology with the goal

of interpreting human gestures via

mathematical algorithms.

Computer vision is the science and

technology of machines that see. It

is concerned with the theory

behind artificial systems that extract

information from images.

Radio Frequency Identification is

basically an electronic tagging

technology that allows the detection,

tracking of tags and consequently

the objects that they are affixed to.

A PP L ICA T I O N S

Make a ca l l

You can use the Sixth Sense to project a

keypad onto your hand, then use that virtual

keypad to make a call.

Call up a map

With the map application we can call up

the map of our choice and then use

thumbs and index fingers to navigate the

map

Check the time

Draw a circle on your wrist to get a virtual

watch that gives you the correct time

Create multimedia reading experiences

Sixth Sense can be programmed to project

related videos onto newspaper articles you

are reading

Drawing application

The drawing application lets the user

draw on any surface by tracking the

fingertip movements of the user’s index

finger

Zooming features

The user can zoom in or zoom out using

intuitive hand movements

Get product information

Sixth Sense uses image recognition or

marker technology to recognize products

we pick up, and then feeds us information

on those products

Get book information

The system can project Amazon ratings on

that book, as well as reviews and other

relevant information

Get flight updates

The system will recognize your boarding

pass and let you know whether your

flight is on time and if the gate has

changed.

Feed information on people

The system will project relevant

information about a person such as what

they do, where they work, and so on.

Take pictures

If you fashion your index fingers and

thumbs into a square ("framing" gesture),

the system will snap a photo.

After taking the desired number of photos,

we can project them onto a surface, and

use gestures to sort through the photos, and

organize and resize them.

Educational Use

Engagement

Teachers may find that Sixth Sense,

because of its unique ability to work

with many of Gardner’s multiple

intelligences, will be a catalyst for

students who might otherwise

remain unengaged in learning.

Sixth Sense uses the visual,

kinesthetic, intrapersonal and

interpersonal among others.

If this technology does nothing else,

the idea that students can simply

use their hands to draw images on

walls and take pictures with their

hands will be (at least initially)

quite motivating

Research

Sixth Sense will give new meaning

to “looking something up.”

Since we are essentially talking

about the internet, students will be

given the opportunity to portably

research the world.

They can all go on a virtual field

trip and project the images on their

own desks, for example.

Collaboration

Students will be given an

opportunity through this technology

to feel more connected to others.

Although there does seem to be the

sense that individuals could walk

through life wired and lonely, I like

to think the classroom would serve

as the environment in which

students and teachers using Sixth

Sense would want to share.

Not only could the sharing take

place in the classroom, I suppose

since the users are connected

online, sharing could take place

around the world.

Cost-Effectiveness

This product currently costs only

$350 (approx. Rs.17000) . From an

educator’s stand point, this is

minimum given the amount of money

spent on textbooks, computer labs and

LCD projectors.

For educator’s, Sixth Sense would be

a way to put technology (literally)

into each student’s hands, because of

its size and its cost.

A D VAN T A G ES

Portable

Inexpensive

Cool

Multi-Sensory

Supports multi-touch and multi- user

interaction

Connectedness between world and

information

Cost effective

Data access directly from machine in

real time

Mind map the idea anywhere

It is an open source

Critical thinking would be somewhat vital

here….allowing teachers a fantastic

opportunity for interdisciplinary

study with critical thinking components

C O NC L U S I O N

Sixth Sense recognizes the objects around

us, displaying information automatically

and letting us to access it in any way we

need

The Sixth Sense prototype implements

several applications that demonstrate the

usefulness, viability and flexibility of the

system

Allowing us to interact with this

information via natural hand gestures

The potential of becoming the ultimate

"transparent" user interface for accessing

information about everything around us

REFERENCES

http://www.pranavmistry.com/projects/

sixthsense/

https://www.google.co.in/search?

q=sixth+sense+pranav+mistry&sa=X&tbm

=isch&tbo=u&source=univ&ei=lGbAUZL

TJc3jrAeahYCgBA&ved=0CDYQsAQ&bi

w=1241&bih=640&sei=p2bAUdraCYnVk

QXerYHoAg

http://en.wikipedia.org/wiki/SixthSense

http://www.ted.com/talks/

pranav_mistry_the_thrilling_potential_

of_sixthsense_technology.html