Dzmitry Tsetserukou Assistant Professor Space  Robo*cs  Lab  Wearable  Media  Lab  

Intelligent  Robo,cs    and  Wearables  

The best way to predict the future is to invent it.

Steve Jobs

Outline

1.  Research Background

2.  Teleoperation system TeleTA and robotic hand ExoPower

3.  NaviGoid: Mobile Robot Navigation with Haptic Vision

4.  LinkTouch: Wearable Fingertip Haptic Device

5.  iFeel_IM! Communication system 2

Susumu Tachi (born January 1, 1946) is a professor of Graduate School of Media Design at Keio University and Professor Emeritus of the University of Tokyo. Prof. Tachi is also a Founding Director of the Robotics Society of Japan (RSJ), a Fellow of the Society of Instrument and Control Engineers (SICE) and is the Founding President of the Virtual Reality Society of Japan (VRSJ).

Invisible  cloaking   Twister  

Telesar  IV  

Teleoperation system TeleTA

LinkTouch: true 2D display

NAVIgoid Robot Navigation with High Level of Immersion

Robot motion control by torso tilt

Tactile, visual and auditory

feedback

Mobile Robot User wearing a Belt Interface

LRF UHG-08LX by HOKUYO

Robot-side PC CPU: Intel Core i7 640M (2.8 GHz) Mem:8GB

Mobile Robot PeopleBot by MobileRobots

Belt Interface

NAVIgoid

(a) Human operator (b) Mobile robot PeopleBot

Single tactor

Vr

xo

yo

Vh

Human

Obstacle

Wall

ro

rh

rw

User bends forward

Belt Interface

yr

xr

xu

yu

0

Flex sensor

LRF

NOVELTY: • engaging the user into teleoperation • a h i g h l e v e l o f immersion through p r o p r i o c e p t i v e telepresence • tactile feedback

GOAL: development a new type of tactile interface a n d t e l e p r e s e n c e robotic system that can make operator of being “embodied”.

Elastic band

User’s waist

Flex sensor I

Flex sensor II Flex sensor III

Flex sensor IV

Accelerometer

Front

Back Right

Left

Geomagnetic sensor

Lug

Accelerometer

Vibration motor

Flex

sens

or

Contact slider

Hol

der

ProInterface

Stationary obstacle

distance direction shape

M o v i n g obstacle

distance location motion direction

The ProInterafce (Proprioception-controlled Interface) detects the trunk stance through the flex sensors. The tactors vibrate to produce the tactile stimuli indicating:

Tracking of Moving Obstacle

(a) 226th frame (b) 227th frame (c) 228th frame (d) 229th frame (e) 230th frame

(a) 231th frame (b) 232th frame (c) 233th frame (d) 234th frame (e) 235th frame

: robot, : new detected obstacle, : moving obstacle, : velocity of moving obstacle

•  Object is tracked by using KF. ( ) ( )Tytxtytxt

TTot

Totot vvpp ,,,, == vpx

( )I0qq0

vvp

vp

x 2

1

11 )8(,, tN

tvt

vtot

otot

ot

otto Δ⎟⎟

⎠

⎞⎜⎜⎝

⎛+⎟⎟⎠

⎞⎜⎜⎝

⎛ Δ+=⎟⎟⎠

⎞⎜⎜⎝

⎛=

+

++ ～

State vector:

Object motion model (motion prediction):

position speed

process noise

Cloud swarm robotics

LinkTouch

Operation principle of PulseTouch

LinkTouch: true 2D display

Inverted five-bar linkage mechanism

Fn C Fn

Ft

F l1

l4 l3 l2

[█■𝐹↓𝑡 @𝐹↓𝑛  ]=[█■𝑆↓12 𝐶↓2 &− 𝑆↓43 𝐶↓3 @− 𝑆↓12 𝑆↓2 &𝑆↓43 𝑆↓3  ]  [█■𝜏↓1 ∕𝑙↓2  @𝜏↓2 ∕𝑙↓5   ]

l5

Motor Faulhaber DC Motor 0615 4,5 S

Gear Ratio GH Series 06/1 64:1

Encoder HXM3-64

D i m e n s i o n s (W)×(H)×(D) [mm] 26.1×32×38.5

Link length l1, l2, l3, l4, l5 [mm] 18, 25, 10, 10,25

Weight [g] 13.5 Normal force [N] 0.58

iFeel_IM! Communication system with rich emotional and haptic channels

The philosophy behind iFeel_IM! (intelligent system for Feeling enhancement powered by affect sensitive Instant Messenger) is

iFeel_IM!: communication system with emotional and haptic channels

“I feel [therefore] I am!”

D. Tsetserukou, et al. Enhancing mediated interpersonal communication through affective haptics, INTETAIN 2009, pp. 246-251.

Architecture of iFeel_IM!

HaptiHeart HaptiHug HaptiTickler HaptiBatterfly

HaptiTemper and HaptiShiver

chat text

3D world Second Life

emotion: intensity

Web-based interface of Affect

Analysis Model

PC of each user

Chat log file

Haptic Devices Controller

D/A

Driver box

Affect Analysis Model: recognition of emotions from text

Test for emoticons, abbreviations, acronyms, interjections, “?” and “!” marks, repeated punctuation and capital letters

Sentence

Estimation of resulting emotion state

Sentence pre-processing for parser

Symbolic cue analysis module

Emoticon or em abbr. “yes” “no”

Sentence annotated by

emotion

emotion category: intensity emotion category: intensity

Syntactical structure analysis module

Stanford Parser

Parser output processing

output of the parser

Word-level analysis module

Phrase-level analysis module

Sentence-level analysis module

I stage

II stage

V stage

III stage

IV stage

Affect database

Emotions: Anger, Disgust, Fear, Guilt, Interest, Joy, Sadness, Shame, Surprise

Website:  h:ps://sites.google.com/site/dzmitrytsetserukou/  Email:  d.tsetserukou@skoltech.ru      

Спасибо за внимание! Thank you for your attention!