©SIProp Project, 2006-2008 1

Auto Chasing Turtle

Hirotaka NiisatoNoritsuna Imamura

©SIProp Project, 2006-2008 2

About ourselves

OurselvesHirotaka Niisato(@hirotakaster)Noritsuna Imamura(@noritsuna)

About SIProp ProjectOpen Source Software ProjectThe world of media (communication medium) is extended.

©SIProp Project, 2006-2008 3

3/11

EarthquakeTsunamiNuclear Plant

©SIProp Project, 2006-2008 4

Agenda

Summary

Part “Software”Detect the faceCalculate the courseCalculate the distance

Part “Hardware”How to control the robotHow to process the analog dataNeed more power supply

©SIProp Project, 2006-2008 5

Summary

This product is "Auto Chasing Turtle".By autonomous control, this robot recognizes people's face and approaches to the detected human.

©SIProp Project, 2006-2008 6

Robot behavior

1. Rotate and look for the human who becomes a target.

2. Detect the human with recognizing people's face by using RGB camera of Kinect.

3. If it can detect the human, it calculates the course which it should follow. And It direction is changed.

4. Calculate the distance to target by using Z(depth) camera of Kinect.

5. Walk toward to target.6. 2-5 are repeated until it becomes a suitable

distance. And if losts target, returns to 1.7. The scene that it is working in real time can be

seen by iPad.

©SIProp Project, 2006-2008 7

Movie

©SIProp Project, 2006-2008 8

YouTubehttp://www.youtube.com/watch?v=8EgfAk5RBVo

Source Code & detail explanationhttp://www.siprop.org/ja/2.0/index.php?product%2FAutoChasingTurtle

Movie

©SIProp Project, 2006-2008 9

complete set parts

RobotKONDO Animal 01

ControllersFor servo

RCB3

For applicationBeagleboard-xM

Linaro-KernelAndroid(Embedded Master)

SensorsKinect

RGB cameraZ(Depth) camera

Connect to outside

Radio waveWi-Fi router

VieweriPad

Power supply12V1A output10V1A output5V3A output

©SIProp Project, 2006-2008 10

Why this project?

DENSUKE projectMobile Internet Device with Cyber-Pet User Interface

©SIProp Project, 2006-2008 11

DENSUKE project

OESF(Open Embedded Software Foundation) 　 Future Systems WG’s mission

http://fswg.oesf.biz/

Partners Project leaderMr. Ohtsuki

The Father of AIBO

©SIProp Project, 2006-2008 12

Detail of “KINECT Turtle world”

©SIProp Project, 2006-2008 13

Why KINECT and Robot ?

Application have changed Game, Motion Controller, NunchukCell Phone, Tablet

©SIProp Project, 2006-2008 14

Why KINECT and Robot ?

KINECT show the next futureCamera, Depth censor.Recognition of human’s data.Mobility Problem ...

©SIProp Project, 2006-2008 15

Why KINECT and Robot ?

KINECT on Robot is result of mobility problem.

Camera dataDepth censor

Analyze

Robot movement

INPUT OUTPUT

©SIProp Project, 2006-2008 16

Part “Software”

©SIProp Project, 2006-2008 17

openFrameworks x

Kinect x

Android

ofxDroidKinect

©SIProp Project, 2006-2008 18

About ofxDroidKinect

©SIProp Project, 2006-2008 19

About ofxDroidKinect

This is the Android Application Framework which runs on openFrameworks & uses Kinect.

Using softwaresopenFrameworks for AndroidLinaro Android

©SIProp Project, 2006-2008 20

Control Robot and KINECT

Linaro Android

openFrameworks

ofxDroidKinect

Robot Kinect

iPad

Serial USB

Wifi

©SIProp Project, 2006-2008 21

Analyze KINECT data

Detect the Face

Calculate the course

Calculate the distance

1. detect

2. course

3. distance

©SIProp Project, 2006-2008 22

Detect the face

What’s data ?640x480 RGB image, using KINECT’s camera, Characteristics of human face.

KINECT Image

©SIProp Project, 2006-2008 23

Detect the face

Face recognitionCharacteristics of human faceStatistical face detection and machine learning.

©SIProp Project, 2006-2008 24

Detect the face

Not all frame, but realtime analyzing.

Auto search around people.

Chase specified human.

©SIProp Project, 2006-2008 25

Calculate the course

Using width.1. Specify center position of face.2. Specify position of the face in 4-sections

relavive to KINECT’s image.

©SIProp Project, 2006-2008 26

Calculate the course

Using height.Change Kinect’s angle.Specify position of the face in image relavive to KINECT’s image.

©SIProp Project, 2006-2008 27

Calculate the distance

1. Mesure the depth of specified point using KINECT.

2. Robots walk forward or back, depending on the mesured distance.

©SIProp Project, 2006-2008 28

Performance problem

Data analyze and image processing

Face recognition

More resource !!GPU, Multi core.

©SIProp Project, 2006-2008 29

Part “Hardware”

©SIProp Project, 2006-2008 30

©SIProp Project, 2006-2008 31

How to control the robot?

Only control servos’ angle.AccuracyReaction velocityTorqueCompact(Size, Weight)

©SIProp Project, 2006-2008 32

How to control a servo?

RCB series24 controls pinsControl commands

Control all servo at same time !

©SIProp Project, 2006-2008 33

More control

Do you know this IC?

©SIProp Project, 2006-2008 34

How to process more sensors?

Analog data is stream data!Need powerful processorNeed to save power for Mobile

Now SolutionARM base Processor

dual Cortex A9MALI 400 GPU

Next trendGPGPUFPGA

©SIProp Project, 2006-2008 35

How to optimize for each sensors?

　　　　　　　　　 http://linaro.org/

MissionOptimize for each SoC & Platform

We are committer!

©SIProp Project, 2006-2008 36

Trouble: Need more power supply

Don’t run beagleboard-xM by 5V/1A power supply.

It needs 5V/2A.Standard USB battery has 5V/1A.

Create power supply by myselfMy battery has 12V/2A.My plan is to create convertor of 12V/2A ⇒ 5V/2A.

Can’t get power IC…Using car’s cigarette adapter which convert to USB.

12V ・ 24V/2A ⇒ 5V/2A

©SIProp Project, 2006-2008 37

MAKE:style

Hardwarebeagleboard-xMKONDO AnimalKinect

SoftwareofxDroidKinect

Linaro KernelAndroid(Embedded Master)

　 Do It Yourself ⇒Do It With Others!