telenoid robot

7/29/2019 telenoid robot

1/72


2/72

We present a humanoid robot thatresponds to human gestures seen bya camera.

The behavior of the robot can be

completely deterministic as specifiedby a Finite State Machine that mapsthe sensor signals to the effectorsignals.

This model is further extended to the

constraints-satisfaction based modelthat links robots vision, motion,emotional behavior and planning.

Use adiabatic quantum computerwhich quadratically speeds-up every

constraint satisfaction problem andwill be thus necessary to solve largeproblems of this type.

We propose to use the remotely-connected Orion system by DWAVE

Corporation.

WHAT HASBEEN DONE?


3/72

Emotional

Robot Helpers

Because humans attribute emotions to other humans andto animals, future emotional robots should perhaps bevisually similar to humans or animals,

otherwise their users would be not able to understand robotsemotions and correctly communicate with them.

Observe that the whole idea ofemotional robot helpers isto enable easy communication between humans and

robots.


4/72

Robot emotions

Simple emotions like fear or anger orbehaviors like obstacle-avoidance forwheeled mobile robots.

Subsumption architecture.

Practically insufficient to cover allnecessary behaviors offuture household

helper robots.

The research on robot emotions

and methods to allowhumanoid robots to acquirecomplex motor skills isrecently advancing at a veryfast pace.


5/72

Larger biped robots are veryexpensive hundreds thousands dollars.

Recent small humanoid robots.

We acquired two KHR-1

robots and integrated them toour robot theatre system withits various capabilities such as: sensors,

vision, speech recognition and

synthesis

Common Robot Language.

Emotions can bebest expressed

by a biped robot withhuman-like face


6/72

Walking bipedrobot canexpress the

fullness ofhuman emotions: body gestures, dancing, jumping, gesticulating

with hands.

Emotions canbe: Emergent -

Arushi Programmed

Martin LukacISMVL

Mimickedthis paper

LearnedMartin LukacReed-Muller

Humanoidrobots toexpressemotions:

M. Lukacuses human-like faces andhead/neck

bodycombinations.

KAIST theatreused whole-body

stationaryrobots withhands.


7/72

KHR-1 from Japan


8/72

Project

Overview

KHR-1 Biped robot

17 servos

2 RCB-1 servo

controllers (each

12 servos) Serial port

connectivity


9/72

Accomplishments

Movements

We worked through many mechanical

challenges

Trim

Balance

Power


10/72Cross Arms and Servo hones

The first objective was to make

the robot executing what isadvertised: walking forward and backward, dancing, doing pushups, etc.

All documentation was inJapanese or Korean

The English translation wasdone only on our request.

Some small components suchas screws, washers, and servohones were missing

Assembly should be verycareful. Is not easy.

KHR-1 Hardware, Assembly and Maintenance.


11/72

RCB-1s controllers and Servo Cable Arrangements.

Labelingof theServo

motors

Be sure that you know the labels of all

servos.

You should understand how this servo

contributes to walking, pushups andother behaviors.

Start from hand movements.

Be sure that you know the connections

of RCB-1 boards, which is which.


12/72


13/72


14/72

more

There are three types of data that make

the KHR-1 so versatile.

Positions

A single list containing position data for each servo

There can be 99 positions stored


15/72

more

Motions

There can be 40 motion files stored

Motions can contain 40 positions

Are used to make things like, walk, turns, and

more.


16/72


17/72

Challenges

We ran into a number of challenges with

the project.

Power needs

Mechanical issues

Communications (Robot to PC)

Vision

Integration


18/72

more

Created our own movements Right turn

Left turn

Fight

Modified old files to work with this robot Walk

Push-ups Dancing

Fights


19/72

Communications

RS232 PC to Robot

All commands can be sent

Calls can be made

Servo Positions

Movement info

Ect

Commands are sent as list of hex values thatrepresent all needed data.


20/72

What we have done We created our own cotrolling software environment in Visual Basic

that we can fully understand and modify

We have written a more inclusive instruction set for setting up theKHR-1

Including basic trouble shooting info.

We have explained in detail the data structure types.

Created a base VB comm setup that will give future students somewhereto start.

We now can make calls for motions and more.

More life like motion using a random numberalgorithm

The ability for the applications to generate random motions to simulatemoods


21/72


22/72

Robot safety while movement

We added limitations programmed into the VB softwarethat controls the KHR-1 so that the robot would not breaka servo by trying to push its arm into its body.

The values are limited based on the physical constraintsof the KHR-1.

Example: If both conditions are in that window then we limit theelbow so that it can not hit the body of the robot.

Without this function the KHR-1 could hit itselfandpossibly break a servo.


23/72


24/72

The Gyroscope We have only one gyroscope, and chose to control side to

side balance.

Our choice forside to side motion was due to the fact thatadditional hardware is necessary to program the servos 22and 16.

In any case, installing the gyro helped with movementstability and we plan to add also the second gyro.


25/72


26/72

Common Robot Language. We developed symbolic approach

to robot specification based on a

Common Robot Language.

While the syntax of this languagespecifies rules for generatingsentences, the semantic aspects

describe structures forinterpretation.

Every movement is described onmany levels, for instance every

joint angle or face muscle are atlow level and complete movementssuch as pushups or joyful handwaving are at a high level.


27/72

Common Robot Language. These aspects serve to describe

interaction with environment at

various levels of description.

It uses also the constraintsatisfaction problem creatingmovements that specify constraints

of time, space, motion style andemotional expression.


28/72

The goal of our Common Robot Language is to describehuman-oriented movements

But it exceeds these behaviors to those likeanthropomorphic animals and fairy tale characters.

We created new GUI interface and robot controllinglanguage specific to KHR-1. Editing functions.

Testing functions.

The ability to read information back from the robot by serialcommunication was added.

There are two main functions that we achieved: mimicking,

behavior state machine.

Describing movements, behaviors

and emotions


29/72

Using HBP robot vision software forhuman mimicking.

Control behaviors mimicked from a human standing in front ofthe camera. (with state machine or not)

We wanted the KHR-1 to mimic human motion that was being

shown on the screen by the HBP software.

The HPB works by taking an image of a persons upper body. Itthen will try and identify the face.

Once it can recognize a face it will then look at the body.

The image that it acquires is converted to a set of feature(parameters) values assigned to several groups of variables.

Wh i i h i i


30/72

What is wrong with our visionsoftware?

HBP is slow

OPENCV is slow

Robot responds with delay

HBP is not accurate

That one great thing about HPB, is that you have theoption ofmodifying the original code to some extent and

make your own features.

To speed up the image recognition we will use theOrion quantum computer in the next project


31/72

C t i t S ti f ti f


32/72

Constraint Satisfaction forEmotional Robotics

Insufficient speed of robot image processing and patternrecognition.

This can be solved by special processors, DSP processors, FPGA

architectures and parallel computing.

Prolog allows to write CSP programs very quickly.

An interesting approach is to formulate many problems using the

same general model.

This model may be predicate calculus, Satisfiability, ArtificialNeural Nets or Constraints Satisfaction Model.

Huffman and Clowes


33/72

ConstraintSatisfaction Image

Analysis by Waltz

created an approach topolyhedral sceneanalysis, scenes withopaque, trihedral solids,next improved

significantly by Waltz

Popularized theconcept of constraintssatisfaction and its usein problem solving,

especially imageinterpretation.

Objects in thisapproach had alwaysthree plane surfaces

intersecting in everyvertex.


34/72


35/72


36/72

Constraint satisfaction model in robotics

Used in main areas of robotics:

vision, knowledge acquisition, knowledge usage.

In particular the following:

planning, scheduling, allocation, motion planning, gesture planning,assembly planning, graph problems including graph coloring, graphmatching, floor-plan design, temporal reasoning, spatial andtemporal planning, assignment and mapping problems, resourceallocation in AI, combined planning and scheduling, arc and pathconsistency, general matching problems, belief maintenance,experiment planning, satisfiability and Boolean/mixed equation

solving, machine design and manufacturing, diagnostic reasoning,qualitative and symbolic reasoning, decision support,computational linguistics, hardware design and verification,configuration, real-time systems, and robot planning,implementation of non-conflicting sensor systems, man-robot androbot-robot communication systems and protocols,contingency-tolerant motion control, multi-robot motion planning, multi-robot

task planning and scheduling, coordination of a group of robots,and many others


37/72

Examples of CSP in robotics Scene recognition

Motion generation in presence ofconstraints

internal (low power, dont hit itself)

external (shape of racing track,

wolf-man-cabbage-goat)

Gesture under emotions

Communication in a swarm ofrobots (graph coloring)

Robot guard (set covering)


38/72


39/72

Robot ReasoningProblem

New Approach

to QuantumRobotics

Adiabatic Quantum Computer

Constraint Satisfaction Problem

Robot VisionProblem

RobotCommunication

Problem

RobotObstacleAvoidanceProblem

Classicalquantumcomputing


40/72

Adiabatic Quantum Computing to solve Constraint


41/72

Adiabatic Quantum Computing to solve ConstraintSatisfaction Problem efficiently.

Will February 13th 2007 be remembered in annals of computing.?

DWAVE company demonstrated theirOrion quantum computing systemin Computer History Museum in Mountain View, California.

The first time in history a commercial quantum computer was presented.

The Orion system is ahardware acceleratordesigned to solve inprinciple a particular NP-complete problem calledthe two-dimensional Ising

model in a magnetic field(for instance quadraticprogramming).

It is built around a 16-qubitsuperconducting adiabaticquantum computer (AQC)processor.


42/72


43/72


44/72

We plan to concentrate


45/72

Adiabatic Quantum Computing was proved equivalent to standard QCcircuit model.

Each of the developed by us methods can be transformed to anadiabatic quantum program and run on Orion.

We developed logic minimization methods to reduce the graph that is

created in AQC to program problems such as Maximum Clique or SAT.

This programming is like on assembly level but with time moreefficient methods will be developed in our group.

This is also similar to programming current Field-Programmable Gate

Arrays.

pon robotic applications ofthe ConstraintSatisfaction Model.

Future work on Adiabatic Quantum


46/72

Future work on Adiabatic Quantum

Controller for a robot

In the second research/development

direction the interface to Orion systemwill be learned

How to formulate front-end formulationsfor various robotic problems as

constraint-satisfaction problems for thissystem?


47/72

Conclusions and future work.

KHR-1 is now able to mimic upper body human motions.

Students who work on this project learn about robot kinematics, robotvision, state machines (deterministic, non-deterministic, probabilisticand quantum - entangled) robot software programming andcommercial robot movement editors.

The most important lesson learned is the integration of a non-triviallarge system and the appreciation of what is a real-timeprogramming.

It is important that the students learn to develop a trial and errorattitude and also how to survive using a non-perfect and incompletedocumentation.

It was also emphasized by the professor that students create a very

good documentation of their work for the next students to use.

Didactic Aspects

N l


48/72

New classes

New class teaches quantumcomputing and quantumrobotics

One of the goals of this

lecture is to help others tostart with this new andexciting research area.

KHR-1 like robot can becomea widely acceptedinternational educationplatform.

.. and finally..


49/72


50/72

Additional

Slides


51/72

Heart to Heart overview

There are several key

components in H2H

that must be used.

Motion creator Learning

Setup


52/72

Motion files

Motion screen

allows you to set

the order of

positions that willbe called. You

can also load the

motions into the

robot.


53/72

Trim

This trim setup allow

you to account for any

discrepancies.


54/72

VB progress (randomness)

Private Sub btnSetServoPos_Click()Dim SetCurrentPos As Variant

Dim TstRandomPosLoad(12) As Long

' This data will be populated from CSV file from Mike

TstRandomPosLoad(0) = Int(Rnd() * 20 + 80)












' Send out to comm port

SetCurrentPos = SetServoPos(TstRandomPosLoad(), 0, 4)

MSComm1.Output = SetCurrentPos

End Sub


55/72

VB progress (Motion call)

Private Sub btnPlayMotion_Click()Dim MotionNumInput As Variant

Dim Motion As Variant

'Get user data and check integrity

MotionNumInput = InputBox("Enter motion bank, valid #'s are 0 to 39","Scenario Data")

If MotionNumInput = "" Then

MsgBox "No data found!", , "Bad Data"

Exit Sub

ElseIf MotionNumInput < 0 Then

MsgBox "The number you entered is too low!", , "Bad Data"

Exit Sub

ElseIf MotionNumInput > 39 Then

MsgBox "The number you entered is too high!", , "Bad Data"

Exit Sub

End If

' Send out to comm port

Motion = PlayMotion(0, Int(MotionNumInput))

MSComm1.Output = Motion

End Sub


56/72

The Gyroscope

The gyroscope installed on this robot is sensitive toacceleration in only one of two possible corrective axes.

One pair of servos controls side to side balance at the

base of the feet.

Another can provide front to back correction by changingthe angle of bend at the knee joints in the legs.

It would be necessary to have two separate gyroscopesto provide balance feedback for both front to back andside to side motions

Describing movements behaviors


57/72

Describing movements, behaviors

and emotions Non-deterministic and probabilistic behaviors are possible within the

framework of constraints

They allow more natural behavior of the robot where the movements arelogical but not exactly the same in similar environmental or emotionalsituations.

Mechanisms for scripting and scenario writing are also necessary.

Humanoid robot movements and emotional behaviors require specialnotations that take their origins from human emotional gestures andmovements such as dances, sport-related and gymnastic movements as wellas theatre-related behaviors.

These notations and languages originate from choreography, psychology andgeneral analysis of human behavior.

Several notations describing human dances exist using Benesh notation,LifeForms and others


58/72

Improvements needed

The openCV software runs slow on a laptop. Gross versus small body movements hand waving or smiling?

This was accomplished by writing a subroutine which tracked the robotsarm positions and mouth size. The commands from this state machine weresent to the robot whenever the avatar from the HBP software ran theShowAvatar routine. Placing a function call to the State Machine function atthe end of the ShowAvatar routine provided the trigger mechanism for the

state machine function. The state machine code is located in the visualbasic project module modKHR1State

There are many variables in the Human Body Project software that indicaterelative position of the eyes, nose, mouth, and arms of the subject.

We used only a small subset

More experimentation with other features and a faster computer are needed.

Motion and Vision as constraint


59/72

Motion and Vision as constraint

satisfaction

A popular approach to solve many motion planning andknowledge-based behavior problems for humanoid robots is theConstraint Satisfaction Model.

Unfortunately, for future robots large problems should be solved

in real time which will require powerful computers.

Observe that while MIT Cog planned to use interaction withenvironment as a base of learning, it has no walking capability,thus its access to environment is limited.

On the other hand the walking robots such as Honda have muchdeveloped walking ability giving them access to powerfulenvironmental information, but they lack learning abilities andsophisticated models of environment.


60/72

Orion computer from DWAVE


61/72

359

Sudoku7

3

2

1

82

5

9

82

3

4

93

75

8

7

66

264

4

5




62/72

The plans are that by the end of year 2008 the Orion systems will be scaled tomore than 1000 qubits.

It is even more amazing that the company plans to build in 2009 processorsspecifically designed for quantum simulation, which represents a bigcommercial opportunity.

These problems include protein folding, drug design and many other inchemistry, biology and material science.

Thus the company claims to dominate enormous markets of NP-completeproblems and quantum simulation.

If successful, the arrival of adiabatic quantum computers will create a need forthe development of new algorithms and adaptations of existing search

algorithms (quantum or not) for the DWAVE architecture.

The arrival of Orion systems is certainly an excellent news for any researchgroup that is interested in formulating problems to be solved on a quantumcomputer.

In this project we plan to concentrate on robotic applications of the ConstraintSatisfaction Model.



63/72

O i t f DWAVE


64/72

To practically design oracles for Grover as quantumcircuits one has first to formulate various NP-completeproblems and NP-hard problems as oracles.

Some robotic problems, especially in vision (such as

convolution, matching, applications of QuantumFourier Transform and other spectral transforms)require quantum circuits that are not permutative butuse truly quantum primitives like the controlled phasegate.

Methods to convert these circuits to AQC modelshould be investigated and the problems should beconverted to AQC model and executed on Orion.



65/72

P bl d ti f O i


66/72

We work also on: SAT, maximum clique, Hamiltonian Path, shortest path, travelling salesman, Euler Path, exact ESOP minimization, maximum independent set, general constraint satisfaction problems such as

cryptographic puzzles,

and other unate/binate/even-odd covering problems, non-Boolean SAT solvers and equation-solvers.

For all these problems we built oracles and we plan toconvert them to AQC.

Problem reduction for Orion

O i t f DWAVE


67/72

Development of new quantum algorithms based on extensions andadaptations of Grover, Hogg and other quantum search and Quantum

Computational Intelligence models.

Generalizations of Grover, Simon and Fourier transforms to multiple-valued quantum logic as implemented in the circuit model of quantumcomputing.

Analysis and comparison with binary quantum algorithms and theircircuits. Conversion to AQC model.

Generalizing well-known quantum algorithms to multiple-valued quantumlogic. For instance,

we generalized the historically famous algorithm by Deutsch and Jozsa

to arbitrary radix and we proved that affine functions can bedistinquished in a single measurement.

Moreover, functions that can be described as affine with noise can bealso distinguished.

This can be used for very fast texture recognition in robot vision. Wework also on generalization of Grover to multiple-valued quantum circuits.



68/72

All these problems are useful in robotics to solve various vision and patternrecognition path-planning, obstacle avoidance and motion generation problems.

Observe that every NP-complete problem can be reduced to Grover algorithm andGrover reduced to AQC model that can be run on Orion.

Similarly the classes of quantum simulation algorithms will be run of future

DWAVE architectures.

Although the speedup of the first of the classes is only quadratic, it will be still adramatic improvement over current computers.

It is also well-known that if some heuristics are known for an NP-completeproblem, one of several extensions and generalizations to Grover can be used,

which may provide better than quadratic speedup, but is problem-dependent.

Since however all classical solvers of NP-Complete problems that are used now inindustry are heuristic and better than their exact versions, we believe that thesame will happen when quantum programming will become more advanced.


69/72


70/72


71/72

Conclusion

Some ideas of quantum computing can beused to build sophisticated robot controllers.

Intelligent biped robots will be an excellentmedium to teach emotional robotics, robottheatre, gait and movement generation,dialog and many other computationalintelligence areas that have been notresearched yet because of high costs ofbiped robots.


72/72

What may be added

More CSP examples More on adiabatic computing

More on Waltz and vision

Image matching etc

Hidden Markov Model for Vision

Avatar how it looks like and its role

Logic design for oracles

Martins lions Interface to Orion

Controversy over Orion

telenoid robot

Documents

Transcript of telenoid robot