PROJECT NAO - Rutgers DCS Hacker Spacehackerspace.cs.rutgers.edu/library/NAO/NAO-programming.pdf ·...

PROJECT NAO

Conception & Programmation

Choregraphy For The Robot NAO

2012

Students: Matthieu Boyault 500656863 Joost Huisman 500605038

Study: E-technology Robotics Technical attendant School: Mr. M. Stolk Project attendant School: Mr. J.D. Bakker Extern initiator: LIKEMINDS

Date of return: 13-01-2012 Project period: September 2011 - January 2012

Project NAO 2012

- 1 - Matthieu BOYAULT, Joost HUISMAN – Teacher: Martin Stolk , 12 credits

Table of Contents

Table of Contents - 1 -

1. Abstract - 1 -

2. Introduction - 2 -

2.1 Initial assignment - 2 -

2.2 Assignment - 2 -

2.3 Background - 3 -

3. Acknowledgement - 4 -

4. Presentation - 5 -

4.1 The client - 5 -

4.2 Project name - 6 -

4.3 Project team - 6 -

4.4 The NAO - 6 -

5. Management - 9 -

5.1 Tools - 9 -

5.2 Global Schedule - 10 -

5.3 Strategy - 10 -

5.4 History - 11 -

5.5 Issues - 13 -

5.6 Solutions - 16 -

5.7 Risks - 16 -

6. Research and development - 17 -

6.1 Equipments - 17 -

6.2 Software - 17 -

6.3 Theorical Emotions - 21 -

6.4 Emotions, motions and movements - 21 -

6.5 Advanced programs - 24 -

6.6 Tools - 25 -

7. Products delivered - 25 -

7.1 Pilot - 28 -

7.2 The N.A.O. 1.0 - 29 -

7.3 The N.A.O. 2.0 - 32 -

7.4 The N.A.O. 3.0 - 34 -

8. Conclusion - 36 -

8.1 Statement - 36 -

8.2 Advices - 38 -

8.3 Future applications - 38 -

9. References - 39 -

9.1 Website - 39 -

9.2 Software - 39 -

9.3 Code - 39 -

9.4 Table of Figures - 40 -

10. Appendix - 41 -

10.1 Plan of approach - 41 -

10.2 Reporting of meeting - 41 -

10.3 Original script - 44 -

10.4 Final Presentation - 46 -

10.5 Poster - 0 -

10.6 Press article - 50 -

10.7 Codes - 51 -

Project NAO 2012


1. ABSTRACT The “Project N.A.O” which stands for “The New Actor Of A’dam”, is a student project from the Hogeschool van Amsterdam on the field of E-Technology. The client LIKEMINDS, a dynamic collective of young urban artists, was interesting by a challenging cooperative robotics project between HVA and LIKEMINDS. The main goal is to program the NAO robot for a theatre play which has been presented during the Festival LIKEMINDS “Bedankt” on 15th, 16th and 17th of December 2011. In the context of our modern society the show is called “Man VS Robot”. It consists of the interaction between a robot and a human actor. Despite the fact that the NAO is one of the most cute, small and impressive humanoid robot, through this project the message is to make people aware about the festival’s theme: authenticity. For that, one of the challenges was to program NAO in a way that it acts as close as possible to a real human actor in terms of emotions and motions. However from the beginning to the end of the project, the show has been considered as an experiment, which has enabled LIKEMINDS to present a new concept and also enabled HVA’s students to work on a concrete project. Thanks to this project, we learned what NAO’s abilities are and how to work with them. Therefore, in this document we have wrote a scholar report which includes our progress, our results, and our advices. We have also made a quick manual for the next person who wants to go deeper. Finally, our code will be available on our under construction website. In order to complete our main challenge, we researched the theatrical way to express convincing emotions. Then we focused on the way humans act. After that we tried to adapt both theories and create one which will fit with our simple humanoid robot. Concerning the programming, we first followed Aldebaran Guide then studied many online tutorials and finally did a lot of tests with the “Choregraphe” and “NaoSim” software. Little by little, we created our toolkit by saving our emotions, motions and other programs in our library and project folder. After the investigation period, we spent one month working on the script for the theater play. In the matter of programming, our main strategy for this task was to keep it simple and safe. Indeed, we chose to divide the script in small fractions that we could activate manually and separately. Like that in the case where something went wrong, we could react quickly, efficiently and without damage. Before getting the final result we first made a first rough program which had been fine tuned a lot in order to respect the initial goal. The NAO performed three times in the ROZENTHEATER. Every time it did some progress but also a few mistakes. However, we can conclude that the experiment was a success. The NAO has the feature to express convincing emotions and motions which can be understood by the audience. Therefore it is possible for a humanoid robot like NAO to act.

Key words: Robot, Emotion, Motion, Recognition, Interaction,

Project NAO 2012


2. INTRODUCTION

2.1 INITIAL ASSIGNMENT The goal of this group is to research a way of defining motion models in the Nao’s own programming language, so poses can be altered dynamically by changing software coordinates. This probably will start using the Naosim simulation package and understanding the concepts of the Nao via the Choreographe visual programming environment. The next step, will be to enable a more direct programming style, in which the Nao can be controlled and positioned not only via the visual block type of programming, but also via the programming language Python. This also includes low level control of joint rotations and automatically maintaining the Naos stability when for example an arm is moved from the left till right extremes of the robot. The main goal is to clearly describe an easy but flexible way to teach the Nao some dance or body movements, sign language and gestures. Also make an interesting proof of concept that can be shown promotions/demos. This is the primary objective. When these goals are met, you can proceed with:

Creating a python script to sync different Naos (one Nao broadcasts an IP packet and the other ones listen/wait for it)

Performing research on how the ROS (Robot Operating System) can be installed on a Nao and (as a star) trying to accomplish the same criteria as mentioned above (but using ROS). With documentation.

Interfacing the Miscrosoft Kinect (via the new official SDK) with the Nao, maybe using ROS, to let Nao take the same pose as the human in front of the Kinect. Or at least facilitate a way of passing pose/coordinate instructions via Ethernet packets, so the Nao can be remote controlled.

2.2 FINAL ASSIGNMENT The assignment is to create and program a choreography for the robot NAO which will be played in a theatre play. This show consists of the interaction between a robot and a real actor. The deadline for this assignment is the 15th of December 2011 in order to perform at the “Bedankt” theatre festival in the Rozengrachttheater. This assignment is initiated by the theatre group LIKEMINDS. The robot that is used goes by the name NAO. The NAO is developed by the French company “Aldebaran Robotics”. The NAO can be programmed by a user friendly way and has lots of possibilities. This report consists of the NAO programmed with the purpose of acting in the theatre play. The academic assignment is to learn how to work with the NAO and to put our theoretic robotic knowledge in practices. In the end we have to present our project and write a report.

Project NAO 2012


2.3 BACKGROUND The main goal is to achieve competences or upgrade competences of the specialism “Robotics”. The NAO project is an IVC project. IVC stands for Internal Received Competences that’s why most of the work will be done on the HVA location. The NAO is developed by the French company “Aldebaran Robotics”. In 2005, Bruno Maisonnier the founder launched this company. It is the first French company focusing on humanoid robotics. Nowadays, the firm counts about 140 employees.

NAO IS THE MOST WIDELY USED HUMANOID ROBOT FOR ACADEMIC PURPOSES WORLDWIDE.

FULLY INTERACTIVE, FUN, AND CONSTANTLY EVOLVING, NAO IS A TRUSTED PLATFORM FOR TEACHING STUDENTS AT DIFFERENT LEVELS.

FIGURE 2 ROBOTS NAO FROM ALDEBARAN

Why teach/learn with robots?

Robotics is one of the latest technological innovations, and a humanoid robot is an ideal learning tool for classes at all levels. Robots allow students to connect theory with practice and discover a wide range of robotics-related fields, such as computer science, engineering, and mathematics. Students gain hands-on experience using NAO, and, when used in the lab, they discover exciting topics such as locomotion, grasping, audio and video signal processing, voice recognition, and much more. NAO also allows teachers to integrate team work, project management, problem solving, and communication skills in a stimulating setting. NAO offers the flexibility for developing interdisciplinary projects.

EDUCATIONAL ROBOTICS IS ...VISIONARY, ENGAGING, FUN, MOTIVATING!

FIGURE 1 LOGO ALDEBARAN

Project NAO 2012


3. ACKNOWLEDGEMENT The special thank goes to our helpful supervisor, Mr. J.D. Bakker and Mr. M. Stolk. The supervision and support that they gave truly help the progression and smoothness of this project. We also wish to express our deep sense of gratitude to the theater group LIKEMINDS, in particular Mr.J. Besse and Mr. C. Nieuwenhuis, of having permitted us to carry out this project work. Mr. M. Duygulu for his time and his precious cooperation for the acting. We would like to give our special thanks for the LIKEMINDS employees who helped us during the Festival. Special thought for Mr J. van Lingen and Mr P. van Loon who took great pictures and send us as a souvenir of this project. Finally, yet importantly, we would like to express our thanks to Mr. A. Grégoire, Mr. E. Steuten, Mr. C. Keyer and J. Schreuder who supported us with the tools of working with the NAO for the successful completion of this project.

Project NAO 2012


4. PRESENTATION 4.1 THE CLIENT

FIGURE 3 LOGO LIKEMINDS

LIKEMINDS is a dynamic collective of young urban artists. The Likeminds theatre writers use the city as inspiration and the theatre as a stage. The performances are personal and relevant. Stories of today, in a form different styles and disciplines are used interchangeably. The philosophy of LIKEMINDS is “learning by doing”. They stand for “minded”, they represent the voice of our changing world. Their company provides not only professional experiences but also a dynamic training, coaching, teaching.

“LIKEMINDS Bedankt” is the annual festival where the actors of LIKEMINDS trying new things and thanking the audience with previews of shows, acts weird, unorthodox films and the best DJs. This year there was the following representations:

Man vs. Robot

Glen Faria – Dutch rapper

Waas thought Yurdakul Dilan right against time

Achmed El Jennouni's protagonist - confronted when his father dies.

Charmaine Walnut - The Future is not what it used to be

Olaf Ait Tami - Everything under control

FIGURE 4 HIERARCHY LIKEMINDS

Project NAO 2012


4.2 PROJECT NAME The name of the project we have chosen is the N.A.O. it stands for:

- New - Actor - Of A’dam

With this name we approach the NAO as a real actor. We want it to become as close as possible to a real human actor in sense of emotions and motions.

4.3 PROJECT TEAM

FIGURE 6 HIERARCHY PROJECT TEAM

The developers are two students from Hogeschool van Amsterdam. They are studying the 4th year E-Technology.

The technical supervisor is also our teacher in Robotics and Artificial Intelligence. He is one of the main reasons that

this project exist. Finally, the management supervisor is also a new teacher in the E-Technology department.

FIGURE 5 PROJECT FIELD

Project NAO 2012


4.4 THE NAO

The goal of “Aldebaran Robotics” is to deliver humanoid robots for a bigger audience against a reasonable price. This robot has to become part of families to assist in daily home jobs. The NAO is programmable in a user friendly way for lots of different applications. In picture 7 you can see the structure of the NAO. The robot is 57cm tall with the following key components:

- Body with 25 degrees of freedom (DOF) whose key elements are electric motors and actuators - Sensor network, including 2 cameras, 4 microphones, sonar rangefinder, 2 IR emitters and receivers, 1

inertial board, 9 tactile sensors, and 8 pressure sensors - Various communication devices, including voice synthesizer, LED lights, and 2 high-fidelity speakers - Intel ATOM 1,6ghz CPU (located in the head) that runs a Linux kernel and supports Aldebaran’s proprietary

middleware (NAOqi) - Second CPU (located in the torso) - 27,6-watt-hour battery that provides NAO with 1.5 or more hours of autonomy, depending on usage

Today, there are more than 1500 NAOs used by laboratories and universities worldwide for research and education in the fields of robotics, computer sciences, cognitive sciences, mathematics, vision, sensors, motion, and more.

FIGURE 7 NAO STRUCTURE

Project NAO 2012


In the following pictures you can see some of the broad usages of the NAO:

FIGURE 8 NAO PLAYING SOCCER IN THE ROBOCUP

FIGURE 9 NAO INTERVIEWED ON CNN

FIGURE 10 NAO DRAWING TRADITIONAL JAPANESE KANJI

CALLIGRAPHY

FIGURE 11 NAO DOING STAND-UP COMEDY ROUTINES

Project NAO 2012


5. MANAGEMENT

5.1 TOOLS For this project there were information processing programs used like Dropbox and Zoho. Zoho offers a suite of online web applications geared towards increasing your productivity and offering easy collaboration. Zoho's online office tools include word processing, spreadsheets, presentations, databases, note-taking, wikis, calendar and

googleapps ... Free plan (1 project, 10 MB storage, & Unlimited users), maximum size upload file 125Mo

FIGURE 13 DROPBOX FOLDER

FIGURE 12 ZOHOO SHARING PLATFORM

Project NAO 2012


5.2 GLOBAL SCHEDULE

37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54

Investigation Programming Pilot

Waiting feedback

Programming Script

Fine tuning

Repetition & adjustments

Performing in "Bedankt"

Writing report

Week 37-41:

Discover the possibilities of the NAO

Investigate the programming of the NAO

Appointment with the theater writer

Fill in the request form for school

Make a plan of approach Week 41-43:

- Working together with the theater writer to program the basic poses

- Program fall detection and how to act on it Week 45-48:

- Working together with the theater writer to program the whole theater script

Week 48-49: - Fine-tuning of the theater program

Week 49:

- Repetition and the opportunity for some small adjustments

Week 50:

- The NAO performing in the theater Week 51-54:

- Put the documentation into a report and present it

5.3 STRATEGY In this small, ambitious and short project, we had a lot of issues about management. In fact, we often had some troubles with our resources and the planning. We didn’t want to spend too much time on the management because we thought we won’t have enough time to program the play. Moreover, we were only with two students working on this project. If one of us would have been the project manager, we might loss a useful developer. Consequently, during this project we decided to use and work on management only when we need it. However, before starting the programming we made a rough estimation in order to inform our supervisor and also to use it as deadline to finish on time. On average, we applied the Deming circle PDCA: plan, do, check, and act. In conclusion, every time we switch and alternate our role: manager, controller of quality and developer. About the quality, it wasn’t really the main idea for the client. They wanted a robot performing with a real actor for their Festival in December. The main goal was to bring a new concept in their background. However, they specifically ask to focus on a working play even if it is short and to show all the potential of the robot. Therefore, we worked step by step with the writer and make first a rough script. Afterwards, we progress on the programming of the script by improve the play by fine tuning of the robot’s actions. In other hand, we had to deliver a report for the school. It should be detailed and understandable for anyone. So, in term of quality we had also to consider this writing assignment. We tried to keep a history of each task done, our

FIGURE 14 PSEUDO GANTT CHART

Project NAO 2012


issues and solutions. It looks like a tutorial thus other students can learn faster the technology of the robot, continue our work and improve their skills on programming.

5.4 HISTORY

ON THURSDAY 12TH ON FRIDAY 13RD

Create icon for motions box Investigate on the transfer python script to Nao box

Contact again Jan Derk for the missing meeting Meeting with Martin

Contact A.H. Gregoire

Investigate on the transfer python script to Nao box

ON MONDAY 17TH ON THURSDAY 20TH

Create more emotions Finish the plan of approach

Create more motions Think about the presentation

Prepare questions for Mustafa

ON THURSDAY 27TH ON SUNDAY 30TH

Progress Meeting with Jan Derk Send report to supervisors after 1 month

ON MONDAY 31ST ON THURSDAY 30TH

Exam Week – Waiting answer from supervisors

ON MONDAY 7TH ON TUESDTAY 8TH ON FRIDAY 12ND

Discussion with Mustafa Preparation of the powerpoint Simulation of the presentation

Discussion with Jens Preparation of videos Testing programs

Progress Meeting with Martin Presentation Project NAO

ON MONDAY 14TH ON TUESDAY 15TH ON THURSDAY 17RD

Investigate on remote control Start the fuzzy logic program Create MP3 script

Investigate on balance control Combine basic motions & emotions

Implement python box

Meeting with LIKEMINDS: exchange first script

ON MONDAY 21ST ON TUESDAY 22TH ON THURSDAY 24TH

Investigation and evaluation

of the first script

Continue the fuzzy logic program Record videos of NAO dancing

(Thriller, Tai-chi, Ballet)

Meeting with Mustafa:

feedback for the first script

Investigate on Naomarks recognition

Update of the script

(motions, songs, reactions)

Investigate on NAO synchronization

ON MONDAY 28TH ON TUESDAY 29TH ON THURSDAY 1ST

Naomark actions, grabbing power,

solutions to manage the script step by step

Finish the fuzzy logic program Finish all main motions

Project NAO 2012


5.5 ISSUES

Communication

Planning meeting

Like you can see in the presentation, in our project there are a lot of person who participate either to the design or

the decision. Of course, you can also imagine that find a free time for a meeting which fit with eight agendas, it is

quite a challenge.

In that context, the first meeting was only with Caspar, Martin, Joost and I. It wasn’t really useful meeting because

we only talk about the concept. Actually, I think we shouldn’t have to come especially because we had to skip one

lecture for this meeting.

The first kick off with everybody was cancelled twice and moved to the 27 September. After that we had our weekly

working time (2 sessions), we almost had a weekly exchange or meeting with the writer via face to face or email or

phone. We difficulty planned progress meeting with our supervisors and kept update LIKEMINDS via email or phone.

Exchange the material

Our work was to program the robot to play an actor. We used specific software with a license paid by Hogeschool

van Amsterdam. But LIKEMINDS didn’t buy one. So, when we want to show our progress to our client either we

have to invite them or we have to visit them. It is a waste of time. Moreover, send videos by email are quite

restrictive because they are heavy. Even if we can use the BSCW server, our client doesn’t have access.

Time

No respect of Deadline

Like I explain before, it is a project ordered by LIKEMINDS. They hire Mustafa (the writer) for this new concept. But unfortunately, he also worked in the same time in several other projects. Consequently, he was really busy and it took too much time to understand what LIKEMINDS expected. At the first deadline, instead of delivering the first script, he gave us only three very general options. With that we couldn’t work. After that our supervisors from LIKEMINDS also took too much time to choose the concept.

Short working time

It was only after the 14th of November, that we could start. Thus, we had two weeks left to program the script. That is to say, we had one month to finish our project.

Project NAO 2012


FIGURE 15 OVERVIEW OF THE SCHEDULE

Project NAO 2012


Material

The robot

In a technical point of view, during this project we had to share one robot with the two

of us. In order to save some time we used to work different tasks in the same time

then when we had something relevant we take the robot and try. Unfortunately even

with this arrangement, we needed a lot time of the robot exactly in the same time.

Another fact is the real autonomy of the robot is not enough to work, test, and check like we did.

Most of time, we had to plug every time the robot if we wanted to do as much as possible. However,

this wasn’t the best solution because we had a lot of warning from the motors. They were too hot

consequently they didn’t work as efficient as they should.

Finally, the last problem that we met is the license conflict. Indeed, we have only one robot, two softwares and one

license. Therefore, we often get this message error.

FIGURE 16 DIFFERENT TECHNICAL ISSUES

The school

One of our difficulties for this project was to find free space to work. This is a main issue because we spent too much time trying to find a room instead of working. Hogeschool van Amsterdam is big but a lot of students also work in the same time in different project. So, it became a challenge to get the robot, the rooter and a quiet room. The theatre

In order to make sure that the robot will perform well the play. We need to test our program for real in the theatre.

Unfortunately, LIKEMINDS could guarantee us this free access. Therefore, at the last minute we had to fix some

issues that we have never met before.

Project NAO 2012


5.6 SOLUTIONS Planning meeting

We have decreased the number of big meeting and focus on our programming. We preferred to send an email of progress every week. We didn’t skip anymore lecture, most of time Mustafa came to us.

Exchange the material We program at home, test at school, record our work then share them with Mustafa via dropbox. We tried to use a sharing platform Zohoo but it didn’t work.

No respect of Deadline We didn’t have any choice except we just worked harder to catch up our delay. We also asked to make the script easier for us in order to respect the final deadline and produce something good.

Short working time We tried to management our time by making a rough estimation of each task and work by priorities. We also divided the work to progress faster.

The robot

We tried to use more the simulation mode then check with the real robot. We prepared programs before our testing session. We made more breaks to let the robot rest a bit. We tried to get access to the second robot in order to have an alternative.

The school We tried to book a room every week in advance.

The theatre We asked LIKEMINDS to use their office because they have a small scene with light and materials.

5.7 RISKS In the table below the risks arranged from most destructive to less can be seen. In the second column a value between 0 and 10 indicates the weight of the risk. And the thirt column shows the possibility that this risk will occur.

Risk Risk weight Possibility

The NAO damaged someone or something in his surrounding 10 1% The NAO program is not finished soon in enough for his performance at the theater 10 5% The NAO isn’t doing anything at the theater play 10 3% A part of the NAO gets defect 9 5% The NAO falls 9 7% There is no license for the NAO to perform in a theater 9 30% The NAO gets disturbed during his play by someone or something 8 2% The theater play of the NAO isn’t 100% percent correctly 7 6%

Project NAO 2012


6. RESEARCH AND DEVELOPMENT 6.1 EQUIPMENTS

FIGURE 17 EQUIPMENTS NECESSARY

Project NAO 2012


6.2 SOFTWARE The NAO can be programmed by the Choregraphe software. It is a cross-platform application that allows you to edit NAO's movements and behaviors. A behavior is an action or couple of actions that you can upload in the NAO itself. In this platform, you can create programs by linking blocks together like you can see in the figure 1. This program starts in the left and goes through the string into the box “stand up”. After executing this block the signal goes on to the end of the string. In the right of the figure 1 you can see the virtual NAO, when executing the program this NAO will do the action of the block in this case stand up.

There already made blocks in Choregraphe, but you can also make the blocks by yourself. The blocks are kept in the box library like you can see in figure 19. To make a new box you can use the application you see in figure 18. In this application you can fill in the name of the block and define the inputs and outputs. Also there is the possibility to put an image in the box. After creating this box the box in still empty, in the next page you can see how to program the box.

FIGURE 18 PROGRAMMING BY BLOCKS

FIGURE 19 NEW BOX FIGURE 20 BOX LIBRARY

Project NAO 2012


When opening the box you will see an empty field. In this field you can put already made blocks. When internally connect this blocks you can make a complex program consisting out of one block. Another way to program is by upload motions and behaviors in the block like you can see in picture 20. On the top you can see a timeline of Motion. This timeline starts at the number 215 and ends at the number 245, this numbers represents frames. So in this view you can see the timeline of 30 frames. For each frame you can define a motion. This motion you can capture in a frame by putting the NAO in a certain position and give the comment save whole body. After this comment a grey block will appear like you can see in de top of the timeline. In this program there are 11 of this grey blocks, thus 11 captured frames.

FIGURE 21 UPLOAD BEHAVIOUR AND MOTIONS IN A BLOCK

Choregraphe automatically connects these frames. How the frames are connected in terms of acceleration and deceleration you can see in picture 21. Each part of the NAO has a separate line. By changing the way the line increase or decrease you can change the acceleration or deceleration. Another future when creating a block is the possibility of adding more behaviors as you can see in the previous picture. The goal of this is that you can do more actions in the same time. In this case play music and adjust the color of the eyes.

FIGURE 22 CONNECTIONS BETWEEN FRAMES

Project NAO 2012


NaoSim is a simulator where you can test your behaviors and algorithms in a virtual world. That way you can already deal with errors that would happen in the real world, this virtual environment can be modified by inserting objects, see picture 22. To test a program, Choregraphe has to be running together with NaoSim. Choregraphe detects the virtual NAO of the NaoSim, so you can connect. When the connection is established the program from Choregraphe can by loaded to the virtual NAO. In the NaoSim the virtual NAO would execute the program. The NaoSim works with respect to balance, time and 3D motions. During the motions of the virtual NAO can be captured by camera’s from different angles.

FIGURE 23 ALDEBARAN SOFTWARE NAOSIM

One of the program languages that run on the NAO is Python. In figure 23 you can see a part of Python script from the NAO for the balance control.

FIGURE 24 PYTHON CODE BALANCE CONTROL

Python runs on Windows, Linux/Unix, and Mac OS X and is a high-level language. It was originally created in the 1980's by Guido van Rossum at CWI. The first public release was in 1991. After the release of Python quickly became one of the preferred programming languages for the creation of web applications in the Internet. Python has the following characteristics that make it different from other program languages:

- It is free to use, because of its OSI-approved open source license. - The language is very easy to learn because its source code resembles pseudo code - I uses elegant syntax, which allows the users to read program code easily and which makes it suitable for

prototype development and different ad-hoc programming tasks. - It is embeddable in applications from other languages like C, C++ and Java. - Python comes with an inbuilt development environment called IDLE and offers a large standard library. - In interactive mode Python can easily test even small portions of code.

Project NAO 2012


6.3 THEORICAL EMOTIONS

Primary

emotion

Secondary

emotion Tertiary emotions

Love

Affection Adoration, affection, love, fondness, liking, attraction, caring, tenderness, compassion, sentimentality

Lust Arousal, desire, lust, passion, infatuation

Longing Longing

Joy

Cheerfulness Amusement, bliss, cheerfulness, gaiety, glee, jolliness, joviality, joy, delight, enjoyment, gladness,

happiness, jubilation, elation, satisfaction, ecstasy, euphoria

Zest Enthusiasm, zeal, zest, excitement, thrill, exhilaration

Contentment Contentment, pleasure

Pride Pride, triumph

Optimism Eagerness, hope, optimism

Enthrallment Enthrallment, rapture

Relief Relief

Surprise Surprise Amazement, surprise, astonishment

Anger

Irritation Aggravation, irritation, agitation, annoyance, grouchiness, grumpiness

Exasperation Exasperation, frustration

Rage Anger, rage, outrage, fury, wrath, hostility, ferocity, bitterness, hate, loathing, scorn, spite, vengefulness,

dislike, resentment

Disgust Disgust, revulsion, contempt

Envy Envy, jealousy

Torment Torment

Sadness

Suffering Agony, suffering, hurt, anguish

Sadness Depression, despair, hopelessness, gloom, glumness, sadness, unhappiness, grief, sorrow, woe, misery,

melancholy

Disappointment Dismay, disappointment, displeasure

Shame Guilt, shame, regret, remorse

Neglect Alienation, isolation, neglect, loneliness, rejection, homesickness, defeat, dejection, insecurity,

embarrassment, humiliation, insult

Sympathy Pity, sympathy

Fear

Horror Alarm, shock, fear, fright, horror, terror, panic, hysteria, mortification

Nervousness Anxiety, nervousness, tenseness, uneasiness, apprehension, worry, distress, dread

Project NAO 2012


6.4 EMOTIONS, MOTIONS AND MOVEMENTS

FIGURE 25 PRACTICAL EMOTIONS

Project NAO 2012


Concerning the acting expression, we first investigate on the theatrical way to express emotions. (cf Table above)

While we learned more about the subject, it wasn’t relevant enough for our project. Indeed, the play is more

practical.

Therefore, we continue to investigate on how to express emotion. But this time we focused on the human point of

view. It was better than before but not as spontaneous as we thought. And in every case there is always several ways

to express a feeling. (Figure 24)

Finally, we had to reduce our research on humanoid robot. Consequently, we lost some great ideas of expression

due to the structure of the robot. In fact, human use a lot their face to express their feelings. Unfortunately, NAO has

only a simple head without mouth, smile, and eyes.

So, little by little we manage to find alternative by using all the body language and the voice.

FIGURE 26 BASIC EMOTIONS PROGRAMMED ON CHOREGRAPHE

FIGURE 27 BASIC MOTIONS PROGRAMMED ON CHOREGRAPHE

FIGURE 28 MOVEMENTS PROGRAMMED ON CHOREGRAPHE

Project NAO 2012


6.5 PRELIMINARY STUDY Another research was the one about interaction. The upcoming dialog is part of the movie “Closer” send by Mustafa to test the possibilities with the NAO.

FIGURE 29 ACTOR FROM THE MOVIE CLOSER,

LARRY = NAO

FIGURE 30 ACTOR FROM THE MOVIE CLOSER,

DAN = MUSTAFA

1.1.1 LARRY: She's made her choice.

DAN: I owe you an apology. I fell in love with her. My intention was not to make you suffer.

1.1.2 LARRY: So where's the apology? You cunt.

DAN: I apologize. If you love her, you'll let her go, so she can be happy.

1.1.3 LARRY: She doesn't want to be happy.

DAN: Everybody wants to be happy.

1.1.4 LARRY: Depressives don't. They want to be unhappy to confirm their depression. If they were happy, they couldn't be

depressed anymore. They'd have to go out into the world and live, which can be depressing.

DAN: Anna's not a depressive.

1.1.5 LARRY: Isn't she?

DAN: I love her.

1.1.6 LARRY: Boo hoo. So do I.

FIGURE 31 SAMPLES MP3 MADE FROM THE MOVIE CLOSER

The first steps were downloading the vocal script of the movie and extracted the parts of Larry (NAO) from it. Finally, we extracted each part of the Larry’s (NAO) dialogue in MP3’s. In the following sentences you can see the results of out tests:

Project NAO 2012


1. What is the quality of the sound out of the NAO when playing MP3 voice recordings?

The quality of the sound was as you can expect the same as the sound out of an common audio speacker. Also the sound was not very close to that of a real human, the sound was loud and clear. The conclusion is therefore that this way of playing the voice MP3’s is implementable in the theater play.

2. What is the quality of the sound out of the NAO when playing text with the NAO speaking box? The voice of the NAO becomes very robotic and some words weren’t pronounced fluently. This way of speaking is not recommendable for the actual theater play.

3. When Mustafa has to speak the NAO has to be quit for a time. Can this delay be configured really tight so the dialog sounds natural?

The delay can be configured in a proper way by using the timeline or with delay block. When testing this, the human actor (Dan) had to be very strict to the script, when doing so the interaction sounded excellent.

4. What is the quality of the NAO’s voice recognition? The quality of the voice recognitions was tested with a standard program included in the NAO. This program detects words like yes, no, thriller (dance), tai chi and several other applications. When using this voice recognition application you had to be closed to the NAO, speak at a good volume and articulate good. Another point was that the sound of the surrounding was interfering sometimes. The conclusion was therefore that this application was not helpful in theater play, keeping in mind the noise of the audience.

6.6 TOOLS

Inputs Manager

ALMemory is a class that implements an event-based storage

base. Users can store data inside ALMemory, they can store

groups, or relationships. Modules can then subscribe to data

and receive notifications when data get changed with callback

ALMemory contains three types of data and provides three

different APIs.

- Mainly data from sensors and joints - Event - Micro-event are mostly use internally for callback system

Fall Manager

Because there is a small risk that the NAO could fall during testing or during the theater play the ‘’fall manager’’ application was investigated. This fall manager activates when the robot is out of balance. Without the fall manager the robot can fall with full stiffness in his body with the risk that parts of the NAO could break. But with the fall manager the NAO will turn of the full stiffness in his body resulting in a softer collision with the ground. A memory event called ‘’robotHasFallen” is raised when the fall management process is completed. This program is an API that’s implemented in the class ‘’ALMotionProxy.

FIGURE 32 ALMEMORY INPUTS

Project NAO 2012


Naomark

FIGURE 33 NAOMARKS ALDEBARAN

We can use those ten Naomark in order to create a recognizable environment for Nao. The program that we use for

this task is vision_landMarkDetection.py. It is a vision module which is activated with Telepathe. This software report

the detected Naomark by circling them on the video and catching the Naomark identifier. We can see in the next

picture the power of the mark detection. It is really fast in recognition capabilities and efficient while the quality of

the video was not really great.

Conditions:

Lightning: 100 to 500 lux.

Tilt : -60° to +60°

Rotation 360°

FIGURE 34 TELEPATHE MANAGER

FIGURE 35 NAOMARK RECOGNITION PROGRAM

Project NAO 2012


Fuzzy logic

We wanted to include in the show an interaction between the robot and the public. NAO should be able to understand and react on the opinion of the public thanks to colored cards (green or red). Every person should have to show either the green or red card. Then, NAO has to count and convert the number of green and red card. For this task, we are using fuzzy logic algorithm in order to interpret the best way the majority of the public’s opinion. According to this number, it should react differently. The inputs can be the surface of each color, or the intensity of the color, or if we use different form of card the number of each kind of card. The outputs should be three possible answers: “YES”, “NO” and “I DON’T KNOW, I NEED HELP”. In this case, a 3D representation should be useful to check if the rule-bases are relevant.

FIGURE 36 FUZZY LOGIC SIMULATION

The key is to use Mustafa’s opinion when NAO doesn’t know. In an enhanced design, NAO should evaluate Mustafa’s

opinion with its real data. And only if the Mustafa’s answer is relevant, NAO will agree.

The second stage can be to implement a smarter fuzzy system with 5 outputs “DEFINITIVELY YES”, “PROBABILY YES”,

“I DON’T KNOW, I NEED HELP”, “PROBABILY NO” and “DEFINITIVELY NO”.

Unfortunately we didn’t have the time to implement the program which has been tested on Matlab Software then

transcript on Python language.

Project NAO 2012


7. PRODUCTS DELIVERED

7.1 PILOT When the script was defined the pilot could be made. We separate the script in small parts as you can see in the next figures. By dividing the script in parts and given each part a value in time, difficulty and subject the program becomes very clear. In the first figure you can see a graph of all subjects in terms of time and difficulty. As you can see the subject that really pups out in time is motion and recognition. So the focus of this project was a lot on the motions and recognition.

FIGURE 37 EVALUATION OF TASKS

In the next figure you can see the graph of all actions of the script in terms of time and difficulty. As you can see the really difficult parts are in the beginning and in the end. Those two graphs helped us and Mustafa to determine on which parts the focus has to be to complete the script. The script is also programmed in parts in the NAO that way we could easily skip a part when going wrong and go on with the rest of the script.

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

EnvironmentMotion

MovementRecognition

Interaction

Difficulty (%)

Total Time (%)

Project NAO 2012


FIGURE 38 EVALUATION OF THE SCRIPT

7.2 THE N.A.O. 1.0 The day of the first theater play was also the first day that the NAO could be tested in the theater. Therefore the following adjustment where made after testing:

- Fine tuning of motions because of the theater stage. - Fine tuning of the orientation of the NAO because of the theater floor. - Fine tuning of interaction with Mustafa in the dialog.

We came early in theater to prepare everything, but we had also to share the stage with other performances. Consequently, we worked little by little each parts of the script. We tested them on the stage when it was possible and correct them in backstage. We tried to adapt the entire environment to the robot but it was not easy to take in account everything. After the fine tuning session the NAO did his first play for the first time all parts together

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

Project NAO 2012


You will be able to see the video on our website soon we upload it.

FIGURE 39 CUSTOMIZED POSTER LIKEMINDS BEDANKT

http://boyault.matthieu.free.fr/Webpage/Pages/Projects/ProjectNAO/Videos/Show1.avi

After the play we looked back on went well and not so well. In table 1 you can see the result. As you can see a few parts didn’t quit work. First, the orientation wasn’t perfect. Secondly, the recognition parts we skip because the NAO had trouble of recognize because of the poor lightning conditions. At last, the dialog with Mustafa was a bit weird due to the delays we programmed between the sentences. But overall the first real play was a success considering the reaction of the public. For the next play the steps will be check the whole play in terms of orientation. Also adjust the recognition part and change the delays in the dialogs.

Project NAO 2012


FIGURE 40 RESULTS SHOW 15/12/2012

Project NAO 2012


7.3 THE N.A.O. 2.0 You will be able to see the video on our website soon we upload it.



After the play we looked back on went well and not so well. In table 2 you can see the result. As you can see the play was much better as last time due to the updates. The dialogs were much better because of the adjustment we did in the delays. And the recognition worked for a part. But there still were some parts not perfect. Like the orientation was still not totally good. The problem of orientation came out of the fact that the light is not good enough in each area of the theater and the problem of orientation lies in the surface of the theater. The surface of the theater is not equal and rough enough so the NAO ends each time on a different position. For the next play the most important steps will be updating the recognition and the orientation. For the recognition the angle and direction that the NAO is looking at is very important to recognize the red objects. Therefore we changed the orientation of the NAO when tracking the objects. The problem of the wrong orientation was tackled by using a carpet. The carpet was much rougher so the NAO’s position was less variable. Also Mustafa wanted to skip some parts of the script because that would upgrade the story of the theater play in his opinion.

Project NAO 2012



Project NAO 2012


7.4 THE N.A.O. 3.0 You will be able to see the video on our website soon we upload it.



After this last play we looked back on went well and not so well. In table 3 you can see the result. As you can see the play was almost perfect. The only thing was again with the red ball tracking this time because of the bat light conditions. Also in the end the NAO didn’t had enough power to execute the last fighting move because to the motors where to hot so the power in the motor was not enough. Overall all three theater shows where a great success considering to the reaction of the public and LIKEMINDS.

Project NAO 2012



Project NAO 2012


8. CONCLUSION 8.1 STATEMENT In the introduction we mentioned the question if it is possible to do a theater play with a robot. Considering the reaction of the public we can conclude that the three theater plays where a great success. The NAO can express convincing emotions and motions which can be understood by the audience. On the other side the NAO was not very naturally in terms of interaction, motions and did not improvise. So the answer on the question if the NAO could duplicate a real small human actor is no. But for the answer can the NAO act we give the exact definition of acting: Acting is the representation of a usually fictional character on stage or in films. At its highest levels of accomplishment acting involves the employment of technique and/or an imaginative identification with the character on the part of the actor. In this way the full emotional weight of situations on stage be communicated to the audience. The actor must be a sharp observer of life and thoroughly trained in voice projection and enunciation and in body movement. [http://encyclopedia2.thefreedictionary.com/Acting] The NAO did transfer emotions by motions and voice to the audience and did represent a fictional character. Therefore the answer to the question it is possible for a humanoid robot like NAO to act is yes.

FIGURE 46 TEAM PHOTO TAKEN IN LIKEMINDS BEDANKT 2011 ©JEAN VAN LINGEN

FIGURE 45 STATISTIC OF EACH SHOW “MAN VS

ROBOT”

Project NAO 2012


FIGURE 47 PHOTOS TAKEN IN LIKEMINDS BEDANKT 2011 ©JEAN VAN LINGEN

Project NAO 2012


8.2 ADVICES On a technical point of view we strongly advice to use a second computer for the show. I was for us one of our

solutions to react in case of something went wrong. We had the same program load and ready to run for any

trouble.

During our training session and also for the three shows, we were very annoying by the battery level. Indeed, we had

to take account that for the efficiency of the choreography. Therefore, we think that a good idea will be not to have

several robots but several batteries.

Finally, we met some trouble with the lightning. In fact, it is really affecting the efficiency of the camera

consequently any movements like tracking or walk-tracking a red ball. For the last show we found the idea to use a

beamer where we can project the camera’s view from the robot. It is really helpful.

FIGURE 48 DIFFERENT ADVICES

8.3 FUTURE APPLICATIONS At the end of this project we can look back for future applications. We can conclude that for the idea of let a robot act with a real actor there are advantages and disadvantages. One of the good points is that the audience really likes to see a robot act. However when the robot is acting with a real actor the robot gets all the attention from the audience, and they are not focusing anymore on the real actor as well. Another inconvenience is that it’s very difficult to create natural interaction between the robot and the real actor. That’s why our suggestion for the future is to do the theater play by two robots instead of a robot and a real actor. In that way you can program the interaction very fine and the focus of the audience will be in both robot actors.

If we would go one with this project and had the opportunity for a second theater show we would like to do the play with two NAO robots and focus more an interaction with each other and the audience.

About the NAO we advise school to do more projects with the NAO because the amount you can learn about robotics is very much. Out of our experience for the programming part we would like to advice future students to establish a library of all parts of the project so your work is very clear and interchangeable for other applications. In terms of the NAO’s software we strongly advice to use the new update. It includes a lot of new and useful tools. We also want to mention one of the famous world challenge with NAO is RoboCup. It is a football competition.

FIGURE 51 NAO PLAYING SOCCER

FIGURE 50 ALDEBARAN SOFTWARE

FIGURE 49 NAO 1 AND NAO 2 FROM HVA

Project NAO 2012


9. REFERENCES

9.1 WEBSITE GENERAL

http://www.aldebaran-robotics.com/documentation/software/choregraphe/index.html

http://changingminds.org/explanations/emotions/basic%20emotions.htm#So

http://www.aldebaran-robotics.com/

RECOGNITION

Controlling Nao's arms with color markers

Color recognition

INTERACTION

Vocal Interaction Tutorial

TRACKING

Object Tracking

ACTING

TAME Project

Intensity of Perceived Emotions in 3D Virtual Human

9.2 SOFTWARE

http://download.cnet.com/BSR-Screen-Recorder/3000-13633_4-10198470.html http://oege.ie.hva.nl/~stolm/downloads/Nao/ http://openbookproject.net/thinkCSpy/ch01.html http://www.ntchosting.com/python/

9.3 CODE

http://boyault.matthieu.free.fr/Webpage/Pages/Projects/ProjectNAO/projectNAO.html

FIGURE 52 NAO DANCING THRILLER

Project NAO 2012


9.4 TABLE OF FIGURES

Figure 1 Logo Aldebaran - 3 -

Figure 2 Robots NAO From Aldebaran - 3 -

Figure 3 Hierarchy Likeminds - 5 -

Figure 4 Logo Likeminds - 5 -

Figure 5 Project Field - 6 -

Figure 6 Hierarchy Project Team - 6 -

Figure 7 NAO Structure - 7 -

Figure 8 NAO Playing Soccer in the RoboCup - 8 -

Figure 9 NAO Interviewed on CNN - 8 -

Figure 10 NAO Drawing traditional Japanese kanji calligraphy - 8 -

Figure 11 NAO Doing stand-up comedy routines - 8 -

Figure 12 Zohoo Sharing Platform - 9 -

Figure 13 Dropbox Folder - 9 -

Figure 14 Pseudo Gantt Chart - 10 -

Figure 15 Overview of the schedule - 14 -

Figure 16 Different technical issues - 15 -

Figure 17 Equipments necessary - 17 -

Figure 18 Programming by blocks - 18 -

Figure 19 New Box - 18 -

Figure 20 Box Library - 18 -

Figure 21 Upload behaviour and motions in a block - 19 -

Figure 22 Connections between frames - 19 -

Figure 23 ALdebaran Software NaoSim - 20 -

Figure 24 Python Code Balance Control - 20 -

Figure 25 Practical Emotions - 22 -

Figure 26 Basic Emotions Programmed on Choregraphe - 23 -

Figure 27 Basic Motions Programmed on Choregraphe - 23 -

Figure 28 Movements Programmed on Choregraphe - 23 -

Figure 29 Actor From The Movie Closer, Larry = Nao - 24 -

Figure 30 Actor From The Movie Closer, Dan = Mustafa - 24 -

Figure 31 Samples MP3 made from the movie Closer - 24 -

Figure 32 ALMemory Inputs - 25 -

Figure 33 Naomarks Aldebaran - 26 -

Figure 34 Telepathe manager - 26 -

Figure 35 Naomark recognition program - 26 -

Figure 36 Fuzzy Logic Simulation - 27 -

Figure 37 Evaluation of tasks - 28 -

Figure 38 Evaluation of the Script - 29 -

Figure 39 Customized Poster Likeminds Bedankt - 30 -

Figure 40 Results Show 15/12/2012 - 31 -





Figure 45 Statistic of each show “Man VS Robot” - 36 -

Figure 46 Team Photo taken in Likeminds Bedankt 2011 ©Jean van Lingen - 36 -

Figure 47 Photos taken in Likeminds Bedankt 2011 ©Jean van Lingen - 37 -

Figure 48 Different Advices - 38 -

Figure 49 NAO 1 and NAO 2 From HVA - 38 -

Figure 50 Aldebaran Software - 38 -

Figure 51 NAO Playing soccer - 38 -

Figure 52 Nao dancing thriller - 39 -

Project NAO 2012


10. APPENDIX 10.1 PLAN OF APPROACH

Plan of approach NAO

IVC project

Study: E-technology Robotics Technical attendant School: Mr. M. Stolk Project attendant School: Mr. J.D. Bakker Extern initiator: Mr. M. Duygulu

Project period: September - January 2011

Students: Matthieu Boyault 500656863

[email protected] Joost Huisman 500605038 [email protected]

Project NAO 2012


TABLE OF CONTENTS

1. Backgrounds 1

2. Project assignment 1

3. Project Activities 1

4. Project Limits 2

5. Products 2

6. Quality 2

7. Project organization 2

8. Global planning 3

9. Costs and benefits 3

10. Risks 4

1. BACKGROUNDS The name of the project we have chosen is the N.A.O. New Actor Of Amsterdam and is about the NAO robot. NAO is a robot developed at the company “Aldebaran Robotics” that can be programmed in an user friendly way. The main goal is to achieve competences or upgrade competences of the specialism “Robotics”. The NAO project is an IVC project IVC stands for Internal Received Competences that’s why most of the work will be done on the HVA location. The originator of the project is the theater group LIKEMINDS.

2. PROJECT ASSIGNMENT

The project should be finished for the 15th of December in order to perform it for the Festival. We are working for LIKEMINDS Theater, represented by Jens Besse, the producer and Casper Nieuwenhuis, the artistic director. We also work with Mustafa Duygulu, the writer of the script. At the end of the project, we should deliver a working program and a report.

3. PROJECT ACTIVITIES The project activities can be subdivided in the next parts:

- Fill in the EVC project application form - Meeting with the project initiator and attendant from school - Make a plan of approach - Investigation on the NAO - Investigate how to program by blocks - Investigate how to program with Python - Program NAO actions - Test programs on NaoSIM - Test programs on the real NAO - Optimize our code - Meeting with our supervisors - Make a presentation about our progress - Write a report

Project NAO 2012


4. PROJECT LIMITS In the text below the absolute deadlines for the project are mansion:

- 1 December: Repetition NAO - 15 December: The NAO performing in the theater - Begin February Deliver report for school

The limit for the project budget depends on the budget that the theater group wants to invest.

5. PRODUCTS To successful accomplish this project the following products have to be delivered:

- The program for the NAO - A paper about the project

6. QUALITY The main goal of this project is to program a robot for a play. Therefore, NAO should pretend to be an actor. This means that NAO should act the more realistic possible. Consequently, one of the criteria about quality should be the human expressions. This one can be checked with a partial judgment or by our supervisors. Another point about the quality should be the code. This means that programs should be included comments. It should be understandable easily by another developer. Once again it can be checked with a partial judgment or by our supervisors.

7. PROJECT ORGANIZATION Because the project team consists out of two persons we choose to take booth the task of product developer and switch weekly the task of manager. The technical attended from school is Mr. M. Stolk and the project attended is Mr. J. D. Bakker. The external initiator is M. Duygulu who is working at the theater group LIKEMINDS.

Project NAO 2012


8. GLOBAL PLANNING Week 37-41:

- Discover the possibilities of the NAO - Investigate the programming of the NAO - Appointment with the theater writer - Fill in the request form for school - Make a plan of approach

Week 41-43:

- Working together with the theater writer to program the basic poses

- Program fall detection and how to act on it

Week 45-48

- Working together with the theater writer to program the whole theater script

Week 48-49

- Fine-tuning of the theater program Week 49

- Repetition and the opportunity for some small adjustments

Week 50

- The NAO performing in the theater Week 51-54

- Put the documentation into a report and present it

9. COSTS AND BENEFITS In this project, we should consider as cost the license for NAO software. It is about ....euros. Of course, we should also count the robot's price. In the case where Nao will be broken, we should add some extras for the reparation. Else about benefits, given that our play will be for a festival, people will pay their entrance. We might have some bonus. Nevertheless, this project will enable us to get an experience with python and Nao language. Without forgetting the fact that we work in collaboration with a theater.

10. RISKS In the table below the risks arranged from most destructive to less can be seen. In the second column a value between 0 and 10 indicates the weight of the risk. And the thirt column shows the possibility that this risk will occur.

Risk Risk weight Possibility

The NAO damaged someone or something in his surrounding 10 1% The NAO program is not finished soon in enough for his performance at the theater 10 5% The NAO isn’t doing anything at the theater play 10 3% A part of the NAO gets defect 9 5% The NAO falls 9 7% There is no license for the NAO to perform in a theater 9 30% The NAO gets disturbed during his play by someone or something 8 2% The theater play of the NAO isn’t 100% percent correctly 7 6%

Project NAO 2012


10.2 REPORTING OF MEETING

MEETING ON 20TH SEPTEMBER

Background

Young people speaking about issues living in Amsterdam. The subjects are about major points life. He wants to create a short play with two NAO for December in Dutch. For the festival, new concepts are presented. How can be myself? How define authenticity without the influence of media.

Requirements

Need a story board Write a manual and think about all weird questions. Good lightning for the camera (normal level)

Balancing, fall, Step by Step, Motion control,

Deadline

8 April inside the museum. Dogs played 30 years ago, 2 weeks

MEETING ON 27TH SEPTEMBER

In the coming month you will find the time to make a plan & a script, that you can present to both LIKEMINDS as Jan Derk & Martin of the HvA. The deadline is hereby set at monday 31st of October. Monday 7th of November you will have the response of both HvA and LIKEMINDS. From the 7th of november 'till the 15th of December is 5 1/2 weeks, in that period you should plan the time for the realization of the accorded plan. To recap:

Jan Derk is process supervisor for the HvA students

Martin is the technical supervisor

Caspar is the artistic supervisor I will be in charge of the production of the festival LIKEMINDS Bedankt, where the performances will start on 15, 16 & 17 december, 20:00hrs at the Rozentheater, Amsterdam. Notes - if the project for some reason will not result in a appr. 10-minute performance, LIKEMINDS will think of another opening act on the festival days. IF necessary decision will have to be made in the week of december 5th. - Jan Derk will check the legal aspect of using the robots in a public performance - liability for use of the robots lies with the HvA. - LIKEMINDS will provide a proper environment for the performance to take place (grote zaal Rozentheater, vlakke vloer etc.) If there are comments/remarks on the short summary above of our meeting tuesday 27th, I'll fladly hear or read them.

Project NAO 2012


MEETING ON 3RD OCTOBER

The goal of this event was to share what really we will try to do for the play.

We had a discussion about how define and describe human emotions. Then how define and describe humanoid

emotions. Finally, what and how NAO can express them.

Mustafa wanted to know what NAO is able to do. Like that he can adapt, focus his story to NAO capacity.

We agree to meet the 17th October in order to exchange our first draft: script and basic emotions.

MEETING ON 27th OCTOBER

First restricted meeting with Jan Derk was about the progress, the management and the issue of communication. We discuss about his expected requirements about our work. Jan Derk advises us about the way to work with a client and also between us. The main point is to warn frequently the client of your progress. The more there is check point, the easier it is not to be surprised by the final product. We shouldn’t hesitate to ask what the client expects and how many details we should give them. The management should be shared by main steps however we should know each other work in order to be able to continue it in case of personal issue.

MEETING ON 7th NOVEMBER

The goal of this event was to make a sum up about our progress to Martin. We receive some advices from Martin

about the delay of the first script. According to him, we should focus on technical point that we should use in

whatever situation like balance control, fall detection and remote control.

Good news: Martin will participate to the programming event in December with Aldebaran. Martin found another

environment for the simulation where the Naoqi can be implemented too. It is less heavy, there are more complex

map and the navigation is faster and easier.

MEETING ON 14th NOVEMBER

Main Ideas for the script: o Being human, doing simple action o Close scene o Mustafa and NAO should tease each other o Inspiration on muppet show o Show that the robot is smart, surprised the public about the intelligence of the robot

Discussion about possible actions to do with the NAO o Recognition of objets thanks to 10 Naomarks o Recognition of scene change thanks to strong difference of contrast (light Black/White/Red) o Show the "no-recognition" o Simplified Matrix scene (left and right sidesteps) or fake falling down o Simple movements

Project NAO 2012


Technical points: o English play: use pre-recorded speech (done sentence by sentence by Mustafa and his friend) o Decor already present: White wall, black floor, grey sofa o Possibility to test whenever in LIKEMINDS office

Deadlines: o 15th November: Mustafa have to the first script o 21st November: Arrangement of the first script with Mustafa at HvA o 24th November: Mustafa have to send us the recording o Week 49: Check in the theater

MEETING ON 21st NOVEMBER

Discussion on our rough evaluation o Mustafa’s priority: interaction and acting o Our priority: recognition (object and colors) and synchronisation (songs, motion and eyes)

Addition details on the script o Think about a map of the scene o Decomposition step by step of each reaction

MEETING ON 5tH DECEMBER

Complication with matrix move and walking back

Make the list of object to buy

MEETING ON 13rd JANUARY

Evaluation of the project together with Casper and Mustafa LIKEMINDS and Martin representing HVA. The goal of this meeting was to closed this project and talk about the future plans. During this appointment we exchange our feedback from the three shows and also and also on the entire project. The conclusion was that this project was a success for both parts. Each of us learned about both subjects: acting and robotics. Mustafa really appreciated our work and the concept of the project. He has already planned a performance for the LIKEMINDS Bedankt 2013 and 2016. Casper also approved the results and asked to Martin for an official agreement and partnership between HVA and their company. For the next show coming, they would like to go deeper and do new thing. For instance, use the new “adult” NAO robot.

Project NAO 2012


10.3 ORIGINAL SCRIPT

Before the audience enters the theater they get mini chocolates red/blue) Music - Moonlight Sonata van Beethoven

There can be different colour cards hanging on a board. Red means drink. Green means sleep. Orange means take a shit etc. and

every object: bed, mini-toilet, thee cup are placed on specific points in the space)

Mustafa is laying on a table. He has a sheet on him and we can’t see his face and body. In the corner of the room a light spot

appears. The NAO-robot enters the room DANCING (ballet-like) on the classical music. The spotligth FOLLOWS him. Next to

Mustafa, on the ground, there is a switch/handle. The NAO-robot DANCES towards the switch/handle en stops.

1. NAO: At last, my love has come along. My lonely days are over. And life is like a song. Oh, yeah, at last (Etta james sample)

The Robot pulls the switch. We see a LIGHTNING flash and hear THUNDER. The robot grabs one point of the sheet and pulls it of

Mustafa. Mustafa comes alive. He stands up slowly and streches.

2. NAO: Welcome to jamrock! Out in the street they call it murder! (Damian marley sample)

Mustafa looks the robot.

MUSTAFA: Who are you? 3. NAO: Lonely, I’m Mr. Lonely. I have nobody for my own. I am so lonely. (bobby vinton sample) MUSTAFA: Me too. 4. NAO: Let’s stay together (Al green sample) MUSTAFA: Let’s. What do you want to do?

The NAO robot GRABS a paintbrush en PUTS some YELLOW paint on it and GIVES it to Mustafa. Mustafa starts to paint on a

sheet (schildersezeltje met doek). The NAO robot comes and stands next to Mustafa.

5. NAO: I think. I think. I think. I like it. (Fake blood sample) MUSTAFA: You do? 6. NAO: YEAH!!! (Lil jon sample) MUSTAFA: I think... Yellow is my favourite colour.

The nao robot looks at the YELLOW and takes a brush and starts to PAINT with a BLUE brush on the sheet. Mustafa looks suprise

to the NAO-robot.

MUSTAFA: What are you doing? 7. NAO: It’s like that and that’s the way it is (Run dmc sample) MUSTAFA: Sorry? 8. NAO: It’s like that and that’s the way it is (Run dmc sample) MUSTAFA: That’s arrogant. Isn’t it? 9. NAO: No no no no no no (Destinys child sample) MUSTAFA: Really? 10. NAO: Yeah yeah yeah yeah yeah (Destinys child sample)

Project NAO 2012


Mustafa puts down the brush. The NAO keeps on PAINTING. Mustafa takes a little ball and helds in front of the NAO so it can see

the ball.

11. NAO: don’t play any game I can’t win. (beastie boys sample) MUSTAFA: Bitch. 12. NAO: Mmmm Watcha say? (Jason derulo sample) MUSTAFA: Nothing.

Mustafa takes the sheet (mini-schildersezel) away and puts a mirror in front of the NAO. The Nao keeps on painting.

Mustafa looks at the cards. He looks at the cards DRINK and SHIT. He walks towards the toilet and the cup and changes their

positions in the space. He takes the DRINK card and helds it in front of the NAO. The NAO stops with painting a walks to the toilet

and "drinks".

MUSTAFA: How does it taste? 13. NAO: I feel good! (James brown sample)

Mustafa takes the SHIT card and holds it in front of the NAO robot. The robot goes to the CUP and "takes a shit". Mustafa sees a

cupcake. He eats the cupcake.

MUSTAFA: What are you doing? 14. NAO: shiiiiiiiiiiitt (The Wire sample)

After finishing the robot walks to the place where the cupcake was. He SEES the cupcake is gone.

15. NAO: Tell me why. (backstreet boys sample) MUSTAFA: Wat? 16. NAO: Tell me why. (backstreet boys sample) MUSTAFA: Hahahaha 17. NAO: Don’t joke with a hungry man (Quantic sample)

Mustafa goes to the audience and ask who has a chocolate? (If everybody ate it he ask for the wrap) Mustafa holds the paper of

a MILK chocolate in front of the robot

MUSTAFA: You like Dark Chocolate? 18. NAO: Oh no no no no don’t lie! (black eyed peas sample) MUSTAFA: I’m not lying. 19. NAO: I started this gangster shit and this is the motherfucking thanks I get? (dr dre sample) MUSTAFA: You want to take this outside? 20. NAO: Bring it! (Dmx sample)

Mustafa and the NAO robot look at a gun in the middle of them. (Maybe the good, the bad and the ugly soundtrack)

They go for the gun. Mustafa is there first. Then he aims the gun to the NAO robot. In movement the do the matrix shoot out in

slow motion. At the end both of them fall on their back. Only the NAO robot is able to stand up by himself. Mustafa can’t.

Project NAO 2012


10.4 FINAL PRESENTATION

Project NAO 2012


10.5 POSTER

10.6 PRESS ARTICLE

Project NAO 2012


10.7 CODES

Emotions

Motions

Movements

Recognition

Project NAO 2012


Script

Interaction

Tools

Database

Fuzzy logic

Library

To download on http://boyault.matthieu.free.fr/Webpage/Pages/Projects/ProjectNAO/projectNAO.html

PROJECT NAO - Rutgers DCS Hacker Spacehackerspace.cs.rutgers.edu/library/NAO/NAO-programming.pdf ·...

Documents

Transcript of PROJECT NAO - Rutgers DCS Hacker Spacehackerspace.cs.rutgers.edu/library/NAO/NAO-programming.pdf ·...