ARPUPPET – A CHARACTER Animation FRAMEWORK for Augmented Reality Istvá n Barakonyi StbZwerg...

ARPUPPET – A CHARACTER Animation FRAMEWORK for Augmented RealityIstvá n Barakonyi

StbZwerg Day’03

/18

ISTVÁ N BARAKONYIBARA @ IMS.TUWIEN.AC.AT

Main goal

Combining AR applications with intelligent, animated characters

• NO augmented paper clip• NO AI, avoiding the term 'agent‘• Interface to reasoning engines, behavior generators

Main focus on

• Researching important aspects and implications for intelligent animated characters Visualization, appearance Behavior Required technology Applications

/18


VisionAR applications with embedded intelligent animated characters

with multiple representations that are aware of

Own Other characters

Users Props

Pose BehaviorContext

Storage

/18


ARPuppet FrameworkSeveral different characters acting together Puppet theater metaphor

Author/Storyteller

Director

Choreographer

Puppeteer 1 Puppeteer 2 Puppeteer n

Puppet 1.1 Puppet 1.P Puppet 2.1 Puppet n.1 Puppet n.Q

…

… …

/18


Different representations

Director

Choreographer

Puppeteer i

Physical Virtual Augmented

/18


Different devices/platforms

Director

Choreographer

Puppeteer i

Projection screen PC monitor PocketPC

/18


Puppeteer features• Command parsing /w type checking

commandType %s|%f|%d• Unified command queue: parallel / sequential• Unified feedback mechanism

If unsupported, provide default feedback• Default implementation of commands

Position, orientation, scale goTo, lookAt, turnTowards, pathMove etc.

• Conversation: command - response• Multi-level compound commands

goTo = { turnTowards, go, stop }, go = { walk_anim, setPosition }• Handling multiple puppets and attributes

Puppet-specific command execution• Tracking functions

Tracking on/off, Relative pose to tracking station, snap-to-station• Default bounding box for collision detection/path planning (IV)• Hotspots

Particular meaning or function (e.g. buttons, rooms) Points of interaction (hand, head etc.)

• External speech & sound support if not featured by puppets

/18


Choreographer features

• Command parsing with type checking: own cmds without type checking:

puppeteer attributes can be params Easy scripting

• Command synchronization Easy handling of group commands

[targetList:] commandType [parameterList]

• Conversation: command – response Choreographer’s own commands

- Getter functions- Events: collision, facing, near, approaching etc.- Utilities: motion planning, path following, terrain

following Puppeteer commands

- Choreography- e.g. goTo pathMove, walk run

/18


Scripting examples

Example 1: { LEGOrobot: motorState +--,LEGOmaxl: goTo legorobot.position 2000 } ,LEGOmaxl: pointAt legorobot.hotspot#motorA

Example 2: TourGuide: pathMove (building.hotspot#signpost.path).position 5000,{TourGuide: pointAt building.hotspot#signpost.currentRoom 3000,TourGuide: speak `This is our seminar room.` }

Example 3:chess.nextStart: goTo chess.nextDest.position 2000, chess.nextStart, chess.nextDest: play `fight` 3000 ,chess.nextStart: setAttribute `name` chess.nextDest

/18


Existing characters

/18


Creating characters a new Puppeteer

• Puppets Prepare to communicate with all representations Adapt commands

• Attributes Sub-scene graphs + field connections

• Supported commands Reimplementing standard commands

(Walk, Run, Grab, Drop, Point_at, Look_at, Turn towards/away, Idle, etc.)

Own commands• Feedback

Which command has finished and when Set up triggering structure

• Bounding box calculation structure e.g. for collision detection, path planning

• Hotspots Particular meaning or function (e.g. buttons,

rooms) Points of interaction (hand, head etc.)

/18


Configuring / Interaction / Embedding

Choreographer

Puppeteer

Puppet

command

command

attributes 1..j

IV, C++, python, …

APRIL

PUCPUC device

component

class, wrapper…

response

response

response

Application

attributes 1..i

Story engine

Conditions, events

wrapper

/18


Personal Universal Controller (PUC)

• Carnegie Mellon University (Pebbles project)+ Thomas Psik: Inventor integration

• Automatically generate control GUI Statically configured Dynamically changing (coming soon)

• Goal: walk up & connect to any object in a running ARPuppet framework to configure

• generate GUI to configure and control Puppet Puppeteer

- Command types (e.g. setPosition)- Command format (e.g. %f %f %f %d)

Choreographer- Currently loaded puppeteers

puppeteerName: puppeteerCmd [parameterList]

choreographerCmd [parameterList]

/18


Enhancing applications /w characters

• Characters Gestures Positioning Highlighting Animation Auxiliary objects

- Arrows- Labels- Markers- Other MM objects

• Interaction, communication Multimodal 2D/3D GUIs (PIP, PUC) Users’ own behavior

/18


Test applications

• Dummy examples• ARSensei – early demo• ARLego• Virtual Tour Guide • Planned demos:

Battlechess- Testing scripting power

“Geist 2”- Ultimate demo- Multiuser role playing game

/18


Contribution and current status

• Combination of a collaborative AR framework and character animation framework has not existed before

• Working prototype• Possibility to experiment with numerous characters with

several representations • Possibility to experiment with many applications to

prove usefulness own tests/apps VRLU projects

• Default values and methods provided very easy and quick to add own character without much extra work

• Easy scripting• Can be smoothly integrated with other projects

Studierstube apps PUC APRIL

/18


Future plans

• Adding lots of extra functionality to the Puppeteer and Choreographer objects

• Lots of implementation work• Closer cooperation with APRIL• Studierstube + APRIL + ARPuppet + PUC

powerful combination for AR storytelling• Searching for new applications

GEIST 2 ?

/18


Questions?

ARPUPPET – A CHARACTER Animation FRAMEWORK for Augmented Reality Istvá n Barakonyi StbZwerg...

Documents

Transcript of ARPUPPET – A CHARACTER Animation FRAMEWORK for Augmented Reality Istvá n Barakonyi StbZwerg...