Media Semantics Character Builder

User’s Guide

Copyright © 2000-2017 Media Semantics, Inc. All rights reserved. Information contained in this document is subject to change without notice, and does not represent a commitment on the part of Media Semantics. The product described in this User’s Guide is provided under the terms of a license agreement. The license agreement specifies the terms and conditions for its lawful use. No part of this document may be reproduced or transmitted in any form, or by any means, without the express written permission of Media Semantics.

2

Contents Getting Started................................................................................................................................. 4

Overview ..................................................................................................................................... 4 Requirements............................................................................................................................... 4 Acknowledgements ..................................................................................................................... 4 Documentation ............................................................................................................................ 5 Installation ................................................................................................................................... 5 A quick tour................................................................................................................................. 5 Obtaining technical support......................................................................................................... 6

Slideshow Tutorial .......................................................................................................................... 7 Creating the scene........................................................................................................................ 7 Creating an opening slide ............................................................................................................ 9 Adding a message...................................................................................................................... 10 Obtaining audio ......................................................................................................................... 10 Adding animation ...................................................................................................................... 11 Understanding actions ............................................................................................................... 12 Adding a second slide................................................................................................................ 13

Sequence Tutorial.......................................................................................................................... 15 Creating a sequence................................................................................................................... 15 Understanding automatic actions............................................................................................... 17 Navigation with sequences ........................................................................................................ 17

Powerpoint Tutorial....................................................................................................................... 18 Preparing for import .................................................................................................................. 18 Running the wizard.................................................................................................................... 18 Dealing with changes to your Powerpoint................................................................................. 19 Adding one slide at a time ......................................................................................................... 19

Screen Capture Tutorial................................................................................................................. 20 Preparing for import .................................................................................................................. 20 Using the wizard........................................................................................................................ 20 How it works ............................................................................................................................. 21

Quiz Tutorial ................................................................................................................................. 23 Building a multiple-choice slide................................................................................................ 23 Prompts and reactions................................................................................................................ 23 A second interaction.................................................................................................................. 25 Keeping score ............................................................................................................................ 26

Dialog Tutorial .............................................................................................................................. 27

Web Greeter Tutorial..................................................................................................................... 28 Adding a character to your web page ........................................................................................ 28 Triggering messages from links ................................................................................................ 28 Synchronizing events with your messages ................................................................................ 29 Triggering URLs ....................................................................................................................... 30 Passing variables from HTML .................................................................................................. 30 Conditional messages ................................................................................................................ 30 Giving your character a transparent background....................................................................... 30

HTML5 and Mobile Browser Tutorial .......................................................................................... 32 Mobile browsers and audio ....................................................................................................... 32

3

HTML5 Animation Tutorial.......................................................................................................... 33

Video Tutorial ............................................................................................................................... 34 Apps versus animation/video .................................................................................................... 34 Rendering to video .................................................................................................................... 34 Output preferences..................................................................................................................... 35

Guided Discovery Tutorial ............................................................................................................ 36 Objects....................................................................................................................................... 36 Conditions and Auto Start ......................................................................................................... 37 The seen() Function................................................................................................................... 38 Reacting to Inactivity ................................................................................................................ 38 Reacting to Interaction .............................................................................................................. 39

PhotoFit Character Tutorial........................................................................................................... 40 Introducing PhotoFit.................................................................................................................. 40 Shape and texture controls......................................................................................................... 41 Obtaining a face from a photo ................................................................................................... 41

PhotoMorph Character Tutorial .................................................................................................... 43 Introducing PhotoMorph ........................................................................................................... 43

4

Getting Started

Overview The Character Builder™ software installs on your PC and lets you generate unlimited, royalty-free animation in HTML5, Flash, and Video formats. The Builder can be used to create linear animation, for insertion into leading video and training tools, as well as non-linear animation, to give characters a continuous presence and the illusion of life. At the heart of the Character Builder is an animation engine that combines specially-prepared vector and bitmap art into an animated character presentation. A “character” is really a library of images that the software can draw upon to produce high-level actions, such as Look, and Point, that are synchronized with your message. The Builder comes with a number of stock characters, and other characters can be downloaded from the Media Semantics site. The Character Builder is more than a tool for creating character animation. Characters can control, and be controlled-by, a wide range of supporting objects, including images, movies, buttons, and more. Supporting visuals can be placed in Scenes and Slides, and a navigation system lets end-users navigate within the content, or be led through the content by a character. The Builder’s media library includes several stock objects, such as quiz buttons for use in online-learning applications, and entry panels for use in question-answering applications. You can specify content in a visual manner using the Builder’s Outline, Design, and Script views. These views are merely graphic representations of a rich underlying markup language, which can also be edited directly in an optional XML View. The underlying XML tag set forms a high-level application model, and programmers can leverage this model with the optional Character API and Character Server products to create solutions in which the message is not known in advance, such as the current weather, or a greeting typed-into a web form. Though you may start by using the Builder purely as a means of creating character animation for use within another application, we hope that you will also have an opportunity to explore the rich standalone applications that can be built using these mechanisms.

Requirements The Character Builder requires a PC running Windows 7 or higher. The Powerpoint import feature requires that you have Powerpoint 2003 or higher installed. The Builder works with recorded voice, as well as with any SAPI 5-compliant Text-to-Speech (TTS) engine. Please note that many TTS engines are licensed for personal use only. See the Media Semantics web site for more information on high-quality TTS engines that permit the distribution of audio with the Character Builder output.

Acknowledgements Recorded speech is lip-synced using technology under license from Annosoft LLC. PhotoFit and Face Design features use technology under license from Singular Inversions, LLC. The Builder uses the LAME encoder module lame_enc.dll (see www.mp3dev.org), when present in the application directory, and any compatible ACM MP3 codec when absent. A compatible version of lame_enc.dll is installed by the installer, for your convenience. The Builder also works with ffmpeg.exe, an implementation of FFMPEG (see www.ffmpeg.org), when present in the application directory, to provide FLV and MOV video output. The ffmpeg.exe file is installed

5

by the Character Builder installer for your convenience, but may be removed and/or independently recreated from FFMPEG sources. Some tutorials in this guide provide information on how to use the Builder with Microsoft Powerpoint and Adobe Flash. Neither product is required, in order to use the Character Builder.

Documentation This User’s Guide contains a series of tutorials designed to introduce you to the Character Builder and the underlying application model. The product also contains the Builder Reference, with information on the Character Builder. All documentation is available from the Builder’s Help menu.

Installation To install the Character Builder, please run the associated installer program. The Builder will install by default under “C:\Program Files\Character Builder”. Your project files can be located anywhere on your machine, but by default the File > New Project command will create projects in your “My Documents\Character Builder Projects” folder. This is also where you will find the tutorial files.

A quick tour The Builder lets you work with content files, image files, and audio files.

The Builder window is divided into three different panes: Editor Pane The top pane lets you edit files. Several files can be open at a time, and you can switch

between open files by clicking on the appropriate file tab. Property Pane The middle pane is used to view an object’s properties.

6

Message Pane The bottom pane is used to display error and status messages, and is typically closed. You can open it using View > Show Message Pane command.

The Editor Pane lets you view files in a number of different views. You can switch between views using the view buttons at the bottom of the Editor Pane. Outline View Outline View lets you view and manipulate the hierarchical structure of your project. Design View Design View lets you directly place and size objects in scenes and slides. Script View Script View lets you edit a character’s words and actions, and access the Builder’s voice

recording, lip-syncing, and audio cleanup features. Preview View The Preview View lets you view your content as it would appear in a web browser. XML View The XML View lets you edit the markup tags that make up your project. All changes

made through the Outline, Design, and Script views will result in changes to the markup, and you can always view and edit this markup directly in XML View. The XML View is turned off by default, but can be turned on through Edit > Preferences (General tab).

You create a new project by invoking File > New Project. The New Project wizard will ask you a series of questions, and then create a new project (.mpr) file. You can open a project with the Character Builder by double-clicking on the project file from any Windows folder. By default, new projects are created in a “Character Builder Projects” folder beneath your “Documents” folder. A project consists of a hierarchy of structural objects, such as Scenes, Characters, Slides, and Steps. Scenes can contain characters, navigation elements, “props” and other visual elements. Slides can be used to organize a sequence of visuals into a slideshow. Steps let you break a longer presentation into a series of smaller steps. Scenes and slides contain visual objects, including characters, images, and text, that are edited in Design View. Scenes can also contain Scripts, which are discrete sequences of actions that a character can perform. Scripts are edited in Script View. You can use the Project > Render (Quick) option, or the Render button, to render your project to the chosen media format. The Render (Full) command differs from Render (Quick) in that it does not cache any intermediate files. You might use it if you suspect that changes, for example due to the installation of an updated character, are not being picked up. The generated files are located in the Output subdirectory of your project file. They will include a sample HTML file, and media files in the chosen format.

Obtaining technical support Registered users of the Character Builder may access technical support via support@mediasemantics.com. Before contacting technical support, please be sure to visit our support area, at http://www.mediasemantics.com/support.htm. Here you will also find a link to our support board. When emailing, please be sure to include any sample project files.

7

Slideshow Tutorial This tutorial steps you through the creation of a simple slideshow presented by a character. Along the way we’ll introduce various aspects of the Character Builder. We’ll create the project in a series of steps. We encourage you to complete these steps yourself, but you can also see the final results by examining the files in the “Character Builder Projects\Tutorials\Slideshow” directory.

Creating the scene To begin, select File > New Project and enter a name for your project (e.g. “Slideshow”). There are some powerful wizards that will let you set up a slideshow, but for this tutorial, please start with the “Empty Project” selection. The result will be an empty project in the “My Documents\Character Builder Projects\Slideshow” folder.

The “Slideshow” file will be open for editing in the Editor Pane in Design view. The Property Pane below shows the properties for the currently-selected object. Switch to Outline View by clicking on the Outline button – you will see your project structure, along with tools for altering the structure.

A Project can contain one or more Scenes. Only one Scene will be visible at any given time, and it occupies the entire project area. Switching between scenes is a bit like switching between pages in an HTML browser. Let’s now add a visual element to our scene. Click on the Design tab to switch back to Design View. Locate the Library panel in the bottom-right area of the window. Expand “Props and Sets” and “Lecture Hall Set”, then select “Lecture Hall Background” and drag it to the scene – a simple two-color background will appear. Repeat and add the object named “Platform” and the object named “Whiteboard”.

8

You can select an object directly, or by clicking on its entry in the Layer panel on the right-hand side of the view.

You can drag the selected object to reposition it, or, if the object is resizable, you can drag the handles to resize it. The properties of the selected object are shown in the Property Pane below. There you can specify coordinates numerically, or alter other properties of an object, such as its name. Objects are positioned on the scene by specifying the x and y positions of the upper-left corner of that object. The coordinates are always specified in pixels, with the origin being the upper-left corner of the Scene or Slide, and with positive y coordinates going downward. You can use the keyboard arrow keys to “nudge” an object, that is move it one pixel at a time. You can also hold down the Shift key to nudge an object in increments of 10 pixels at a time. Each object is conceptually on its own “layer”. Layers work like a stack of transparencies, with one object on each transparency. You can use the Layers panel to hide or lock these layers, making them invisible, or inaccessible for selection, respectively. This can be important when you have a busy scene or slide. The Hidden and Locked state of an object in the palette has no effect on how your slide or scene actually runs, and is not to be confused with the “Starts Hidden” checkbox available on the property panel for most objects. You can reorder the layers (i.e. move a “transparency” forward or backward in the stack) using the Move Forward ( ) and Move Backward ( ) buttons. Another useful feature when dealing with complex slides is the ability to select multiple objects and drag them together as a unit. To select multiple objects, click and hold the left mouse button to create a selection marquee, or Control-click on multiple objects on the scene or in the Layer pane. We still need to add a Slideshow object to act as a container for our slides. Like the Stage, the Slideshow object has no appearance of its own, since its entire area is covered by the current slide. Use the Add Slideshow button ( ) to add a slideshow. The slideshow will appear as a gray rectangle in Design View. The Slideshow and the whiteboard prop are independent objects – you can use one or the other, or both. To complete our scene, we’ll need a character. From the Library pane, choose Characters > Realistic Body > Steve and drag him to the stage so that he appears to stand on the platform, with the whiteboard to his right. Note that you may be asked to install the Steve package – because of their size, most realistic characters are packaged as separate installers that can be installed as needed. If you now switch to the Preview Pane, your scene should look something like this:

9

Creating an opening slide Next we’ll create a slide for Steve to present. Switch to Outline View, select the Slideshow object, and click the Add Content button ( ). The Content wizard is a quick and easy way to import your content. For now, choose “Empty Slide”.

To edit the contents of this slide, return to Design View and, if necessary, select the slide from the sidebar. Note that you can quickly view a given slide in Design View by double-clicking on it in Outline View. Now add a title by clicking on the Add Text button ( ). You can either click-and-release at the location where the text should start, or you can click and drag out a rectangle that the text will flow into. In the first case you give the text frame no fixed width, so it simply resizes to conform to the text. In the second case you give it a size, so it wraps the text over multiple lines to conform to this width. When you create a Text object you can immediately begin typing “My First Slideshow”. When you are done typing, press Esc or click outside of the text object. You can return to “edit mode” on a text object by double-clicking on it. When a Text object is selected you will see the text formatting controls become active in the toolbar. You can use these controls to modify the selected text object. You can also double-click and edit the object to apply the formatting to specific ranges of text. Note that the Builder uses pixels and not points as a unit of font height.

Next, let’s try adding a slide background. To do so, use the Library pane to navigate to Backgrounds > Slide Backgrounds and choose “Blue Wash”. With the blue background, you might want to change the color of the text to white. You can select the text object and locate the Text Color button in the text formatting toolbar, then pick a white color from the Color dialog.

10

In the Preview view, your whiteboard might look something like this:

Adding a message Next we’ll add some voice and animation. Returning to Outline View, select the Slide object and press the Add Message button ( ). You can use the “Empty message” template. The Message object goes in your slide, but it doesn’t have any visual appearance of its own. Message objects won’t appear in Design View.

To edit a message, you switch to Script View. You can use the Import Text ( ) feature to import several lines of text. Or you can simply begin typing to start a new line of text. Try entering “Welcome to my first slideshow”. As you type, the Script View area will update to reflect your changes:

Obtaining audio If you don’t do anything else, audio will be created for you using a Text-to-Speech voice. Alternatively you can use the Record button () to record your own voice. Note that this button is “push-to-talk”, i.e. you hold it down while you are speaking. For best results we recommend using recorded audio, however several high-quality Text-to-Speech options are also available for purchase from the Media Semantics site. These “speech packs” are specifically licensed for use with the Builder, and include distribution rights to the resulting audio when embedded in the Character Builder output. Depending on your operating system you may already have the Text-to-Speech voices Microsoft Sam (Windows XP), Microsoft Anna (Windows Vista and Windows 7), or Microsoft David and Hazel (Windows 8, 10). These voices are free, but tend to sound robotic – they are a great way to obtain a quick voiceover for testing purposes.

11

If you are using Text-to-Speech then you may need to use some creative spellings and insert extra punctuation to make your transcript come out right. Depending on your speech engine, you may also have access to a Lexicon feature that lets you customize the pronunciation for specific words, under the Tools menu. If you are using recorded audio and forget to record a line, then you may be surprised to hear a synthesized voice deliver the missing audio. If you are using Text-to-Speech then we recommend that you use the Play button frequently while you are editing the message, as it lets you catch pronunciation errors early.

Adding animation Next we’ll add an animation action. Click just before the “W’ in “Welcome” to place the insertion point at the beginning of the action. Insert a “Hands Up” action, by clicking on the Add Hands Action button and selecting the first icon from the drop-down list (Hands Up).

Your Script View will now look something like this.

Each line with text is a Say action. Any other actions on that same line are synchronized with that text. You can click on an action icon to select the action. When an action is selected, its properties will appear in the Property Pane. Many actions have a Target property that allows a target for an action to be selected. Now click right below this line and notice how the blue insertion mark in the margin indicates where your next insertion will occur. You can click between lines to insert a new line, or below the last line to add a line. Now press the Insert Pause Action button to create a line containing a Pause action. You will often want a Pause action between distinct Say actions. The default duration of a pause is ¾ of a second. A line will often consist of one or more action icons, and no text. These actions are not synchronized with any audio, and are simply played silently. Next, create a third line: “My name is Steve, and I will be your host today.”

You synchronize an action with the voice by placing the action where you would like the action to begin – in this case it will begin at the same time as the Welcome audio starts. Finally, switch to the Preview view to see the result of your work.

12

Note that you can place several actions in a row. In many cases two successive actions will actually run concurrently. For example a Look Left and a Gesture Left can be played concurrently since they involve different parts of the body. However a Look Left and a Look Right action cannot be played concurrently. If a second action becomes ready for execution when the first action is still running, and the two cannot play together, then the second action will simply wait for the first to complete. All actions embedded in a Say action get executed in order – if needed the Say action is extended with silence in order to let all actions play. You can try moving the Hands Up action within the Say action. You can do this in Script View by clicking on the action and then dragging it to its new position. How important is it to keep the transcript in sync with the audio? In the case of Text-to-Speech, the transcript is read to create the audio when you press the Play button or when you press Render. In the case of recorded audio, a new () button will appear to the left of the Play button that lets you select whether or not the text transcript is accurate (i.e. matches the audio), and may be used to improve the lipsync process. When your project is running smoothly in Preview view, you can use the Render button ( ) to produce distributable media files. Next, click the Explore Output Files ( ) button. A File Explorer window should appear showing the contents of the Output subdirectory of your project. You are now ready to copy these files to a standard web server. The HTML file serves as a test container page – you can cut and paste the HTML embed code from this HTML file into your own HTML page.

Understanding actions The Builder does not re-render Steve from 3D primitives, but chooses from a set of pre-rendered images that are carefully designed to let him perform a range of common actions. The process of creating a new character, or extending the range of motion of an existing character, requires a deeper understanding of the mechanism that drives the character’s animation, and access to the required 3d modeling and/or vector art tools. However you can use a “stock” character from the library, or download additional characters from the Media Semantics web site. Many characters, including Steve, can also be modified through properties in the Customize pane. Additional compatible hair and clothing for Steve can be purchased separately. Media Semantics defines a Standard Animation Model that characters are expected to conform to where possible. This model includes many of the actions that you would expect a “presenter” to use, including turning, looking, pointing, speaking, hand movements, etc. When a given action is not available, such as when a hand action is used with a Head-only character, the character will simply ignore the action. Actions can affect a character’s body, arms, head, mouth, or eyes. Some actions, such as “Palm Up”, put the body part into a particular state, or “pose”. Other actions, such as “Palm Wave” animate a body part and then return to the same state. Actions are designed to be largely independent of one another. For example you can have your character look at an object, point at it, begin speaking, then look back at the user while still pointing, then raise its eyebrows, etc. You can play many actions concurrently for greater effect. For example you could create a “shrug” action by combining a Hands Out, and an Eyes Wide action. Since the actions involve independent body parts, the system can play both actions concurrently, even though they are specified sequentially in the script. Media Semantics characters tend to have many small, independent actions that can be combined to produce fluid character animation. However independence of motion can come at a high cost to the animator. Consider the body position for example. In most full-body characters, the character can be turned to face the user (front) or turned partially to the left or right (half-profile). For each position, the animator might need to draw a completely new set of images for all arm actions. For this reason, some actions are limited

13

to certain body positions. For example many hand actions are only implemented for the “front” position. The half-profile position is used primarily when the character is drawing the attention to some supporting visuals – he can look and point at the content, and may glance back at the user. Most characters will implement the mouth and eye animations for all head positions, and “preserve” these expressions as the head turns. This allows a character to talk as it looks back and forth between the user and any supporting visuals.

Adding a second slide You can use the Character Builder to create a standalone application that lets the end-user start, stop, and navigate within your content. To do this we’ll add a navigation bar to our scene. Switch to Design View, and be sure you are viewing the entire Scene, and not the slide. You can do this from the sidebar selector:

Use the Library pane to navigate to “Objects > Navigation Bars”, then choose “Standard Navigation Bar”. You’ll see a navigation bar appear at the bottom of the scene. To create a second slide, you might want to take advantage of the work you’ve already done. You can go to Outline View, the select the first slide, right-click Copy, then select the slideshow, and then right-click Paste. Next go to Design View and change the text of the second slide to, say “My Last Slide”. Next, go to Script View – you can select the first line by clicking in the margin, then shift-clicking before the last line to select all the lines, then hit the Delete key to remove them. Create a new Say action with the words “Goodbye!”. When it is not presenting, a character plays an “idle” behavior. An idle behavior will normally be created automatically based on the character’s Idle Behavior property. When the project starts, the character will normally begin presenting the first message. You can change this behavior through the Initially Presenting property of the Project object, in Outline View. A character normally plays through a message until the end, then stops, i.e. goes idle. Message objects are “atomic units of presentation”: they can be interrupted, however they can only be restarted from the beginning. For this reason, longer presentations should normally be broken into distinct steps using the Sequence and Step objects. This will be covered in the next tutorial. A character can transition from one message to another, however to ensure that there is no jump in this transition, the Character Builder ensures that all messages start and end with the character in the same “default” position, usually with the character facing the user and its hands at rest. If you test the current Slideshow project, you’ll find that the character presents the first slide, then stops. You can navigate to the second slide, and get Steve to present it by pushing Next, and then Start. You can also have the character control the navigation. To do so, add a Next action as the last line in the first presentation.

If a Next action is reached at the end of a message, then the character will “move the focus” (i.e. the current slide or step), just like when the end-user presses the Next button. Since the character was presenting when it encountered the Next, by default it will continue presenting on the next slide. When the focus is moved, the new slide is normally shown immediately – then the character returns to its default position in order to make the transition to the next message. A nice effect is to end your slide with the following sequence:

14

This makes the character pause slightly for the last few words to sink in, then look at the slideshow and “will” the slide transition to occur. In subsequent tutorials you’ll also see how the character can cause all kinds of changes to occur in the slide, and also to react to the user’s actions.

15

Sequence Tutorial In this tutorial we’ll build a slightly longer presentation that is broken down into a sequence of steps. Each step is associated with a visual change, and the user can navigate through the steps using the navigation bar. We’ll also show how a character can control objects on the screen and discuss automatic character actions. You can see the final result in the “Character Builder Projects\Tutorials\ Sequence” directory.

Creating a sequence We’ll create a series of 3 text bullets that Brad will reveal in a step-by-step manner. We’ll have 4 steps – one for each text bullet, and one to wrap up. In this case we’ll put the visuals on the scene itself, though you could also place them in a slide.

Note that the Content Wizard will let you easily create slides using bullets, but in this tutorial we will be starting from first principles to explain the concepts involved. Start by creating a new project using File > New Project with the “Empty Project” template. In Design view, use the Library pane to navigate to Backgrounds > Slide Backgrounds, and drag the “Blue Wash" background to the scene. Next, use the Library pane to navigate to Characters > Cartoon, and drag Brad to the scene. You will also want to add a “Standard Navigation Bar” from Objects > Navigation Bars. Next, use the “Add Text” button ( ) to create a title and 3 bullets. You can use the Alt-0149 keystroke sequence (hold the Alt key while pressing 0 – 1 – 4 – 9 on the keypad) to insert the bullet character – a complete list of special character can be found from Windows > Start, Accessories, System Tools, Character Map application. After creating the first bullet, you can rapidly create the second and third using Copy and Paste. You can use the arrow keys and control-arrow keys to nudge an element into place using the keyboard.

Next, set the Starts Hidden checkbox for each of the three bullet text objects.

16

Note that you can control whether an object is hidden (or locked) from the layer panel to the right of the view, but these settings are completely independent from the Starts Hidden property. The layer panel simply lets you work with complex layouts in Design View by locking down or hiding those objects that you are not currently editing. Now switch back to Outline View, and, with the Scene object selected, use the Add Sequence button () to add a Sequence object. Select the Sequence object and choose Add Step (). Repeat so as to add 4 steps in total. Then select each Step and add an empty Message to it using Add Script. The resulting outline should appear as follows:

Next, we fill in the message for each step by switching to Script view. For the first step:

We added a slight head-nod to go with the “Hello”. Then, coinciding with “This is my first bullet item”, we make Brad look and gesture to his right, and then Show the (initially hidden) bullet 1. The Show action requires a Target property, which you can set in the property pane below. If you haven’t renamed your text objects, then this would be “Text 2”. When we are done, we move the focus to the next step using the Next Focus action, depicted by the arrow icon. Here is the second message.

Here we have the character turn to half profile – from this position he can point to items, and also glance back at the user – useful when you want to get the user to focus on the content. Note that, in addition to Look Right and Point Right, you can use Look At and Point At, along with a target, to let the character determine the optimal head and arm position based on the position of the character relative to the target. These “targeted” actions are often only implemented from the “half-profile” position, as they appear more natural from this position.

17

And the third message:

We end with a follow-up message.

Understanding automatic actions Before we continue, let’s discuss some of the “automatic actions” that are taken by the character. In the previous messages, we concerned ourselves with getting the character into its different poses, but not so much on returning Brad’s head to look at the user, or lower his arms again. You can always return the gaze

to the user with the Look User action ( ), and lower the arms with the Hands By Side action ( ). But in many cases this won’t be necessary because of the Automatic Actions feature. To simplify the authoring process, most characters have some “automatic”, or “reflexive” actions. Generally you can expect a character to:

1. Return the hands to a resting (hands-by-side) position if left too long in an extended position, 2. Return to look at the user when talking with the head turned away from the user for too long, 3. Blink 4. Use subtle head movements 5. Use subtle arm or body movements.

Behaviors 1 and 2 mean that you just have to “push” the character in different ways, and the character returns to a default, “natural” state by itself, if you haven’t already instructed it to do so explicitly. Note

that you can also specify that the character “hold” a certain position using the Hold action (), which “holds” the last specified body part in its current position until another action is explicitly taken on that body part. Behavior 3 through 5 will interleave additional subtle animation, such as blinking, to provide a sense of life to your character. You can selectively turn off different automatic behaviors in the Advanced Properties panel of your Character.

Navigation with sequences You’ll find that Brad presents each bullet in turn, as we would expect him to. You can also stop Brad at any time using the Stop button on the navigation bar. When a message is stopped, Brad will go idle, but notice what happens when you press Start again. Since Message objects are “atomic units of presentation”, Brad restarts at the beginning of the current message, which is the beginning of the current step. This is one good reason to break a long message into several short ones using steps. Another reason is to provide navigation within the sequence of steps, to let a user skip backward or forward. Navigation within a sequence of steps can be made to appear very natural to the end user, as long as you follow two principles. First, each step should ideally be associated with some visual change – in this case a bullet is revealed during each step. Second, a “wrap-up” step, such as our fourth step is also recommended, in which all visuals are showing.

18

Powerpoint Tutorial In this tutorial we’ll use the Powerpoint Import feature to create a slideshow presentation directly from a Powerpoint file. You will need Microsoft Powerpoint 2013 or higher installed in order to complete this tutorial. You can see the final result in the “Character Builder Projects\Tutorials\ Powerpoint” directory.

Preparing for import A simple, two slide Powerpoint file “Powerpoint.ppt” is located in the “Character Builder Projects\Tutorials\ Powerpoint” directory.

Running the wizard Start a new project with the File > New Projects wizard and the “Slideshow” template. When you are asked, indicate that you want to start from a Powerpoint file:

You will be asked for your Powerpoint file.

When you press Finish, the Builder will read the Powerpoint file and create a series of images in the Images subdirectory.

19

When the import is complete, you will have a Character Builder slide for each of your original Powerpoint slides. Switching to Outline view, the project should look like this:

Once imported, each Builder slide becomes an independent unit. You can rearrange the order of the slides, as well as delete any slides you no longer need. You can even add additional slides, such as a Multiple-choice question slide, using the Slideshow's Add Slide right-click command or the Add Content button ( ).

Dealing with changes to your Powerpoint In Outline View you can right-click on any slide that was originally imported from Powerpoint to access the “Update from Powerpoint” command. This feature works by remembering the index of the original Powerpoint slide, so if you add or remove a slide from the Powerpoint deck, then the index may have shifted, and Update from Powerpoint may update to the wrong slide entirely. It is generally best to avoid removing or inserting slides in your Powerpoint after the import. However it is quite safe to add new Powerpoint slides at the end of the Powerpoint deck. Note that Character Builder also creates default slide names of the form "Slide N", and these numbers will actually line up with the Powerpoint slide indices when you create a new slideshow project from a Powerpoint file, however this is purely coincidental. You might consider renaming your Character Builder slides based on their content after importing.

Adding one slide at a time If you add a new slide to your Powerpoint then you can import that slide directly using the Slideshow's Add Slide right-click command. Select “Powerpoint slide”, then select the Powerpoint file and the slide to import.

20

Screen Capture Tutorial In this tutorial we’ll use the New Project wizard to create a Video Commentary project, in which a character “narrates” a movie. The movie might be a traditional movie used to illustrate a concept, or it might be a “screen capture” movie of a software application or web site. This tutorial will focus on the latter. While there are many ways to integrate a character with a movie, the technique outlined in this tutorial is simple, and allows you to maintain good synchronization. You can see the final result in the “Character Builder Projects\Tutorials\Capture” directory.

Preparing for import Since the output format for this project will be HTML, you will need an MP4 movie. Several third-party programs are available that can create movies directly from your screen as you demonstrate a software program or web site. One free tool is called CamStudio. Though the output is AVI, this can be converted to MP4 using other freely available tools, such as ffmpeg. The key thing to understand with this technique is that your original movie will not be played continuously, but rather in short bursts, under the precise control of the character. Think of the movie as a “prop” that remains under the character’s complete control. For this tutorial we used Adobe’s PDF reader to fill in an online form. We recorded the actions using CamStudio, without any audio, using the Intel IYUV codec, and then compressed it using ffmpeg settings ffmpeg -i w4.avi -r 24 w4.mp4. You can use the resulting movie file to experiment with – it is located in the My Documents\Character Builder Projects\Tutorials\Capture\Movies folder and is called W4.mp4.

Using the wizard To get started, choose File > New Project, enter a project name, and select the “Video Sync” configuration. You will be asked for an initial visual configuration, and then you will be asked for your Movie.

Select the W4.mp4 movie located in your “Character Builder Projects\Tutorials\Capture” directory. The movie will be copied to your Movies subfolder when the wizard completes. In order to assist with the synchronization between the character and the movie, you will then annotate the movie with information that will drive the script.

21

You are asked to break the movie into a series of steps. Each step corresponds to a time range that represents some logical action in the movie. In the W4 movie, for example, each step represents a box being filled in. For each step you are also asked to provide a step name and a paragraph of text that the character will speak during this step. The step name is for authoring convenience only, and is not normally shown to the end-user. You can the use the Add New button to add a new step to the end of the list, or the Insert New button to insert a new step before the selected step. If you start with the W4 movie then you may find the appropriate annotations already filled-in. When you annotate your movie, the data is saved in an "Annotations.txt" file located in the same folder as the original movie. If this file is present, then it is used during this wizard. Each step represents a time range within the movie. The ranges should not overlap, but there can be space between them. For example you might pick up an action from 4.0 to 6.0 seconds, then another from 10.0 to 12.0 seconds.

How it works A Sequence is created out of the steps specified in the annotations. If you look at the resulting Step objects in Outline View, you will see that they use the “Sync Movie with Step” feature, so that the movie will snap to the indicated time whenever that step begins. This means that you can use the navigation bar to quickly move to a certain step in the movie, then press Play to begin presenting at that point.

Looking at the Message generated for each step, you will see that the each step begins with a Play Until action. The Play Until action is just like the Play action, except that it automatically stop at the appointed time. Importantly, the character continues to speak while the movie is playing. You can also use a second movie action, Watch Until, to have the character pause until a certain point is reached in the movie. You can even start the movie, talk for a bit, watch for a bit, then talk some more. The Watch Until command has no effect on the movie: it simply controls the character. For naturalness, be sure to look in the direction of the action before doing a Watch Until.

22

Selecting the Movie action we see:

In typical screen capture movies there is a lot of talking for every bit of action. But if there is a chance that the action might run on after the character is done talking, then it is a good idea to precede the Next command with a Watch Until with the same value as the Play Until. Without this the character might move to the next step before the time index specified in the Play Until. When creating screen capture movies for use with the Character Builder, you can just focus on performing all of the actions, leaving just enough of a pause between them so that you can easily find the breaks. Most screen capture programs let you pause the recording with a keystroke, and this can be useful to catch your breath and think about the next action to record. The project created by the New Project wizard is a good start, but you will still want to review the generated actions. For example you will want to make the character look at where the action is taking place. Also, there is no reason why you need to place the Play statement at the beginning: you might wait until the character makes an opening statement, for example. You can preview each step individually to make sure that the character is looking at the right place, and that the timing is right.

23

Quiz Tutorial

In this tutorial we’ll introduce the Character Builder’s interaction capabilities by building a simple quiz, hosted by a cartoon character.

Building a multiple-choice slide Start by creating a new project, using “Empty Project”. We called ours “Quiz” – you can see the final result in the “Tutorials/Quiz” subdirectory of the Character Builder Projects directory. Switching to Design View, we added a character (Kim), and a slideshow. There are no other props – the slideshow simply lets us swap-in different supporting visuals beside the character.

Next, switch back to Outline View and use the Add button to create a slide. Normally we recommend using the wizard to select one of the Quiz slides, but for this tutorial we’ll start from first principles and use an Empty Slide. Create the following layout, using the Text object and the Button object. To choose the Button object, use the Library pane to navigate to Nav Buttons > Misc Quiz Buttons, and select “Radio Button”. Here is the question we created:

Prompts and reactions When you have completed the slide, switch back to Outline View, select the slide, and use the Add Message button ( ) to create a new message within that slide. Double-click on the message to bring it up in Script View. An interaction normally consists of a “prompt” message and two or more “react” messages. An example of a prompt would be to ask the user a question. After the prompt is played, the character goes idle waiting for the user to respond. Each time the user presses a button a “reaction” message is played.

24

Here is the Prompt script we used.

We chose to create multiple reactions – one for each button. Use the same steps that you used to create your prompting message to create three more messages. This time, be sure to give them names, such as “React A”, “React B”, and “React C”.

Now return to Design View, select each button, and set the Action to “Goto and Start” the appropriate React message:

We also chose to take advantage of automatic score tracking by indicating whether each button represents a correct or incorrect answer. Finally, switch to Script View. For the first, and correct answer, we used the following reaction script.

A nice effect is to first make the character look at what the user selected, and then react, even though she technically already “knows” what the user has selected. Similarly, we have for B:

And for C:

25

You need to use the “Render to Flash” and “Run In Browser” buttons to experience the proper flow of your interaction. On a correct response, the character acknowledges the response and moves to the next question. On an incorrect response there is no Next action, so the character goes idle waiting for the user to try again. That covers the first example. In the next example we’ll see a slightly different exercise, and a slightly different approach to reacting to the user’s input.

A second interaction For our next interaction we’ll use the Checkbox buttons, also from the “Misc Quiz Buttons” category of the library. We’ll add some follow-up information at the bottom of the slide – an image and a text object. You can copy the file “Dodecahedron.png” from the Tutorials\Quiz directory to your own project directory. These follow-up objects should have the “Starts Hidden” box checked. Here is how our slide looks:

The image and text annotation at the bottom are marked “Starts Hidden”. Next, switch to Outline View and add a Prompt message:

Next we’ll add two reactions as we did in the first slide. For True:

Here we use the Clear Chosen and Set Chosen actions to choose the correct answer for the user. For False:

26

We want to reinforce the correct answer with our image in both cases, so we'll make both reaction messages follow on with a Teach message, using the Goto action.

Here we use two successive Show actions to show both the image and the text caption.

Keeping score To complete our project we chose to add a Score slide, using the Add Content wizard’s “Quiz score” slide. The slide contains three “Field” objects – a field is a text object whose value can change at runtime.

When you select a field, you specify an expression to display – in this case we are using the built-in functions getTotalScore(), getTotalCompleted(), and getTotalQuestions(). These function work by looking at the scoring information in the buttons.

Switching to Script View you will notice that the Add Content wizard added not one, but four alternative messages that can be used in presenting the results. All except the last message defines a Condition. For example the Score 3 message says:

Looking at the Message panel for Score 3 you will see:

When the focus arrives at a scene or slide in which several messages are available to choose from, then the first message whose condition is satisfied will be chosen.

27

Dialog Tutorial You might want your slideshow to have two presenters rather than one. Many customer-service training simulations work by showing how a “simulated” subject reacts to the user’s choices, while a third character, or “coach”, comments on these choices. In either case you will want to create a dialog between two or more characters. To see how this works, create a new project using File > New Project with the “Dual Presenter”. You will be asked for two characters – we chose Tasha and Tom in the “Character Builder Projects\Tutorials\Dialog” project.

In Outline View you should see something like this:

Slide 1 contains a special kind of script called a Dialog. You can change between a regular message and a dialog using the Type dropdown on the message object:

When editing a dialog in Script view, a column to the left of the transcript area will show the character for which the line is intended, just like in a theatrical script.

You can change the character associated with a line by right-clicking on the name, or by double-clicking on the name to cycle through the characters in the scene.

28

Web Greeter Tutorial In this tutorial we’ll examine how to place a character on a web site and make it react to web page events. In this case, there will be no supporting visuals – the only object on the scene will be our character.

Adding a character to your web page To begin, create a new Character Builder project using the “Message with API”. We used Greeter as our project name and chose Tom (Head) as our character. The resulting files can be found in the “Character Builder Projects/Tutorials/Greeter” directory. To deploy your character, simply upload the files from the Output subdirectory to your web server. The Builder will create an HTML page that can be used to test the corresponding embedding. You can also cut and paste the tag from this sample HTML page to your own HTML page – please see the comments within the generated HTML file for details on how to do this.

The Initially Presenting checkbox (on the Project panel) indicates whether the character speaks the first message immediately, or whether it waits until it is triggered.

Triggering messages from links In the Slideshow tutorial we saw how the “focus” could be moved from one slide to another. In this case there is no slide to provide a visual feedback for the focus, but we can still move the focus to a Message. To illustrate this, we’ll give our first message the name “Opening”, and add a second message called “Products”, from the text “May I recommend the product page. Just click on products.”

Your Outline should like this:

Normally the character returns to the “idle” mode when a message completes. One way to control a multiple-message project is by using button objects in the scene. You would simply set the button action to “Goto and Start”, and select a target message. However in this tutorial we’ll show how to trigger a message from the surrounding web page. The “Message with API” template started us out with the following HTML Options settings (click the Options button on the Project panel).

29

A sample HTML file is created with the same name as the project name. This file has a block of HTML code that can be pasted into another HTML file: <script src="MSHTML5.min.js" type="text/javascript "></script> <script type="text/javascript"> msEmbed('Movie1','MyProject','MyProject_Files',250 ,200); </script>

You can think of this block as being equivalent to an image tag that places an image on your site. In addition to the sample HTML file there is a MSHTML5.min.js file and a MyProject_Files folder. Both should be uploaded to the same directory as your HTML file. Alternatively, if you use a subdirectory, you must adjust the script src and msEmbed arguments to include the relative path to that subdirectory. The script reference line loads MSHTML5.min.js, the JavaScript required to implement the HTML5 API, including the msEmbed() command. It should be included once on each page containing HTML5 Character Builder output. The msEmbed() function creates an HTML DIV with the id 'Movie1'. MSHTML5.min.js adds an API that lets you control the app programmatically. Setting the “Include sample control JavaScript” and “Include sample event JavaScript” options in the Options dialog on the Project page results in the following sample output. <!-- Character Builder API examples --> <a href="javascript:msPlay('Movie1','Message 1')"> msPlay('Message 1')</a><br>

The msPlay() API takes either the name or the programmatic id of a message, and causes it to begin playing. The programmatic id of a message is normally chosen automatically by the Builder, but can be seen in XML view. For a complete list of javascript API for Builder-generated Flash files, please see Help > API Reference.

Synchronizing events with your messages You can use the External Command action to raise an event to the containing web page. This allows you to control aspects of your page in a manner that is synchronized with your character message. To do so, be sure to check the “Include sample external event JavaScript” checkbox to have the Builder create sample event handling logic in the generated HTML file. Some sample event functions are created for you. For example: function onExternalCommand(id, cmd, args) { alert("External Command cmd=" + cmd + ", args=" + args); }

We added an “external command” action to the second message – when this action is run you should see the “alert” window of the onExternalCommand event handler.

30

Triggering URLs Characters can also use the Get URL action to trigger a page change in the surrounding browser. Your containing page might have a content frame, in which case the character can trigger page changes in the content frame using the Target field. Alternatively a character might offer to take the user to another page:

Having the character look in a consistent direction (e.g. up) before the link is taken is a nice effect.

Passing variables from HTML You can use a single embedding on multiple pages but start each instance off with a different variable, to provide some context. One way to do this is to use the Variables dialog on the Project Properties panel to add a variable. If you name your variable "Page", then you can set the initial value directly from HTML. Here we sit it to 1: msEmbed('Movie1','MyProject','MyProject_Files',250, 200,'Page=1');

You can set multiple variables by separating them with a semicolon.

Conditional messages You can create several messages that are each guarded by a condition expression. For example you might have a web greeter that greets users with one of two messages, one for use on Fridays and one for use every other day. To do so, simply create two messages:

Give the first message a condition “dayOfWeek() == 6”. (Sunday is considered the first day of the week.)

You might open the first message with “T.G.I.F!”. The second message has no condition, and contains the normal message. When the focus moves from the scene or slide down to a message, the first message whose condition expression evaluates to true will be selected to receive the focus. You can also use the random() function, which returns a random number between 0 and 1, to add a bit of randomness. For example you could give a first message the test “random() < 0.5” so that on average half of all views will get the first message and half will get the second. Finally, you can make messages that check the value of variables passed in from HTML.

Giving your character a transparent background In some cases you will want your character to “float” on top of the site, rather than occupy a fixed area.

31

The Builder’s Flash output actually has a transparent background by default, but the containing HTML page needs some special syntax in order for the transparency effect to be seen. The “Include sample transparent background syntax” checkbox on the HTML Options dialog (Options button on the Project panel), will result in an HTML file with the correct syntax:

The resulting sample page might look like this:

If you look at the HTML you will see that the Flash file is placed in an absolutely-positioned ‘div’ element. The generated HTML includes JavaScript code that allows the user to drag the character to a new location on the page. The character can also move automatically upon a scroll to remain in view. Comments in the HTML show how these features can be adjusted or disabled. The HTML generated for a floating character also contains an External Command event handler () that allows you to move the character on the HTML page under script control. To use this feature, you can use “move” as the command and the x, y coordinates as the argument.

32

HTML5 and Mobile Browser Tutorial The Character Builder can render apps to both the HTML5 format for use on mobile browsers. However you should be aware of one important difference between how HTML5 plays on the desktop and on mobile.

Mobile browsers and audio Mobile browsers, such as Safari on iPad and Chrome on Android, disallow "autoplay" for audio and video. This means that any audio or video play actions must be directly triggered by a touch event, such as the user pressing a Play button or a link. The reasoning behind this is that audio and video may incur a significant data charge, and should explicitly be authorized by the user, to avoid letting ads consume all of your data budget. In practice this means that, on mobile, the Initially Presenting option will not work as it does on the desktop, since there is no touch event to initiate it (the user's act of navigating to the containing page does not count as a touch event, as the touch must directly start the audio). If Initially Presenting is selected, then a Play button is shown on mobile browsers, as in the following example:

Pressing the white Play button is a proper touch event, and will cause the project to begin playing just as it would automatically play on a regular desktop browser. You can also choose to turn Initially Presenting off, in which case the character can begin in idle mode on all browsers, even on mobile, since idle behavior requires no audio. You can then choose to trigger the first message using a button or a link. The other behavior difference occurs at the end of a slide in a slideshow: while on desktop the character can chain from one slide to another with the Next command, on mobile, the character appears to advance to the next slide and then go idle. In other words, a Goto and Play action is interpreted on mobile as Goto and Stop, if it is triggered by the character, and not by a user touch event. If you have a navigation bar, then your mobile user may soon realize that they can get the character to restart by pressing Play. But if you are serious about mobile, then you might avoid having the character invoke Next at the end of a slide altogether, and simply have it end each slide with a statement such as "Press Next to continue," so that the user can trigger the navigation (and ensuing audio), with a real touch event. You can simulate mobile browser behavior on a desktop browser using the Project panel's "Allow Autoplay" setting, on the Advanced tab. Setting "Allow Autoplay" to "Never" causes a desktop browser to act like a mobile browser. The normal setting is "Detect", which will use autoplay on desktop browsers only. A third setting, "Never", is useful for generating content that plays inside a mobile app's web view, which is not subject to autoplay limitations.

33

HTML5 Animation Tutorial The HTML5 Animation is aimed primarily at JavaScript game developers. Projects must consist of a single scene with a single character, an optional background image or rectangle, and one or more messages. The output contains a "sprite sheet" or "texture map". This is an image that contains all the image "pieces" needed to recreate all of the messages in the project.

The output folder also contains audio files for each message and a JavaScript file that implements an API. The API is implemented using PixiJS, a popular library for HTML5 web games. To use the API, include the PixiJS library and the generated script. If your project is named "Test" then the API is declared in a Test.js file.

<script src="Test.js"></script>

Create a div with the same size as your project, and with the id "divTest": <div id="divTest" style="width:250px; height:200px; "></div>

When the page is loaded, the opening scene is loaded into the div. You can then trigger messages using the msPlayTest() API, assuming your project was named "Test". You can also use msStopTest() to smooth-stop your message and return to the initial state.

<a href='javascript:msPlayTest("Msg1")'>msPlayTest( "Msg1")</a><br/> <a href='javascript:msStopTest()'>msStopTest()</a>

There are three events that you can react to: <script> function onTestSceneLoaded() { console.log("onTestSceneLoaded"); // can kick off animation msPlayTest("Msg1"); } function onTestAnimationComplete() { // can chain on another animation, start an idle t imer, etc. console.log("onTestAnimationComplete"); } function onTestExternalCommand() { // can use to drive other changes in sync with ani mation console.log("onTestExternalCommand"); } </script>

With HTML5 Animation, the responsibility for maintaining "idle" behavior is squarely on the JavaScript developer. You can create short messages (e.g. "Blink") that can be played at random intervals, for example. Any supporting content is implemented in HTML. You can use the External Command action to embed actions inside your messages, and the onTestExternalCommand() function to drive changes in your HTML page. HTML5 Animation can also be generated dynamically, using the Character API service.

34

Video Tutorial This tutorial describes the differences between App and Video output. As an exercise, we suggest changing the Slideshow tutorial (Character Builder Projects\Tutorials\Slideshow) to render to Video rather than to Flash.

Apps versus animation/video The default output of the Character Builder is a Flash or HTML5 “App” in which the user has the sense of interacting with a live character, and not a movie. For example when you press the Stop button, the character smoothly interrupts its message and goes “idle”. When you restart the character, it goes back and resumes from a logical point. The user can interact with the material with buttons or other controls, and the character will react. When rendering to video, the result is a linear movie. If you pause the movie then the character stops in mid-action, and when you resume, the character continues precisely where it left off. The difference between Flash App output and Video output is more than just about the file format – it is also about the psychology of the user-experience. A user will tend to treat a video clip as a media object, whereas a character that reacts to the user and displays idle behavior is more likely to be treated as a social entity. The “Flash Animation” setting is similar to the “Video” setting – it results in a linear sequence of animation frames, with no smooth recovery or idle behavior. Flash Animation output is useful when integrating the Character Builder output into other software that supports Flash embeddings, such as Adobe Captivate. The simple linear nature of the generated file normally allows the containing application to integrate the animation within its own timeline. There are some limitations when rendering to video. Some of these limitations are structural (for example it is meaningless to have a button in your video output), and some of them have to do with the current implementation (for example embedded movies and sound clips are not currently supported).

Rendering to video To render to video, use Outline View to select the Project node, and set the Output Type selector to “Video”. You can then choose from several popular formats.

With the Video setting, your toolbar changes to include a Render to Video button. Before you render, you should review your project to remove any interactive elements. For example you should remove any navigation bar and other buttons. Your project should play in a continuous manner from beginning to finish, by way of the Next action. The resulting video file is placed in your Output subdirectory, and is ready for integration into movie-editing software, or for upload to a movie service such as YouTube. The Run in Browser button becomes Run in Player, and will open the default video player for each format.

35

Note that regardless of the output format, the Preview pane will use the Flash Animation format, so small differences in output quality may be apparent between the Preview and the final product.

Output preferences You can also use the Video tab of the Edit Preferences dialog (Edit > Preferences) to further adjust the output quality.

The first section is used when you render to AVI. AVI is actually a video container standard, and you can select from a number of different Windows codecs. AVI is not commonly used on the web, but it is a common interchange format for raw, uncompressed video. Since the resulting AVI files are not compressed at all, they tend to be very large. There is a 2GB maximum file size for AVI files, and this limit is actually quite easy to hit in practice. The uncompressed AVI format is also the only format that supports an alpha channel. If you render a character to uncompressed AVI with alpha, then you can import the resulting file into a video editing tool such as Adobe Premiere, where you can place it in front of a still image or another piece of video. You will then get a perfect mat, with no halo or fringing, and without the use of green-screen removal techniques. When rendering to all non-AVI formats, the Builder renders to an internal MPEG-based format first, and then transcodes to the final output file using the FFMPEG tool. The default settings should be adequate for most uses, however it is possible to alter the parameters used in the transcoding step to change the size and quality of the resulting output. When rendering to a video service such as YouTube, we recommend the MP4 video format, and an output size of 640x360 (for 16:9 ratio) or 480x360 (for a 4:3 ratio).

36

Guided Discovery Tutorial The most powerful way to communicate a concept is to have your users discover it for themselves by manipulating a model. The role of the character, in this case, can be to guide the discovery, suggesting actions to take, or reinforcing the learning that has taken place. To do this effectively, the Character Builder introduces features to take occasional initiative, and to keep track of what the user has seen and done to provide appropriate comments and to avoid being repetitive. These features are only available with the HTML5 Output setting.

Objects The Character Builder contains primitive user interface elements such as Buttons, Edit Controls, and Text Fields. Using the Extensibility Kit, it is also possible to create more complex objects that combine several primitives. An example of this is the Navigation Bar. Using JavaScript, it is also possible to create completely new objects of arbitrary complexity. An example of this is the Graph object. Objects can be dragged into Design View from the Library Pane. Objects can expose properties that change their appearance or behavior. Properties are set using the Advanced tab of the property panel.

For this tutorial, we are using a Graph object, found under Objects > Teaching Aids, in the Library pane. We give it the function: Amplitude * sin(Coefficient * x + Horizontal) + Ve rtical

We then call out 4 constants and give them initial values.

37

Amplitude=1,Coefficient=1,Horizontal=0,Vertical=0

These are constants as far as the equation is concerned, but if they are declared as variables, then the actual values used for the constants will take on the value of the variable. We declare variables as follows:

Now whenever we change the value of a variable, the equation for the graph will change, and the graph will be redrawn. You could use this in a slideshow presentation that describes the effect of each constant on the equation, using the Set Variable action. For example you could create a line such as:

But in this case we are interested in letting the user control the constants, and we do this with an Up/Down Selector. This object is similar to an Edit Control, except that you alter the value by clicking on the up or down buttons. Each selector is bound to one of the variables. For example for Amplitude we have, on the Advanced tab:

Now when we run the project, we are able to adjust any of the 4 constants using the Up/Down Selectors. Since the selectors are bound to variables, they serve both to display the current value of the variable, and to let you change it. Since the Graph object is also bound to those same variables, each time a variable is changed, the graph is instantly updated.

Conditions and Auto Start There are two key features that allow us to react intelligently to the user's actions: Conditions and Auto Start. When Presenting is True and the focus begins at (or moves to) a Step, Slide, or Scene, then the character looks in that Step, Slide, or Scene for a message to present. If there are multiple messages, then it looks through the messages in order and evaluates the Condition expression, if any. If there is no Condition, then the message is picked immediately, and searching stops. If there is a Condition, then the message is picked only if the expression evaluates to True, otherwise it continues searching through the list of messages. In the Web Greeter tutorial, this was the mechanism that allowed us to offer a different greeting on Friday, for example, with a "dayOfWeek() == 6" condition on the first message (for Friday), and no condition on the second message (for all other days).

38

If you select the current Scene, you will see some Auto Start parameters:

In this case we have place the Graph and Up/Down selectors in a single scene, so this is where the focus will start. With the above settings, we will automatically start presenting 1 second after any interaction, and also after 10 seconds of inactivity. It is then up to the Conditions on our messages to help pick an appropriate message. In previous tutorials we would often have only one message per Scene or Slide, and the last action of a message was often to have the character move the focus using a Next action (in the case of a slideshow) or a Goto And Start action (in the case of a multiple choice button). In this Tutorial the message ends with no action, so the Focus returns to the Scene and the character is no-longer Presenting (it goes idle). The next time it Auto Starts the message selection process repeats, and the condition of the first message is tested again, etc.

The seen() Function The Character Builder keeps track of when a message has been seen. You can test if a message has been seen using the seen() function. The Condition !seen("Message 1")

evaluate to True if the given message has not already been seen. When referring to itself, it is convenient to use the 'this' keyword: !seen(this)

A message is judged to be seen if the user has listened to at least half of it. A slide has been seen if it contains any messages that have been seen. Adding !seen(this) as the condition to any message will ensure that it only gets presented once.

Reacting to Inactivity We have set our scene to auto start 10 seconds after inactivity. This allows us to play a suggestion. But the scene was also set to auto-start 1 second after an interaction: how do we know what we are reacting to? The answer is through the functions navigated() , interacted() , and inactive() , which each return True only if we were auto-started by the corresponding event. You could create a message with the condition inactive() && !seen(this) to offer a one-time suggestion when the user appears to be stuck. You will find that if you don't also check for 'seen', the Scene will keep auto-starting so long as the user continues to be idle. So if you don't do anything else, the same inactive message will repeat every 10 seconds. As the user continues to be idle, the system keeps track of how many times it has auto-started, and this value is available through the inactiveCount() function. So another way of responding just once with the inactive message would be to use the condition inactive() && inactiveCount() == 1 . You might have a second message that triggers after a full minute of inactivity, with inactive() && inactiveCount() ==

6. We chose 6 in this case because we chose to restart after every 10 seconds of inactivity. (You might find it helpful to think of a boxing referee who counts up with each "cycle" of an "inactive" boxer.)

39

Reacting to Interaction Conditions can include tests on values of variables. For example you could write a message that reacts to interacted() && Amplitude == 2 && !seen(this) to react specifically when the user has doubled the amplitude, and then only once. The Builder also keeps track of which objects were touched. We can use this to prompt the user with a suggested action. For example we can suggest that they vary the Amplitude with the condition: inactive() && inactiveCount() == 1 && !touched("Am plitude")

The suggestion only occurs if they are idle and haven't already touched the Amplitude control. When the user does interact with the amplitude, we can react using the condition: interacted() && lastTouched() == "Amplitude" && !s een(this)

You can examine the Character Builder Projects\Tutorials\Graph tutorial to see how we use these ideas to make our character prompt the user to try the different controls, and then explain some key concepts once they do.

40

PhotoFit Character Tutorial The Character Builder offers two different character styles that work with photos. The PhotoFit style lets you extract the face from a photo and map it onto a stock body. You can then choose matching hair and clothing, and the resulting character has a full range of motion. The PhotoMorph style derives the entire appearance from the photo, including the hair and clothing. It creates limited head-turn animation from the photo itself.

Introducing PhotoFit When a PhotoFit-capable character is selected, the character’s Customize tab will contain an Import Photo and an Edit Face button.

Let’s look at this second button first. The Edit Face button brings up the Edit Face window.

The Edit Face window lets you edit the face of a character only – all other aspects, such as hair, clothing, and glasses, are altered in the Customize tab. The stock characters come with only a few options for these parameters – other options are available through Hair Pack and Clothing Pack addons. When you edit a face using the Edit Face window you will find that a small “FG” file is added to your project directory. This file contains the parameters of the face, and is used by the software to generate previews and renders. The resulting imagery is created “just-in-time” from the model, and then cached by the Character Builder. You may find the first few renders of a PhotoFit character to be a little slower, but once the cache is populated the render time approaches that of a realistic character.

41

Shape and texture controls The appearance of a face is controlled by two types of parameters. Shape parameters control the shape of the face, such as the strength of the jaw or the shape of the nose. The shape parameters are further divided into symmetric and asymmetric parameters. While all faces have some degree of asymmetry, we tend to judge symmetric features as being more attractive. Texture parameters control the detail of the skin, such as the skin coloring, makeup, or facial hair. You can control all Shape parameters from the Shape tab and all Texture parameters from the Texture tab. While the Shape and Texture tabs provide complete control over the face, in practice you may find it easier to work with the General tab. The General tab lets you control “meta-parameters”, such as Gender, Age, and Race. These broad parameters tend to affect all of the parameters on the Shape and Texture tabs at once. For example changing the gender slider from more “male” to more “female” will have several effects at the level of the head shape, eyes size, cheekbones, etc. You can control both the shape and texture independently. For example, in addition to shape difference, males generally do tend to have darker skin than females. The Caricature sliders control the degree to which differences from the average are amplified. The Asymmetry slider is similar to Caricature, but operates only on the asymmetric shape parameters. Finally, the Race sliders are used to select the ethnic origins of a face. For example a “pure” African character would be high on the African and low on the European, South East Asian, and East Indian parameters. The General tab also has a Random Face button for generating random faces. When generating random faces, you can choose to “lock” any of the other broad parameters, so that only the unlocked parameters will vary.

Obtaining a face from a photo The Import Photo button provides an alternative way to set the Shape and Texture parameters by analyzing an actual photograph. The first step in creating a character from a photo is to locate a photograph with the following properties. Essential:

• Use a color photo with good resolution • Have the subject look straight at the camera, or slightly above it • Pull hair away from face • Mouth should be relaxed and closed - no smile.

Recommended:

• Hair should not cover the forehead or any part of the face. • The camera should be at least 5 feet away from the face. • The face should be at least 500 pixels high in the image. • In the frontal shot, the chin should be slightly elevated as if looking just above the camera. • Focus your eyes on something just above the camera, rather than into the distance, to avoid a

spacey look. OK:

• Facial hair is OK. • Flashes on both sides of the camera can be used. • Flash diffusers can be used.

42

Bad:

• The face should not be painted (makeup is OK). • Avoid taking photos outdoors in daylight. • Drawings of faces may not yield good results.

For a quick test you can try the photo in your “Documents\Character Builder Projects\ Tutorials\PhotoFit” folder. In the second step you adjust the position of several key points to specific locations on the face, as shown in the guide image.

Hints:

• Click and drag a point to move it. • The eye points should be in the centre of the eyes. • The outer cheekbone points should be inside of any sideburn hair. • The nose flap points should be on the outermost edges of the nose flaps. • The mouth corner points should be on the outermost edges of the mouth. • The outer jaw points should be horizontally in line with the mouth corner points. • The chin point should be at the bottom centre of the chin.

In the third step you let Builder analyze the photo. Press Start to begin the analysis. You will be able to see the progress of the analysis in a separate window. The face will start out small and look nothing like your photo, but over time it will get larger and look more like your photo. The analysis step takes advantage of any graphics acceleration hardware that you may have installed. With a good graphics card (such as found in a gaming or Vista-capable system), the analysis will take approximately three minutes. With no graphics acceleration it can take much longer. Once the analysis is complete, the preview should update as the sliders take on the specific values that represent the parameters of the face. You may want to alter the face further by adjusting the sliders. For example you might adjust the age or asymmetry level of the face, or alter the strength of the texture detail, at the bottom of the Texture tab.

43

PhotoMorph Character Tutorial The PhotoMorph style derives the entire appearance from the photo, including the hair and clothing. It creates limited head-turn animation from the photo itself, but it there are no hand actions and only limited head-turning action.

Introducing PhotoMorph To create a PhotoMorph character from a photo, begin with any PhotoMorph character, such as PhotoMorph Man or PhotoMorph Woman. Select the character in Design view and click the Customize tab in the property pane below.

Click the Import Photo button. You will need to locate a photograph in JPEG format, with the following properties.

• The photo guidelines are similar to those for PhotoFit, namely: • Use a color photo with good resolution • Have the subject look straight at the camera, or slightly above it • Pull hair away from face • Remove glasses • Mouth should be relaxed and closed - no smile.

In addition, you will typically want to remove the background. There are two reasons for this. First, by removing the background now, you will be able to place your character over a different background later in the Builder. Second, if you don't remove the background, and the background is anything other than a solid color, then some distortion will be noticeable behind the character when it moves its head. There are three approaches you can take to removing the background:

• Perform the background removal beforehand using image-editing software and provide a 32 bit (24 bit with alpha) PNG file as the source for Import Photo.

• Photograph the subject on a green screen or edit the photo with an image-editing tool to fill the background with pure green (#00FF00), then use the built-in green-screen removal feature (more on this shortly).

• Perform the Import without worrying about the background, then perform background removal, if desired, on the resulting PNG image.

In all cases your job will be simplified by selecting a photo that has as little background detail as possible, especially around the face and hair. Image-editing tools such as Adobe PhotoShop include a “smart select” tool that works well for removing the background, particularly if you begin with a photo on a white or uniformly-colored background. For a quick test you can try the photo in your “Documents\Character Builder Projects\Tutorials\PhotoMorph” folder.

44

Press Load to load your photo, then press Next. In the second step you adjust the position of several key points to specific locations on the face, as shown in the guide image. When the points are placed, press Next to proceed to the third step.

It is important to provide a high-resolution original image for analysis, however the final character will typically be displayed at a fraction of the original size. In Step 3 you use the Scale and Crop controls to scale down and/or crop your image so that the face appears in the center of the image. The size shown will be the actual size of your character – this can be any size, and it need not match the original size of the PhotoMorph character. If your photo has a “green screen” background, then you can check the “Remove background” checkbox - all green portions of the image should get replaced with a checkerboard pattern, which indicates that these portions of the image are now transparent.

45

Press Next, then press Start on the final step – analysis. When complete, the character should be similar to the original photo. Conceptually, an animatable face model, defined by an FG file, is drawn with “feathered” edges over an underlying image, defined by the PNG file. Both the FG file and the PNG file are created by Import Photo, and placed in your project directory. Take a moment to review the properties in the Customize panel. These properties, along with the FG and PNG files, represent all the information required to recreate your character - they are what makes Joe look like Joe and not like Evan or Emily. The analysis step does its best to automatically match the face model to the underlying face, but the match is often not perfect, and it will take some tweaking to yield a better match. The following techniques generally produce good results:

• Locate a parameter called Alpha Multiplier. By setting it to the value 0.5 you make the face model semi-transparent. This will allow you to see both the underlying image and the face model, and your eye will quickly make out differences in alignment. In the case of the Joe.jpg image, the face was slightly rotated round the Y and Z axes. (The Z axis is an imaginary line emerging at right angles from the screen). Note that only the “Adjust” parameters should be modified - the Face Scale, Pixels Per Unit, Cheek, Eye, Nose, and Chin parameters are all generated for you by Import Photo and should rarely be altered. One occasional exception to this rule is the Face Scale parameter, which can be underestimated by the analysis step when the marker points are not placed at the very edge of the face. If you change the scale at all then the change should be very slight. Return the Alpha Multiplier to 1.0 (or leave it blank) when you are done with this step.

• Often the lighting or coloring of the face model will not quite match the underlying image. You

will tend to notice this most in the forehead area. You can use the Gamma Adjust and Light Adust parameters to alter the color tone and brightness of the face model. In some cases you will need to alter the color of the face model more drastically. To do this, you can use the Edit Face button and click on the Texture tab. Scrolling down you will find sliders that adjust the color of the face texture. While in the Texture tab, you may want to alter some other color values – for Joe we increased the brightness of the eyewhites, as this detail was not picked up well by Photo Import.

• Though normally not required, you can also adjust the bounds for the feathering. You can see the

feathering in action most easily by temporarily setting Light Adjust to a large negative value such as –10, which essentially makes the model black. You can use the Feather Adjust parameters to increase the top, bottom, or sides of the feathering shape.

• We already discussed how to use the Texture tab on the Edit Face dialog to adjust the face texture.

For subjects with a large chin or a long face, you may also want to use the Shape tab to manually adjust the shape of the face, as this is not always picked up correctly from the photo. The General tab also has an Asymmetry slider - Import Face occasionally exagerates the asymmetry of a face.

• The Twist and Pitch adjustment parameters are used to derive the “morph” animations from the underlying PNG image file. Their effect can only be seen by having the character twist and pitch its head, by looking left, right, up, and down. These values will typically need no further adjustment.

• Though you generally want your subject's hair to be pulled back from the face, it is possible to

sandwich the face model between the generated “Morph” image and a second “Foreground” image. If used, the Foreground image must be the same size as the Morph image, and should be mostly transparent, except for areas that cover the face, such as a wisp of hair that covers the eyes.

Here is the result before and after these adjustments:

46

When you are happy with your adjustments, you can use the Save As button on the Customize tab to save the parameters as a new character, e.g. “Joe”. The Joe character will then appear along with your other characters, for use in future projects.