Microsoft Robotics Studio Simulation Kyle Johns Software Development Engineer Microsoft Corporation.

Microsoft Robotics Microsoft Robotics StudioStudioSimulationSimulationKyle JohnsKyle JohnsSoftware Development EngineerSoftware Development EngineerMicrosoft CorporationMicrosoft Corporation

Presentation OverviewPresentation Overview

Why a simulator?Why a simulator?

Simulator Simulator architecturearchitecture

Using the simulatorUsing the simulator

Entities and Entities and servicesservices

Creating simulator Creating simulator scenesscenes

Articulated armsArticulated arms

Why a Simulator?Why a Simulator?

Robotics hardware Robotics hardware is expensiveis expensive

Hardware can be Hardware can be difficult to debugdifficult to debug

Hard for a team to Hard for a team to work concurrently work concurrently with limited with limited hardwarehardware

Why a Simulator?Why a Simulator?BenefitsBenefits

Low barrier to Low barrier to entryentry

Staged approachStaged approach

Easy prototypingEasy prototyping

Useful for Useful for educationeducation

Good learning and Good learning and research toolresearch tool

Why a Simulator?Why a Simulator?LimitationsLimitations

Lack of noisy dataLack of noisy data

Incomplete or Incomplete or inaccurate modelsinaccurate models

Accurate tuning Accurate tuning takes timetakes time

Simulator ArchitectureSimulator ArchitectureThe Simulator Engine ServiceThe Simulator Engine Service

Implemented as a Implemented as a serviceservice

Maintains world stateMaintains world state

Manages input devicesManages input devices

3D rendering using 3D rendering using XNAXNA

Ageia Physics Ageia Physics SimulationSimulation

Graphical User Graphical User InterfaceInterface

Editor for modeling Editor for modeling and debuggingand debugging

SimulationEngineService

XNAGraphicsLibrary

Display Hardware

AgeiaPhysicsEngine

User Interface / Editor

Simulator ArchitectureSimulator ArchitectureEntitiesEntities

Entities are objects Entities are objects in the simulated in the simulated worldworld

Visual componentVisual component

Physics shapesPhysics shapes

Represent:Represent:CamerasCameras

Sky and GroundSky and Ground

Robot componentsRobot componentsMotors, sensorsMotors, sensors

Buildings & Buildings & FurnitureFurniture

Anything visible or Anything visible or physicalphysical


XNAGraphicsLibrary

Display Hardware

AgeiaPhysicsEngine


Sky

Ground

Robot

Camera

Simulator ArchitectureSimulator ArchitectureServices associated with entitiesServices associated with entities

Services link with Services link with some entitiessome entities

Drive motorsDrive motors

Gather sensor dataGather sensor data

Modify the worldModify the world


XNAGraphicsLibrary

Display Hardware

AgeiaPhysicsEngine


Sky

Ground

Robot

Camera

Drive Service

Webcam Service

Simulator ArchitectureSimulator ArchitectureOrchestration servicesOrchestration services

Provides the Provides the “intelligence”“intelligence”

Interprets sensor Interprets sensor datadata

Commands motorsCommands motors

Can directly control Can directly control the simulated worldthe simulated world


XNAGraphicsLibrary

Display Hardware

AgeiaPhysicsEngine


Sky

Ground

Robot

Camera Webcam Service

OrchestrationService

Drive Service

Starting the SimulatorStarting the Simulator

From the Start MenuFrom the Start Menu

From the Control PanelFrom the Control Panel

From a manifestFrom a manifest

From another serviceFrom another service

Starting the SimulatorStarting the SimulatorFrom a manifestFrom a manifest

<?xml version="1.0" ?><Manifest xmlns="http://schemas.microsoft.com/xw/2004/10/manifest.html" xmlns:dssp="http://schemas.microsoft.com/xw/2004/10/dssp.html" xmlns:simcommon="http://schemas.microsoft.com/robotics/2006/04/simulation.html" > <CreateServiceList>  <ServiceRecordType> <dssp:Contract>http://schemas.microsoft.com/robotics/2006/04/simulationengine.html </dssp:Contract> <dssp:PartnerList> <dssp:Partner> <dssp:Service>LEGO.NXT.Tribot.SimulationEngineState.xml</dssp:Service> <dssp:Name>dssp:StateService</dssp:Name> </dssp:Partner> </dssp:PartnerList> </ServiceRecordType> </CreateServiceList></Manifest>

Starting the SimulatorStarting the SimulatorFrom another serviceFrom another service

[DisplayName("Simulation Tutorial 2")] [Description("Simulation Tutorial 2 Service")] [Contract(Contract.Identifier)] public class SimulationTutorial2 : DsspServiceBase { State _state = new State();

// partner attribute will cause simulation engine service to start [Partner("Engine", Contract = engineproxy.Contract.Identifier, CreationPolicy = PartnerCreationPolicy.UseExistingOrCreate)] private engineproxy.SimulationEnginePort _engineServicePort = new engineproxy.SimulationEnginePort();

Basic Simulator Basic Simulator OperationOperation

NavigationNavigationRendering modesRendering modesMenu optionsMenu optionsThe simulator editorThe simulator editor

Creating a MeshCreating a Mesh

Use a 3Use a 3rdrd party party modeling tool such modeling tool such as Maya, 3D Studio as Maya, 3D Studio MaxMax

Must export Must export objects to Alias .obj objects to Alias .obj formatformat

Entities Represent Entities Represent HardwareHardware

Entities usually Entities usually represent motor or represent motor or sensor hardwaresensor hardware

Services can Services can interact with interact with simulation entities simulation entities or real-world or real-world hardwarehardware

Only a manifest Only a manifest change requiredchange required

Lego NXT Entity

Differential Drive Service Lego NXT Drive Service

Orchestration Service

Editing EntitiesEditing Entities

CamerasCamerasSky and groundSky and groundEditing shapesEditing shapesSpecifying meshes and texturesSpecifying meshes and texturesSpecial entity typesSpecial entity types

Using Services with Using Services with EntitiesEntities

Each entity can specify a service Each entity can specify a service contractcontract

Saving a sceneSaving a sceneSaves simulator state and all entity stateSaves simulator state and all entity state

Saves a manifest which will start Saves a manifest which will start servicesservices

Manifest links a service to an entity Manifest links a service to an entity by specifying a partner which is the by specifying a partner which is the entity nameentity name

Using Services with Using Services with EntitiesEntitiesEntity name specified as partnerEntity name specified as partner

 <ServiceRecordType> <dssp:Contract> http://schemas.microsoft.com/robotics/simulation/services/2006/05/simulateddifferentialdrive.html </dssp:Contract> <dssp:PartnerList> <dssp:Partner>  <dssp:Service>http://localhost/LegoNXTMotorBase</dssp:Service> <dssp:Name>simcommon:Entity</dssp:Name> </dssp:Partner> </dssp:PartnerList> </ServiceRecordType>

Using Services with Using Services with EntitiesEntities

Hierarchical entitiesHierarchical entitiesSpecifying a service for an entitySpecifying a service for an entityDifference between state and Difference between state and manifestmanifestUsing the Simple Dashboard Using the Simple Dashboard serviceserviceThe DifferentialDrive serviceThe DifferentialDrive serviceThe LaserRangefinder serviceThe LaserRangefinder serviceThe SimulatedWebcam serviceThe SimulatedWebcam service

Making a SceneMaking a Scene

Add entities to the simulation Add entities to the simulation environmentenvironment

Using the simulation editorUsing the simulation editorSimulation Tutorial 6Simulation Tutorial 6

Editing simulator stateEditing simulator stateSimulation Tutorial 3Simulation Tutorial 3

Using a manifestUsing a manifestLego NXT Tribot simulation manifestLego NXT Tribot simulation manifest

From a serviceFrom a serviceSimulation Tutorials 1,2,4,5Simulation Tutorials 1,2,4,5

Making a SceneMaking a SceneAdding new types of entitiesAdding new types of entities

Creating a completely new type of Creating a completely new type of entity (such as a legged robot) entity (such as a legged robot) requires defining a new entity classrequires defining a new entity class

The entities.cs file in the SDK shows The entities.cs file in the SDK shows examples of how entity classes are examples of how entity classes are defineddefined

This topic is outside the scope of this This topic is outside the scope of this presentationpresentation

Making a SceneMaking a SceneAdding entities with the editorAdding entities with the editor

Turn on the simulation editor (F5)Turn on the simulation editor (F5)

Select Entities->New Entity…Select Entities->New Entity…Select desired assemblySelect desired assembly

Select entitySelect entity

Select parent entity if desiredSelect parent entity if desired

Specify constructor parametersSpecify constructor parameters

Optionally edit stateOptionally edit state

Making a SceneMaking a SceneCopy entity from one scene to Copy entity from one scene to anotheranother

Load the source sceneLoad the source scene

Turn on the simulation editor (F5)Turn on the simulation editor (F5)

Check the desired entity or entitiesCheck the desired entity or entities

Select Entities->Save Entities…Select Entities->Save Entities…

Load the destination sceneLoad the destination scene

Select Entities->Load Entities…Select Entities->Load Entities…

Making a SceneMaking a SceneAdding entities by editing simulator Adding entities by editing simulator statestate

Save the scene to an XML fileSave the scene to an XML file

Use a text editor to open the fileUse a text editor to open the file

Copy and paste the desired entityCopy and paste the desired entity

Modify state as necessaryModify state as necessary

Save the fileSave the file

Re-load the scene from the modified Re-load the scene from the modified filefile

Can also copy entities between scene Can also copy entities between scene files in a similar wayfiles in a similar way

Making a SceneMaking a SceneLoading simulator state from a Loading simulator state from a manifestmanifest

<?xml version="1.0" ?><Manifest xmlns="http://schemas.microsoft.com/xw/2004/10/manifest.html" xmlns:dssp="http://schemas.microsoft.com/xw/2004/10/dssp.html" xmlns:simcommon="http://schemas.microsoft.com/robotics/2006/04/simulation.html" > <CreateServiceList>  <ServiceRecordType> <dssp:Contract>http://schemas.microsoft.com/robotics/2006/04/simulationengine.html </dssp:Contract> <dssp:PartnerList> <dssp:Partner> <dssp:Service>LEGO.NXT.Tribot.SimulationEngineState.xml</dssp:Service> <dssp:Name>dssp:StateService</dssp:Name> </dssp:Partner> </dssp:PartnerList> </ServiceRecordType> </CreateServiceList></Manifest>

Making a SceneMaking a Scene

Using the simulation editorUsing the simulation editorCopying an entity between scenesCopying an entity between scenesEditing simulator stateEditing simulator stateLoading simulator state from a Loading simulator state from a manifestmanifest

Making a SceneMaking a SceneAdding entities from a serviceAdding entities from a service

Service creates the scene Service creates the scene

Specifies simulation engine as Specifies simulation engine as partnerpartner

Creates entities and sets stateCreates entities and sets state

Inserts entities into simulatorInserts entities into simulator

Starts associated servicesStarts associated services

Simulation tutorial 2 is a good Simulation tutorial 2 is a good exampleexample

Adding Entities from a Adding Entities from a ServiceServiceRequired referencesRequired references

SimulatedBumper.Y2006.M05.Proxy SimulatedDifferentialDrive.2006.M06.Proxy SimulatedLRF.Y2006.M05.Proxy SimulatedWebcam.Y2006.M09.Proxy

using drive = Microsoft.Robotics.Services.Simulation.Drive.Proxy;using lrf = Microsoft.Robotics.Services.Simulation.Sensors.LaserRangeFinder.Proxy;using bumper = Microsoft.Robotics.Services.Simulations.Sensors.Bumper.Proxy; using simwebcam = Microsoft.Robotics.Services.Simulations.Sensors.SimulatedWebcam.Proxy;

Add a reference to the proxy for each Add a reference to the proxy for each service which will be used.service which will be used.

Adding Entities from a Adding Entities from a ServiceServiceSet the camera viewSet the camera view

private void SetupCamera() { // Set up initial view CameraView view = new CameraView(); view.EyePosition = new Vector3(2.491269f, 0.598689f, 1.046625f); view.LookAtPoint = new Vector3(1.873792f, 0.40983f, 0.2830455f); SimulationEngine.GlobalInstancePort.Update(view); }

Adding Entities from a Adding Entities from a ServiceServiceAdd a SkyAdd a Skyvoid AddSky()

{ // Add a sky using a static texture. We will use the sky texture // to do per pixel lighting on each simulation visual entity SkyEntity sky = new SkyEntity("sky.dds", "sky_diff.dds"); SimulationEngine.GlobalInstancePort.Insert(sky);

// Add a directional light to simulate the sun. LightEntity sun = new LightEntity(); sun.Name = "Sun"; sun.Type = LightEntityType.Directional; sun.Color = new Vector4(0.8f, 0.8f, 0.8f, 1); sun.Pose.Position = new Vector3(1.0f, 1.0f, -0.5f); SimulationEngine.GlobalInstancePort.Insert(sun);}

Adding Entities from a Adding Entities from a ServiceServiceAdd the GroundAdd the Groundvoid AddGround()

{ HeightFieldShapeProperties hf = new HeightFieldShapeProperties( "height field", 64, // number of rows 100, // distance in meters, between rows 64, // number of columns 100, // distance in meters, between columns 1, // scale factor to multiple height values -1000); // vertical extent of the height field.

// create array with height samples hf.HeightSamples = new HeightFieldSample[hf.RowCount * hf.ColumnCount]; for (int i = 0; i < hf.RowCount * hf.ColumnCount; i++) { hf.HeightSamples[i] = new HeightFieldSample(); hf.HeightSamples[i].Height = (short)(Math.Sin(i * 0.01)); }

Adding Entities from a Adding Entities from a ServiceServiceAdd the Ground (2)Add the Ground (2)

// create a material for the entire field. // We could also specify material per sample. hf.Material = new MaterialProperties("ground", 0.8f, 0.5f, 0.8f);

// insert ground entity in simulation and specify a texture SimulationEngine.GlobalInstancePort.Insert( new HeightFieldEntity(hf, "03RamieSc.dds"));}

Adding Entities from a Adding Entities from a ServiceServiceAdd a motor baseAdd a motor base

// use supplied entity that creates a motor base // with 2 active wheels and one casterPioneer3DX robotBaseEntity = new Pioneer3DX(position);

// specify mesh. robotBaseEntity.State.Assets.Mesh = "Pioneer3dx.bos";

// the name below must match manifestrobotBaseEntity.State.Name = "P3DXMotorBase";

// Start simulated motor servicedrive.Contract.CreateService(ConstructorPort, Microsoft.Robotics.Simulation.Partners.CreateEntityPartner( "http://localhost/" + robotBaseEntity.State.Name));

Entity source is provide in entities.csEntity source is provide in entities.cs

Adding Entities from a Adding Entities from a ServiceServiceAdd a laser rangefinderAdd a laser rangefinder

// Create a Laser Range Finder Entity .// Place it 30cm above base CenterofMass. LaserRangeFinderEntity laser = new LaserRangeFinderEntity( new Pose(new Vector3(0, 0.30f, 0)));

laser.State.Name = "P3DXLaserRangeFinder";laser.LaserBox.State.DiffuseColor = new Vector4(0.25f, 0.25f, 0.8f, 1.0f);

// Create LaserRangeFinder simulation service and specify// which entity it talks tolrf.Contract.CreateService(ConstructorPort, Microsoft.Robotics.Simulation.Partners.CreateEntityPartner( "http://localhost/" + laser.State.Name));

Adding Entities from a Adding Entities from a ServiceServiceInsert the new entitiesInsert the new entities

// insert laser as child to motor baserobotBaseEntity.InsertEntity(laser);

// Insert the motor base and its children into the simulationSimulationEngine.GlobalInstancePort.Insert(robotBaseEntity);

Child entities are inserted into their Child entities are inserted into their parentparent

The composite entity is inserted into The composite entity is inserted into the simulation enginethe simulation engine

The engine initializes the entitiesThe engine initializes the entities

Articulated ArmsArticulated Arms

Simulation Tutorial 4Simulation Tutorial 4

Joints and SegmentsJoints and Segments

Articulated arms Articulated arms are built from are built from entities (segments) entities (segments) attached by Jointsattached by Joints

Joints are physics Joints are physics objectsobjects

Up to 6 degrees of Up to 6 degrees of freedomfreedom

3 linear and 3 3 linear and 3 rotationalrotational

Each degree of Each degree of freedom can be freedom can be locked or freelocked or free

Joint PropertiesJoint Properties

A Joint object has the following A Joint object has the following properties:properties:

The degrees of freedom locked or freeThe degrees of freedom locked or free

The entities which are joinedThe entities which are joinedThe point on each entity where the joint The point on each entity where the joint attachesattaches

The axis of attachment for each entityThe axis of attachment for each entity

The joint normal (specifies joint The joint normal (specifies joint orientation)orientation)

An optional drive and spring mechanism An optional drive and spring mechanism for each unlocked degree of freedomfor each unlocked degree of freedom

Example JointExample Joint

One rotation degree of freedom One rotation degree of freedom unlockedunlocked

PhysicsJoint jointInstance = null;JointAngularProperties commonAngular = new JointAngularProperties();commonAngular.TwistMode = JointDOFMode.Free;

commonAngular.TwistDrive = new JointDriveProperties( JointDriveMode.Position, new SpringProperties(500000, 100000, 0), 1000000);

jointInstance = PhysicsJoint.Create( new JointProperties(commonAngular, null, null));

// joints must be namedjointInstance.State.Name = "Joint0";

An Articulated ArmAn Articulated Arm

Built from multiple segments and Built from multiple segments and jointsjoints

Each joint may have a different normal Each joint may have a different normal and axisand axis

Articulated Arm Articulated Arm InitializationInitialization

Segment entities must be initialized Segment entities must be initialized before joints can be initializedbefore joints can be initialized

Requires a 2-step processRequires a 2-step process

Good example in entities.csGood example in entities.csKukaLBR3EntityKukaLBR3Entity

Example Link InitializationExample Link Initialization commonJoint.State.Connectors[0] = new EntityJointConnector( null, new Vector3(0, 1, 0), // joint normal (+Y) new Vector3(0, 0, -1), // joint axis (-Z) new Vector3(0, ARM_LENGTH2, 0)); // entity 0 connection point commonJoint.State.Connectors[1] = new EntityJointConnector( null, new Vector3(0, 1, 0), // joint normal (+Y) new Vector3(0, 0, -1), // joint axis (-Z) new Vector3(0, 0, 0)); // entity 1 connection point ArmLinkEntity link = new ArmLinkEntity( "arm2", // the name of the link entity "lbr3_j1.obj", // the visual mesh for the link entity new Pose(new Vector3(0, MIDPOINT_Y, 0), // initial orientation Quaternion.RotationAxis(new Vector3(1, 0, 0), (float)Math.PI)));

The KUKA LBR3 ArmThe KUKA LBR3 Arm

Using Simple Dashboard to move Using Simple Dashboard to move jointsjoints

SummarySummary

Simulator architectureSimulator architecture

Basic Simulator OperationBasic Simulator Operation

Modifying and Creating EntitiesModifying and Creating Entities

Using Services with EntitiesUsing Services with Entities

Articulated Arm EntitiesArticulated Arm Entities

Microsoft Robotics Studio Simulation Kyle Johns Software Development Engineer Microsoft Corporation.

Documents

Transcript of Microsoft Robotics Studio Simulation Kyle Johns Software Development Engineer Microsoft Corporation.