Robotic System Simulation and ModelingStefan JörgRobotic and Mechatronic Center
Folie 2Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 3Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
DLR‘s Mirosurge: A versatile MIRS scenario
Slave Master
Folie 4Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
3 kg payload; 10 kg weight7 torque-controlled joints
DLR MIROTM
Folie 5Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Various control concepts
Telepresent Semi-autonomous Autonomous
DLR MIROTM
Folie 6Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Open surgeryMinimally invasive surgery
ButtonShaft
Plate
Magnet
DLR MIROTM
Folie 7Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
A10
A9A8
Manipulation forces up to 6 N; 0,85 kg weight3 actuated axes; Cardanic joint: ± 40°10 mm diameterIntegrated electronicsSensing of manipulation forces/torques
DLR MICA
Folie 8Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
DLR‘s Mirosurge: A versatile MIRS scenario
Folie 9Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Simulator (RSS)(for MIRS)
Motivation
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
RobotDynamics
Model
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Increase patient safety through the accurate simulation of the surgical robotic system.
Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot
Folie 10Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 11Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Simulator (RSS)(for MIRS)
The SAFROS Robotic System Simulator (RSS)
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
RobotDynamics
Model
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot
adaptable to various use casesfocus on real-time simulation withhardware-in-the-loop ( haptics ►1kHz )scaling of simulation effort (layers of detail)accurate dynamics modelsUse application control ofthe real system
Folie 12Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Motion
RSS - Simulation Platform For Workflow Design
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
Robotic System Simulator (RSS)(for MIRS)
SimulatorDesign Under Test
Folie 13Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForWorkflow Design
Relevant to surgical robotic workflow: change the surgical tool
Folie 14Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Parameterization
Motion
RSS - Simulation Platform For User Interface Design and Monitoring
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/Torques
Positions
Monitoring
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
SimulatorDesign Under Test
Folie 15Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForUser Interface Design and Monitoring
Folie 16Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForUser Interface Design and Monitoring
Compare simulation with reality during operation.
First tele-operated robot in space
Folie 17Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Parameterization
Motion
RSS - Simulation Platform For Surgeon Training
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
TrainingApplication
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...Surgeon
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
TRAINING
Robotic System Simulator (RSS)(for MIRS)
SimulatorUser UnderTest
Folie 18Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
WorldModel
HapticRenderingParameterization
Motion
RSS – Modular Simulation Platform For Various Use Cases
TrainingApplication
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Surgeon
TRAINING
SimulatorUser UnderTest
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
Folie 19Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
WorldModel
HapticRenderingParameterization
Motion
RSS - Simulation at Different Levels Of Modeling Detail
TrainingApplication
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Surgeon
TRAINING
SimulatorUser UnderTest
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
Folie 20Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
Scaling of effort (Implementation and Computation)Different use cases require different modeling detailsDefine component interfaces for each layer
Folie 21Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: kinematics, position, motion
abstracts: dynamics (masses and forces)
Folie 22Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: forces, torques w.r.t. motion, motor ripple, latency, friction, compliance
abstracts: implementation and computing hardwaredetails
Folie 23Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: implementation details (cycle-true)
abstracts: the real system
Folie 24Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
robotic hardware (e.g. MIRO, MICA,…)user interface (e.g. GUI, 3D Display, Omega Device)patient, animal model, phantomLayer 0 requires real-time simulation ( 1kHz )
Folie 25Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Implementation Design:Main Components of a MIRS Simulator
Folie 26Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
db: GeometryDataBase
ibd [block] MIRS Simulator [Layer 3]
user interface:Surgeon Console
robot:Robotic System
Simulator
world:World Simulator
in:UiMotion
out:MotionList
in:MotionList
out:UiMotion
cmd:Commands
monitor:Monitor
monitor:Monitor
3dImage:DVI Link
:Objects
robotGeometry:Objects robotGeometry:Objects
Discrete Event communication shall be implemented asUDP sockets @ 1kHz rate
The two main interfaces are defined for each Layer of modeling detail (Example shows Layer 3:Kinematics)
Implementation Design:Interface Definition for each Layer
Folie 27Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
ibd [block] Surgeon Console [Adapter Layer 0 to Layer 2]
right:Omega Force
Feedback Device
:Omega DI
:Display:DVI
RealSystemToLayer2:Adapter
left:Omega Force
Feedback Device
:Omega DI
left:Omega DI
right:Omega DI
out:UiMotion
in:UiForcesTorques
cmds:Commands
monitor:Monitor
Example: Adapt Omega Input Devices (Layer 0) to Layer 2
Note: Real hardware binds the virtual simulation time to the physical real time.
Implementation Design:Adapters between Layers
Folie 28Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Example: RSS with three MIRO modelsOne robot model for each robotApplication Control connects to the Surgeon ConsoleWorld Interface connects to the World Simulator
Implementation Design:Robotic System Simulator
Folie 29Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 30Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling
Robot Dynamics Model (RDM)Sensor/Actuator Models
Together, RDMs, Sensor, and Actuator models simulate the robot‘s physical (Layer 2,3) and implemented (Layer 1) behavior.
Robotic System Simulator (RSS)(for MIRS)
Motion
World Simulator (Patient)
Virt
ual R
ealit
y
VisualRendering
Robot Simulator 1H
ardw
are
Abs
tract
ion
Laye
r
Inte
rface
to w
orld
sim
ulat
or WorldModel
HapticRendering
Motion/Torques
Robot Dynamics Model
(kinematics, stiffness, damping, inertia)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Forces/Torques
HA
L Motion/Torques
(measured)
TorquesMotor
current Actuator Model
(friction, ripple)
Sensor Model
(gain, offset,latency)
Folie 31Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Robot Dynamics Model (RDM)
Depending on the use-case different level of detail within a layer:
1. Find the right model2. Develop a calibration procedure to find
the paramters for a specific robot
Degree of Details
Computational Costs,
Implementation Time,
High
LowSingle Body (TCP)Layer 2Dynamics
Finite Elements (robot-structure)
Multi-Body (robot-segments)
Folie 32Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Kinematics
Joint and structure stiffnesses/damping
Inertia
Scheme of the MIRO kinematics
Influence of inertia on forces and torques
Stiffness of a structure component
Robotic System Modeling:Robot Dynamics Model (RDM)
Folie 33Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Calibration Methods
Static calibration is performed stepwise reduces complexity
In 3 steps, different subsets of parameters are optimizedStatic robot poses for measurements are plannedThe measurement setup and a calibration tool are planned Steps are repeated to account for their dependencies
[Klodmann et al., Static Calibration of the DLR Medical Robot MIRO, a Flexible Lightweight Robot with Integrated Torque Sensors,IROS 2011]
Folie 34Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Calibration Methods
Folie 35Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Conclusions
Dynamic Model of the Robotic System enables Accurate SimulationAccurate Simulation of the Robotic System allows better
Workflow Design User Interface Design Monitoring during operations Training
Increased Patient Saftey
Top Related