Robotic System Simulation and Modeling · Folie 12 Robotic System Simulation and Modeling > Stefan...
Transcript of Robotic System Simulation and Modeling · Folie 12 Robotic System Simulation and Modeling > Stefan...
Robotic System Simulation and ModelingStefan JörgRobotic and Mechatronic Center
Folie 2Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 3Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
DLR‘s Mirosurge: A versatile MIRS scenario
Slave Master
Folie 4Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
3 kg payload; 10 kg weight7 torque-controlled joints
DLR MIROTM
Folie 5Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Various control concepts
Telepresent Semi-autonomous Autonomous
DLR MIROTM
Folie 6Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Open surgeryMinimally invasive surgery
ButtonShaft
Plate
Magnet
DLR MIROTM
Folie 7Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
A10
A9A8
Manipulation forces up to 6 N; 0,85 kg weight3 actuated axes; Cardanic joint: ± 40°10 mm diameterIntegrated electronicsSensing of manipulation forces/torques
DLR MICA
Folie 8Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
DLR‘s Mirosurge: A versatile MIRS scenario
Folie 9Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Simulator (RSS)(for MIRS)
Motivation
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
RobotDynamics
Model
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Increase patient safety through the accurate simulation of the surgical robotic system.
Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot
Folie 10Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 11Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Simulator (RSS)(for MIRS)
The SAFROS Robotic System Simulator (RSS)
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
RobotDynamics
Model
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Goal: Develop a surgical simulator perfectly interchangeable with the real surgical robot
adaptable to various use casesfocus on real-time simulation withhardware-in-the-loop ( haptics ►1kHz )scaling of simulation effort (layers of detail)accurate dynamics modelsUse application control ofthe real system
Folie 12Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Motion
RSS - Simulation Platform For Workflow Design
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
Robotic System Simulator (RSS)(for MIRS)
SimulatorDesign Under Test
Folie 13Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForWorkflow Design
Relevant to surgical robotic workflow: change the surgical tool
Folie 14Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Parameterization
Motion
RSS - Simulation Platform For User Interface Design and Monitoring
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/Torques
Positions
Monitoring
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
SimulatorDesign Under Test
Folie 15Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForUser Interface Design and Monitoring
Folie 16Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
RSS - Simulation Platform ForUser Interface Design and Monitoring
Compare simulation with reality during operation.
First tele-operated robot in space
Folie 17Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Parameterization
Motion
RSS - Simulation Platform For Surgeon Training
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
Robot Simulator 1
TrainingApplication
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
WorldModel
HapticRendering
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...Surgeon
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
TRAINING
Robotic System Simulator (RSS)(for MIRS)
SimulatorUser UnderTest
Folie 18Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
WorldModel
HapticRenderingParameterization
Motion
RSS – Modular Simulation Platform For Various Use Cases
TrainingApplication
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Surgeon
TRAINING
SimulatorUser UnderTest
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
Folie 19Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
World Simulator (Patient)
Virt
ualR
ealit
y
VisualRendering
WorldModel
HapticRenderingParameterization
Motion
RSS - Simulation at Different Levels Of Modeling Detail
TrainingApplication
Forces/Torques
User Interface(Surgeon Console)
Real • Omegas • Display• Pedals• GUI• Planning
Forces/torques
Positions
Monitoring
Surgeon
TRAINING
SimulatorUser UnderTest
Robot Simulator 1
Har
dwar
e A
bstra
ctio
nLa
yer
Inte
rface
to w
orld
sim
ulat
or
Inte
rface
to s
urge
onco
nsol
ean
d tra
inin
gRobot
DynamicsModel
Sensor/ActuatorModel
Har
dwar
e A
bstra
ctio
nLa
yer(
HA
L)Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
WORKFLOW DESIGN
USER INTERFACE DESIGN and MONITORING
Robotic System Simulator (RSS)(for MIRS)
Folie 20Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
Scaling of effort (Implementation and Computation)Different use cases require different modeling detailsDefine component interfaces for each layer
Folie 21Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: kinematics, position, motion
abstracts: dynamics (masses and forces)
Folie 22Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: forces, torques w.r.t. motion, motor ripple, latency, friction, compliance
abstracts: implementation and computing hardwaredetails
Folie 23Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
models: implementation details (cycle-true)
abstracts: the real system
Folie 24Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Layers of modeling details
Layer 3: KinematicsLayer 2: DynamicsLayer 1: ImplementationLayer 0: Real System
High abstraction
High detail
robotic hardware (e.g. MIRO, MICA,…)user interface (e.g. GUI, 3D Display, Omega Device)patient, animal model, phantomLayer 0 requires real-time simulation ( 1kHz )
Folie 25Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Implementation Design:Main Components of a MIRS Simulator
Folie 26Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
db: GeometryDataBase
ibd [block] MIRS Simulator [Layer 3]
user interface:Surgeon Console
robot:Robotic System
Simulator
world:World Simulator
in:UiMotion
out:MotionList
in:MotionList
out:UiMotion
cmd:Commands
monitor:Monitor
monitor:Monitor
3dImage:DVI Link
:Objects
robotGeometry:Objects robotGeometry:Objects
Discrete Event communication shall be implemented asUDP sockets @ 1kHz rate
The two main interfaces are defined for each Layer of modeling detail (Example shows Layer 3:Kinematics)
Implementation Design:Interface Definition for each Layer
Folie 27Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
ibd [block] Surgeon Console [Adapter Layer 0 to Layer 2]
right:Omega Force
Feedback Device
:Omega DI
:Display:DVI
RealSystemToLayer2:Adapter
left:Omega Force
Feedback Device
:Omega DI
left:Omega DI
right:Omega DI
out:UiMotion
in:UiForcesTorques
cmds:Commands
monitor:Monitor
Example: Adapt Omega Input Devices (Layer 0) to Layer 2
Note: Real hardware binds the virtual simulation time to the physical real time.
Implementation Design:Adapters between Layers
Folie 28Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Example: RSS with three MIRO modelsOne robot model for each robotApplication Control connects to the Surgeon ConsoleWorld Interface connects to the World Simulator
Implementation Design:Robotic System Simulator
Folie 29Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Outline
IntroductionThe SAFROS Robotic System SimulatorRobotic System ModelingConclusions
Folie 30Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling
Robot Dynamics Model (RDM)Sensor/Actuator Models
Together, RDMs, Sensor, and Actuator models simulate the robot‘s physical (Layer 2,3) and implemented (Layer 1) behavior.
Robotic System Simulator (RSS)(for MIRS)
Motion
World Simulator (Patient)
Virt
ual R
ealit
y
VisualRendering
Robot Simulator 1H
ardw
are
Abs
tract
ion
Laye
r
Inte
rface
to w
orld
sim
ulat
or WorldModel
HapticRendering
Motion/Torques
Robot Dynamics Model
(kinematics, stiffness, damping, inertia)
Robot Simulator 2
...
Application Control(MIRS)
• Workflow• Safety Checks• Control
Forces/Torques
HA
L Motion/Torques
(measured)
TorquesMotor
current Actuator Model
(friction, ripple)
Sensor Model
(gain, offset,latency)
Folie 31Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Robot Dynamics Model (RDM)
Depending on the use-case different level of detail within a layer:
1. Find the right model2. Develop a calibration procedure to find
the paramters for a specific robot
Degree of Details
Computational Costs,
Implementation Time,
High
LowSingle Body (TCP)Layer 2Dynamics
Finite Elements (robot-structure)
Multi-Body (robot-segments)
Folie 32Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Kinematics
Joint and structure stiffnesses/damping
Inertia
Scheme of the MIRO kinematics
Influence of inertia on forces and torques
Stiffness of a structure component
Robotic System Modeling:Robot Dynamics Model (RDM)
Folie 33Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Calibration Methods
Static calibration is performed stepwise reduces complexity
In 3 steps, different subsets of parameters are optimizedStatic robot poses for measurements are plannedThe measurement setup and a calibration tool are planned Steps are repeated to account for their dependencies
[Klodmann et al., Static Calibration of the DLR Medical Robot MIRO, a Flexible Lightweight Robot with Integrated Torque Sensors,IROS 2011]
Folie 34Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Robotic System Modeling:Calibration Methods
Folie 35Robotic System Simulation and Modeling > Stefan Jörg > IROS 2011 > 30.09.2011
Conclusions
Dynamic Model of the Robotic System enables Accurate SimulationAccurate Simulation of the Robotic System allows better
Workflow Design User Interface Design Monitoring during operations Training
Increased Patient Saftey