Towards Simulation-based Verification of Autonomous...
18
SAFER, SMARTER, GREENER DNV GL © 2019 13 November 2019 Tom Arne Pedersen and Jon Arne Glomsrud DNV GL Group Technology and Research Towards Simulation-based Verification of Autonomous Navigation Systems The International Maritime and Port Technology and Development Conference (MTEC)
Transcript of Towards Simulation-based Verification of Autonomous...
DNV GL © 2019 13 November 2019 SAFER, SMARTER, GREENERDNV GL © 2019
13 November 2019
Tom Arne Pedersen and Jon Arne Glomsrud
DNV GL Group Technology and Research
Towards Simulation-based Verification of Autonomous Navigation SystemsThe International Maritime and Port Technology and Development Conference (MTEC)
DNV GL © 2019 13 November 20192
Motor_vessel:99% Motor_vessel:97% Motor_vessel:93%
Ship external / internal condition detection
Situational awareness of operating environment
and ship capabilityAction Planning Action Control
RISKMinimum Risk Condition
DNV GL © 2019 13 November 20193
Test System for Autonomous Vehicles
Ship external / internal condition detection
Situational awareness of operating environment
and ship capabilityAction Planning Action Control
DNV GL © 2019 13 November 2019
Test System for Autonomous Vehicles
4
Scenario Manager
MetoceanconditionsTraffic
MapVesselconfiguration
Test Evaluation
COLREGS
Safety
Rules & regulations
White box
White box
DIGITAL TWIN / System
Black box
Vessel dynamics Radar Lidar
Camera IR
Gyro HPR
GPS MRU
Thruster dynamics
Generator dynamics
Hotel loads Ballast system
DP System
Situational awareness
Autonomous navigation
MotionSensorinput
OPERATING ENVIRONMENT
Grey box
WindWavesCurrent
Geographic location
Interactive traffic
situation
1
2
Alarms, actions
3
Scenario log4
TEST MANAGEMENTSYSTEM
VIRTUAL WORLD
TEST IN
TERFA
CE
DNV GL © 2019 13 November 2019
How to interpret COLREG?
Not suitable for direct implementation in code–Due regard, immediate danger, appropriate distance,
ample time readily apparent etc.
Interpretation required– How can we ensure that the interpretation is correct?
How to evaluate COLREG performance?– At what distance should maneuver start? What maneuver is
correct? Course change/speed change/combination? What passing distance is acceptable?
It depends on the situation
DNV GL © 2019 13 November 2019
Scenario Evaluation - Woerner
DNV GL © 2019 13 November 2019
Scenario Evaluation – Nakamura & Okada
score =∑𝑡𝑡=0𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒 − (2 ∙ 𝐷𝐷𝐷𝐷𝑒𝑒𝐷𝐷𝑒𝑒𝐷𝐷𝐷𝐷𝐷𝐷𝑠𝑠𝑡𝑡 + 1 ∙ 𝐶𝐶𝐷𝐷𝐷𝐷𝑡𝑡𝐶𝐶𝐷𝐷𝑒𝑒𝐷𝐷𝐷𝐷𝐶𝐶)
𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒
DNV GL © 2019 13 November 2019
Vendor scopeDNV GL scope
Test System for Autonomous Vehicles
8
Scenario Manager
MetoceanconditionsTraffic
MapVesselconfiguration
Test Evaluation
COLREGS
Safety
Rules & regulations
White box
White box
DIGITAL TWIN / System
Black box
Vessel dynamics Radar Lidar
Camera IR
Gyro HPR
GPS MRU
Thruster dynamics
Generator dynamics
Hotel loads Ballast system
DP System
MotionSensorinput
OPERATING ENVIRONMENT
Grey box
WindWavesCurrent
Geographic location
Interactive traffic
situation
1
2
Alarms, actions
3
Scenario log4
TEST MANAGEMENTSYSTEM
VIRTUAL WORLD
TEST IN
TERFA
CE
Situational awareness
Autonomous navigation
DNV GL © 2019 13 November 2019
Vendor scope
Objectivity / Independence
9
DIGITAL TWIN / System
Black box
Vessel dynamics Radar Lidar
Camera IR
Gyro HPR
GPS MRU
Thruster dynamics
Generator dynamics
Hotel loads Ballast system
DP System
Automation systems
Autoremotesystems
MotionSensorinput
OPERATING ENVIRONMENT
Grey box
WindWavesCurrent
Geographic location
Interactive traffic
situation
VIRTUAL WORLD
TEST IN
TERFA
CE
No restrictions on test scope
Possible to access all relevant signals
Possible to validate the correctness of the simulator and all its
components
Possible to validate the correctness of the interface between simulator, test interface and control system
software
DNV GL © 2019 13 November 2019
Testing the test system
DNV GL © 2019 13 November 2019
Vendor scopeDNV GL scope
Testing the test system – Generic setup
11
Scenario Manager
MetoceanconditionsTraffic
MapVesselconfiguration
Test Evaluation
COLREGS
Safety
Rules & regulations
OPERATING ENVIRONMENT
MotionSensorinput
Grey box
Black box
White box
Alarms, actions
Scenario log
White box
DIGITAL TWIN / System
Vessel dynamics Radar Lidar
Camera IR
Gyro HPR
GPS MRU
Thruster dynamics
Generator dynamics
Hotel loads Ballast system
DP System
WindWavesCurrent
Geographic location
Interactive traffic
situation
1
2
3
4
TEST MANAGEMENTSYSTEM
VIRTUAL WORLD
TEST IN
TERFA
CE
Situational awareness
Autonomous navigation
DNV GL © 2019 13 November 2019
DNV GL scope FFI scope
DNV GL scopeDNV GL scope
Testing the test system – FFI
12
Scenario Manager
MetoceanconditionsTraffic
MapVesselconfiguration
Test Evaluation
COLREGS
Safety
Rules & regulations
OPERATING ENVIRONMENT
MotionSensorinput
Grey box
Black box
White box
Alarms, actions
Scenario log
White box
DIGITAL TWIN / System
Vessel dynamics Radar Lidar
Camera IR
Gyro HPR
GPS MRU
Thruster dynamics
Generator dynamics
Hotel loads Ballast system
DP System
Autoremoteystems
WindWavesCurrent
Geographic location
Interactive traffic
situation
1
2
3
4
TEST MANAGEMENTSYSTEM
VIRTUAL WORLD
TEST IN
TERFA
CE
Situational awareness
Autonomous navigation
DNV GL © 2019 13 November 2019
Testing the test system – Test setup
DNV GL © 2019 13 November 2019
Contacts:- Head on
DNV GL © 2019 13 November 2019
Contacts:- Head on- Crossing
DNV GL © 2019 13 November 2019
Possible test scope for an autonomous navigation system
Generic, predefined & stylistic single COLREG scenarios
Generic, predefined & more complex scenarios with multiple COLREGs and multiple target ships
Location and operation specific scenarios, including historical cases and known incidents
Automated, adaptive search for critical scenarios
DNV GL © 2019 13 November 2019
Next step - Co-simulation in the cloud
Sim node
F G H
Sim node
D E
Sim node
O P
Sim node
L M N
Sim node
I J
Sim node
K
Sim node
A B C
DNV GL © 2019 13 November 2019
Tom Arne Pedersen
Jon Arne Glomsrud
DNV GL Group Technology & Research
Thank you for your attention
