20140318 cisec-critical-hmi
-
Upload
cisec -
Category
Technology
-
view
1.009 -
download
2
description
Transcript of 20140318 cisec-critical-hmi
Interaction Homme-Machine et Ingénierie des Systèmes Interactifs Critiques
Philippe Palanque
Interactive Critical Systems research grouphttp://www.irit.fr/ICS/palanque - [email protected]
March 18th, 2014
Interaction Homme-Machine et Ingénierie des Systèmes Interactifs Critiques
Philippe Palanque
Interactive Critical Systems research grouphttp://www.irit.fr/ICS/palanque - [email protected]
March 18th, 2014
• Air Trafic Management (enroute ATC workstations) 1995-2001& 2010-2014 HALA! Network of excellence & SPAD (System Performance under Automation Degradation)
Dynamic instantiation of widgets, Post WIMP interfaces Time constraint about 3mn (speed vector) Automation and Automation Degradation
• Drones (UAVs) 2001-2003 Management of fleet of aircrafts Authority sharing Cooperation and collaboration problems
• Military aviation 2003-2006 Multimodal systems for military cockpits (evolutions of RAFALE fighter) Specification of multimodal fusion engines, “real time” (20 ms)
• Space domain : R&T IMAGES (2004-2006) R&T TORTUGA (2008-2011) R&T ALDABRA (2011-2012) R&T MARACCAS (2012-2014)
Multimodal interfaces for ground segments Specification of satellite ground segments with multimodal interfaces Target application: AGENDA & spacecraft collision avoidance system
• Civil aviation 2004-2006 & 2009-2016 (Airbus – dependable interactive cockpits) Interactive Cockpits (ARINC 661 standards) Specification of all the embedded elements (widgets, UA, UI server) Specification of system architectures for dependable interactive systems (fault tolerance) Touch interaction in cockpit
3
Past-Current Research Projects
Human-Computer Interaction (HCI)
• ACM▫ ACM SIGCHI main SIG (36) at ACM (4600 members)▫ ~20% of downloads ACM DL ($510k return to
SIGCHI)▫ Main conference CHI (in 2013 = 3442 participants)
• IFIP▫ IFIP TC 13 on HCI ▫ Main conference INTERACT (2011=500 participants)
• Main research interests/contributions ▫ Exploration of the jungle of possibilities ▫ Focus on Usability and User Experience
4
Human-Computer Interaction (HCI)
• ACM▫ ACM SIGCHI main SIG (36) at ACM (4600 members)▫ ~20% of downloads ACM DL ($510k return to
SIGCHI)▫ Main conference CHI (in 2013 = 3442 participants)
• IFIP▫ IFIP TC 13 on HCI ▫ Main conference INTERACT (2011=500 participants)
• Main research interests/contributions ▫ Exploration of the jungle of possibilities ▫ Focus on Usability and User Experience
5
Human-Computer Interaction (HCI)
• ACM▫ ACM SIGCHI main SIG (36) at ACM (4600 members)▫ ~20% of downloads ACM DL ($510k return to
SIGCHI)▫ Main conference CHI (in 2013 = 3442 participants)
• IFIP▫ IFIP TC 13 on HCI ▫ Main conference INTERACT (2011=500 participants)
• Main research interests/contributions ▫ Exploration of the jungle of possibilities ▫ Focus on Usability and User Experience
6
• Human Computer Interaction : Usability of computing systems (effectiveness, efficiency, satisfaction – ISO 92 41- part 11)
Basic principle: user centered designProcess: iterative design/development
• Initial approach in computer science: We design/develop the system and THEN usability is evaluated
• HCI domain contribution: We design/develop the system and FOR usability
7
A bit of history: What is HCI?
8
Beaudouin-Lafon, M. 2004. Designing interaction, not interfaces. In Proceedings of the Working Conference on Advanced Visual interfaces (Gallipoli, Italy, May 25 - 28, 2004). AVI '04. ACM, New York, NY, 15-22.
iPhone iPAD
• In one sentence: Designing Interactive Systemsneither Interaction, nor Interfaces
• Principle: Usability is NOT more important thanReliability, Dependability, Security, Resilience, Safety, User eXperience, others Privacy, Trust, Accessibility, …
• Proposal: Design methods, processes and tool to design/develop interactive systems FOR these properties
10
Beaudouin-Lafon, M. 2004. Designing interaction, not interfaces. In Proceedings of the Working Conference on Advanced Visual interfaces (Gallipoli, Italy, May 25 - 28, 2004). AVI '04. ACM, New York, NY, 15-22.
They are not Orthogonal !?• Usable & reliable then safer?▫ Planes ▫ Command and control systems
• Usable & reliable then less safe!!▫ The less usable the more safe▫ The less reliable the more safe
• Safer for some less for others • Less Reliability less User eXperience• More Secure and more Reliable then less Usable • More Privacy then less Security• More Security less reliability (cockpits & satellites)
There is a need for a holistic view on these properties and not for a reductionist one (even though this supports progress)
11
12
Do We Need New Integrated Processes?Usability/User eXperienceengineer
Software engineer
Reliability engineer
Safety engineer
…
13
Current Situation• Low hanging fruits already been collected• Foundations identified many years ago▫ Annett & Duncan HTA in 1967▫ Petri nets C.A. Petri in 1962
• Refinement and deeper understanding over the years
• Need for long term detailed smaller refinements• Need for support to the design, development of
safe, usable and dependable interactive systems
• Introduction (HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
• Dependability for Interactive Systems/Cockpits
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
15
Aircraft Systems
Display System DataCrew
members
System
Monitor systems
Input manage
ment Display system was
not interactive
No USER INPUT related to display system
INPUT and OUTPUT are independent(Segregation, (Separation and Isolation) and
Diversity)
The Past: Input vs OutputCommand systems
Command + data
Control and
Display System (CDS)
Events
SetParametersCrew members
Actions
Monitor system
System
User Applications for Aircraft
SystemsUA
With ARINC 661 the command and display system is interactive
Execution of system depend strongly on user activity (and expect user input)
What about usability?
ARINC 661: Input and Output Intertwined
17
DU: Display Unit
KCCU: Keyboard and Cursor Control Unit
CDS : Control and Display SystemStandard ARINC 661 Specification
A380 Cockpit
19
Current State of ARINC 661• AEEC PP661 adopted October 2001/published April 2002
▫ Met Airbus critical need requirement (161 pages)• Supplement 1 (Dec 10, 2002, 141 pages)
▫ Vertical map display capability▫ Eight new widgets added▫ Airbus A380 CDS versus needs for future CDSs▫ ARINC 661-1 published June 26, 2003
• Supplement 2 (292 pages)▫ Draft 1 published 1st September 2004▫ Changes to ARINC 661 necessary for the Airbus A380
(NextFocusedWidget) and Boeing 787 cockpit display system development
▫ Seven new widgets (57 widgets in total)▫ Addition of state diagrams for interactive objects (p196)
• Supplement 3 draft 1 released May 21st 2007 (356 pages)▫ Eight new widgets
• Supplement 4 released May 10th 2010 (466 pages)▫ Three new widgets
ARINC 661 Principles• Client-server • Very similar to previous old work in HCI▫ IBM Common User Access 1989 standard for UI,
OSF/MOTIF, … ▫ X Window
Display Unit - Screen -
Window (managed by the CDS) Layer
(owned by one User Application)
Widget
Format
Application 1Application 3Application 2Application 1
Widget
Layer
20
• Introduction (HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
▫ System models
▫ Task models
▫ Integrated models
• Dependability for Interactive Systems/Cockpits
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
21
• “Formal” description techniques for the specification, design and construction of interactive systems ▫ Support better dependability of the system▫ Support better usability of the system Can provide contextual help Can support the production of training material
▫ Support diversity (compatibility of various models)
▫ Can take into account evolvability▫ Can support safety by e.g. providing tools to
prevent incident and accident from re-occurring
22
Our Research Proposal
23
Overview of Interactive Cooperative Objects: a formal description technique • Set of cooperating classes • For each class ▫ Behavior (Petri nets)▫ Services (availability)▫ State (distribution and value of tokens)▫ Presentation Activation (how users' actions on the input devices
trigger systems methods) Rendering (how state changes are presented to the
users• Extensions▫ Asynchronous multicast communication mechanism ▫ Quantitative temporal information (temporal window)
reuse of previous work in Petri nets theory
This is not the first work in that field
Dragicevic & Fekete . ICMI 04
This is not the first work in that field
David Carr et al. CHI 94
Goal of ICOs and PetShop• The user interface requires the same dependability
as the rest of the software• Completeness (model the entire UI)▫ the complex parts must be dealt with too▫ the more complex the UI the more likely the notation is
to be not able to deal with it• Concurrency, “infinite” number of states, temporal
aspects, objects and behavior integrated, …• Verification, validation, certification, … of the
interactive software• Bridge the edition-execution gap (Navarre D. et al. A
Model-Based Tool for Interactive Prototyping of Highly Interactive Applications. 12th IEEE, International Workshop on Rapid System Prototyping ; Monterey (USA), IEEE, 2001.)
26
26
27
A Small Example – Double click
dud
uDC
Idle DownOne_Click Two_Down
tC
28
Multimodal Interaction & ATM
Unexpected Double Clicking
29
A Small Example
du / StartTimerd
uDC
Idle DownOne_Click Two_Down
tC
tC
Adding Time
30
A Small Example
Taking Movements into account + Threashold
mD
uE
mC,B
du / StartTimer
mC,M
d, target=this
uDC
Idle
mB
DownOne_Click
Moving
Two_Down
tC
tC
31
A Small Example
Taking Movements into account + Threashold
mD
uE
mC,B
du / StartTimer
mC,M
d, target=this
uDC
Idle
mB
DownOne_Click
Moving
Two_Down
tC
tC
Einstein: "Things should beas simple as possible but not more simple"
32
Multimodal Interaction & ATM
33
Multimodal Interaction & ATM
34
A Small ExamplemD
mC,B
IdleDown One_Click
Moving
Two_Down
uE m
B tC
du / StartTimerd, target=this
tC
mC,M
uDC
CDC
CCComb_Click
Comb_Double_Click
Multimodal Part
Monomodal Part
Multimodality
Interaction Technique
35
Who said it is not readable?
36
Who said it is not readable?
37
Who said it is not readable?
m
38
39
An example: the MPIA application
Weather Radar
The issues
The user interfaces (output)
43
MPIA Application• Available in several cockpits▫ Switch between modes▫ The tilt angle: a numeric edit
box permits to select its valueinto range [-15°; 15°]
▫ Modifications are forbidden when in AUTO tilt selection mode
• Simple behavior but realistic• Tasks are simple enough too• Used in our group for
dependability and scalability studies of interactive applications
44
Behavioral description of the application: system model
45
PetShop and the system model
Modelling the Entire Interactive System• User Application• Widgets• User inteface server ▫ Objects, widgets▫ Applications▫ Input and output devices
47
Formal Description of a "simple" widget: ARINC 661 PushButton p.98-101• Informal presentation• Formal Description of the PushButton▫ Services and Events▫ Behaviour▫ Activation and Rendering functions
• Thales CDS Look & Feel (21 other ones modelled)
48
PushButton : The Behavior
Modelling the Entire Interactive System• User Application• Widgets• User inteface server ▫ Objects, widgets▫ Applications▫ Input and output devices
• Towards zero-default interactive systems
50
Dealing with Look&Feel changes
51
More about ICOs • Navarre et al. ICOs: a Model-Based User Interface Description Technique
dedicated to Interactive Systems Addressing Usability, Reliability and Scalability. ToCHI, ACM SIGCHI, Vol. 16 N. 4, p. 1-56, 2009
• Bastide, Sy & Palanque. A formal notation and tool for the engineering of CORBA systems. Concurrency: practice and experience (Wiley) Special issue "Selected papers from ECOOP'99" Vol. 12, n° 14, pp. 1379-1403, 2000
• Bastide, et al. Formal specification of CORBA services: experience and lessons learned. ACM Conference OOPSLA'2000, Minnesota USA. ACM Press; 2000.p105-117.
• Bastide & Palanque Modelling a groupware editing tool with cooperative objects "Advance in Petri nets on Object Orientation", 2001, G. Agha & F. De Cindio (Eds.), Springer Verlag, Lecture Notes in Computer Science n° 2001
• Bastide, Palanque A Petri Net Based Environment for the Design of Event-Driven Interfaces. 16th International Conference on Application and theory of Petri Nets (ATPN'95) Torino, Italy, 20-22 June 1995, LNCS.
52
There is a need for adequate tools
53
• Introduction (ICS group and HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
▫ System models
▫ Task models
▫ Integrated models
• Dependability for Interactive Systems/Cockpits
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
54
Problem ter and last (real last now)
• Engineering Interactive Systems▫ Processes, methods, techniques
and tools for the design, construction and validation of interactive systems
▫ Design prototyping ▫ Construction programming ▫ Properties usability
• Usability ▫ Efficiency▫ Satisfaction▫ Effectiveness
• User eXperience▫ Fun▫ Pleasurable▫ Desirable▫ Stimulating
55
task/artefact vicious cycleCaroll/Rosson 1991
Interactive System model
Users’ Tasksand Goals
model
56
57
58
59
Goals of HAMSTERS• Remain similar to the main task modeling tools ▫ Factorization of operators ▫ Handle low-level tasks (related to interaction techniques)
• Extends expressive power of existing tools▫ Handle object information (preconditions, processing, …)
ECCE 2013▫ Support refinement INTERACT 2011
• Make it possible to ▫ Connect to a system model (TAMODIA 2007/AMBOSS▫ Co-execution of models EICS 2010▫ Support performance evaluation (EICS 2009)▫ Formally check the compatibility of tasks and system models
(EHCI 1995, IwC 1997)▫ Support training (EICS 2011)
Task models:HAMSTER(S)
- Decomposition of a user’s goal- Hierarchical- Temporally ordered
61
Simulation with HAMSTERS
There is a need for adequate tools
62
• Introduction (HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
▫ System models
▫ Task models
▫ Integrated models
• Dependability for Interactive Systems/Cockpits
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
63
• Strong integration (co-execution of models)• One single platform (PetShop with HAMSTERS
inside) • Two modes▫ Task driven (performing a task makes the system
evolve)▫ System driven (acting on the system changes the
current task in the task model)
64
Integration Principles
65
66
Objectives of the work
• Increase reliability ▫ Complete and unambiguous description of the entire
interactive system (cockpit)▫ Including interaction (eventually multimodal)▫ Support context-tolerance (interruptions, failures,
errors, …)• Reduction of costs▫ Faster iterations to support task/artefact virtuous
circle ▫ Support for testing (software and usability)
• Improved operations▫ Faster and safer interactions in the cockpit▫ Faster recovery from system failure (MTTR)
67
Conclusions on the example• 4 views of the same real world▫ System (including interaction and interface)▫ Tasks (of each operator and of the cooperating
operators)▫ Training and User Manual (e.g. Elect. Flight Bag)
• Support for task-based construction and testing• Not presented▫ Construction of training program, assessment of
trainee and online contextual help (EICS 2011) ▫ Dealing with errors and failures (human and systems)▫ Dealing with “user over the loop” issues (automation)▫ Configurations switching following failures
Integration within ADDIE
68
There is a need for adequate tools
69
• Introduction (ICS group and HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
• Dependability for Interactive Systems/Cockpits
▫ Zero default
▫ N-version programming
▫ Self-checking widgets
▫ Impact of hardware/software architecture on usability
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
70
• “The dependability of a system is the ability to avoid service failures that are more frequent and more severe than is acceptable” Avizienis A., Laprie J-C., Randell B., Landwehr C: Basic Concepts and Taxonomy of Dependable and Secure Computing. IEEE (2004)
• Failure Condition Severity DO 178B and Probability Objectives
Dependability
FailureCondition Severity
Probability Objective
Probability descriptive
Catastrophic <10-9 Extremely Improbable
Hazardous <10-7 (very) Improbable
Major <10-5 Improbable
Minor <10-3 Reasonably probable
Redundancy is required to provide design protection from catastrophic failure conditions (ARP 4761) safety civil airborne systems
71
• Software side of it ▫ If the systems exhibit zero default then the interactive cockpit is
dependable ▫ Formal description techniques (complete and unambiguous
specification)▫ No gap between code and implementation ▫ Models can be used to support exhaustive testing
• Hardware side of it▫ Hardware failures still possible (KCCU is a single point of failure)▫ Network failure/bugs
• Environment side of it ▫ Bit flips (altitude), memory errors, memory leaking (flight time 18 hours)
…
• Human side of it ▫ ~80% of accidents are attributed to human error▫ Increase dependability level should not have a negative impact on
usability of interactive system▫ New mechanisms and methods to make cockpits dependable without
increasing task difficulty for crew
Several Views on the Problem72
• Introduction (HCI in Critical Contexts)
• Examples from the Interactive Cockpits domain
• A Research Contribution based on Models
• Dependability for Interactive Systems/Cockpits
▫ Zero default
▫ N-version programming
▫ Self-checking widgets
▫ Impact of hardware/software architecture on usability
• Similarities with other domains (Space, ATM & Entertainment)
• Conclusions and perspectives
Outline of the talk
First Architecture: User input of non critical data
74
Other Architecture: User input of critical data - Pilot as Monitor
75
Third Architecture: User input of critical data - Pilot as Monitor
76
Usability Assessment (envisioned efficiency)
77
Architectures for user input
Total number of
tasks
Number of input –output
devices to use
Number of
information and its
use
Number of user and interactive tasks
Computed tasks difficulty
Non-critical -Figure 5
(Figure 6)-13
1 Input + 1 Output (used 7
times)
2 (used 8 times)
10 (3 cognitive, 2 perceptive, 1 motor, 1
interactive, 3 abstract)
(8)+(3+2+1+1+(3*3))
24Critical System
monitored -Figure 7
(Figure 13) -15
+ problem management
1 Input + 1 Output (used 7
times)
2 (used 8 times)
10 (3 cognitive, 2 perceptive, 1 motor, 1
interactive, 3 abstract) + problem management
(8)+(3+2+1+1+(3*3))
24+ problem management
Critical Pilot monitored -
Figure 8
(Figure 14) - 27 +
problem management
1 Input + 2 Output (used
19 times)
3 (used 12 times)
21 (5 cognitive, 5 perceptive, 1 motor, 1
interactive, 9 abstract) + problem management
(12)+(5+5+1+1+(3*9))
51+ problem management
Critical Fail safe - Figure 10
(Figure 15) – 25 +
problemmanagement
2 Input + 2 Output (used
14 times)
3 (used 12 times)
19 (5 cognitive, 5 perceptive, 1 motor, 1
interactive, 6 abstract) + problem management
(12)+(5+5+1+1+(3*6))
42+ problem management
Summary : without system error
78
• Introduction (HCI in Critical Contexts)
• Introduction to the Interactive Cockpits domain
• A Research Contribution based on Models
• Dependability for Interactive Systems/Cockpits
• Dealing with automation
• Conclusions and perspectives
Outline of the talk
79
Iterative design process
ICO –PetShop
Formal System modelling
Preliminary System Model
HAMSTERS
Formal Task modelling
Preliminary Task Model
Correspondence between System & Task models co-execution
Analysis
Check Objectivies
Towards User
Testing
OKProposal for mending the
System Model
Not OK
ith Iteration Ith Iteration
80
Iterative Process Including Automation
81
WXR Task model
82
WXR System model
83
Task model: second iteration
84
System model: second iteration
85
Analyzing Gains and Losses
• More detailed case study of satellite ground segment (PICARD satellite)
• More complex tasks migrations based on information
• Assessment of tasks complexity using scenarios• Assessment of the current task-system design
with respect to Sheridan/Parasuraman levels (task-based and scenario-based)
86
Benefits of the approach• Very detailed description of function migration▫ At the user levels▫ At the system level
• Simulation▫ Of the various designs ▫ Connected to the models
• Support for performance evaluation ▫ On the tasks ▫ On the systems ▫ On the couple tasks/systems
• Decision support tools for identifying candidates for migration
• Integration of formal notations
87
Towards an Integrated Process (1/2)
Legend
Needs and requirements
analysis
Interactive critical system
deployment
Phase of the process covered by our contribution
Phase of the process not covered by our contribution
Phase of the process partly covered by our contribution
Minimal flow
Training program development
I
II
III
IV
Needs and requirements
Proposal for redefining or mending requirements
Proposal for redefining or mending requirements
Proposal for redefining or mending models and/
or prototype
Proposal for redefining or mending
models and/or prototypeInteractive Critical
System Design
Interactive Critical System: very high-fidelity prototype and specification (task and system
models)
Interactive Critical System and associated training
program
Proposal for modifying the
training programOptional flow
Towards an Integrated Process (2/2)
• Introduction (HCI in Critical Contexts)
• Examples from the Interactive Cockpits domain
• A Research Contribution based on Models
• Dependability for Interactive Systems/Cockpits
• Similarities with other domains (Space, ATM &
Entertainment)
• Conclusions and perspectives
Outline of the talk
90
• Dependability and usability are intrinsically related, but often studied independently in the literature
• Increase dependability level can have a huge (possibly negative) impact on usability of interactive system
• Necessity to design new mechanisms or methods which can make critical interactive system reliable assessing▫ Impact on usability▫ Impact on training ▫ Impact on performance▫ Potential for automation (impact of degradation)
HCI In Critical Contexts
91
Combining Design and Engineering of Interactive Systems
92
0 100% of time spent on design vs engineering
DependabilitySafety
Market Push (Pilots, Airlines, …)
Regulation Push –ARINC 661, 653 DO178B
InnovationUX
Operation performance Usability
Thoughts for the future• Construction▫ Adequate tools▫ Adequate machines▫ Adequate factories
• Product characteristics▫ Properties / qualities▫ Handling and managing conflicts rationally
• Understanding and handling the borders▫ Formal and informal ▫ Critical / public ▫ Work environment / entertainment-social
93
Thank you very much …
for the invitation for your attention
AcknowledgementsThe work presented is partly funded by:ResIST EU Network of Excellence on Resilience for IST CNES R&T projects TORTUGA & ALDABRA Airbus contract UPS/ CNRS/AIRBUS PBO D08028747-Thèse 788/2008EUROCONTROL HALA! (Higher Automation Level in Aviation research network)
Thanks to my colleagues: Yannick Deleris (Airbus), Jean-Charles Fabre (LAAS) and David Navarre, Célia Martinie and Eric Barboni (ICS-IRIT)
HAMSTERS v2 Available for download
• Feedback greatly appreciated• Tech support for 3 years (min)• Requests for extensions will be all processed• Collaborative aspects under way • Will be open source when scientifically stable
(already built with Maven)
96
http://www.irit.fr/recherches/ICS/softwares/hamsters/
Google Hamsters + ICS IRIT
HAMSTERS v2 Available for download
• Feedback greatly appreciated• Tech support for 3 years (min)• Requests for extensions will be all processed• Collaborative aspects under way • Will be open source when scientifically stable
(already built with Maven)
97
http://www.irit.fr/recherches/ICS/softwares/hamsters/
Google Hamsters + ICS IRIT
Why Another Tool for Task Modelling
• HAMSTERS deals with user goals • HAMSTERS is more accurate• It is stronger than CTTE• It deals with small case studies (lab)• It deals with real case studies (companies)• It requires very little training• It will support collaboration• It is connected to other tools• It has been designed by d’Artagnan relatives