20140318 cisec-critical-hmi

Interaction Homme-Machine et Ingénierie des Systèmes Interactifs Critiques

Philippe Palanque

Interactive Critical Systems research grouphttp://www.irit.fr/ICS/palanque - [email protected]

March 18th, 2014

• Air Trafic Management (enroute ATC workstations) 1995-2001& 2010-2014 HALA! Network of excellence & SPAD (System Performance under Automation Degradation)

Dynamic instantiation of widgets, Post WIMP interfaces Time constraint about 3mn (speed vector) Automation and Automation Degradation

• Drones (UAVs) 2001-2003 Management of fleet of aircrafts Authority sharing Cooperation and collaboration problems

• Military aviation 2003-2006 Multimodal systems for military cockpits (evolutions of RAFALE fighter) Specification of multimodal fusion engines, “real time” (20 ms)

• Space domain : R&T IMAGES (2004-2006) R&T TORTUGA (2008-2011) R&T ALDABRA (2011-2012) R&T MARACCAS (2012-2014)

Multimodal interfaces for ground segments Specification of satellite ground segments with multimodal interfaces Target application: AGENDA & spacecraft collision avoidance system

• Civil aviation 2004-2006 & 2009-2016 (Airbus – dependable interactive cockpits) Interactive Cockpits (ARINC 661 standards) Specification of all the embedded elements (widgets, UA, UI server) Specification of system architectures for dependable interactive systems (fault tolerance) Touch interaction in cockpit

3

Past-Current Research Projects

Human-Computer Interaction (HCI)

• ACM▫ ACM SIGCHI main SIG (36) at ACM (4600 members)▫ ~20% of downloads ACM DL ($510k return to

SIGCHI)▫ Main conference CHI (in 2013 = 3442 participants)

• IFIP▫ IFIP TC 13 on HCI ▫ Main conference INTERACT (2011=500 participants)

• Main research interests/contributions ▫ Exploration of the jungle of possibilities ▫ Focus on Usability and User Experience

4






5






6

• Human Computer Interaction : Usability of computing systems (effectiveness, efficiency, satisfaction – ISO 92 41- part 11)

Basic principle: user centered designProcess: iterative design/development

• Initial approach in computer science: We design/develop the system and THEN usability is evaluated

• HCI domain contribution: We design/develop the system and FOR usability

7

A bit of history: What is HCI?

8

Beaudouin-Lafon, M. 2004. Designing interaction, not interfaces. In Proceedings of the Working Conference on Advanced Visual interfaces (Gallipoli, Italy, May 25 - 28, 2004). AVI '04. ACM, New York, NY, 15-22.

iPhone iPAD

• In one sentence: Designing Interactive Systemsneither Interaction, nor Interfaces

• Principle: Usability is NOT more important thanReliability, Dependability, Security, Resilience, Safety, User eXperience, others Privacy, Trust, Accessibility, …

• Proposal: Design methods, processes and tool to design/develop interactive systems FOR these properties

10

Beaudouin-Lafon, M. 2004. Designing interaction, not interfaces. In Proceedings of the Working Conference on Advanced Visual interfaces (Gallipoli, Italy, May 25 - 28, 2004). AVI '04. ACM, New York, NY, 15-22.

They are not Orthogonal !?• Usable & reliable then safer?▫ Planes ▫ Command and control systems

• Usable & reliable then less safe!!▫ The less usable the more safe▫ The less reliable the more safe

• Safer for some less for others • Less Reliability less User eXperience• More Secure and more Reliable then less Usable • More Privacy then less Security• More Security less reliability (cockpits & satellites)

There is a need for a holistic view on these properties and not for a reductionist one (even though this supports progress)

11

12

Do We Need New Integrated Processes?Usability/User eXperienceengineer

Software engineer

Reliability engineer

Safety engineer

…

Current Situation• Low hanging fruits already been collected• Foundations identified many years ago▫ Annett & Duncan HTA in 1967▫ Petri nets C.A. Petri in 1962

• Refinement and deeper understanding over the years

• Need for long term detailed smaller refinements• Need for support to the design, development of

safe, usable and dependable interactive systems

• Introduction (HCI in Critical Contexts)

• Introduction to the Interactive Cockpits domain

• A Research Contribution based on Models

• Dependability for Interactive Systems/Cockpits

• Dealing with automation

• Conclusions and perspectives

Outline of the talk

15

Aircraft Systems

Display System DataCrew

members

System

Monitor systems

Input manage

ment Display system was

not interactive

No USER INPUT related to display system

INPUT and OUTPUT are independent(Segregation, (Separation and Isolation) and

Diversity)

The Past: Input vs OutputCommand systems

Command + data

Control and

Display System (CDS)

Events

SetParametersCrew members

Actions

Monitor system

System

User Applications for Aircraft

SystemsUA

With ARINC 661 the command and display system is interactive

Execution of system depend strongly on user activity (and expect user input)

What about usability?

ARINC 661: Input and Output Intertwined

17

DU: Display Unit

KCCU: Keyboard and Cursor Control Unit

CDS : Control and Display SystemStandard ARINC 661 Specification

A380 Cockpit

19

Current State of ARINC 661• AEEC PP661 adopted October 2001/published April 2002

▫ Met Airbus critical need requirement (161 pages)• Supplement 1 (Dec 10, 2002, 141 pages)

▫ Vertical map display capability▫ Eight new widgets added▫ Airbus A380 CDS versus needs for future CDSs▫ ARINC 661-1 published June 26, 2003

• Supplement 2 (292 pages)▫ Draft 1 published 1st September 2004▫ Changes to ARINC 661 necessary for the Airbus A380

(NextFocusedWidget) and Boeing 787 cockpit display system development

▫ Seven new widgets (57 widgets in total)▫ Addition of state diagrams for interactive objects (p196)

• Supplement 3 draft 1 released May 21st 2007 (356 pages)▫ Eight new widgets

• Supplement 4 released May 10th 2010 (466 pages)▫ Three new widgets

ARINC 661 Principles• Client-server • Very similar to previous old work in HCI▫ IBM Common User Access 1989 standard for UI,

OSF/MOTIF, … ▫ X Window

Display Unit - Screen -

Window (managed by the CDS) Layer

(owned by one User Application)

Widget

Format

Application 1Application 3Application 2Application 1

Widget

Layer

20




▫ System models

▫ Task models

▫ Integrated models




Outline of the talk

21

• “Formal” description techniques for the specification, design and construction of interactive systems ▫ Support better dependability of the system▫ Support better usability of the system Can provide contextual help Can support the production of training material

▫ Support diversity (compatibility of various models)

▫ Can take into account evolvability▫ Can support safety by e.g. providing tools to

prevent incident and accident from re-occurring

22

Our Research Proposal

23

Overview of Interactive Cooperative Objects: a formal description technique • Set of cooperating classes • For each class ▫ Behavior (Petri nets)▫ Services (availability)▫ State (distribution and value of tokens)▫ Presentation Activation (how users' actions on the input devices

trigger systems methods) Rendering (how state changes are presented to the

users• Extensions▫ Asynchronous multicast communication mechanism ▫ Quantitative temporal information (temporal window)

reuse of previous work in Petri nets theory

This is not the first work in that field

Dragicevic & Fekete . ICMI 04

This is not the first work in that field

David Carr et al. CHI 94

Goal of ICOs and PetShop• The user interface requires the same dependability

as the rest of the software• Completeness (model the entire UI)▫ the complex parts must be dealt with too▫ the more complex the UI the more likely the notation is

to be not able to deal with it• Concurrency, “infinite” number of states, temporal

aspects, objects and behavior integrated, …• Verification, validation, certification, … of the

interactive software• Bridge the edition-execution gap (Navarre D. et al. A

Model-Based Tool for Interactive Prototyping of Highly Interactive Applications. 12th IEEE, International Workshop on Rapid System Prototyping ; Monterey (USA), IEEE, 2001.)

26

26

27

A Small Example – Double click

dud

uDC

Idle DownOne_Click Two_Down

tC

28

Multimodal Interaction & ATM

Unexpected Double Clicking

29

A Small Example

du / StartTimerd

uDC

Idle DownOne_Click Two_Down

tC

tC

Adding Time

30

A Small Example

Taking Movements into account + Threashold

mD

uE

mC,B

du / StartTimer

mC,M

d, target=this

uDC

Idle

mB

DownOne_Click

Moving

Two_Down

tC

tC

31

A Small Example

Taking Movements into account + Threashold

mD

uE

mC,B

du / StartTimer

mC,M

d, target=this

uDC

Idle

mB

DownOne_Click

Moving

Two_Down

tC

tC

Einstein: "Things should beas simple as possible but not more simple"

32


33


34

A Small ExamplemD

mC,B

IdleDown One_Click

Moving

Two_Down

uE m

B tC

du / StartTimerd, target=this

tC

mC,M

uDC

CDC

CCComb_Click

Comb_Double_Click

Multimodal Part

Monomodal Part

Multimodality

Interaction Technique

35

Who said it is not readable?

36


37


m

38

39

An example: the MPIA application

Weather Radar

The issues

The user interfaces (output)

43

MPIA Application• Available in several cockpits▫ Switch between modes▫ The tilt angle: a numeric edit

box permits to select its valueinto range [-15°; 15°]

▫ Modifications are forbidden when in AUTO tilt selection mode

• Simple behavior but realistic• Tasks are simple enough too• Used in our group for

dependability and scalability studies of interactive applications

44

Behavioral description of the application: system model

45

PetShop and the system model

Modelling the Entire Interactive System• User Application• Widgets• User inteface server ▫ Objects, widgets▫ Applications▫ Input and output devices

47

Formal Description of a "simple" widget: ARINC 661 PushButton p.98-101• Informal presentation• Formal Description of the PushButton▫ Services and Events▫ Behaviour▫ Activation and Rendering functions

• Thales CDS Look & Feel (21 other ones modelled)

48

PushButton : The Behavior

Modelling the Entire Interactive System• User Application• Widgets• User inteface server ▫ Objects, widgets▫ Applications▫ Input and output devices

• Towards zero-default interactive systems

50

Dealing with Look&Feel changes

More about ICOs • Navarre et al. ICOs: a Model-Based User Interface Description Technique

dedicated to Interactive Systems Addressing Usability, Reliability and Scalability. ToCHI, ACM SIGCHI, Vol. 16 N. 4, p. 1-56, 2009

• Bastide, Sy & Palanque. A formal notation and tool for the engineering of CORBA systems. Concurrency: practice and experience (Wiley) Special issue "Selected papers from ECOOP'99" Vol. 12, n° 14, pp. 1379-1403, 2000

• Bastide, et al. Formal specification of CORBA services: experience and lessons learned. ACM Conference OOPSLA'2000, Minnesota USA. ACM Press; 2000.p105-117.

• Bastide & Palanque Modelling a groupware editing tool with cooperative objects "Advance in Petri nets on Object Orientation", 2001, G. Agha & F. De Cindio (Eds.), Springer Verlag, Lecture Notes in Computer Science n° 2001

• Bastide, Palanque A Petri Net Based Environment for the Design of Event-Driven Interfaces. 16th International Conference on Application and theory of Petri Nets (ATPN'95) Torino, Italy, 20-22 June 1995, LNCS.

52

There is a need for adequate tools

53

• Introduction (ICS group and HCI in Critical Contexts)



▫ System models

▫ Task models





Outline of the talk

54

Problem ter and last (real last now)

• Engineering Interactive Systems▫ Processes, methods, techniques

and tools for the design, construction and validation of interactive systems

▫ Design prototyping ▫ Construction programming ▫ Properties usability

• Usability ▫ Efficiency▫ Satisfaction▫ Effectiveness

• User eXperience▫ Fun▫ Pleasurable▫ Desirable▫ Stimulating

55

task/artefact vicious cycleCaroll/Rosson 1991

Interactive System model

Users’ Tasksand Goals

model

59

Goals of HAMSTERS• Remain similar to the main task modeling tools ▫ Factorization of operators ▫ Handle low-level tasks (related to interaction techniques)

• Extends expressive power of existing tools▫ Handle object information (preconditions, processing, …)

ECCE 2013▫ Support refinement INTERACT 2011

• Make it possible to ▫ Connect to a system model (TAMODIA 2007/AMBOSS▫ Co-execution of models EICS 2010▫ Support performance evaluation (EICS 2009)▫ Formally check the compatibility of tasks and system models

(EHCI 1995, IwC 1997)▫ Support training (EICS 2011)

Task models:HAMSTER(S)

- Decomposition of a user’s goal- Hierarchical- Temporally ordered

61

Simulation with HAMSTERS


62




▫ System models

▫ Task models





Outline of the talk

63

• Strong integration (co-execution of models)• One single platform (PetShop with HAMSTERS

inside) • Two modes▫ Task driven (performing a task makes the system

evolve)▫ System driven (acting on the system changes the

current task in the task model)

64

Integration Principles

66

Objectives of the work

• Increase reliability ▫ Complete and unambiguous description of the entire

interactive system (cockpit)▫ Including interaction (eventually multimodal)▫ Support context-tolerance (interruptions, failures,

errors, …)• Reduction of costs▫ Faster iterations to support task/artefact virtuous

circle ▫ Support for testing (software and usability)

• Improved operations▫ Faster and safer interactions in the cockpit▫ Faster recovery from system failure (MTTR)

67

Conclusions on the example• 4 views of the same real world▫ System (including interaction and interface)▫ Tasks (of each operator and of the cooperating

operators)▫ Training and User Manual (e.g. Elect. Flight Bag)

• Support for task-based construction and testing• Not presented▫ Construction of training program, assessment of

trainee and online contextual help (EICS 2011) ▫ Dealing with errors and failures (human and systems)▫ Dealing with “user over the loop” issues (automation)▫ Configurations switching following failures

Integration within ADDIE

68


69

• Introduction (ICS group and HCI in Critical Contexts)




▫ Zero default

▫ N-version programming

▫ Self-checking widgets

▫ Impact of hardware/software architecture on usability



Outline of the talk

70

• “The dependability of a system is the ability to avoid service failures that are more frequent and more severe than is acceptable” Avizienis A., Laprie J-C., Randell B., Landwehr C: Basic Concepts and Taxonomy of Dependable and Secure Computing. IEEE (2004)

• Failure Condition Severity DO 178B and Probability Objectives

Dependability

FailureCondition Severity

Probability Objective

Probability descriptive

Catastrophic <10-9 Extremely Improbable

Hazardous <10-7 (very) Improbable

Major <10-5 Improbable

Minor <10-3 Reasonably probable

Redundancy is required to provide design protection from catastrophic failure conditions (ARP 4761) safety civil airborne systems

71

• Software side of it ▫ If the systems exhibit zero default then the interactive cockpit is

dependable ▫ Formal description techniques (complete and unambiguous

specification)▫ No gap between code and implementation ▫ Models can be used to support exhaustive testing

• Hardware side of it▫ Hardware failures still possible (KCCU is a single point of failure)▫ Network failure/bugs

• Environment side of it ▫ Bit flips (altitude), memory errors, memory leaking (flight time 18 hours)

…

• Human side of it ▫ ~80% of accidents are attributed to human error▫ Increase dependability level should not have a negative impact on

usability of interactive system▫ New mechanisms and methods to make cockpits dependable without

increasing task difficulty for crew

Several Views on the Problem72


• Examples from the Interactive Cockpits domain



▫ Zero default

▫ N-version programming

▫ Self-checking widgets

▫ Impact of hardware/software architecture on usability

• Similarities with other domains (Space, ATM & Entertainment)


Outline of the talk

First Architecture: User input of non critical data

74

Other Architecture: User input of critical data - Pilot as Monitor

75

Third Architecture: User input of critical data - Pilot as Monitor

76

Usability Assessment (envisioned efficiency)

77

Architectures for user input

Total number of

tasks

Number of input –output

devices to use

Number of

information and its

use

Number of user and interactive tasks

Computed tasks difficulty

Non-critical -Figure 5

(Figure 6)-13

1 Input + 1 Output (used 7

times)

2 (used 8 times)

10 (3 cognitive, 2 perceptive, 1 motor, 1

interactive, 3 abstract)

(8)+(3+2+1+1+(3*3))

24Critical System

monitored -Figure 7

(Figure 13) -15

+ problem management

1 Input + 1 Output (used 7

times)

2 (used 8 times)


interactive, 3 abstract) + problem management

(8)+(3+2+1+1+(3*3))

24+ problem management

Critical Pilot monitored -

Figure 8

(Figure 14) - 27 +

problem management

1 Input + 2 Output (used

19 times)

3 (used 12 times)



(12)+(5+5+1+1+(3*9))


Critical Fail safe - Figure 10

(Figure 15) – 25 +

problemmanagement

2 Input + 2 Output (used

14 times)

3 (used 12 times)



(12)+(5+5+1+1+(3*6))


Summary : without system error

78







Outline of the talk

79

Iterative design process

ICO –PetShop

Formal System modelling

Preliminary System Model

HAMSTERS

Formal Task modelling

Preliminary Task Model

Correspondence between System & Task models co-execution

Analysis

Check Objectivies

Towards User

Testing

OKProposal for mending the

System Model

Not OK

ith Iteration Ith Iteration

80

Iterative Process Including Automation

81

WXR Task model

82

WXR System model

83

Task model: second iteration

84

System model: second iteration

85

Analyzing Gains and Losses

• More detailed case study of satellite ground segment (PICARD satellite)

• More complex tasks migrations based on information

• Assessment of tasks complexity using scenarios• Assessment of the current task-system design

with respect to Sheridan/Parasuraman levels (task-based and scenario-based)

86

Benefits of the approach• Very detailed description of function migration▫ At the user levels▫ At the system level

• Simulation▫ Of the various designs ▫ Connected to the models

• Support for performance evaluation ▫ On the tasks ▫ On the systems ▫ On the couple tasks/systems

• Decision support tools for identifying candidates for migration

• Integration of formal notations

87

Towards an Integrated Process (1/2)

Legend

Needs and requirements

analysis

Interactive critical system

deployment

Phase of the process covered by our contribution

Phase of the process not covered by our contribution

Phase of the process partly covered by our contribution

Minimal flow

Training program development

I

II

III

IV

Needs and requirements

Proposal for redefining or mending requirements

Proposal for redefining or mending requirements

Proposal for redefining or mending models and/

or prototype

Proposal for redefining or mending

models and/or prototypeInteractive Critical

System Design

Interactive Critical System: very high-fidelity prototype and specification (task and system

models)

Interactive Critical System and associated training

program

Proposal for modifying the

training programOptional flow

Towards an Integrated Process (2/2)


• Examples from the Interactive Cockpits domain



• Similarities with other domains (Space, ATM &

Entertainment)


Outline of the talk

90

• Dependability and usability are intrinsically related, but often studied independently in the literature

• Increase dependability level can have a huge (possibly negative) impact on usability of interactive system

• Necessity to design new mechanisms or methods which can make critical interactive system reliable assessing▫ Impact on usability▫ Impact on training ▫ Impact on performance▫ Potential for automation (impact of degradation)

HCI In Critical Contexts

91

Combining Design and Engineering of Interactive Systems

92

0 100% of time spent on design vs engineering

DependabilitySafety

Market Push (Pilots, Airlines, …)

Regulation Push –ARINC 661, 653 DO178B

InnovationUX

Operation performance Usability

Thoughts for the future• Construction▫ Adequate tools▫ Adequate machines▫ Adequate factories

• Product characteristics▫ Properties / qualities▫ Handling and managing conflicts rationally

• Understanding and handling the borders▫ Formal and informal ▫ Critical / public ▫ Work environment / entertainment-social

93

Thank you very much …

for the invitation for your attention

AcknowledgementsThe work presented is partly funded by:ResIST EU Network of Excellence on Resilience for IST CNES R&T projects TORTUGA & ALDABRA Airbus contract UPS/ CNRS/AIRBUS PBO D08028747-Thèse 788/2008EUROCONTROL HALA! (Higher Automation Level in Aviation research network)

Thanks to my colleagues: Yannick Deleris (Airbus), Jean-Charles Fabre (LAAS) and David Navarre, Célia Martinie and Eric Barboni (ICS-IRIT)

HAMSTERS v2 Available for download

• Feedback greatly appreciated• Tech support for 3 years (min)• Requests for extensions will be all processed• Collaborative aspects under way • Will be open source when scientifically stable

(already built with Maven)

96

http://www.irit.fr/recherches/ICS/softwares/hamsters/

Google Hamsters + ICS IRIT

HAMSTERS v2 Available for download

• Feedback greatly appreciated• Tech support for 3 years (min)• Requests for extensions will be all processed• Collaborative aspects under way • Will be open source when scientifically stable

(already built with Maven)

97

http://www.irit.fr/recherches/ICS/softwares/hamsters/

Google Hamsters + ICS IRIT

Why Another Tool for Task Modelling

• HAMSTERS deals with user goals • HAMSTERS is more accurate• It is stronger than CTTE• It deals with small case studies (lab)• It deals with real case studies (companies)• It requires very little training• It will support collaboration• It is connected to other tools• It has been designed by d’Artagnan relatives

20140318 cisec-critical-hmi

Technology

Transcript of 20140318 cisec-critical-hmi