Episode 3 / CAATS II joint dissemination event Lessons Learnt Episode 3 - CAATS II Final...

Episode 3 / CAATS II joint dissemination event

Lessons Learnt

Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

Philippe LeplaeEUROCONTROLEpisode 3

Brussels, 13 & 14 Oct 2009


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

2

Content

• Project objectives• Lessons learnt method• Lessons learnt in:

management validation concept clarification result integration validation techniques

• Conclusion


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

3

Project objectives

• Provide detail on key concept elements in SESAR• Initial operability and performance assessment• Initial technical impact assessment• Analysis of available tools for SESAR validation• Report on validation methodology……

• Hence specific emphasis on lessons learnt


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

4

Lessons learnt method

• 2 workshops, Madrid and Brétigny• 9 different sessions on 5 main topics involving more than

30 participants Validation strategy and project DOW Project management Concept clarification Integration and Consolidation EP3 validation techniques and levels of maturity

• Most project members also working in SESAR• Strong desire to provide feedback to SESAR


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

5

High level lessons learnt

• System view should drive validation activities Need to adapt DOW and management to required

flexibility Need time to develop system driven project We have developed artefacts to drive it

• Many different techniques tried on Episode 3 Suited to low maturity level concept elements Guidelines on when and how to use it are produced

• Importance of support tools Good strategy useless without tools to support it Tools also require investment in training This provides harmonized working methods, key to make

distributed project work


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

6

Lessons learntin management

• Our challenges: Wide range of disseminated partners Crossed responsibilities and thinly spread effort Innovative tasks High quality requirements Tight deadlines

• Our answers: Development of tools for distributed work Training on the tools, and specific delivery process A lot of co-ordination activities

• Follow-up actions: Build more flexibility in the programme (DOW and contract)


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

7

Lessons learntin validation

• Our challenges: Maturity lower than expected at programme onset E-OCVM V2 not well adapted to concept wide validation Lack of E-OCVM expertise inside team Wide range of results

• Our answers: We built a framework with guidelines, principles, templates. Developed assumptions management Performance framework and DOD’s provided at start Guidance provided through system WP and co-ordination cell Thorough review of experimental plans

• Open issue: Maturity assessment at start of each validation activity

WP6

Increasing Level of Concept Maturity

V1 V2 V3

Overall Validation Strategy

Updated Overall

Validation Strategy

Updated Overall

Validation Strategy

Operational Context

Validation Strategy

Updated Operational

Context Validation Strategy

Updated Operational

Context ValidationStrategy

Target Concept

Operational Context

Concept Level

OIs Validation Exercises

EPISODE 3

WP5WP4

WP3

WP2

WP6WP6

Increasing Level of Concept Maturity

V1V1 V2V2 V3V3

Overall Validation Strategy

Updated Overall

Validation Strategy

Updated Overall

Validation Strategy

Operational Context

Validation Strategy

Updated Operational

Context Validation Strategy

Updated Operational

Context ValidationStrategy

Target Concept

Operational Context

Concept Level

OIs Validation Exercises

EPISODE 3

WP5WP4

WP3

WP2


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

8

Lessons learntin concept clarification

• Our challenges: Multiple sources from SESAR (CONOPS, OI’s,etc…) Lack of details for validation Multiple sources to update DOD’s

• Our solutions: Propose a structure, around process model and DOD’s Develop different modes of descriptions

- DOD’s - Operational scenarios - Story boards

Develop collaborative working methods to update documents

• Follow-up: Establish a recognised authority to select among options Use hot topics to drive validation


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

9

Lessons learntin result integration

• Our challenges: Exercises not providing consistent results Different granularity of results Various degree of validity of quantitative data

• Our solutions: Influence diagrams help understanding benefit mechanism

and identify where performance is improved Influence model has been developed ready to be fed with

results Management of assumptions to interpret results

• For subsequent coverage: Steer validation activities from the performance framework Use of models developed in EP3 to provide more quantitative

data to performance model


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

10

Lessons learnt in validation techniques

• Successful used of different techniques Prototyping Macro-modelling Various sorts or gaming Expert groups

• Work packages integrated different techniques Combine results of very different nature Various sequences of activities tried Expert groups key to success

• Benefits of industry assessment: “Reality check” Should run as a complementary process to operational

validation


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

11

Validation techniques:Expert Groups

• Lessons learnt Useful at all maturity levels Need good facilitation techniques by somebody

with OPS expertise Need current operation experts + concept

experts Storyboards good method to structure EG

debates

• Our input to SESAR Storyboards technique Various facilitation techniques Hot topics Guidelines document


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

12

Validation techniques:Gaming

• Lessons learnt To be associated with Expert Groups and modelling Suited to clarify roles and responsibilities for all maturity levels More flexible than prototyping as not in real time (interruptions

feasible) Progression from paper-based gaming to platform gaming

• Our input to SESAR Different platforms

- CHILL - DARTIS- ACCES platform

Paper based gaming and associated web based tool

Guidelines document


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

13

Validation techniques:Prototyping

• Lessons learnt: Not a cheap RTS! Allowed to fail Concept clarification-no quantitative

data Successive runs with same team

of OPS staff

• Our input to SESAR First results on:

- PRNAV, CDA in descent phase.- Acceptability of 4D trajectory to

controllers Guidelines document


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

14

Validation techniques:Modelling

• Lessons learnt: Use of process modelling and

network modelling Process modelling complement

gaming Need to work on precise concept

elements Macro-modelling

• Our input to SESAR: 3 innovative tools developed in EP3

- One abandoned (unsatisfactory results)- One providing results when applied

to mature concept elements- One promising for network wide assessments

Guidelines document

Node type I (airport)

Node type II (congestion zone)

Link: air traffic flow


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

15

Conclusion

• Episode 3 is the first ATM-wide concept validation programme

• We had to develop tools and methods adapted to this new challenge

• Another 2 months to complete our work (WP and final report)

• We provide to SESAR: New tools New methods An integrated approach First validation results… See www.episode3.aero

• And our most precious output: A team with the expertise and concrete experience already

working on the SESAR projects:

http://www.episode3.aero/


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

16

Thank you

Episode 3 / CAATS II joint dissemination event Lessons Learnt Episode 3 - CAATS II Final...

Documents

Transcript of Episode 3 / CAATS II joint dissemination event Lessons Learnt Episode 3 - CAATS II Final...