Post on 03-Jun-2018
8/12/2019 Raytheon (Powell) - Behaviour Trees
1/44
Dan Powell24 August 2011
Behaviour
TreesGrowing Understanding,
Taming Complexity,
Minimising Risk
Copyright 2011Raytheon Company. All rights reserved.Customer Success Is Our Mission is a registered trademark of Raytheon Company.
8/12/2019 Raytheon (Powell) - Behaviour Trees
2/44
Page
Topics in this Presentation
The Case for Behaviour Engineering
What is Behaviour Engineering?
The Behaviour Engineering ProcessThe Behaviour Modelling Language
Outcomes so far
Applications of Behaviour Engineering
8/12/2019 Raytheon (Powell) - Behaviour Trees
3/44
Page
The Case for Behaviour EngineeringThe Case for Behaviour Engineering
What is Behaviour Engineering?
The Behaviour Engineering ProcessThe Behaviour Modelling Language
Outcomes so far
Applications of Behaviour Engineering
8/12/2019 Raytheon (Powell) - Behaviour Trees
4/44
Page
Large Projects/ProgramsGreatest Risks
Cost Overruns
Schedule Overruns
Ending up with a system that does not meet the needs andexpectations of stakeholders.
Ending up with a system that requires a lot of maintenance(defect fixing)
Project failure or abandonment
8/12/2019 Raytheon (Powell) - Behaviour Trees
5/44
Page
Early Life Cycle the Biggest Issue
Upto6
0%ofalld
efects
inasys
tems
lifetime
origina
tefromde
ficient
requ
irement
s
Reworkingrequirementscostsbetween
40%and80%oftotalprojectcosts
8/12/2019 Raytheon (Powell) - Behaviour Trees
6/44Page
The Problem
Critical Defect
Implied/Assumed KnowledgeOK
Industry average specification defectiveness metrics (from analysis of 15000+ requirementsfrom mature sectors of the industry on large specifications used for tender or development)
Vocabulary - Inconsistent, undefined
Missing conditional cases, events
Unstated preconditions for system Behaviour
Multiple possible interpretations
Inconsistent requirements
Over-specification
Requirements not independently verifiable
Interpretation dependent on subject matter expertise
8/12/2019 Raytheon (Powell) - Behaviour Trees
7/44Page
The Communication Problem
Have we got this right?
Can this be built and how longwill it take / how much will itcost?
Can we verify and validate this
when it is built?
About 46% of the pieces are missing, dont fit, belong to a different puzzle or require significant interpretation
(glue and scissors) to fit into the puzzle
Existing system and software engineering methods dont let us find these problems early enough - they donteven provide a way to assemble the puzzle!
That is, existing methods dont assist us in understanding the huge volume of information presented.
8/12/2019 Raytheon (Powell) - Behaviour Trees
8/44Page
Critical Success Factor
Achieved by:
Identification/removal of ambiguity,incompleteness and inconsistency inrequirements
Provision of human readable modelenabling understanding (and hencevalidation) of the integrated set of allrequirements.
Customer
8/12/2019 Raytheon (Powell) - Behaviour Trees
9/44Page
The Case for Behaviour Engineering
What is Behaviour Engineering?What is Behaviour Engineering?
The Behaviour Engineering ProcessThe Behaviour Modelling Language
Outcomes so far
Applications of Behaviour Engineering
8/12/2019 Raytheon (Powell) - Behaviour Trees
10/44Page
Customer
What is Behaviour Tree Analysis
A Discipline (Behaviour Engineering)
An Associated Modeling Language (BML)
Translate
Integrate
Resolve
Refine
8/12/2019 Raytheon (Powell) - Behaviour Trees
11/44Page
Behaviour Engineering
Used in industry to:
provide early life-cycle evaluation and correction of specifications; enable shared understanding of a specification;
explore concepts and architectures;
analyse models (FMECA, Reliability Analysis, ...); incrementally grow and validate specifications/scenarios/stories/use
cases;
incrementally refine, implement and test models.
8/12/2019 Raytheon (Powell) - Behaviour Trees
12/44Page
Behaviour Tree Analysis
Inherently Manages Complexity
Improves Project Outcomes
Reduces Risk
Customer
8/12/2019 Raytheon (Powell) - Behaviour Trees
13/44Page
The Behaviour Engineering Processfor Evaluating Requirements
8/12/2019 Raytheon (Powell) - Behaviour Trees
14/44Page
Step 1- Requirements Translation
Translate each requirement 1 at a time to a Behaviour tree.
Provides a means to manage complexity by localising complexity -can translate one requirement at a time Translation of natural language to a formal language (while trying to
maintain as much original language as possible)
Formal languages do not tolerate ambiguity Need to use common vocabulary during translation - produce anintegrated composition tree at same time as translation - aids inunderstanding
8/12/2019 Raytheon (Powell) - Behaviour Trees
15/44Page
Formalise One Requirement at a TimeRetain vocabulary of requirement
Step 1- Requirements Translation
8/12/2019 Raytheon (Powell) - Behaviour Trees
16/44Page
Order is not important BUT position where placed is Information about f is spread across THREE pieces Only see there are missing pieces when integrate pieces
Only see some pieces dont fit when integrate pieces
Step 2- Integration
8/12/2019 Raytheon (Powell) - Behaviour Trees
17/44Page
Integration EnablerThe Precondition Axiom
Each and every functionalrequirement expressed as aBehaviour tree, BTi, has associated
with it a precondition, Pi, that needsto be satisfied in order for theBehaviour encapsulated within it tobe exhibitable
8/12/2019 Raytheon (Powell) - Behaviour Trees
18/44Page
Why Integrate
When we can analyse the whole, emergent properties becomeapparent - we can reason about them.
Reviews/inspections of large specifications are ineffective at findingdefects that can only be found by understanding the wholespecification - the requirement on page 3 that is inconsistent withthe requirement on page 253; the business requirement that should
constrain the user story that was pasted on the wall 3 weeks ago.
Again, this is due to the limitations of human short-term memory
The tool supported integration of, and analysis of integratedBehaviour Engineering models facil itates finding these types ofissues.
8/12/2019 Raytheon (Powell) - Behaviour Trees
19/44Page
Example Behaviour Tree
8/12/2019 Raytheon (Powell) - Behaviour Trees
20/44
Page
Why BE Works
What differentiates Behaviour Trees from any other modellinglanguage
8/12/2019 Raytheon (Powell) - Behaviour Trees
21/44
Page
Why BE Works - Representation
7
+ 5
-----
This simple sum is easy to do in our head.For simple problems representation maters little.
This is harder to do in our head (for most of us). This is due to thelimitations of human short-term memory.
However, on paper, this is a trivial (although lengthy) sum - therepresentation allows us to localise complexity. We simply repeat aunit level operation that does not tax short term memory.
15476735386
+ 37932914987
+ 79109026544
+ 33987110011
+ 44488730928
-------------------------The Behaviour Modelling Language has this property
Contrast this to a representation that does not permit unit leveloperation
MMDCCLI
+ CCCLXIX
------------------
8/12/2019 Raytheon (Powell) - Behaviour Trees
22/44
Page
Why BE Works - Representation
The Behaviour Modelling Language supports the use, managementand resolution of the language used by the stakeholders (it even
supports managing aliases where different stakeholders usedifferent terms)
This helps to preserve intent
This enables validation by model walkthrough
The Behaviour Modelling Language is not large but is veryexpressive
8/12/2019 Raytheon (Powell) - Behaviour Trees
23/44
Page
Behaviour Modelling Language (BML)A Lesson from Chemistry
Ethane MoleculeRepresentations
8/12/2019 Raytheon (Powell) - Behaviour Trees
24/44
Page
Why BE Works - BE is Scalable
The basic unit-level operations of requirement translation andrequirement integration scale (900+ requirements, 200+ pages)
8/12/2019 Raytheon (Powell) - Behaviour Trees
25/44
Page
The Case for Behaviour Engineering What is Behaviour Engineering?
The Behaviour Engineering ProcessThe Behaviour Modelling Language
Outcomes so farOutcomes so far
Applications of Behaviour Engineering
8/12/2019 Raytheon (Powell) - Behaviour Trees
26/44
ROI on a
8/12/2019 Raytheon (Powell) - Behaviour Trees
27/44
Page
ROI on aLarge SW Development Project(cost data from an industry pro ject (non-Raytheon) applied to ave. size and defectiveness metrics)
Estimated Cost per Requirements Defect = $3K For defects not detected in traditional reviews and found at System Test
Based on the organisations own and accessible industry data
Number of requirements defects detected by BTA on Requirements = 307 These were major requirements defects found afterclose out of traditional review process That is, these issues would have been missed at the requirements stage
Cost of fixing Requirement Defects from BTA = $300 / requirement
Note from this data the cost of defect at System Test is 10 times the cost at Requirements
Some studies report 60-80 times!!!
Payback from BTA on Average Project = ($3000 - $300) x 307 = $828,900
Cost of Behaviour Engineering on Average Project = $30,000
ROI = ((828,900 - 30,000) / 30,000)*100 = 2663%
8/12/2019 Raytheon (Powell) - Behaviour Trees
28/44
Page
Results - Behaviour Engineering
Defect Detection Rate
2 to 3 times higher than the defect detection rates reported for scenario-based and checklist-based inspectionaccording to the most often cited data
8/12/2019 Raytheon (Powell) - Behaviour Trees
29/44
Page
CommunicationEngineering PM
Engineering Users (validation of requirements)
Engineering (SYS) other Engineering DisciplinesProject Customer
Across Integrated Product Teams
Common vocabulary through Composition Trees Real increase in understanding across the project
Relatively small investment for the return
Risk reduced to Project, Customer and Stakeholders
BE Other BenefitsImprovement in Behaviour?
8/12/2019 Raytheon (Powell) - Behaviour Trees
30/44
Page
Validations with the User / Capability.Cannot have requirements that are vague, unconnected etc
beyond Systems Requirements Review defects can be
contained early in the development life cycle Improve Estimating (System Engineering Test andAcceptance)
Changes (new / changed requirements or cannot achievespecification) fully analysed to determine the impact on theSystem
Non-domain experts can detect material errors inrequirements
low cost, high rate of detection
BE Other BenefitsImprovement in Behaviour?
8/12/2019 Raytheon (Powell) - Behaviour Trees
31/44
Page
The Case for Behaviour Engineering What is Behaviour Engineering?
The Behaviour Engineering Process
The Behaviour Modelling Language
Outcomes so far
Applications of Behaviour EngineeringApplications of Behaviour Engineering
8/12/2019 Raytheon (Powell) - Behaviour Trees
32/44
Page
Behaviour Engineering Centric Life Cycle
8/12/2019 Raytheon (Powell) - Behaviour Trees
33/44
Page
Representing Architecture Artefacts
8/12/2019 Raytheon (Powell) - Behaviour Trees
34/44
Page
Supporting FMEAProduce Models from Specifications / Design Descriptions
8/12/2019 Raytheon (Powell) - Behaviour Trees
35/44
Page
Supporting FMEAIntroduce Failure Into Models
Introduce Failure:Engine 1 Lost
8/12/2019 Raytheon (Powell) - Behaviour Trees
36/44
Page
Supporting FMEAAnalyse Consequences of Introduced Fai lure
Loss of Engine 1:Hydraulic system, Yellow and Blue Distribution
lines still providing pressure to hydraulic systemdirectly, If PTU operating, Green Distribution Lineproviding pressure through PTU from Yellow EDP
8/12/2019 Raytheon (Powell) - Behaviour Trees
37/44
Page
Supporting FMEAAnalyse Consequences of Introduced Fai lure
Loss of Engine 2:Hydraulic system, Green and Blue Distribution
lines still providing pressure to hydraulic systemdirectly, If PTU operating, Yellow Distribution Lineproviding pressure through PTU from Green EDP
8/12/2019 Raytheon (Powell) - Behaviour Trees
38/44
Page
Supporting FMEAAnalyse Consequences of Introduced Fai lure
Loss of Engine 1 and 2:Green and Yellow Distribution lines not providing
pressure to hydraulic systemPTU wont be operating
Blue Channel has pressure driven by RAT if flyingand doing > 100 kts
8/12/2019 Raytheon (Powell) - Behaviour Trees
39/44
Page
Supporting FMEAAnalyse Consequences of Introduced Fai lure
3 Independent Failures Example:Engine 1 Lost & PTU Failed & Blue Dist Line Cut
Hydraulic system able to control surfaces throughYellow Channel
8/12/2019 Raytheon (Powell) - Behaviour Trees
40/44
Page
Using Behaviour Engineering to Understand the Problem Space
Growing Requirements
Before
After
Initialdescrip
tionofp
roblem
Descrip
tionofPro
blemafte
r
queries
andiss
uesresolve
d.
Resolving Requirements Defects
8/12/2019 Raytheon (Powell) - Behaviour Trees
41/44
Page
Resolving Requirements DefectsCurrent Situation
Requirements Quality
Initial BTA ReviewRequirements Quality
Subsequent BTA Review
Have we hit a wall with our current methods?
Resolving Requirements Defects
8/12/2019 Raytheon (Powell) - Behaviour Trees
42/44
Page
Resolving Requirements DefectsBESituation
Requirements Quality
Initial BTA ReviewRequirements Quality
Subsequent BTA Review
Use BT to Resolve
Summary
8/12/2019 Raytheon (Powell) - Behaviour Trees
43/44
Page
SummaryBehaviour Engineering Benefits
Higher Yield than current early lifecycle defect detection...
...at 2-3 times defect detection rate.
Systematic and Scalable
Results in a model that facilitates shared understanding...
...and understanding of emergent properties
Results in a model that supports validationResults in a model that supports further engineering/development
effort
IS A KEY RISK MITIGATION STRATEGY
Significant saving throughout the development life cycle
8/12/2019 Raytheon (Powell) - Behaviour Trees
44/44
You cant do things differently,
Until you see things differently.Unknown