System-of-Systems Architecting: Critical Success Factors Dr. Azad M. Madni Chief Executive Officer...

System-of-Systems Architecting:Critical Success Factors

Dr. Azad M. MadniChief Executive Officer

USC-CSSE Convocation: Executive Workshop Presentation

October 23-26, 2006

3250 Ocean Park Blvd., Suite 100, Santa Monica, CA 90405310-581-5440 Fax: 310-581-5430 www.IntelSysTech.com

Copyright © 2006 Intelligent Systems Technology, Inc.

Copyright © 2006 Intelligent Systems Technology, Inc.Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.

Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.

Madni/2

Outline

• First Hand Experiences

• System-of-Systems (SoS) Definition

• SoS Architecture

• SoS Architecting Complexities

• Promising Approaches

• Implementation Options

• Challenges Ahead

• Critical Success Factors



Madni/3

First-Hand Experiences• Principal Investigator on DARPA, Army, Navy, Air Force, DoE,

NIST Research Initiatives– Ontology-enabled Agile Visualization of Multi-domain SoS data (sponsor: MDA)– Development of Toolset for DoDAF – compliant SoS Architecting (sponsor: U.S.

Army RDECOM)– Process-enabled Formation and Management of Agile Virtual Enterprises

(sponsor: DARPA DSO)– Virtual Prototyping of Combat Information Systems (sponsor: NSWCDD)– Integrated Product-Process Development (sponsor: NSWCDD, DARPA ISTO)– Development of Simulation-enabled Activity-Based Costing System (sponsor:

AFRL)– Development of ANSI-EIA-632 systems engineering process library (sponsor:

ONR, NSWC, DARPA ISTO)– Executable Models in Computer-aided Concurrent Engineering (sponsor: DARPA

DSO DICE) – Simulation-based Design and Analysis of Agile Manufacturing Systems (sponsor:

DoE TEAM Program)– Semantics of Process Description (sponsor: NIST PSL Program)– Design Process Management for complex products (sponsor: DARPA ESTO)



Madni/4

SoS Definitions Vary

• Arrangement of interdependent systems connected to provide a capability greater than the sum of the member systems.– GAO, January 2006

• A federation of systems, whose context, mission, and operational logic are required for handling the allocation and coordination of shared, integrated resources– Dr. Ray Paul, OASD/NII

• Several others



Madni/5

My Definition of SoS(Architecting Perspective)

• A large, complex, ensemble … … of interdependent systems

… developed and introduced over different time-frames

… by multiple independent authorities

… to provide multiple, interdependent capabilities

… in support of multiple missions

•An SoS capability typically exceeds the sum of the capabilities of the member systems

(Madni, 2006)



Madni/6

DoN Operational Forces

Systems-of-Systems support these forces.



Madni/7

Principal Supported MCP (determines Lead FA CHENG)

Secondary supported MCP (determines Supporting FA CHENGs)

IA Integrated Architecture MCP Military Capabilities Package NR-KPP Net-Ready Key Performance Parameter

Legal Portfolio

Sea Enterprise

System-Level

Sub-Functional Areas

Functional Areas

Enterprise

ISR/BA MCP C&N/I MCP Logistics MCP EMW IntelTAMD MCP

Strike MCP C2/DS MCP Force Deployment

MCP

EMW C2

NP21 Integrated Architecture

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from

NR-KPP)

System 5 IA (from

NR-KPP)

System 6 IA (from

NR-KPP)

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from

NR-KPP)

System 5 IA (from

NR-KPP)

System 6 IA (from

NR-KPP)

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from

NR-KPP)

System 5 IA (from

NR-KPP)

System 6 IA (from

NR-KPP)

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from

NR-KPP)

System 5 IA (from

NR-KPP)

System 6 IA (from

NR-KPP)

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from

NR-KPP)

System 5 IA (from

NR-KPP)

System 6 IA (from

NR-KPP)

System 1 IA (from

NR-KPP)

System 2 IA (from

NR-KPP)

System 3 IA (from

NR-KPP)

System 4 IA (from NR-

KPP)


KPP)


KPP)

Sea Strike IA EMW IA

FORCEnet IA

Sea Shield IA Sea Basing IA

Comms, Networking, and Services Infrastructure

Surface Warfare MCP

System-to-Capabilities Mapping

One system can serve in different SoSs to satisfy requirements of different military capability packages.



Madni/8

System Versus SoS

Comparison Criteria System System-of-System (SoS)

Focus

Lifetime

Governance

Boundaries

Membership

Information Flows

creation of the system

finite; can be extended

single, dominant oversight

well-defined

fixed; parent-offspring

well-understood, mostly internal to the system

creation of a capability

indefinite; continually extended through replacement/addition of member systems

multiple, overlapping stakeholders

continually changing as new systems become part of SoS, while others leave

dynamic; based on SoS capability requirements

changing; based on changes in information sharing requirements

(Madni, 2006)



Madni/9

System Versus SoS (cont’d)

Comparison Criteria System System-of-System (SoS)

Development

Complexity

Autonomy

Unintended Consequences

Biggest Challenge

COTS or custom-developed under control of system authority

designed out

ceded by the components/parts to the system

negligible; foreseen/tested and designed out

satisfaction of all functional and performance requirements subject to cost, schedule and “ility” constraints

systems procured from/developed by 3rd party organizations; some COTS

mostly ignored; imposed by dynamic mix of systems, emergent behavior

retained by member systems which contribute to SoS

high likelihood; changing boundaries; emergence

interoperability at the programmatic, constructive and operational levels

(Madni, 2006)



Madni/10

An Apt AnalogyCity SoS

• Collective action of multiple individuals acting locally over time

• City emerges and changes over time through loosely coordinated and regulated action of multiple individuals

• Exhibits coherence without central control--mechanisms that regulate local action

• Mechanisms include city regulations, communications, distribution, emergency services

• Built gradually in parts by people, companies, communities to serve their own purpose

• Grows and thrives based on cultural and economic necessities

• City is always in a state of perpetual changes (construction, repairs, mods, demolitions, operations)

• Collective action of multiple entities acting locally over time

• SoS behavior emerges and changes over time through loosely coordinated and regulated action of multiple systems

• Exhibits coherence without central control--mechanisms to regulate local action

• Mechanisms include acquisition policies, communications, distributed governances

• Built and introduced gradually by organizations to satisfy mission capability packages

• Grows and is sustained based on mission requirements

• SoS has dynamically changing boundaries (systems enter, leave, are modified, while operating)

Adapted from SEI ULS report, 2006



Madni/11

SoS Realities

• Different missions require different Mission Capability Packages (MCP)

• Different capability packages require different aggregations of systems, with the potential for some overlap

• Interoperability is key to realizing the desired capability– syntactic interoperability assures ability to exchange data– semantic interoperability assures ability to use and understand data

• Mission Capability Package can vary with locale and type of threat– urban environment versus jungle environment versus desert

environment– type of threat (e.g., NBC, IED,…)

• Operational documents not in step with current understanding• Acquisition process not in step with SoS interoperability

requirements



Madni/12

SoS Architecture

• A framework for structuring nodes that define the SoS– regardless of their source (i.e., COTS, NDI, custom, legacy)

• Captures scope of the nodes, and how they are intended to interact within the SoS

• A vehicle for:– incorporating existing/planned nodes into the SoS

– performing tradeoffs and making decisions

– communicating and discussing issues

– documenting SoS state/status



Madni/13

SoS Architecting Complexities

• SoSs are dynamic entities– systems are added, modified, or removed as mission requirements

change

– such changes are handled a priori or “on-the-fly”

• The integration infrastructure needs to change with– technological advances

– changes in Quality-of-Service (QoS) requirements

• These considerations imply that a SoS architecture needs to enable the evolution of the SoS and be evolvable itself

• Evolvability requires up front engineering because it is costly and difficult to introduce later



Madni/14

SoS Architecting Challenges

• Overcoming development friction– complex, independent-overlapping governance

• Getting all stakeholders to develop shared interests– multiple, independent concurrent developments, overlapping

governances• Maintaining coherence at SoS level

– independently planned programs pulling in different directions• Achieving robustness despite inability to perform critical

tradeoffs– complexity renders certain tradeoffs incalculable

• Maintaining interoperability (syntactic, semantic)– dynamically changing, uncertain information sharing

requirements



Madni/15

SoS Architecting Myopia

• Focus solely on technical issues at the expense of operational mission goals

– evident in the rush to migrate to service-oriented architectures (SoA)

• Ignore critical issues such as:– SoS architecture coverage relative to SoS mission requirements

– likely changes to mission and SoS ability to handle these changes

– mission-imposed QoS requirements and implications for SoS architecture

– most likely and most challenging operational scenarios and the ability of the SoS architectures to support them



Madni/16

SoS Architecture Tradeoffs (examples)

• Mission Coverage vs. Lifecycle Costs– how well does the SoS architecture satisfy mission requirements and

at what cost (lifecycle, recurring)

• Security vs. Performance– how does the SoS address security when dealing with heterogeneous,

remote nodes– do these measures come at the expense of reduced performance

• Functionality vs. QoS– granularity of node interface– risk of extraneous data or multiple trips to acquire data

• Standard vs. Proprietary Communication Mechanisms– do standard mechanisms adequately satisfy QoS requirements– do proprietary mechanisms preclude new nodes from joining the SoS



Madni/17

Promise of Executable Representations

• SoS architecture and design will necessarily evolve iteratively to respond to critical capability requirements

• Executable representations of the architecture produced/refined in each iteration can potentially: – clarify understanding and reduce development risks– provide key early insights into the behavior of the target SoS– explore sensitivities of SoS attributes to changes in mission

requirements (“Mission Capability Packages”) – validate operational scenarios using “what-if” perturbations– verify technical realizability of the architecture



Madni/18

SoS Architecture Implementation Options

• The three major implementation options are:– Service-oriented Architecture

– Grid Architecture

– Component-based Architecture

• Each has its advantages and disadvantages

• Technological maturity to implement each tends to vary



Madni/19

Challenges Ahead

• No guidance on how to architect a SoS– especially when goal(s) need to be created “on the fly”

• Creating a SoS on the fly involves dynamic discovery, composition, and invocation– therefore, the architecture may not be known until run-time…risky!

• Open questions– how to determine architecture of such a SoS? is it only determined at run time

or is there sufficient static information to determine the architecture a priori?– what is the potential impact of not knowing the architecture on the quality

attributes and performance of the SoS? is it possible to evaluate the desired quality attributes of such a system?

– is it possible to guarantee some level of service for a SoS under development?– how can a SoS that is put together at run-time distinguish between legitimate

and illegitimate service? … can this knowledge be used to guarantee an acceptable QoS level?



Madni/20

Critical Success Factors

• Do as much up-front engineering as possible– evolvability is costly and difficult to infuse later

• Avoid “SoS architecture myopia”– attend to mission capability requirements– map them onto SoS architecture nodes to assess coverage, and

to identify and fill gaps• Focus on mostly likely and most challenging operational

scenarios in defining architectural requirements• Define fundamental vocabulary, semantics, and models to

build and share best practices• Employ iterative process in SoS architecting to increase

understanding and reduce risks• Experiment with “guided” emergence by creating conditions

and policies that result in desired SoS capabilities



Madni/21

Critical Success Factors (cont’d)

• Employ executable representations of the architecture at the appropriate levels of abstraction in each iteration to: – clarify understanding and reduce risks in each iteration– acquire early initial insights into the behavior of the target SoS– explore sensitivities of critical SoS attributes to perturbations – validate operational scenarios– verify technical realizability of the architecture

• Simulate, analyze and resolve human-system integration issues arising from:– spikes in cognitive load– frequent context-switching on the part of humans

• Aid/Train humans to effectively function in scenarios where:– changes in collaborators and collaboration perspectives occur

frequently– context-switching occurs periodically– cognitive load spikes suddenly

System-of-Systems Architecting: Critical Success Factors Dr. Azad M. Madni Chief Executive Officer...

Documents

Transcript of System-of-Systems Architecting: Critical Success Factors Dr. Azad M. Madni Chief Executive Officer...