Post on 26-Apr-2018
Measuring Complexity of
Aerospace Systems
Shashank Tamaskar1 Kartavya Neema
2 Tatsuya Kotegawa
3 Daniel
Delaurentis4
School of Aeronautics and Astronautics
Purdue University, West Lafayette
stamaska@purdue.edu
In this paper, a framework for measuring system complexity is proposed, marrying structural
and functional representations of the system and addressing cross-domain interactions. We
identify seven different aspects of complexity in aerospace systems, namely, levels of
abstraction, type of representation, size, heterogeneity of components and interactions, network
topology factors such as coupling, modes of operation and off-design interactions. A system is
decomposed into different levels of abstraction and within each level the system is represented
using structural (network of components and interactions) and functional graphs (network of
functions). Networks provide a natural framework for system representation and facilitate use
of topological metrics such as size, coupling and modularity. Synthetic examples are developed
to study how the different aspects of network complexity aggregate and the relationship
between structural and functional graphs. The authors investigate the relationship between
different levels of abstraction. We also study how cross-domain interactions, modes of
operation and off-design interactions affect complexity. We also generate some preliminary
results to show the relevance of the proposed framework by applying it to a spacecraft design
problem and highlight some of its advantages and disadvantages.
1 Introduction
Over the past several decades, many industries have seen a significant improvement in the performance and capability of their products. One such example is the fifth generation F-35 tactical fighter which offers roughly three to eight times the operational capability of a fourth generation aircraft such as the F-16 or F-18. This superior capability comes at the cost of a substantial increase in technical complexity. For example, the F-35 has roughly 130 subsystems, ~10
5 interfaces and 90% of its
903
functions managed by software. In comparison, the F-16 has 15 subsystems, ~103
interfaces and 40% of its function managed by software (Arena, 2008). Automotive and electronics industry reflect similar trend as well (Eremenko, 2009). The increase in technical complexity has resulted in greater time being spent on design, verification and validation of these systems. Another important consequence of increasing complexity is the appearance of emergent behaviors, which are not apparent during the design and development phase but become visible during system validation or operation. These potentially undesirable behaviors, can lead to expensive redesign and result in cost and schedule overruns (Eremenko, 2009). An analysis conducted by the US Government Accountability Office (GAO) of major defense acquisition program found that in the recent years, the research and development costs are on average 42 % higher than what was originally estimated and the average time required in delivering the initial capability has increased by 22 months (Assessments of Selected Weapon Programs, March 2009). Thus, complexity not only leads to higher cost and effort for a system but also decreases our ability to predict its development cost and schedule accurately.
A brief literature survey reveals that numerous approaches for measuring
complexity have been proposed. These techniques address one or more aspects of this
multifaceted problem and can be classified into four different categories-
1. Metrics based on software complexity metrics
2. Metrics based on information entropy
3. Network theoretic metrics
4. Traditional/ Empirical metrics
Many metrics to measure the complexity of software architectures are available
from the realm of computer science. Software architectures define the flow of
information and control between the different modules of the software. Flow of
materials in the field of manufacturing is similar to the flow of information within the
software. Consequently, several metrics, which measure the complexity of
manufacturing plans, are derived from the field of computer science (Phukan, 2005).
Further, some software complexity metrics are used for measuring complexity of
manufacturing processes (Ahn, 1994). Models of many manufacturing processes
closely resemble the graphical representations of software (directed flow-graph). The
software complexity metrics employed include cyclomatic, nesting level, program
complexity, and cyclomatic flow. Cyclomatic complexity measures the difference
between the number of the program statements (design tasks) and the number of the
control flows (connections between the design tasks) (Oviendo, 1980). Nesting level
complexity focuses on the depth (distance) the design actor is from the final solution
and is found by summing the scope of each of the design actors, providing an insight
into the depth to which each actor has an influence in the entire process network
(Harrison, 1981). Program complexity measures the coupling between the design
actors with respect to the number of times that a declaration is used as an input or
definition for another actor, thus measuring the functional relations between the data
elements involved in the solution process (Stetter, 1984). The cyclomatic flow
904
complexity measures the complexity of the program or process by combining the
control flow and data relations into a single metric.
The information axiom of Suh’s axiomatic design process states that a good
design is one, which has minimal information content (functional requirements and
design parameters). In this context, complexity is defined as the measure that relates
the desired objective against what is known and unknown about the system (Suh,
2001). This forms the basis of application of information-based approaches for
measuring the complexity of the design. While some of the methods suggest that
complexity is fundamentally a characteristic of the information content within a
design, others try to incorporate the designer’s lack of understanding while measuring
the complexity of system (Braha, 1996). Haik and Yang further extend the idea of
information entropy as a measure of system complexity to highlight some of the
components of system complexity of design process such as variability, vulnerability
and correlation (Yang, 1999). Braha and Maimon describe the measurement of
structural and functional aspects of design complexity (Maimon, 1998). Gell-Mann
and Lloyd provide information measures for measuring the effective complexity and
total information content of the system (Lloyd, 1996). Hornby adopts a similar
approach to measure modularity, reuse and hierarchy within the system (Hornby,
2007). While the information-based methods are good for measuring the size aspect
of system complexity, they do not provide an intuitive understanding of the
complexity due the topology of interactions.
Network theoretic approaches are useful for measuring coupling aspect of system
complexity. In these techniques, we represent a complex system as a network of
nodes and links with nodes representing the components and links representing
interactions between the components. Complexity values are assigned to nodes and
links and a roll-up operation is performed which generates a single number
representing the complexity of the entire network. Different authors adopt different
approaches for the roll-up operation. Most recent among this effort is by Zeidner,
Reeve and Khire from United Technologies Research Center (UTRC) who describe a
metric to capture the size, heterogeneity and connectivity aspects of system
complexity. UTRC has also proposed different approaches for roll-up operation based
on graph energy and spectral graph partitioning (Abstraction based complexity
management, 2010). Barabasi and Ravasz have used the network approach to
describe the characteristics of networks having hierarchy and modularity (Barabasi,
2003). DeLaurentis and Ayyalasomayajula (DeLaurentis, 2009)
looked at
complexity's role in system-of-systems engineering for sustainable air transportation,
using network topologies as a source of practical information for complexity metrics,
yet a generic metric was not put forward. One of the shortcomings of the network
theoretic metrics is the lack of a consistent scheme for system representation.
Apart from these more formal developments, many traditional design examples
exist where the complexity metrics is qualitatively and subjectively estimated. These
metrics typically measure the coupling between the performance parameters and
design variables. The most notable in this regard is the work of Bearden (Bearden,
2003). This metric is based on empirical data collected from small satellites
developed between 1990 and 1999. The approach used by the author is the following:
905
Identify the parameters that drive or otherwise contribute to a spacecraft design,
quantify the identified parameters; and combine the parameters into an aggregate
complexity index. While the author has confined himself to space domain the same
basic approach can be applied to other domains as well. A limitation of the
classification is its failure to address the design process, which might include the
number, type, and repeatability of tasks. Furthermore, the complexity of the
individual tasks (process elements) was not addressed. In addition, this might not
work for designs that are radically different from conventional designs.
The literature survey reveals the lack of a comprehensive metric, which measures
the complexity of a design. Recently there has been some effort to unify these
numerous complexity metrics. The efforts of Shah and Summers (Jami J. Shah, 2010)
are notable in this regard. The authors have proposed three measures of design
complexity: solvability, size and coupling. The authors suggest that these measures
capture different aspects of design and hence should be kept independent of each
other. Metrics are developed to capture problem, process and product aspects of the
design. However, the analysis assumes a binary un-weighted network for calculating
coupling and assumes the independence of size, coupling and solvability.
This paper represents another small step in unifying different aspects of
complexity. Seven different aspects of complexity in aerospace systems are identified
namely, levels of abstraction, type of representation, size, heterogeneity of
components and interactions, network topology factors such as coupling, modularity,
modes of operation and off-design interactions. A framework for measuring system
complexity that accounts for these aspects is proposed. Some of the unique aspects
about this framework are its focus on marriage of structural and functional
representations of the system and a unified treatment of the cross-domain
interactions. A system is decomposed into different levels of abstraction and within
each level; the system is represented using structural (network of components and
interactions) and functional graphs (network of functions). Networks provide a
natural framework for system representation and facilitate use of network topology
metrics to capture size and coupling aspects of complexity. Synthetic examples are
developed to study how the different aspects of network complexity aggregate and
the relationship between structural and functional graphs. The authors investigate the
relationship between different levels of abstraction and study how cross-domain
interactions, modes of operation and off-design interactions affect complexity. We
also generate some preliminary results to show the relevance of the proposed
framework by applying it to a spacecraft design problem and highlight some of its
advantages and disadvantages.
2 Approach
2.1 Different Aspects of System Complexity
Based on the literature survey described in the previous section, it can be concluded
that some of the important aspects that govern system complexity, are-
Level of Abstraction
System Representation: Structure/Function
Size which is indicative of number of components and interactions
906
Heterogeneity of components and interactions
Network Topology
Dynamics involving different modes of operation
Off-design interactions: Off-design interactions are the interactions, which
occur due the operation of components, and interactions outside their design
range. They are different from un-modeled interactions (interactions that
cannot be modeled) which occur due to emergent behavior within the
system.
Figure 1: Different aspects of System Complexity
Figure 1 is a pictorial representation of the different aspects of system complexity,
which are considered in the proposed framework.
2.2 Proposed framework for measuring system complexity
Based on consideration of the aforementioned aspects of system complexity our
framework is described below-
1. Representing the system at appropriate level of abstraction: The choice of the
level of abstraction depends on the fidelity of analysis specified by the designer.
In the component level of abstraction, the functions in functional representation
of the system are assumed to have unit complexity. Complexity of components
and interactions at the higher level of abstraction are a result of aggregation of
complexity at the lower level of abstraction. Complexity responsible for cost
and schedule overruns of modern aerospace system is attributed to the complex
interactions between the simple components and functions, which give rise to
emergent behaviors.
2. Generating the structural and functional system representation: At each level of
abstraction, the system is represented via structural and functional graphs. A
CAD model would likely represent the lowest level of structural representation.
907
The functional graph represents the relationship between the system functions.
Fig. 2 describes the structural and functional graphs for an aircraft at the
subsystem level of abstraction.
3. Mapping the functional representation onto the structural graph: In this step,
each function within the functional graph is mapped to components and
interactions in the structural graph. The complexity of components and
interactions in the structural graph is defined by the number and complexity of
functions associated to them. In case, groups of components perform a function,
the complexity associated with this function is divided uniformly between the
components. In a similar fashion, the complexity associated with off-design
interactions is also mapped on to the structural graph. The result of this
mapping is a weighted network where the nodes represent the components and
the links represent the interactions within the system.
Figure 2: Marriage of Structural and Functional Representations for an aircraft represented in
the subsystem level of abstraction
908
4. Calculate the complexity of the weighted network: After generating the
weighted network in step 3, system can be analyzed using a variety of metrics,
which capture different aspects of complexity of networks. Some of the
aspects that are captured in our framework are size and coupling. Coupling
complexity is the product of the topology of interactions within the system.
Summers and Shah (Jami J. Shah, 2010) have proposed an algorithm for
measuring coupling by testing the decomposability of an entity relation graph.
One of the disadvantages of their approach is that it assumes a binary,
undirected graph for calculating coupling, which is a restrictive assumption for
analysis of engineering systems. In addition, it ignores the presence of
feedback loops or cycles, which play an important role in determining the
system complexity. Our proposed approach for measuring coupling
complexity accounts for directivity and weights within a network. It also tests
for the presence of feedback looks within the system. Another key feature of
the proposed metric for coupling is its ability to measure coupling between
nodes, which are not directly connected. Sometimes, design changes in one
part of the system often affect other parts of the system not directly connected
with it. Our proposed metric apart from accounting for direct connectivity
between the components also accounts for all the indirect interactions, which
are associated with a particular link. The metric for coupling is as follows:-
1. Represent the system using directed and weighted structural graph of the
system.
2. To capture coupling between indirectly connected nodes, we propose to
calculate how frequently each links of the network are used while finding
all the paths to connects all the pairs of the nodes in the network. Any
indirectly connected node of the network can only interact using the paths
laid between them. This step will ensure higher weightage to links that are
used larger number of times and thus capture coupling between indirectly
connected nodes.
3. Define refined weights of the system by multiplying the weights of the
network by the frequency of each link calculated in step 2.
4. Calculate coupling complexity using the following formula.
𝐶𝑜𝑢𝑝𝑙𝑖𝑛𝑔 𝐶𝑜𝑚𝑝𝑙𝑒𝑥𝑖𝑡𝑦 = 𝑗 𝑊𝑖 + 𝑊𝑘𝑚𝑘=1
𝑛𝑖=1
𝑐𝑗=1 (1)
Where: c and j denotes the number of cycle and size of the cycle
respectively. Wi denotes the weights of the links of the cycle. m is the
number of links which are not the part of any cycle. Wk denotes the
weights of those links that are not the part of any cycle.
Structural graph of an artificial network is shown in figure 3 (a). We are trying to
calculate the coupling complexity of the network. Therefore, we calculate the
frequency of each links being used when finding all the paths to connects all the
pair of nodes. Frequencies of the links are shown in red color in figure 3 (b).
909
Using the equation given in step 4, the coupling complexity of the network
comes out to be 171.
Figure 3: (a) Artificial Network (b) Frequency of the links
Size is an excellent measure for providing a rough measure for system
complexity. A metric for measuring size complexity is described in Shah and
Summers (Shah, February 2010). It is based on the information content of the
system, modeled in a particular representation. The information content of a
system is related to size of the vocabulary (λ) necessary to describe the system
and the number of language instances (Λ). The number of unique components
and relationships is a measure of the size of the vocabulary of the system, while
the total number of components and interactions gives the number of language
instances. The complexity of the system is defined as-
C=Λln (λ) (2)
However, since the metric for coupling proposed above already accounts for number and heterogeneity of interactions hence these are omitted from (2) to avoid double counting.
5. Aggregate different aspects of complexity into a single measure for system
complexity: After computing the size and coupling of the weighted network, they
are combined to produce a single number, which represents an overall
complexity score for the system. For the sake of simplicity, it is assumed that the
overall system complexity is a weighted sum of size and coupling aspects. An
important aspect to note is that aggregating size and coupling aspects into a
single numeric score is important as it facilitates easy comparison between two
designs. This aggregation however, can lead to loss of valuable insights about the
sources of complexity in a design. We are investigating a potential approach for
developing a heat map, which can be used to identify the regions of high and low
complexity in a design.
3 Analysis
In this section, some examples, which illustrate some of the aspects of the framework
mentioned in the previous section, are described. Fig. 3 shows synthetic examples,
(a) (b)
910
which have been created to demonstrate the effectiveness of the size and coupling
metrics. For the ease of demonstration, nodes and link are assumed to be of unit
complexity.
Figure 3: Synthetic Examples
Table 1 describes the complexity scores for each of the networks shown in Fig. 3.
Increasing the number of nodes will result in increase in size and total complexity.
Another important observation is that as the networks with feedback have
significantly higher coupling than those without feedback. In addition, complexity
significantly increases with increase in number of components within a feedback
cycle. Thus, network B has significantly higher coupling than networks E or F. This
is expected, as increasing the number of components in a feedback path makes it
increasingly difficult to accurately predict the behavior of the system.
Table 1: Calculating complexity of synthetic examples
Networks Size Complexity Coupling Complexity Overall Complexity
A 8.04 4 12.04
B 13.62 1029 1042.62
C 13.62 56 69.62
D 13.62 64 77.62
E 13.62 150 163.52
F 13.62 208 221.62
While the synthetic examples demonstrate the reasonableness of the metrics and the
proposed scheme for the aggregation of different aspects of complexity, a real
aerospace design example is required to demonstrate some of the merits and demerits
of the proposed framework. Fig. 4, 5 shows a conceptual sketch of two satellites
which have been used to illustrate the framework. As described in the previous
section, any complexity analysis begins with choosing an appropriate level of
911
abstraction. For this problem, we have chosen to analyze the system in the component
level of abstraction. The choice of component level of abstraction is dictated by the
fact that both satellites look almost the same at subsystem level of abstraction. Fig. 6
shows the unified system representation for satellite B, which is obtained after
mapping the functional graph on the structural graph. Since the analysis is carried out
at the component level, hence the weights of the complexity of the components and
interactions can be assumed to be one.
Figure 4: Satellite A
Figure 5: Satellite B
912
Fig. 6 shows that our analysis considers four different types of interactions, namely-
matter, energy, force and information. For sake of uniformity, all these interactions
have been assumed to be of unit complexity.
Figure 6: Unified system representation for satellite B (component level)
Table 2: Complexity of the Satellite Example
Satellite Size Complexity Coupling Complexity Overall Complexity
A 184.2 349 533.2
B 212.0 584 796.1
Table 2 shows the complexity calculations for satellites A and B. From this
analysis, it can be seen that satellite B is more complex because not only it has more
components, but also because it has significantly higher coupling than satellite A.
Table 2 also describes complexity of individual modules. This analysis is important
as it helps us identify the distribution of complexity across different subsystems.
For large networks, which are observed in most modern aerospace systems, it is
difficult to draw inferences by visualizing the interactions between the components.
By color coding the components as shown in fig. 6, it can be easily observed that the
system has a hub and spoke architecture for data handling and storage. All the
components act as sources or sinks of information.
913
3 Conclusions
In this paper, a comprehensive framework for measuring system complexity is
proposed. Some of the salient features of the proposed framework are:
Greater components/interactions should imply more complexity
Heterogeneity of components/interactions should result in greater system
complexity
Network topology, which is a measure of coupling within a system, should
be related to the complexity of the system
Complexity of a system should depend on the level of abstraction
The metric should account for coupling between different subsystems
The metric should account for un-modeled interactions happening within the
system
One of the unique aspects of this framework is its intuitive mapping of functional
graph on the structural to generate a unified system representation. This weighted
network is then analyzed to evaluate the size and coupling aspects of complexity,
which are aggregated to calculate the overall complexity score. Synthetic examples
are described to demonstrate the effectiveness of coupling and size metrics and a
satellite example is presented to help us evaluate the merits and demerits of the
framework. Future work will focus on incorporating the effect of uncertainty in the
framework.
Bibliography
[1] Abstraction based complexity management. (2010).
[2] Ahn, J. a. (1994). Complexity Analysis of Computational Engineering. 68.
[3] Arena, M. V. (2008). Why Has the Cost of Fixed Wing Aircraft Risen. RAND
Corporation.
[4] Assessments of Selected Weapon Programs. United States Government
Accountability Office. (March 2009).
[5] Balazs, M. a. (2002). Design Simplification by Analogical Reasoning. From
Knowledge Intensive CAD to Knowledge Intensive Engineering (pp. 29-44). U.
Cugini and M. Wozny, eds., Kluwer.
[6] Bashir, H. a. (2001). Models for Estimating Design Effort and Time. Des. Stud ,
Volume 22 issue 2 pages 141-155.
[7] Bearden, D. A. (2003). A complexity based risk assessment of low-cost planetary
missions: when is the mission too fast and too cheap. 52 (2-6).
[8] Braha, D. a. (1996). On the Complexity of the Design Synthesis Problem. 28.
[9] DeLaurentis, D. A. (2009). Exploring the Synergy between Industrial Ecology
and System-of-Systems to Understand Complexity: A Case Study in Air
Transportation. 13 (2).
[10] Dixon, J. D. (1988). A Proposed Taxonomy of Mechanical Design Problems.
Computers in Engineering Conference, (pp. 41-46). San Francisco, CA.
[11] El-Haik, B. a. (1999). The Components of Complexity in Engineering Design.
IIE Trans , 925-934.
914
[12] Eremenko, P. (2009). META, Novel Methods for Design & Verification of
Complex Systems. DARPA.
[13] Fitzhorn, P. (1994). Engineering Design as a Computable Function. Artif. Intell.
Eng. Des. Anal. Manuf , Volume 8 page 35-44.
[14] Harrison, W. a. (1981). A Complexity Measure Based on Nesting Level. 16.
[15] Hornby, G. S. (2007). Modularity, reuse, and hierarchy: Measuring complexity
by measuring structure and organization. 13 (2).
[16] Jami J. Shah, J. D. (2010). Mechniacal Engineering Design Complexity: Size,
Coupling and Solvability. 132.
[17] Kolmogorov, A. (1983). Combinatorial Foundations of Information Theory and
the Calculus of Probabilities. Russ. Math. Surveys, , 29-40.
[18] Lloyd, M. G.-M. (1996). Information Measures, Effective Complexity and Total
Information. 2 (1).
[19] Maimon, D. B. (1998). The Measurement of a Design Structural and Functional
Complexity. 28 (4).
[20] Oviendo, E. (1980). Control Flow, Data Flow and Program Complexity.
Chicago,IL: Proceedings of the COMPSAC80.
[21] Pahl, G. B. (2007). Engineering Design: A Systematic Approach. London:
Springer-Verlag.
[22] Phukan, A. (2005). Complexity metrics for manufacturing control architectures
based on software and information flow. 49 (1).
[23] Sedgewick, R. (1998). Algorithms in C++. Menlo Park, CA: Addison-Wesley,.
[24] Simon, H. (1998). The Sciences of the Artificial. Cambridge, MA: MIT Press.
[25] Stetter, F. (1984). A Measure of Program Complexity. 9.
[26] Suh, N. (2001). Axiomatic Design: Advances and Applications. NewYork, NY:
Oxford University Press.
[27] Varma, D. a. (1990). On the Estimation of Logic Complexity for Design
Automation Applications. International Conference on Computer Design: VLSI in
Computers and Processors. Cambridge,MA: IEEE.
[28] Yang, B. E.-H. (1999). The components of complexity in engineering design. 31
(925-934).
915