Final Report of the Model Based Engineering (MBE) Subcommittee

Final Report of the

Model Based Engineering (MBE)

Subcommittee

NDIA Systems Engineering Division

M&S Committee

10 February 2011

Final Report of the MBE Subcommittee

10 February 2011 Page i

Table of Contents

Executive Summary ......................................................................................................................... 1

Background ..................................................................................................................................... 4

Genesis of the MBE Subcommittee ............................................................................................ 5

MBE Subcommittee Charter ....................................................................................................... 6

MBE Subcommittee Membership ............................................................................................... 7

MBE Definition ............................................................................................................................ 9

Characteristics of Models Used in MBE .................................................................................... 11

Potential MBE Benefits, Costs, Risks ............................................................................................. 13

High Level MBE Benefits ........................................................................................................... 14

MBE Benefits Across the Acquisition Life Cycle ........................................................................ 16

Virtual Integration to Manage Risk Throughout the Life Cycle ................................................ 18

Risk and Cost Reduction Through Earlier Verification .............................................................. 20

Potential MBE Costs and Risks .................................................................................................. 21

Objective MBE Framework ........................................................................................................... 23

MBE Current State .................................................................................................................... 24

MBE To Be State ........................................................................................................................ 26

Primary Gaps That Must Be Closed .......................................................................................... 28

Policy, Guidance and Contracting Mechanism Impediments and Issues ..................................... 31

Preliminary Policy Findings ....................................................................................................... 32

Required Policy and Regulations to Enable MBE ...................................................................... 33

Recommendations ........................................................................................................................ 39

Collaboratively Work With Stakeholders to Develop the MBE Business Model ...................... 40

Work with Industry, Vendors and Standards Organizations to Develop MBE Standards ........ 44

Develop the Workforce ............................................................................................................. 47

MBE Roadmap ........................................................................................................................... 50

Conclusions and References ......................................................................................................... 53

Conclusions ............................................................................................................................... 54

References and Literature Surveyed ........................................................................................ 56


10 February 2011 Page 1

Executive Summary

Purpose

Numerous recent studies and reports [3, 7, 8, 9, 10] have documented the increasing

complexity of defense systems. The growth in complexity has increased risk and development

time to the point where the time to field new systems and evolve existing systems is not

acceptable. In addition, defense systems are operating in a rapidly changing environment, and

the ability of systems to respond to those changes requires higher degrees of system

adaptability. Traditional systems engineering methods, processes, and tools need significant

improvement to meet the challenges posed by the increasing system complexity trend [10] , as

do traditional software engineering methods [3]. Additionally, increasing complexity arising

from the interdependence of large numbers of component suppliers is posing integration

problems that challenge the limits of tradtional approaches [1].

Model Based Engineering (MBE) is an emerging approach to engineering that holds great

promise for addressing the increasing complexity of systems, and systems of systems, while

reducing the time, cost, and risk to develop, deliver, and evolve these systems. The purpose of

this study was to assess the current state of MBE, identify the potential benefits, costs, and

risks of MBE within the context of the DOD acquisition life cycle, and provide recommendations

that would enable the widespread adoption of MBE practices across the DOD acquisition life

cycle.

Findings and General Recommendations

MBE, as defined by the subcommittee, is an approach to engineering in which models:

Are an integral part of the technical baseline,

Evolve throughout the acquisition life cycle,

Are integrated across all program disciplines (e.g., systems engineering, operations

analysis, software engineering, hardware enginnering, manufacturing, logistics, etc.),

and

Can be shared and/or reused across acquisition programs, including between

Government and Industry stakeholders.

As described within the report, this would enable significant benefits across all phases of the

acquisition life cycle resulting in accelerated development and delivery of capabilities to the

Warfighter that are more timely, reliable, interoperable, and affordable.



The subcommittee found that the current state of practice, methods, processes, and tools has

significant gaps that must be closed to enable the widespread adoption of MBE. We found that

modeling is currently used extensively across multiple domains to support engineering a

product, system, or capability, but that there is a wide variation in the maturity of practice

across engineering disciplines. While there is wide use of models across many domains, they

are used in a stove-piped fashion and are not well integrated from one acquisition phase to

another. The lack of integration also occurs across programs, as there is limited formal model

reuse from one program to another, and a lack of infrastructure to support efficient reuse. The

subcommitte also noted that there is growing interest in MBE as evident in many activities

across industry, academia and standards bodies to help move the state of practice forward.

These activities must be leveraged to enable widespread adoption of MBE.

Transitioning from the current state of practice to an MBE approach will be complex and must

be addressed on multiple fronts. The transition can only be successful if approached in a

collaborative manner with the involvement of the Government, Industry, tool vendors,

standards organizations, and academia. The subcommittee developed a broad set of

recommendations that are organized into three focus areas. First, we recommend that the

Government work collaboratively with Industry and other stakeholders to develop the MBE

Business Model. While many agree with the compelling vision of MBE, there is a lack of

quantitative data to support a compelling the Business Model and move it beyond academics.

This recommendation includes:

Defining the data required to generate the business case / value proposition required by

each stakeholder;

Launching a small number of model based contracting pilot projects;

Conducting a “Grand Challenge” like project to accelerate the cross-discipline end-to-

end MBE implementation;

Implementing a model registry concept;

Using operational models to facilitate capability integration across programs; and

Developing sensitive information protection guidelines.

Second, the subcommittee recommends that the Government work with Industry, tool

vendors, and standards organizations to collaboratively develop MBE standards. The cross-

discipline, cross-acquisition life cycle phase, and cross-program collaboration and reuse that are

at the heart of MBE can only be achieved with a robust set of standards for meta data, model

interconnection and interchange, domain specific modeling languages, and other technologies

common to MBE stakeholders. This recommendation includes:



Developing an MBE Common Reference Model;

Developing an MBE standards roadmap;

Initiating a research program to close high-priority technical gaps;

Developing the standards identified in the standards roadmap;

Providing seed funding for the development of reference implementations of select

MBE standards; and

Developing MBE program planning guidance.

Finally, the subcommittee recommends that the Govement, Industry, and tool vendors develop

the workforce. The MBE approach represents a shift away from individual disciplines working

in isolation, and towards collaboration across the disciplines. This will require training and

education that informs the workforce as to the roles played by all disciplines in each phase of

the acquisition life cycle, and how their discipline interacts with the other disciplines. As MBE is

an emerging approach, there is a dearth of “greybeards” and best practices which less

expereinced staff can draw from. Overcoming this shortfall will require the formalization of

MBE-specific mentoring programs and the means to capture and share knowledge across the

MBE community.

The subcommittee believes that the implementation of the recommendations contained within

this report will enable MBE to become common practice across the life cyle for the acquisition

and evolution of systems and solutions. While the implementation of the recommendations

will be neither quick, nor without investment, we believe that the broad acceptance and use of

MBE will enable the DOD, Industry, and the Warfighter to realize the benefits of accelerated

delivery of capabilities that are more reliable, interoperable, and affordable.



Background

The following subsections provide background for this report. The genesis of the MBE

Subcommittee is discussed, followed by the subcommittee’s charter and membership. Our

definition of MBE is presented along with characteristics of models used in MBE.



Genesis of the MBE Subcommittee

During the NDIA Systems Engineering Division’s Strategic Planning Meeting in December 2009,

The OSD Director, Systems Engineering (now DASD/SE) identified numerous challenges. Among

these were:

• Identifying opportunities to leverage Model-based engineering practices to improve

systems engineering productivity and completeness. Included in this challenge was

developing an understanding of whether existing DoD and Service policies, guidance,

and contracting mechanisms were supportive, or hindered, model-based engineering

and model-based collaboration.

• Reinvigorating exploration and exploitation of M&S Systems Engineering enablers to

identify, assess, and mitigate acquisition program risks.

The NDIA Systems Engineering Division determined that these two challenges are important to

the practice of Systems Engineering and that they should be investigated. The investigation

was assigned to the System Engineering Division’s Modeling & Simulation (M&S) Committee.

During the M&S Committee’s April 2010 meeting, a Model Based Engineering Subcommittee

was formed to conduct the investigation and report findings and recommendations.



MBE Subcommittee Charter

The first activity for the MBE Subcommittee was to develop a charter to specify the scope of

the investigation and the information and recommendations that would be developed by the

subcommittee. After completion of an initial version of the charter, a telecon was conducted

with the leadership of the Systems Analysis organization within the Director, Systems

Engineering (now DASD/SE) Systems Engineering Directorate to review the charter. Minor

changes were made to the initial charter as a result of the telecon, and the final version of the

MBE Subcommittee charter is shown above.



MBE Subcommittee Membership

The subcommittee members are identified above. The organization each member is affiliated

with is shown in parenthesis. Highlighted in blue, are organizations that individual members

belong to that are engaged in model-based initiatives.

The Simulation Interoperability Standards Organization (SISO) is an international organization

dedicated to the promotion of modeling and simulation interoperability and reuse for the

benefit of a broad range of M&S communities. MBE related activities at SISO include the Core

Manufacturing Simulation Data (CMSD) product that defines a data interface specification for

efficient exchange of manufacturing life cycle data in a simulation environment, and the System

Life Cycle (SLC) forum that focuses on M&S and related enablers of integrated, collaborative

enterprises for system/vehicle or weapon system product development, particularly from a life-

cycle wide, mission capability/system-of-systems perspective.

The International Council on Systems Engineering (INCOSE) is a not-for-profit membership

organization founded to develop and disseminate the interdisciplinary principles and practices



that enable the realization of successful systems. INCOSE’s mission is to share, promote and

advance the best of systems engineering from across the globe for the benefit of humanity and

the planet. INCOSE established a Model-Based Systems Engineering (MBSE) Initiative in 2007,

and began conducting MBSE Workshops in 2008. MBSE is part of INCOSE’s overall SE Vision

2020.

The Aerospace Vehicle Systems Institute (AVSI) is a cooperative of aerospace companies, the

Department of Defense, Federal Aviation Administration and NASA working together to

improve the integration of complex subsystems in aircraft. AVSI addresses issues that impact

the aerospace community through international cooperative research and collaboration

conducted by industry, government and academia. One AVSI project, the System Architecture

Virtual Integration (SAVI), is addressing virtual system integration.

The Network Centric Operations Industry Consortium (NCOIC) is an international organization

for accelerating the global implementation of network centric principles and systems to

improve information sharing among various communities of interest for the betterment of their

productivity, interactivity, safety, and security. NCOIC has an M&S Functional Team that is

developing net-centric patterns for improving the interoperability of models across company

and national boundaries. It recently approved a capability pattern for an integrated middleware

environment for live, virtual, and constructive models and simulations. Other patterns are in

development, for example, one on using M&S to improve C3I capabilities, and using M&S to

support T&E of complex net-centric systems of systems.

PDES, Inc. is an international industry/government consortium committed to accelerating the

development and implementation of standards that enable enterprise integration and PLM

interoperability for its member companies. Its members represent leading manufacturers,

national governmental agencies, software vendors and research organizations. PDES, Inc.

supports the Digital Enterprise through the development and implementation of information

standards to support Model-based Engineering, Model-based Manufacturing, and Model-based

sustainment. Testing of implementations and data exchange using the ISO 10303 standard is an

integral part of PDES, Inc.



MBE Definition

The subcommittee defines Model-Based Engineering (MBE) as an approach to engineering that

uses models as an integral part of the technical baseline that includes the requirements,

analysis, design, implementation, and verification of a capability, system, and/or product

throughout the acquisition life cycle.

The subcommittee recognizes and acknowledges that MBE has been defined differently by

DASD(R&E) in its Systems 2020 initiative (“Application of modeling and simulation throughout

the development process to foster more effective concept engineering and concurrent design,

development, manufacturing, deployment and evolution.” *9+) and by BAH in their Systems-

2020 Study (“Model-based engineering (MBE) is defined as leveraging modeling and simulation

techniques to deal with the increasing complexity of systems. This approach facilitates the

interaction of the different domains encountered in the concept creation-design-manufacturing

cycle. Models can assist with all aspects of the complex system life cycle, from the interaction

of stakeholders in an easy to use environment, to enabling the automatic interaction of sub-



modules at different physical scales, to facilitating the manufacturing process as a network of

services.” *8+).

We identified several preferred MBE practices, and note that there is tension among the

preferred MBE practices:

• Models that are scoped and/or appropriate to a specific context may not need to

be interoperable across domains and/or the full product life cycle. Example: a

developmental version of a component may only need to operate in a particular

IT environment for a limited time. One of the challenges to MBE will be to

determine the extent to which models have to be maintained and updated over

time and across platforms. This preferred practice cannot apply to everything

that satisfied the broad definition for “model” quoted above.

• The statement “Models represent the technical baseline to customers…” has

several very different connotations, depending on who (Government or Industry)

developed the model in question. If the DoD developed the model, then

“baseline” would refer to a description of DoD requirements and would become

the basis for soliciting bids, evaluating deliverables, etc. If a contractor

developed the model, then the “baseline” would be either a description of an

item to be delivered or the deliverable itself.



Characteristics of Models Used in MBE

MBE requires the use of many types of models to address different aspects of a product or

capability. The characteristics of these models are heavily dependent on the modeling domain.

The modeling domain may address the systems, mechanical, electrical or software aspects of

the product, and/or the specification, design, analysis, test, manufacturing, or maintenance of a

product. The characteristics may also depend on the application domain, such as

telecommunications, aerospace vehicle design, and automotive design. Domain specific

modeling languages are often used to address the concerns of specific modeling domains.

The model characteristics also depend on whether the model is predominantly computational

or descriptive. A computer-interpretable computational model is a model that is used to

perform quantitative analysis on some aspect of the system. This includes time varying analysis

models such as a performance simulation or thermal analysis of heat flow, and includes static

computation models such as a reliability prediction analysis model. The models may be

deterministic or they may represent parameter uncertainty such as with a monte-carlo



performance simulation. The models may also be integrated with hardware, software, and

humans in the loop.

The computational model contrasts with a human-interpretable descriptive model that is used

to describe the system and design characteristics. A graphical modeling language with a formal

syntax and semantics is typically used to represent the model. The model information is

generally captured in a model repository provided by the modeling tool. Typical design models

include a 3 dimensional geometric model of mechanical designs, schematic capture tools for

electrical design, software design models represented in UML or AADL, and system architecture

design models represented in SysML, UPDM, and others.

In addition to the model itself, a model is characterized by additional information sometimes

referred to as metadata. This may include information on its version, intended purpose, scope

of the model in terms of its depth, breadth, and fidelity, range of applicability, and modeling

assumptions, as well as many other types of information. This information is essential for not

only understanding the results from the model, but also for understanding the appropriateness

of the model relative to addressing the specific concerns at hand.

The use of physical models such as wind tunnel models or other physical mockups and

prototypes are extremely important and fundamental to engineering a product or capability.

These models are considered part of MBE, and must be integrated with other computational

and descriptive models. However, the focus of the recommendations is on enhancing the

computational and descriptive modeling capability.



Potential MBE Benefits, Costs, Risks

The following subsections provide descriptions of the potential MBE benefits, costs, and risks.

First, the high-level benefits of MBE are described in the context of the DASD/SE Systems 2020

initiative. Next, the various benefits that MBE will provide are shown across the phases of the

acquisition life cycle. MBE’s ability to aid in risk identification and mitigation is then shown

through virtual integration and through early and continuous verification. Finally, the potential

costs and risks of MBE are discussed.



High Level MBE Benefits

In the Systems 2020 initiative, the DASD/SE has identified a set of objectives and a set of

constraints that must be maintained, or enhanced, as the objectives are achieved [9]. The

institutionalization of MBE will directly support the achievement of two of the three Systems

2020 objectives:

• Reduce the time to acquisition of first article for systems and solutions

• Reduce the time to implement planned and foreseen changes in systems

The institutionalization of MBE will also result in enhancing two of the three Systems 2020

constraints:

• Reliability

• Interoperability

For each of these objectives and constraints, the major ways in which MBE will support the

achievement of the objective or enhancement of the constraint are identified. MBE benefits

that are common to more than one objective and/or constraint include:



• Performing more complete evaluation of the solution and design trade space to

support not only more rapid acquisition of the first article, but also to support

more rapid system evolution to respond to a changing threat

• Reusing designs as the system or solution evolves, and across programs, to

reduce design and development time

• Identifying, mitigating, and resolving risks, issues, and errors earlier to reduce

development time and cost, and enhance reliability

• Verifying requirements and interfaces earlier and continuously throughout the

life cycle to enhance reliability and interoperability.

Each of the high level MBE benefits also directly enhance affordability by reducing the time and

cost to design, develop, deliver, and support capabilities.



MBE Benefits Across the Acquisition Life Cycle

The subcommittee reviewed a number of papers, studies, and reports to develop an

understanding of how MBE could provide benefits in each of the phases of the DOD system

acquisition life cycle. Above, we list the various benefits which MBE will provide in each phase

of the acquisition life cycle. Following the benefit, we have included the reference to the paper,

study, and/or report that described the benefit. (The studies, papers, and reports reviewed by

the subcommittee appear at the end of this report). In one case (annotated as Boeing 787), the

benefit was provided by a subcommittee member, however, we were not able to obtain a

publicly releasable report.

Many of the benefits identified in the early acquisition life cycle phases result in measurable

benefits during later phases. For example, the development and use of manufacturing models

would allow for manufacturing feasibility to be evaluated during the Material Solution Analysis

phase, and would allow for manufacturing processes to be evaluated during the Technology

Development phase. Gaps and issues uncovered in these earlier evaluations could then be



rectified during the Engineering and Manufacturing Development phase, leading to reduced

manufacturing related costs and schedule in the Production & Deployment phase. [2]

Because MBE is an emerging approach, there are few studies or reports where the benefits

have been calculated in quantified terms. Those benefits that have been quantified range from

calculations of the time and cost savings during integration for a complex air vehicle [1], to

labor savings during the Operations & Support phase by using model-based approaches for a

complex product family [5], to quantification of design and interface errors uncovered earlier

and software auto-generated from models [7], and finally a very significant increase in the size

of the trade space evaluated, with no increase in time [Boeing 787].



Virtual Integration to Manage Risk Throughout the Life Cycle

At least two classes of risk need to be addressed to support DoD goals for improving system

acquisition. Technical risk involves the unknowns associated with the introduction of new

technologies in new systems designs. Program risk involves the effects on development cost

and schedule arising from technical risk and other programmatic unknowns such as

requirements stability, funding and availability of shared resources. Application of emerging

MBE tools and techniques can help contain these risks by collapsing the traditional

development “Vee.”

An analysis of program delays and cost overruns in commercial aerospace development

programs indicates that a major cause of these issues is requirements errors that are only

discovered during the system integration phase of development. The trend toward increasing

delays and cost overruns correlates well with the increase in system complexity over

generations of commercial aircraft as measured by total onboard source lines of code. The

issues arise largely due to the fact that existing “waterfall” systems engineering practices were



created to effect the development of systems that were much less complex than modern

systems.

Model-based engineering tools and practices enable an adaption of the classic development

“V.” Rather than simply decompose the system into its component down the left side of the V

and then integrating and testing these components up the right, MBE enables incremental

testing of the components during development, well before designs have been committed to

hardware. This introduces the ability to manage risk incrementally. At each step in refinement

of the detailed design, systems, subsystem, and component requirements can be virtually

tested against the current understanding of the design requirements and assumptions. Tested

and validated subsystem elements increase the confidence that the system will operate as

intended, while tested and validated components elements increase the confidence that the

subsystem will operate as intended. Thus testing and validation can be more tightly coupled to

the design activities allowing more rapid design convergence and validation of derived

requirements in the same phase of the development lifecycle.

To improve the process further, there is a need to perform system integration tasks virtually as

early in the development lifecycle as possible. This addresses the need to not only have

components behave according to their specifications, but also that all the various component

will integrate and lead to a system that behaves as intended. Issues that are discovered during

system integration often arise from unknown unknowns, that is requirements for components

may be considered adequate, but the interactions between are often insufficiently or

improperly specified. This can lead to emergent behaviors that were not foreseen. Virtual

integration of a potentially disparate set of models and design artifacts from the various

traditional design disciplines (avionics hardware, software, hydraulics, electrical systems,

aerodynamics, fuel systems, landing gear, engines, …) may help contain risk by improving the

confidence in the system design at earlier stages of development than is afforded by the

traditional waterfall process. The technologies necessary to do this in a modern development

environment comprised of OEM’s and suppliers stretched across the globe, does not currently

exist. Though there is current research addressing this problem, the scope is sufficient that a

significant, coordinated effort is needed.



Risk and Cost Reduction Through Earlier Verification

The cost to correct requirements errors increases exponentially with development phase. A

requirement error introduced during the requirements development phase that is detected and

corrected during system test is 25 to 90 times more costly to correct than if it was corrected

during the phase in which was introduced. Much of this cost escalation is derived from the

expanded impact of change. The trends toward more tightly integrated and increasingly

complex systems, coupled with more decentralized development environments, make

understanding the full scope of change impact difficult to determine using current development

practices. Thus “early and often” integration using increasingly capable system models is

necessary to not only help contain risk, but to enable much more agility to the changing design

requirements typical of most modern systems developments. Accurate, virtual representations

of systems components enable early integration testing without the need to commit

preliminary designs to hardware. This mitigates the effects of requirements instability by

reducing the cost of change and by producing higher levels of design confidence at earlier

stages in the design lifecycle.



Potential MBE Costs and Risks

Realizing the full benefit of model-based engineering, including model based systems

engineering will require investment in tools, training, and infrastructure. MBE tools have

already been incorporated to a degree in existing system design processes. Their adoption has

largely been driven by the acute need for better tools to handle increasing system complexity

by human engineers. Models represent another level of abstraction, much the way that high

level programming languages enabled the design of increasingly complex software systems by

abstracting the details of the assembly language instructions. However as this need was

immediate, the application of MBE is system design has been largely siloed in different

engineering disciplines.

To address the increasing complexity of systems, the application of MBE must be

institutionalized throughout the development environment. This implies that communication of

design intent via models must traverse both internal, departmental boundaries, but also the

boundaries between OEM and supplier. Such a deployment must necessarily be planned and

coordinated to be cost effective. Furthermore, the initial investment may be prohibitive if the



entire cost is borne by the first project on which such a coordinate process is implemented. The

prospect of each OEM individually developing such a coordinate MBE approach would surely be

reflected on their common customer, just as the development by a supplier of unique

processes for each OEM customer would be reflected back on all OEM customers. Furthermore,

the capability to produce, analyze, verify, validate, exchange, integrate, and apply models must

extend from the end customer down through the supply chain to the Tier II and III suppliers.

The infrastructure to facilitate this must be developed to enable accurate and precise

communication of design intent across organizational boundaries while supporting a mutually

beneficial business model and protection of proprietary and other sensitive information.

A potential risk lies in placing excessive expectation on MBE tools to deliver the cost and

schedule savings and design agility desired by the DoD. As tools simply represent automation of

design processes, they will not replace strong, rigorous, and disciplined enterprise processes. A

rational design process that is supported by and compliant with government acquisition policies

must first be developed by the defense industry stakeholders. This may make use of some

existing guidance and comply with some existing regulations, but the change in the design an

acquisition paradigm afforded by MBE is significant enough that some change will be necessary

to achieve the full potential benefit of MBE technologies. Adequate tools must be developed to

support and integrate with these emerging processes such that a mature set of tools and

processes can be deployed on real development programs with minimum risk.

The significance of the change to the development paradigm implies that a workforce must be

developed that is versant in the emerging tools and processes. Training for proficiency in new

tools is necessary, but not sufficient. MBE provides abstraction of system complexity to a level

that enables humans to better understand the “big picture” and to thus design their

component informed of its relationship to other components. Specific training activities need to

be implemented as part of a change of the design culture that moves people away from the

archaic “waterfall” thinking and towards more continuous design/integrate/test processes. The

culture must also accommodate changes that have become prevalent such as the movement

away from the expectation of a complete set of requirements prior to program launch.

Finally, the full benefits promised by MBE can be achieved only if we are successful in

incorporating the full design processes. Precise and accurate communication of design intent

must transcend stove-piped responsibilities. Model artifacts must be able to flow across

organizational and discipline boundaries. Interacting electronic, software, and mechanical

subsystems must be virtually integrated to allow analysis of system properties. This will

ultimately require a strong interdisciplinary team to support the development of concurrent

engineering processes and practices.



Objective MBE Framework

The following subsections provide MBE Subcommittee’s vision of the objective MBE framework.

We provide a summary of the MBE current state, a graphical depiction of the MBE future (to

be) state, and a discussion of the primary gaps that must be closed to transition from the

current state to the “To Be” state.



MBE Current State

In order to establish a desired future state of MBE, it is important to understand the current

state of practice of MBE. The difference between the current state and future state is used to

identify gaps, which serve as the basis for defining improvement plans.

Modeling is currently used extensively across multiple domains to support engineering a

product, system, or capability. The current state is characterized by a wide variation in maturity

of practice across engineering disciplines. For example, three dimensional geometric modeling

and electrical design modeling have become common practice with well defined modeling

methods, tools, and techniques throughout the aerospace and defense industry, as well as

many other industries. Their model serves as the basis for defining the bill of materials for the

overall system. Simulation modeling is also a common practice, although the specific practices

for developing simulation models vary widely. Systems of systems, system, and software

architecture and design modeling are emerging practices that are used only in pockets. Some

aspects of system and software modeling, such as the use of Simulink models for design of

control systems and control algorithms, and auto generation of the implementing code, are



reaching high levels of maturity within some organizations. In other areas, the modeling

languages, methods, tools, and the skill of the workforce are in an early stage of maturity, but

are advancing rapidly.

Although in the current state of MBE there is wide use of models across many domains, they

are used in a stove-piped fashion and not well integrated from one model to another. The

system, software and hardware design models are generally developed independently with

little established integration among the models, tools, and methods. The multitude of

engineering analysis models and other simulation models are also generally developed and

implemented independently of one another. There are significant opportunities for inconsistent

input data and assumptions about the system that are used for different models. Effectively

managing the consistency between the diverse range of descriptive and analytical models is

one of the challenges associated with the current state of MBE.

The lack of integration also occurs across programs. There is limited formal model reuse from

one program to another unless a particular model is required by a customer, or the same

engineer is modeling across different programs. There are many contractual and technical

reasons that contribute to this, resulting in each model being developed from scratch, with the

resultant cost, schedule, and model validation challenges.

There is growing interest that MBE provides potential to address some of the challenges faced

by DoD acquisition to reduce risk, make our systems more affordable, and reduce the timeline

to field a system or system upgrade, as described in the benefits section of this presentation.

This growing interest is evident in many activities across industry, academia and standards

bodies. Many of these activities recognize of some of the challenges with the current state and

the promise of MBE, and are working to help move the state of practice forward. These include

organizations such as INCOSE that has a Model-based Systems Engineering (MBSE) Initiative,

and the Consortium of Aerospace companies that are working on the AVSI project to

demonstrate the benefits of virtual integrations. Standards bodies such as PDES, OMG, NIST,

and SISO are all addressing different standards in support of MBE. At the same time, there is

increasing focus on tool interoperability among modeling tools. One example of this is the OMG

Model Interchange Working Group that is working with multiple vendors to increase the model

interchange capability among UML, SysML and UPDM tools.

Aerospace and Defense companies from across industry are also increasing their internal efforts

internally to improve and deploy their modeling practices. Modeling also continues to be an

essential part of the engineering curriculum within academia, and there is increased emphasis

on teaching these practices to industry.



MBE To Be State

The MBE to-be state leverages MBE across the acquisition life cycle to enhance affordability,

shorten delivery time, and reduce risk. In the to-be state, the models become an integral part of

the technical baseline, and evolve throughout the programs life cycle. The current state is

characterized by gaps between domain silos and lifecycle development phase hand-offs that

are often the source of errors until later in the development process when they are more

expensive to fix. The future state of MBE seeks to reduce these errors though seamless

integration of model data across domains and across the lifecycle by aligning shared model

properties and assumptions. Different engineering disciplines concurrently operate on different

facets of the system and/or product design, such that the impact of a change in one model can

be readily assessed in another model. Engineering and programmatic knowledge is shared

through a common technical baseline. The models developed by each discipline evolve in

maturity throughout the life cycle, and are not thrown away and redeveloped as the program

transitions from one phase of development to another. This includes the up-front mission

analysis models, the system requirements and architecture models, the detailed hardware and

software design models, and the detailed simulation models used to assess and verify all



aspects of the system as it evolves. Early validation of requirements including those for

manufacturing and support, and efficient development of a fully integrated technical baseline

result in significant improvement to the development process.

The collaborative foundation provides a means to share the information from the model

registry across the extended enterprise of customers, teammates and suppliers. The foundation

includes the modeling standards that enable information exchange, the model registry that

enables ready access to the different models, and a trusted environment which enforces

protection of intellectual property and secure access to sensitive and classified data. The

collaborative environment also enables reuse from one program to another to enable sharing

across a family of products and system of systems.

The MBE to-be state includes a workforce that is skilled in the use of the matured modeling

methods and tools, an infrastructure that supports this capability, and policies that enable it.



Primary Gaps That Must Be Closed

MBE envisions the extensive use, sharing, and reuse of models and digital products in a

collaborative environment — both within Industry (including between Industry partners),

between Government and Industry, and across acquisition programs. Moreover, the models

and digital products will represent authoritative elements of the technical baseline and be

authorized to serve as contractually binding mechanisms similar to how text-based documents,

and perhaps physical mockups, do today. The use of digital products in this way will likely

require a few new policies and changes to some existing DoD policies and the DFAR. For

example, changes will be necessary to facilitate authorized access to, and the sharing of,

models and other digital products (e.g., DoD-approved scenarios) while controlling access and

safeguarding Government and Industry sensitive information including the intellectual property

contained in models; legitimizing the use of models within the acquisition process; ensuring

open and adequate competition in an MBE marketplace; and protecting the Government from

potential legal challenges brought about by the increased use of information technology in

acquisition contracts.



There are gaps between the current state and the desired To Be state relative to the availability

of mature MBE processes and methods. With the exception of CAD/CAE models, other models

such as simulation, analysis and architecture models are not fully integrated into the technical

baseline. If fully integrated, these models would be used in a consistent and comprehensive

part of the development process to specify, design, analyze, verify, and validate the system. In

addition, the models would be used by the Engineering Review Board (ERB) in to assess the

impact of proposed specification and design changes. The current MBE processes and methods

do not adequately support management of highly complex cross-domain models, including

configuration, version, and variant management and reuse of models and the modeling

environment, or the ability to propagate changes from one model to changes in other models.

The cross-discipline, cross-acquisition life cycle phase, and cross-program collaboration and

reuse of the MBE To Be state can only be realized by closing specific technology gaps and

developing robust tools and standards. Specific gaps to be closed include domain specific

languages and data standards, and formal semantics to encourage and enable model

interoperability and reuse. Standards must be developed to support model interconnect and

interchange, and any standards initiative should leverage the ongoing efforts by INCOSE, AVSI,

SISO, STEP PDES, OMG, and NIST. Finally, the MBE To Be state resembles an open architecture

environment, which will require tools, policy, and approaches for data rights protection.

In the end, it is people that perform systems engineering and acquisition, spread across many

stakeholder organizations, both on the customer (Government) side and the supplier (Industry)

side. MBE is an emerging approach to engineering, resulting in a shortage of qualified model-

based practitioners across stakeholder communities, particularly in leadership positions. There

is also a lack of general acceptance of the routine use of models and simulations as part of

typical business practices. Additionally, there is reasonable concern about the degree to which

models are valid across the potential range of their use, and about the level of confidence (both

qualitatively and statistically) that can be placed in model-based results.

Models and simulations must be sharable across traditional program boundaries, but shareable

in a way that both protects intellectual property and incentivizes innovation. This has significant

business model implications. For example, there is currently no equivalent to the "app store"

paradigm we see in the commercial sector for government and industry models. The

subcommittee believes that the solution to this gap can build on and extend the current DoD

Meta Data Standard and M&S Catalogue. It is also important to ensure that models of

operational capabilities, and associated use cases and scenarios, are shareable across program

boundaries to ensure system consistency with operational needs and interoperability with

other systems in related operational domains. Today these operational scenarios are often



stovepiped, inconsistent with one another, and generally not shared outside specific programs

or narrow customer domains because there is no infrastructure and environment for doing so.

Lastly, a business case must be developed to document and quantify the value propositions for

all stakeholders. MBE is an emerging practice, and little evidence exists that quantifies the

benefits. The few quantified examples are compelling [1, 5, 7], but are not broad enough to

encompass all stakeholders.



Policy, Guidance and Contracting Mechanism Impediments and

Issues

The following subsections provide the preliminary findings of the MBE Subcommittee’s review

of existing policy, guidance, and contracting mechanisms, and provide recommendations on

policy and regulations that we believe are required to enable MBE.



Preliminary Policy Findings

The subcommittee reviewed the applicable set of current OSD and service-specific policies and

guidance related to weapon system acquisition (listed in the appendix). Although existing

regulations are generally consistent with, and in many cases actually supportive of, the aims

and approach to MBE, these policies may not go far enough in:

• Ensuring open and adequate competition, and

• Protecting the Government from potential legal challenges brought about by the

increased use of information technology (models and other digital products) to create

authoritative and authorized elements of the technical baseline.

The aspects of MBE that will lead to changes in current policy and guidance include access

control to models and databases, electronic record keeping and retention, digital vice text-

based products, software interface standards, and data rights. In some cases we were unable to

find any existing policy addressing the issue (e.g., access control to MBE products). In other

cases, the existing policy was simply inadequate or needed to be updated to reflect a different

application (e.g., electronic/digital signatures and certifications). In the remaining areas, the

existing policy appears adequate and users simply need to be made aware of, and trained in,

the current policy.



Required Policy and Regulations to Enable MBE

Fostering the MBE collaborative environment presents a few challenges that can be addressed

through new or updated policy and guidance. For example, MBE is built on a foundation of

collaboration and openness, including the sharing and exchange of ideas, concepts, models,

data, test results, etc. between all of the stakeholders (government and industry teams)

involved in development of weapon systems, and extended in many cases to include

stakeholders of other systems with which this weapon system is expected to interoperate. To

facilitate the extensive use of heterogeneous models developed by different organizations and

different disciplines, and the ready exchange of information between models and between

organizations will require establishing and then enforcing model and data interoperability

standards. The focus should be on interface standards and externally visible and accessible,

behavioral and protocol standards. Standards should be chosen to allow a broad range of

industry participants.

Requirements for the retention of electronic records are outlined in DoD 7000.14-R (DoD

Financial Management Regulation) Volume 5, Chapter 21 and Volume 10, Chapter 8. The



federal statute governing record retention is 44 USC 3302. The Navy's records retention policy

can be found at this Link: http://www.fas.org/irp/doddir/navy/secnavinst/m5210_1.pdf. In

addition to the government requirement, there is a parallel requirement for government

contractors. That requirement is contained in the FAR...and is to retain for three years after

the contract is closed out. In addition to retention requirements, more specific procedures are

needed governing the form of electronic retention to support any possible litigation

downstream. For example, in January 2011 the Supreme Court agreed to review a two-decade

old civil dispute between the Navy and two defense contractors over the A-12 aircraft. MBE

policy on electronic archiving would have to address both the time-period for retention and the

electronic form of retention. The electronic records have to be retrievable and (machine)

readable. These requirements may levy additional requirements on retention of software

applications and hardware to ensure that the models are accessible and useable during the

time period needed.



To support the need to revisit and review the decisions and activities associated with an

acquisition program, including in response to formal protests and other litigation associated

with a program, the models and other digital products need to remain accessible (and

operational) through the necessary computer hardware and software. Whereas text and

graphics associated with acquisition programs today can be maintained for millennia if

recorded in a fixed medium, computer models (because of the hardware and software required

to run the models) have a half-life measured in years. New acquisition policy and regulations

are needed to define sunset provisions to ensure the necessary configurations of hardware and

software to access and operate the models and digital products remain available during this

period. This includes provisions to ensure “backwards compatibility” of files saved across

software versions or policy stating the extent to which a model needs to be fully functional

across platforms (defined by relevant combinations of hardware and software).

Current acquisition guidance and policy (including JCIDS) requires written (text-based)

submission of documents to support the acquisition process. These include the Capability

Development Document (CDD), the Capability Production Document (CPD) and other written



documents to support the Materiel Development Decision (MDD), the Systems Engineering

Plan (SEP), the Test and Evaluation Master Plan (TEMP) and others. Changes are needed in

policy and guidance (including the DFAR) to authorize the use of models and other digital

products as opposed to text and graphics only and stating where and how these digital

products can be incorporated by reference into required text-based documents. Also, the

Weapon System Acquisition Reform Act (2009) requires competitive prototypes of systems and

critical subsystems in the Technology Development phase. There is no policy on whether

“virtual” prototypes, developed through MBE, could substitute for physical prototypes and

satisfy the WSARA requirement.

The models, databases, and other digital products developed within the MBE collaborative

environment and made accessible to acquisition stakeholders will include proprietary and

business sensitive information in the form of intellectual property (e.g., algorithms, databases,

technical approaches, etc). MBE emphasizes the collaboration among government and industry

teams throughout the acquisition process and the sharing of models, data, and interim

products between phases of program development and among stakeholders. The stakeholders

are likely to include a number of industry developers, vendors, and suppliers, several which

could represent competitors. Access to digital products means they are prone to reverse

engineering and stealing trade secrets. Additional policy is needed to protect the intellectual

property that is embedded in MBE tools and products.



To ensure the authenticity, integrity, and confidentiality of MBE products these need to be

encrypted and digitally signed. Electronic/Digital Signatures and Certifications are addressed in

DoD 7000.14-R (DoD Financial Management Regulation) Volume 10, Chapter 17). All MBE users

will need to be trained in the Assistant Secretary of Defense for Networks and Information

Integration ASD (NII) guidelines for digital signature and the DoD-wide interoperability

requirements (for Public Key Infrastructure) prescribed in DoD Instruction 8520.2.

The implementation of MBE must ensure a level playing field among potential industry

participants. This starts with ready access to electronic files on contract announcements, and

proceeds through solicitation documents and, once the contract is awarded, onto the models

and databases used in the design and development of the system. The information technology

(software applications and hardware) required for a firm to participate in an MBE program

must not give an unfair advantage to large firms capable of making the needed investment in

technology, nor hinder competition among potential bidders. For example, today some military

web sites require a CAC card for access. Other sites are not compatible with a particular version



of a web browser. DoD would have to develop a policy on the information technology required

to participate in MBE acquisitions.

Sharing and collaboration should also include government-initiated actions to share

government-controlled models and databases, including formally “approved” scenarios and

CONOPs. Industry access is important especially if these resources will be used in evaluating

proposals or to support other government activities, such as test and evaluation. Classified

models and databases should be made available to industry participants that meet the

appropriate security criteria.

Support for the collaboration and sharing of models and other digital products that is at the

center of the MBE concept will require some form of registry (that points to numerous

repositories) to register the modeling tools and digital products and other components of the

MBE infrastructure such as scenarios, CONOPs, environmental data and other databases. To

make these tools useful to others will require metadata be provided by the original developer

on all models and digital products. Once the models and databases have been registered, policy

and procedures will be needed governing access to these products by others working MBE.

The MBE approach to acquisition may replace some traditional text-based acquisition products

(including contracting paperwork) with computer models or may incorporate digital products

(such as models) into new forms of acquisition products. In such cases, there may be

challenges with interpreting, evaluating, and then enforcing “soft standards” (non-text based)

digital products. While there are current legal conventions on construing written documents,

including written contracts, the task of determining compliance with something that is only

machine-readable and cannot be printed out and shown to a judge adds a new dimension for

disagreement. Established conventions developed for interpreting text and engineering

drawings will have to be updated to account for the digital products associated with MBE.



Recommendations

The following subsections provide the recommendations that were developed by the MBE

Subcommittee. Our recommendations fall into three major areas:

Collaboratively work with stakeholders to develop the MBE Business Model,

Work with Industry, Vendors, and Standards Organizations to develop MBE standards,

and

Develop the workforce.



Collaboratively Work With Stakeholders to Develop the MBE Business Model

While many agree with the compelling vision of MBE, there is a lack of significant data to

support a compelling the Business Model and move it beyond academics. Without the

supporting data, DOD would be more reluctant to mandate MBE, Industry would be more

cautious of infrastructure investment, and Vendor development would be slower.

In order to develop the data required to support the Business Model, the Subcommittee makes

a series of recommendations for collaborative actions between the Government, Industry, and

Vendor Stakeholders. The steps establish the criteria for success; generate data through

projects and challenges; mature the process to increase benefits; and, as appropriate,

institutionalize MBE through incentives.

1. The first step is to understand what success looks like and the supporting data required.

The broad-based Collaborative Team (Government, Industry, Vendor) would agree on

metrics and methods to capture them.



2. Next, the Government would develop MBE pilot programs – perhaps similar to Model-

Based Acquisition programs such as USMC LVSR and MPC - encouraged by award fee.

Program deliverables could be model-based with CDRLs being package, performance,

and descriptive models. Increased Industry investment would encourage Vendor

development in the areas of interoperability and standards.

3. If the data produced from pilots is encouraging but gaps remain, we recommend a

“Grand Challenge” to sponsor revolutionary, high-payoff research to bridge the gaps,

perhaps, similar to the 2005 and 2007 DARPA prize competitions for driverless cars.



4. In addition to interoperability and standards, there must be a reliable source of

authoritative models. Trusted models must be developed and thoroughly verified

through physical test. For each program, models, meta data, and test data must be

traceable to known configurations. A model registry, or catalog, would provide pointers

to authoritative models and their developers as well as operational environments and

the intended uses. Two ongoing DOD initiatives, the M&S Catalog and the Defense

Meta Data Standard, should be assessed for applicability.

5. The next step in maturing MBE and generating even greater value is would be to extend

the span from a single program to multiple programs.

6. While interoperability and registries will promote greater sharing, sensitive data will at

the same time require protection. The Collaborative Team will need to establish

guidelines, new technologies, and integrate them into the tools.



TSCP – Transglobal Supply Chain Program who is working on how to express IP, ITAR, and

security classification in terms of information rights in standard ways, and how to incorporate

that into existing modeling standards using security technologies like encryption, digital

signatures, and digital rights management.



Work with Industry, Vendors and Standards Organizations to Develop MBE

Standards

Realization of the MBE To Be State can only occur with a broad base of technical and

programmatic standards that fully support collaboration and reuse across disciplines, across life

cycle phases, and across programs. The subcommittee makes a series of recommendations for

collaborative actions between Government, Industry, and Vendor Stakeholders to develop the

standards. The recommendations call for the development of the overarching technical

guidance, development of the technical standards and reference implementations, and

development of program planning guidance to support the successful contracting and

execution of MBE-based programs.

1. The initial step is to develop an MBE Common Reference Model that will provide an

abstract framework for understanding the relationships among MBE stakeholders, and

for the development of consistent standards and specifications that support MBE. We

suggest that a Consensus Conference be conducted to initiate the development of the

Common Reference Model. It is important that the Consensus Conference attracts a



wide number of participants that are currently engaged in model-based activities and

research.

2. Next, the Government should lead the development of an MBE Standards Roadmap,

involving Industry, Associations, and the international community so that ongoing

model-based standards initiatives can be leveraged to the greatest extent possible.

3. Should the Common Reference Model and Standards Roadmap identify any high priority

technical gaps that must be closed, the Government should initiate a research program

to close the gaps and transition the technology.



4. Development of MBE standards should follow, in accordance with the Roadmap and the

Common Reference Model. Industry and Vendor participation is vital to ensure

widespread buy-in, and early implementation of the standards in vendor tools. One, or

more, standards organizations should be engaged to formalize the standards.

5. Widespread adoption of new standards can be achieved more easily if reference

implementations are developed and made available. An example of this type of effort is

the seed funding provided by DOD for the reference implementations of the High Level

Architecture’s (HLA) Runtime Infrastructure (RTI) and Realtime Platform Reference

Federation Object Model (RPR FOM). The availability of the RTI and RPR FOM made it

easier for the Services and Industry to implement the initial HLA compliant simulations.

6. Finally, MBE Program Planning Guidance must be developed. Key information, lessons

learned, and best practices will be gleaned from the Pilot Projects (Business Model

recommendation #2) and early adopters. Industry, Vendors, and Academia will provide

input as well to help develop and evolve the Program Planning Guidance.



Develop the Workforce

The subcommittee agreed that all professionals engaged in a Model Based Engineering program

need to be made aware of the strengths and weaknesses of the MBE approach, and that the

needed awareness is a function of their role in the program. As described in other sections of

this report the committee is aware that the principles of model based engineering can be

misapplied, or even misused to misrepresent product capabilities. As an example, all

participants in the process must be aware that a model or simulation proven to be valid in one

application may not be valid or even relevant in another. Relatively fewer participants need to

understand the details of the M&S verification process.

1. The subcommittee suggests that, rather than establishing MBE as a specialty in the

acquisition workforce, the workforce development process should ensure that the

members of that workforce have the level of understanding of MBE appropriate to their

role. The MBE Common Reference Model will be used to help educate the holders of

each role about the needs and contributions of the other roles. M&S has a significant

and growing role in system development and acquisition, making it critically important



that government and industry members have a common, shared, and articulated

understanding.

2. M&S, including MBE has matured to become a normal aspect of the acquisition

landscape and our acquisition professionals should include M&S as part of their normal,

role based professional development. We in the community must work together to

ensure that the needed training is available, is relevant, and is of high quality.

3. While no different than current practice, an understanding of needed skills must be

applied during workforce selection. This should include MBE training and experience.



4. Training, alone, is not sufficient. The concepts of MBE can be quite complex, and

individual staff members, and organizations, new to MBE will require mentoring. Tools

and capabilities should be developed that support MBE collaboration. We believe

Vendors can play an important role in providing mentoring services to the Government

and Industry.

5. Knowledge must be captured on MBE successes and pitfalls. This captured knowledge

can be incorporated into training programs, best practices, and MBE tool improvements

that can be available to the MBE community.



MBE Roadmap

The MBE Roadmap represents the proposed maturity of MBE practice over time. The maturity

of the practice reflects both how well the practice is defined, and the extent to which the

practice has been adopted by industry. The roadmap is intended to help guide investment and

resources from government, industry and academia to advance the practice of MBE and apply it

to system acquisition. The suggested roadmap shown above is adapted from the MBSE

roadmap developed by the International Council on Systems Engineering (INCOSE). The left

vertical axis represents increase capability/maturity of MBE tools and processes. The right

vertical axis represents increasing capability/maturity of a workforce versant in the MBE tools

and processes. Note that there is significant overlap in the timing of the development activities

shown as blue boxes in the diagram. It is intended that the steps along the maturation path

happen largely concurrently rather than sequentially.

The path starts with emerging MBE standards. The extent to which a coherent MBE process is

deployed to all stakeholders in the defense/aerospace industry largely depends on the

development and adoption of adequate standards. Unambiguous communication of design



intent across organizational boundaries via models must be based on a common understanding

of key attributes. These will form the basis of a robust MBE system acquisition/development

paradigm.

The next step involves maturing MBE tools and methods to incorporate more of the systems

design lifecycle. As previously discussed, modeling techniques is relatively mature within

distinct development disciplines. Significant development of the technologies necessary to

integrate these domains into a consistent, analyzable system model is still required. This activity

is a first step toward a fully integrated model. Model tools and processes in the avionics HW

and SW are relatively mature and are being extended to incorporate more systems design

elements. This is evident by development of technologies such as AADL and SysML, which build

off formalisms such as VHDL (electronic hardware design) and UML (software design).

The next step builds on the integration of component models by placing them in the context of

a coherent architectural model. Moving to an architecture-centric design process affords

several benefits. Platform-Based Engineering (PBE) or Product Line Engineering can leverage

existing, proven architectures in developing solutions for new missions. A central architecture

which is used to generate specific views of the systems design (logical, physical, thermal, etc.)

maintains a “single truth” for the system design, thus avoiding errors arising from mismatched

assumptions between stove-piped design organizations. A coherent architecture further

provides a means to maintain an integrated system throughout the design and test

development phases. The key is to start a system design integrated and maintain the

integration throughout development. This continuity implies that simulation and analysis of the

system benefits from the current level of maturity of all components.

The activity defining MBE theory, ontology, and formalisms will lead to improved techniques to

virtually integrate systems. This may potentially enable optimization of architecture-based

designs for metrics such as complexity, cost, adaptability, or performance. It may lead to

further capability to abstract complexity and produce correct-by-construction system designs

reducing the need for current VV&A and/or T&E practices.

The next development is another step toward full integration, integrating formerly stove-piped

design departments within a given organization. The standards, tools, and techniques necessary

to unite the myriad of modeling efforts that already exist within an organization will allow

system-level analyses and potentially uncover emergent system properties.

The final step represents a fully mature MBE process that is widely deployed and adopted by

the relevant stakeholders. In this case, the boundaries of the integrated system model extend



beyond the organizational boundaries. This facilitates accurate and precise communication of

design intent from the end customer to the system integrator to lower level suppliers. It further

allows analysis of system properties in a physically distributed, but logically integrated system

model that is unified by a “single truth” system architectural model. This capability must exist in

an infrastructure that simultaneously ensures controlled access and protection of intellectual

property and secure data. Finally, this environment is support by legal and business policies and

practices that motivate development of the capabilities.



Conclusions and References

The following subsections provide the conclusions reached by the MBE Subcommittee and a list

of the documents, studies, reports, and regulations that were reviewed by the subcommittee.



Conclusions

MBE has the potential to provide significant benefits to the DOD and to Industry. These

benefits will only be achieved if MBE becomes common practice across the life cycle for the

acquisition and evolution of systems and solutions. The MBE Subcommittee strongly believes

that successful wide-scale adoption and sustainment of MBE can only occur with:

• A business model that is supportive of all stakeholders

• The development and evolution of the technology and standards required by the MBE

To Be state

• A workforce that is skilled in using MBE across the acquisition life cycle, to include

acquisition/contracting skills, technical skills, and program management skills

The MBE Subcommittee believes that there are three near-term actions that can be taken to

provide a solid foundation for future MBE activities. The first near-term action is for DOD, in

collaboration with Industry, to develop a detailed MBE Roadmap that provides the plan of

action and milestones for the implementation of the business model, standards, and workforce

recommendations that are detailed in the Recommendations section of this report. The high-



level roadmap provided in the Recommendations section of this report can be used as an initial

guide for the development of the detailed MBE Roadmap.

The second near-term action is to launch, as soon as is practical, the Business Model

Collaborative (see MBE Business Model recommendation 1) and the Standards Consensus

Conference (see MBE Standards recommendation 1). The wide-scale use of MBE across the

acquisition life cycle will necessitate adaptations within current business models and

adaptations in current acquisition and engineering processes. It is critical to initiate the

collaborative work to adapt and evolve the current models and processes at the beginning of

the MBE initiative.

The third near-term action is for the DOD to rapidly establish the means to actively collaborate

with companies, associations, and organizations that are actively engaged in model-based

initiatives throughout the world. As noted in the description of the MBE Current State, several

industries and associations in the US, Europe and Asia are actively involved in model-based

activities and initiatives. Active collaboration with these companies and associations will allow

the DOD to more easily leverage and adapt tools, technologies, methodologies, best practices,

and lessons learned.



References and Literature Surveyed

1. Proof-of-Concept Project AFE #58 Summary Final Report produced under the System

Architecture Virtual Integration (SAVI) Program 8 October 2009

2. Modeling & Simulation Investment Needs for Producible Designs and Affordable

Manufacturing, Systems Engineering Implications; NDIA JCSEM M&S Sub-Committee Final

Report, February 2010

3. Software Intensive Systems, Naval Research Advisory Committee Report, July 2006

4. Preliminary Observations on DoD Software Research Needs and Priorities: A Letter Report,

Committee on Advancing Software-Intensive Systems Producibility, National Research

Council, 2008

5. Complex Product Family Modeling for Common Submarine Combat System MBSE, Lockheed

Martin July 2010

6. Deployment of MBSE Processes Using SysML, US Army ARDEC and HPTI, presentation at the

2010 NDIA Systems Engineering Conference, October 2010

7. Use of a Model Based Approach to Minimize System Development Risk and Time-to-Field

for New Systems, US Army AMRDEC SED, presentation at the 2010 NDIA Systems

Engineering Conference, October 2010

8. Systems-2020 Study, Final Report, Booz Allen Hamilton, 16 August 2010

9. Systems 2020 Strategic Initiative Overview Briefing, Office of the Director, Defense Research

and Engineering, presentation at the NDIA SE Division M&S Committee Meeting, June 2010

10. Development of 3-Year Roadmap to Transform the Discipline of Systems Engineering,

Systems Engineering Research Center, SERC-2009-TR-006, 31 March 2010

11. Technology Tools for Rapid Capability Fielding: Final Outbrief, Office of the Director,

Defense Research and Engineering, presentation at the NDIA SE Division M&S Committee

Meeting, April 2010

12. The Economic Impacts of Inadequate Infrastructure for Software Testing, National Institute

of Standards and Testing, Planning Report 02-3, May 2002

13. 21st Century Strategic Technology Vectors, Volume IV – Accelerating the Transition of

Technologies into US Capabilities, Defense Science Board 2006 Summer Study, April 2007

14. Toward Model-Based Embedded System Validation Through Virtual Integration, DOD

Software Tech News, Volume 12, Number 4, January 2010

15. Model Based Engineering for Medical Device Software, Ray, et. al., 2008

16. Model Based Engineering Design Pilots at JPL, Kordon, et. al., Proceedings of the 2007 IEEE

Aerospace Conference

17. Simulation Based Engineering Science, Revolutionizing Engineering Science through

Simulation, Report of the National Science Foundation Blue Ribbon Panel on Simulation

Based Engineering Science, May 2006



18. INCOSE MBSE Initiative Summary, S. Friedenthal, presentation at the NDIA SE Division M&S

Committee Meeting, June 2010

19. Core Manufacturing Simulation Data Product Development Group, S. Leong, presentation at

the NDIA SE Division M&S Committee Meeting, June 2010

20. Department of Defense Directive 5000.01, The Defense Acquisition System, certified current

as of 20 November 2007

21. Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, 8

December 2008

22. Chairman of the Joint Chiefs of Staff Instruction 3170.01G, Joint Capabilities Integration and

Development System, 12 October 2010

23. Defense Acquisition Guidebook, 12 October 2010

24. Modeling and Simulation Guidance for the Acquisition Workforce, Office of the Deputy

Under Secretary of Defense for Acquisition and Technology, October 2008

25. Army Regulation 70-1, Army Acquisition Policy, 31 December 2003

26. Department of the Army Pamphlet 70-3, Army Acquisition Procedures, 1 April 2009

27. Air Force Policy Directive (AFPD) 63-1/20-1, Acquisition and Sustainment Life Cycle

Management, 3 April 2009

28. Air Force Instruction (AFI) 16-1002, Modeling and Simulation (M&S) Support to Acquisition,

1 June 2000

29. AFI 63-1201, Life Cycle Systems Engineering, 23 July 2007

30. AFI 63-101, Acquisition and Sustainment Life Cycle Management, 17 April 2009

(Incorporating Through Change 2, 16 June 2010)

31. Air Force Pamphlet 63-128, Guide to Acquisition and Sustainment Life Cycle Management, 3

October 2009

32. SECNAV M-5000.2, Department of the Navy Acquisition and Capabilities Guidebook,

December 2008

33. SECNAVINST 4105.B, Independent Logistics Assessment and Certification Requirements, 18

December 2008

34. SECNAVINST 5000.2D, Implementation and Operation of the Defense Acquisition System

and the Joint Capabilities Integration and Development System, 16 October 2008

35. SECNAVINST 5400.15C, Department of the Navy Research and Development, Acquisition,

Associated Life-Cycle Management and Logistics Responsibilities and Accountabilities, 13

September 2007

36. SECNAVINST 5430.7Q, Assignment of Responsibilities and Authorities in the Office of the

Secretary of the Navy, 17 August 2009

37. Department of the Navy Acquisition Plan Guide, March 2007

38. Navy Marine Corps Acquisition Regulation Supplement (NMCARS), 13 July 2010



39. OPNAVINST 3811.1D, Threat Support to Weapon and Information Technology Systems

Planning and Acquisition, 5 June 2008

40. SECNAVINST 5200.38A, Department of the Navy Modeling and Simulation Management, 28

February 2002

41. SECNAVINST 5000.36A, Department of the Navy Information Technology Applications and

Data Management, 19 December 2005

42. DON Policy on Digital Product/Technical Data, 23 October 2004

Final Report of the Model Based Engineering (MBE) Subcommittee

Documents

Transcript of Final Report of the Model Based Engineering (MBE) Subcommittee