An Integrative Method for Appraising Modular Open System Maturity Cyrus Azani Open Systems Joint...

An Integrative Method for Appraising An Integrative Method for Appraising Modular Open System MaturityModular Open System Maturity

Cyrus AzaniCyrus Azani

Open Systems Joint Task Force/NGOpen Systems Joint Task Force/NG

November 2003November 2003

AgendaAgenda

Part IPart I The Modular Open System ProcessThe Modular Open System Process

Part IIPart II The Open System Maturity ModelThe Open System Maturity Model

Part IIIPart III Proposed Appraisal MethodologyProposed Appraisal Methodology

Part IVPart IV The Appraisal Instrument The Appraisal Instrument

Part V Questions and AnswersPart V Questions and Answers

Proposed by Cyrus Azani at OSJTF/NG

Why Modular Open systems?Why Modular Open systems?

Shift in customer taste/expectationShift in customer taste/expectation Evolving needs/requirements Evolving needs/requirements Rapidly changing technologiesRapidly changing technologies Growing need for situational awarenessGrowing need for situational awareness Rising significance of Real-Time informationRising significance of Real-Time information Aging legacy of stove-pipe systems Aging legacy of stove-pipe systems System development cost escalation trends System development cost escalation trends Rising support and maintenance costsRising support and maintenance costs Increased complexity Increased complexity (e.g.,(e.g., Increasing Operator Workload / Increasing Operator Workload /

Information Overload)Information Overload) Reducing decision time line and response timeReducing decision time line and response time Emerging standardsEmerging standards

MOSA Vision MOSA Vision

Affordable, Producible, Sustainable, Interoperable,

Reconfigurable, and Growing System Capability based on

Modular Architecture and Open Standards for

Appropriate

Key Interfaces


Open System: A system that uses modular architecture and implements open standards for appropriate key interfaces.

Open Standard: A widely implemented standard, the use of which is non-proprietary.

Interface Standard: A standard that specifies the physical and logical characteristics of an interface.

Key Interface: The interface of a module that is characterized by the need to be interoperable, easily changed, replaced or isolated because of complexity, obsolescence, or cost.


What is an Open System?What is an Open System?

An integrated business and technical strategy that employs a modular design and, where appropriate, defines key interfaces using widely supported, consensus-based standards that are published and maintained by a recognized industry standards organization.

Encompasses practices that are basic to implementingEncompasses practices that are basic to implementing Effective MOSA processEffective MOSA process Advanced open system practicesAdvanced open system practices


WHAT IS MODULAR OPEN WHAT IS MODULAR OPEN SYSTEM APPROACHSYSTEM APPROACH (MOSA)?MOSA)?

FUNDAMENTAL MOSA PRINCIPLES (BUILDING BLOCKS)

Affordable & Adaptable Capability

Sound Systems Engineering Process

Modular Architecture

Designing Self- contained & Functionally Cohesive Parts

Using Consensus-based & widely-used Standards

Interface Management

Ensuring Openness and Rigorous Def. Of Interfaces

Identifying Interfaces that are Subject to Change

Key Interface Designation

Open Interface Standards


MOSA ProcessMOSA Process

A set of business and engineering activities, A set of business and engineering activities, methods, practices, and transformations that methods, practices, and transformations that organizations use to develop and maintain organizations use to develop and maintain affordable, producible, interoperable, affordable, producible, interoperable, reconfigurable, and adaptable systems and reconfigurable, and adaptable systems and associated products (e.g., design documents, associated products (e.g., design documents, architectures, test cases, user manuals, interface architectures, test cases, user manuals, interface management plans, etc.).management plans, etc.).

A Means of predicting the likely outcomes to be A Means of predicting the likely outcomes to be expected from the next system or product expected from the next system or product development project the organization undertakes development project the organization undertakes

A common process that applies to the A common process that applies to the development of an adaptable product or service in development of an adaptable product or service in the engineering development domainthe engineering development domain


The MOSA Process

5. Establish Level of Openness & Select

Appropriate Standards for Selected Key Interfaces

1..Explore Alternative Concepts & Formulate Migration, Acquisition, Technology, and Support Strategies

4. Assess the Feasibility ofMaking the Key Interfaces

Open

7. Finalize the System Architecture & Implement

Consistent Interface Management Practices

Assess the Impact and Document the Justification

3. Identify Key

Interfaces

Identify modules with rapidly changing technologies

Identify modules which are likely to grow or evolve over the course of the systems life

Identify modules with high cost, failure rate, and interoperability impacts

6. Prepare Test & Evaluation and Technology Transition Plans

YES

Redefine, Modify & Reallocate

Needed Capability and

Performance Requirements

2. Develop the Initial System Architecture by Partitioning the

Candidate System into functional and logical Modules

Yes

NoOpen Interface?

Continuing Market Research and Interface Management

Suitable Environment & Sound Systems Engineering Process

Feedback

Open System Vision


Modular Open System MaturityModular Open System Maturity

Measures an organization’s overall open Measures an organization’s overall open system maturity and the extent to which a system maturity and the extent to which a specific MOSA process is explicitly defined, specific MOSA process is explicitly defined, managed, measured, controlled, and effective. managed, measured, controlled, and effective.

As an organization gains in modular open As an organization gains in modular open system maturity, it institutionalizes its MOSA system maturity, it institutionalizes its MOSA Process within its knowledge and corporate Process within its knowledge and corporate infrastructure and culture to ensure its infrastructure and culture to ensure its endurance and effectiveness. endurance and effectiveness.


The MOSA CMM Purpose The MOSA CMM Purpose

Understand process areas and specific practices required Understand process areas and specific practices required for launching open system improvement processesfor launching open system improvement processes

Identify process areas that contribute to each step of Identify process areas that contribute to each step of organizational Modular Open System Process maturity organizational Modular Open System Process maturity

Provide valuable information to technical staff and Provide valuable information to technical staff and process improvement groups to:process improvement groups to:

– Identify strengths and weaknesses of system integrators, Identify strengths and weaknesses of system integrators, contractors and supplierscontractors and suppliers

– Determine the risks of selecting among different contractors for Determine the risks of selecting among different contractors for awarding business and to monitor performanceawarding business and to monitor performance

– Provide a set of specific evaluation criteria against which the Provide a set of specific evaluation criteria against which the contractors proposals will be evaluated contractors proposals will be evaluated

The integration of Modular Open System practices within the CMMI Framework. The effort is aimed at enabling organizations to:


MOSA Performance MOSA Performance

The actual results achieved by an organization The actual results achieved by an organization That follows a Modular Open System Approach That follows a Modular Open System Approach Process.Process.

Examples of Expected performance:Examples of Expected performance:– More effective change managementMore effective change management– Ease of technology insertionEase of technology insertion– Growing capability Growing capability – Improved interoperabilityImproved interoperability– Less integration risksLess integration risks– Lower cycle timeLower cycle time– Vendor independence Vendor independence – Reduced total ownership costsReduced total ownership costs

Bottom Line: Greater chance to more effectively

adapt an organization and its constituent systems to changes in

the internal and external environment.


MOSA Maturity ModelMOSA Maturity Model

A Staged Representation Model that identifies the A Staged Representation Model that identifies the practices that are basic to implementing effective practices that are basic to implementing effective MOSA process and advanced open system MOSA process and advanced open system practices. practices.

Assigns to those practices associated maturity Assigns to those practices associated maturity levels ranging from negligible to mature and well-levels ranging from negligible to mature and well-managed organization-wide application of open managed organization-wide application of open systems.systems.

Identifies the path through the various practices Identifies the path through the various practices for achieving higher levels of modular open for achieving higher levels of modular open system maturity and improvement of system maturity and improvement of organizational capability and adaptability. organizational capability and adaptability.


Modular Open System CapabilityModular Open System Capability

The range of expected results that will most likely The range of expected results that will most likely be achieved by following a MOSA Processbe achieved by following a MOSA Process..

Measures an organization’s MOSA Process Measures an organization’s MOSA Process improvement achievement. improvement achievement.

The MOSA Capability Maturity Model delineates The MOSA Capability Maturity Model delineates the characteristics of a mature, capable MOSA the characteristics of a mature, capable MOSA Process. Process.


Open SystemVision

Standardized Modular

Open System Process

Well-established Strategies, Guidance, Measures, and Change

Management Plans

Continuing Performance

Measurement andMonitoring

Highly Integrated Process,

Structure, andCulture

Major Characteristics of Mature Modular Open System Capability


Examples of Capabilities/Requirements Examples of Capabilities/Requirements Necessitating MOSA ImplementationNecessitating MOSA Implementation

Dominant performance that requires access to the latest technologies Dominant performance that requires access to the latest technologies Need for continuing integration and reconfiguration capabilityNeed for continuing integration and reconfiguration capability Unstable longer-term requirements that are addressed as evolving incrementsUnstable longer-term requirements that are addressed as evolving increments When technologies/engineering for full capability still need to be developed When technologies/engineering for full capability still need to be developed

and a spiral development strategy is pursuedand a spiral development strategy is pursued When products or spare parts are competitively produced and the When products or spare parts are competitively produced and the

organization must maintain continuing access to multiple sources of supply organization must maintain continuing access to multiple sources of supply Desire to mitigate the intensity and magnitude of risks associated with Desire to mitigate the intensity and magnitude of risks associated with

proprietary standards and sole source of support throughout a system life proprietary standards and sole source of support throughout a system life cycle cycle

Need for design flexibility, modularity, and interface control Need for design flexibility, modularity, and interface control System of systems interoperability requirements that unequivocally call for System of systems interoperability requirements that unequivocally call for

compliance with specific open standards, protocols, or standardized data compliance with specific open standards, protocols, or standardized data formatsformats

When the system has to have connectivity with other systems to properly When the system has to have connectivity with other systems to properly functionfunction

Need for commonality of hardware, software, and support systems Need for commonality of hardware, software, and support systems


Modular Open System Maturity LevelsModular Open System Maturity Levels

Level 5: Complete (Total Organization) OpennessLevel 5: Complete (Total Organization) Openness Level 4: Extensive (System of Systems) OpennessLevel 4: Extensive (System of Systems) Openness Level 3: Reasonable (Individual System) OpennessLevel 3: Reasonable (Individual System) Openness Level 2: Limited (Subsystem) OpennessLevel 2: Limited (Subsystem) Openness Level 1: Negligible OpennessLevel 1: Negligible Openness


OS Maturity Level 5: Complete (Total Organization) OpennessOS Maturity Level 5: Complete (Total Organization) Openness

Organization has institutionalized the Modular Open System Process within its Organization has institutionalized the Modular Open System Process within its overall organization structure and cultureoverall organization structure and culture

Modular Open System Process is managed as a regular business activityModular Open System Process is managed as a regular business activity The organization is pursuing a disciplined, standardized and creative Modular Open The organization is pursuing a disciplined, standardized and creative Modular Open

System ProcessSystem Process Continuous improvement of the organizational processes including Modular Open Continuous improvement of the organizational processes including Modular Open

System Process System Process Negligible difference is observed between targeted and actual open system benefits Negligible difference is observed between targeted and actual open system benefits Pursuit of a pro-active management philosophy in system development and Pursuit of a pro-active management philosophy in system development and

sustainment, in establishing measures, and in monitoring the progress on sustainment, in establishing measures, and in monitoring the progress on implementing a standardized modular open system processimplementing a standardized modular open system process

Well-defined roles and responsibilities for adopting and influencing open standards Well-defined roles and responsibilities for adopting and influencing open standards across the organization. across the organization.

Lessons learned on modular open system application are disseminated widely and Lessons learned on modular open system application are disseminated widely and effectivelyeffectively

Organization-wide openness provides capability to: Organization-wide openness provides capability to: – Continuously and affordably modernize systems, and processes Continuously and affordably modernize systems, and processes – Connect systems and organizational stakeholders, and create commonality across platforms Connect systems and organizational stakeholders, and create commonality across platforms

and systemsand systems– Enable affordable, long-lasting, and adaptable systems and productsEnable affordable, long-lasting, and adaptable systems and products– Reconfigure systems and applications as needs and technology changesReconfigure systems and applications as needs and technology changes– Effective and continuing knowledge sharing Effective and continuing knowledge sharing

CharacterizedCharacterized by openness ofby openness of

all the appropriate all the appropriate key interfaces key interfaces

within all the systems within all the systems in the organization in the organization

Examples of Expected Results:


OS Maturity Level 4: Extensive (System of Systems) OpennessOS Maturity Level 4: Extensive (System of Systems) Openness

A majority of key interfaces within selected system and system of systems A majority of key interfaces within selected system and system of systems (collaborating systems) are defined by open standards(collaborating systems) are defined by open standards

Emphasis is placed on reuse and commonality of products across domainsEmphasis is placed on reuse and commonality of products across domains Use of open standards in family of systems is being consciously assessed Use of open standards in family of systems is being consciously assessed Metrics for measuring the openness of systems and processes have been Metrics for measuring the openness of systems and processes have been

established and being usedestablished and being used An interface management plan has been established An interface management plan has been established Organization is capable to predict total cost of ownership, development Organization is capable to predict total cost of ownership, development

cycle time, and system capabilities at different blocks of improvement cycle time, and system capabilities at different blocks of improvement within reasonable margin of error.within reasonable margin of error.

Most of the benefits of open systems are being realized Most of the benefits of open systems are being realized An organization-wide repository of open system lessons learned existsAn organization-wide repository of open system lessons learned exists Selection of interface standards based on well-defined criteria and Selection of interface standards based on well-defined criteria and

prioritiespriorities Quantitative open system objectives and indicators have been establishedQuantitative open system objectives and indicators have been established MOSA Process has been integrated with all the other appropriate MOSA Process has been integrated with all the other appropriate

organizational processesorganizational processes

CharacterizedCharacterized by openness ofby openness ofappropriate keyappropriate key

interfaces within interfaces within and among selected and among selected

systemssystems



OS Maturity Level 3: Reasonable (Individual Systems) OpennessOS Maturity Level 3: Reasonable (Individual Systems) Openness

Key interfaces within certain systems have been defined and Key interfaces within certain systems have been defined and feasibility of using open standards for such interfaces is assessed feasibility of using open standards for such interfaces is assessed

A well-defined MOSA Process based on sound system engineering A well-defined MOSA Process based on sound system engineering processes, and common and organization-wide understanding of the processes, and common and organization-wide understanding of the activities, roles, and responsibilities has been establishedactivities, roles, and responsibilities has been established

MOSA Process has been integrated with the overall acquisition and MOSA Process has been integrated with the overall acquisition and system engineering processessystem engineering processes

A roadmap for MOSA implementation has been prepared A roadmap for MOSA implementation has been prepared Within established systems/platforms, cost, schedule, and Within established systems/platforms, cost, schedule, and

functionality are under control, and progress is successfully being functionality are under control, and progress is successfully being trackedtracked

The criteria for selecting interface standards have been establishedThe criteria for selecting interface standards have been established Limited measures for tracking the progress on MOSA Limited measures for tracking the progress on MOSA

implementation are establishedimplementation are established An organization-wide open system training program has been An organization-wide open system training program has been

established to ensure adequate open system knowledge and skillsestablished to ensure adequate open system knowledge and skills Verification and validation testing mechanisms have been developedVerification and validation testing mechanisms have been developed


interfaces within interfaces within selected systems selected systems



OS Maturity Level 2: Limited (Subsystem) Openness OS Maturity Level 2: Limited (Subsystem) Openness

An open system czar has been appointed and his/her areas of An open system czar has been appointed and his/her areas of responsibility have been assignedresponsibility have been assigned

A basic Modular Open System Process management discipline has been A basic Modular Open System Process management discipline has been established on an ad-hoc basis established on an ad-hoc basis

The process for implementing open systems in selected subsystems is The process for implementing open systems in selected subsystems is being monitored and controlled being monitored and controlled

A plan to assess the feasibility of the open system strategy has been A plan to assess the feasibility of the open system strategy has been preparedprepared

Open system policies and procedures for their implementation have been Open system policies and procedures for their implementation have been establishedestablished

Limited open system training is providedLimited open system training is provided A majority of obstacles toward open system implementation have been A majority of obstacles toward open system implementation have been

identified and removedidentified and removed Stakeholders responsible for implementation or being impacted by open Stakeholders responsible for implementation or being impacted by open

systems are identified systems are identified Some resources for open system implementation have been allocatedSome resources for open system implementation have been allocated Suppliers are encouraged to use open interfaces for certain components Suppliers are encouraged to use open interfaces for certain components Some of the barriers toward open system implementation are identified Some of the barriers toward open system implementation are identified

and removed.and removed. Earlier successes in open system implementation are repeated as limited Earlier successes in open system implementation are repeated as limited

lessons are gathered, analyzed, and documentedlessons are gathered, analyzed, and documented


interfaces within interfaces within limited subsystems limited subsystems



OS Maturity Level 1: Negligible Openness OS Maturity Level 1: Negligible Openness

The organization has identified the benefits of open systems and The organization has identified the benefits of open systems and has adopted open standards for a few key interfaces within one has adopted open standards for a few key interfaces within one or more components comprising a subsystem (e.g., use of open or more components comprising a subsystem (e.g., use of open interface standard for a display processor) interface standard for a display processor)

Open system application at this level is characterized by:Open system application at this level is characterized by:– No Modular Open System Process management disciplineNo Modular Open System Process management discipline– Scattered open interfaces due to individual system Scattered open interfaces due to individual system

engineer’s preferenceengineer’s preference– Firefighting (e.g., reacting to immediate interoperability or Firefighting (e.g., reacting to immediate interoperability or

integration problems) integration problems) – excessive risk takingexcessive risk taking– Lack of short and long-term open system strategies and Lack of short and long-term open system strategies and

plans (reactionary management philosophy) plans (reactionary management philosophy) – Orientation towards proprietary products, interface, and Orientation towards proprietary products, interface, and

systems systems – Absence of organization-wide policies for open system Absence of organization-wide policies for open system

implementationimplementation– Over the budget and schedule products and systems.Over the budget and schedule products and systems.

CharacterizedCharacterized by openness of a by openness of a few components few components

within a subsystem within a subsystem



Proposed MOSA CMM Appraisal MethodProposed MOSA CMM Appraisal Method

The proposed method is based on 3 simple steps:The proposed method is based on 3 simple steps:

1.1. Reviewing the tangible output of the MOSA process (e.g., tangible Reviewing the tangible output of the MOSA process (e.g., tangible MOSA benefits, hardware and software interface descriptions, MOSA benefits, hardware and software interface descriptions, type of standards used, conformance management activities, etc.)type of standards used, conformance management activities, etc.)

2.2. Conducting structured interviews and gathering detailed relevant Conducting structured interviews and gathering detailed relevant objective evidences from a total of 116 questions that provide objective evidences from a total of 116 questions that provide ability to "tailor" the parameters of the appraisal to the needs of the ability to "tailor" the parameters of the appraisal to the needs of the organizationorganization

3.3. Using the gathered evidence as the basis for appraising the Using the gathered evidence as the basis for appraising the maturity of MOSA Process implementation in the sponsor maturity of MOSA Process implementation in the sponsor organizationorganization


The Appraisal InstrumentThe Appraisal Instrument

Contains a total of 125 structured questions Contains a total of 125 structured questions with the following MOSA Maturity level with the following MOSA Maturity level breakdown:breakdown:

Maturity Level 5 – 17 questionsMaturity Level 5 – 17 questions Maturity Level 4 – 50 questionsMaturity Level 4 – 50 questions Maturity Level 3 – 38 questionsMaturity Level 3 – 38 questions Maturity Level 2 – 20 questionsMaturity Level 2 – 20 questions

Appraisal Areas Covered by the Appraisal Areas Covered by the QuestionnaireQuestionnaire

Capability DevelopmentCapability Development Acquisition StrategyAcquisition Strategy Contracting Contracting Design ProcessDesign Process ResourcesResources Risk MitigationRisk Mitigation ToolsTools Test and EvaluationTest and Evaluation

Core Tasks For Implementing MOSA Process Core Tasks For Implementing MOSA Process


Establish Infrastructure(e.g., Strategy, Policy, Guidanceand Lessons Learned Repository

Conduct Market Research & Identify Key Interfaces

Support Conformance Management, Appraisal

and Reviews

Influence RequirementDevelopment Process &

Strategies

Support Architecture Development and Source Selection

Support System Design and Architecture

Assessment

Open SystemVision

FEEDBACK

Use Widely SupportedOpen Standards

for Key Interfaces

Appraisal RatingAppraisal Rating

Based on the mean response to the specific Based on the mean response to the specific maturity level questions, the appraisal team may maturity level questions, the appraisal team may assign the following MOSA process assign the following MOSA process implementation rating: implementation rating:

5 - Fully implemented5 - Fully implemented 4 - Largely implemented4 - Largely implemented 3 - Moderately implemented3 - Moderately implemented 2 - Partially implemented2 - Partially implemented 1 - Not implemented 1 - Not implemented

MOSA CMM Appraisal MethodMOSA CMM Appraisal MethodAreas of ApplicationAreas of Application

Assess MOSA Process improvement of Assess MOSA Process improvement of Programs Programs

– NewNew– LegacyLegacy

SuppliersSuppliers Organizations of all sizes involved in engineering Organizations of all sizes involved in engineering

and development of :and development of :– Defense and other IT Intensive Systems comprising of mechanical, Defense and other IT Intensive Systems comprising of mechanical,

electrical, electronic, hydraulic, power, sensor, and other electrical, electronic, hydraulic, power, sensor, and other componentscomponents

– Tools and test facilitiesTools and test facilities


Proposed MOSA CMM Appraisal Method Proposed MOSA CMM Appraisal Method AdvantagesAdvantages

1.1. Simplicity Simplicity – The appraisal process only uses 3 steps and a number of – The appraisal process only uses 3 steps and a number of interviewsinterviews

2.2. Flexibility Flexibility – The questions that can be tailored provide to the needs of – The questions that can be tailored provide to the needs of the organizationthe organization

3.3. QuicknessQuickness – the process appraisal could be implemented in less than 3 – the process appraisal could be implemented in less than 3 weeksweeks

4.4. Standard GuidanceStandard Guidance – The appraisal uses well accepted definitions – The appraisal uses well accepted definitions5.5. Cost effectiveCost effective – The main focus of the appraisal is on using the MOSA – The main focus of the appraisal is on using the MOSA

Process as a means toward fulfillment of benefitsProcess as a means toward fulfillment of benefits6.6. Ease of implementationEase of implementation - the method could be easily deployed with a - the method could be easily deployed with a

minimum of required consultants and outside vendor support.minimum of required consultants and outside vendor support.7.7. UnderstandableUnderstandable – The method avoids heavy jargon and use of – The method avoids heavy jargon and use of

acronyms The terminology used is grounded in common business acronyms The terminology used is grounded in common business vernacular.vernacular.


Appraisal GuidelinesAppraisal Guidelines

Ensure the involvement of the sponsor organization and other stakeholders Ensure the involvement of the sponsor organization and other stakeholders throughout the appraisal process throughout the appraisal process

Ensure that the appraisal team has proper skillsEnsure that the appraisal team has proper skills Tailor the proposed appraisal method to specific needs and the level of MOSA Tailor the proposed appraisal method to specific needs and the level of MOSA

understandingunderstanding Immediately address the high risk areas and accentuate the positives found Immediately address the high risk areas and accentuate the positives found Do not make the appraisal or MOSA Process improvement an end into itself -Do not make the appraisal or MOSA Process improvement an end into itself -

Translate the appraisal findings to specific business gainsTranslate the appraisal findings to specific business gains Record the traceability between the data collected during the appraisal and the Record the traceability between the data collected during the appraisal and the

findings and/or ratings findings and/or ratings Findings should specify the appraisal constraints, and serve as a basis for Findings should specify the appraisal constraints, and serve as a basis for

organizational and MOSA process improvements organizational and MOSA process improvements Ensure the confidentially of the information Ensure the confidentially of the information Ensure open communicationEnsure open communication Do not forget to document throughout the appraisal processDo not forget to document throughout the appraisal process Be objectiveBe objective

An Integrative Method for Appraising Modular Open System Maturity Cyrus Azani Open Systems Joint...

Documents

Transcript of An Integrative Method for Appraising Modular Open System Maturity Cyrus Azani Open Systems Joint...