PMI/OPM3 and CMMI Assessment

Alan McSweeney

April 18, 2023 2

Objectives

• Provide customer with an understanding of the approach to using PMI project methodology to use to implement IT quality management

April 18, 2023 3

Agenda

• PMI/OPM3 and CMMI in the context of COBIT• Assessing PMI/OPM3 and CMMI• Approach• Indicative financial analysis• Next steps

April 18, 2023 4

Background

• Maturity models allow organisations to identify and assess areas in need of process improvement

• IT Controls− IT must implement internal controls around how it operates− The systems IT delivers to the business and the underlying

business processes these systems actualise must be controlled – these are controls external to IT

• CMMI and OPM3 are two such maturity models− CMMI focuses on software engineering− OPM3 focuses on project management across any project based

activity

• The de-facto standard for IT governance is COBIT− Control Objectives for Information and related Technology

April 18, 2023 5

IT Service Delivery Issues and Challenges

• Keeping up with business needs

• User and IT dissatisfaction with products and services

• High costs of delivery

• Delivery cycles too long

• Technology infrastructure out-dated

• Projects late and over budget

• Meeting service levels

• Regulatory requirements

April 18, 2023 6

OPM3

• OPM3− Organizational Project Management Maturity Mode (OPMMM or

OPM3)− Part of PMI – project maturity standard for organisations

• OPM3 focuses on knowledge, assessment and improvement− Knowledge - why organisational project management and

maturity are important and how to recognise enterprise competency

− Assessment - the procedure an organisation uses to determine its maturity

− Improvement - provides information on how an organisation can increase its organisational project management maturity

April 18, 2023 7

PMI – Project Management Areas

ProjectIntegration

Management

Project Scope

Management

Project Time

Management

ProjectCost

Management

Project Quality

Management

ProjectHuman Resource

Management

ProjectCommunications

Management

ProjectRisk

Management

ProjectProcurementManagement

April 18, 2023 8

Many Quality Management Frameworks

Baldridge QAI/QM COSO COBIT

COQ SIX SIGMA ISO

ITIL CMMI V-Model

April 18, 2023 9

SEI Capability Maturity Model Integrated (CMMI)

Initial

Repeatable

Defined

Managed

Optimising

Ad Hoc

DisciplinedProcesses(Project)

StandardDisciplinedProcesses(Organisation)

PredictableProcesses

ContinuousImprovement

April 18, 2023 10

Comparison of Standards

April 18, 2023 11

What is COBIT?

• The de-facto industry framework for the management of Information Technology standards and processes

• All other frameworks and standards are a sub set of the COBIT framework

• COBIT comprises− 4 Domains− 34 Processes− 318 Control Objectives

April 18, 2023 12

COBIT

• COBIT aims to be different from other quality and governance approaches in two ways1. It is an IT governance framework and supporting

set of tools that IT can use to bridge the gap between control requirements, technical issues and business risks

2. It provides a detailed implementation structure and toolset that translates the framework theory into a practical and achievable deliverables

April 18, 2023 13

COBIT and Other Standards

• COBIT provides a framework and an associated toolset that allow IT implement controls and address technical issues and business risks and communicate that level of control to IT business stakeholders− By providing a toolset COBIT enables the development of policy

and practice for IT control throughout the enterprise.

• COBIT is integrated with other standards and thus can become an umbrella framework for IT governance− It assists in understanding and managing the risks and benefits

associated with IT− The process structure of COBIT and its business-oriented

approach provides an end-to-end view of IT

April 18, 2023 14

COBIT Domain and Process Structure

April 18, 2023 15

COBIT Structure

April 18, 2023 16

Maturity Models and COBIT

• Typically when an organisation undertakes a maturity assessment, it achieves a single (scored) rating that summarizes appraisal results and makes comparisons among the projects and processes via a staged representation format

• Each stage indicates the level of maturity in a graded scale of process improvement

• The model starts with basic management practices and progresses through a path of successive levels. No stages can be skipped

• To fully map and understand a maturity model, you must place the model in an IT governance context hence the COBIT framework

April 18, 2023 17

COBIT Process Domains and The Delivery of Information to Meet Objectives

``

Monitor andEvaluate

Plan andOrganise

Deliver andSupport

Acquire andImplement

Information

GovernanceObjectives

BusinessObjectives

April 18, 2023 18

COBIT Domains and Processes

Plan and Organise (PO) Acquire and Implement (AI ) Deliver and Support (DS) Monitor and Evaluate (ME) PO1 Define a strategic IT plan AI1 Identify automated

solutions DS1 Define and manage service levels

ME1 Monitor and evaluate IT performance

PO2 Define the information architecture

AI2 Acquire and maintain application software

DS2 Manage third-party services

ME2 Monitor and evaluate internal control

PO3 Determine technological direction

AI3 Acquire and maintain technology infrastructure

DS3 Manage performance and capacity

ME3 Ensure regulatory compliance

PO4 Define the IT processes, organisation and relationships

AI4 Enable operation and use DS4 Ensure continuous service ME4 Provide IT governance

PO5 Manage the IT investment AI5 Procure IT resources DS5 Ensure systems security PO6 Communicate management aims and direction

AI6 Manage changes DS6 Identify and allocate costs

PO7 Manage IT human resources

AI7 Install and accredit solutions and changes

DS7 Educate and train users

PO8 Manage quality DS8 Manage service desk and incidents

PO9 Assess and manage IT risks

DS9 Manage the configuration

PO10 Manage projects DS10 Manage problems DS11 Manage data DS12 Manage the physical

environment

DS13 Manage operations

April 18, 2023 19

COBIT Information Measurement Criteria

• COBIT defines seven measurement criteria:1. Effectiveness - Deals with information being relevant and pertinent

to the business process as well as being delivered in a timely, correct, consistent and usable manner

2. Efficiency - Concerned with the provision of the information through the optimal use of resources

3. Confidentiality - Concerned with the protection of sensitive information from unauthorised disclosure

4. Integrity - Relates to the accuracy and completeness of information as well as to its validity in accordance with business values and expectations

5. Availability - Relates to the information being available when required by the business process now and in the future

6. Compliance - Deals with complying with laws, regulations and contractual arrangements

7. Reliability - Relates to the provision of appropriate information for the workforce of the organisation

April 18, 2023 20

COBIT Process Goals and Metrics

• Goal

• Activity Goals

• Process Goals

• IT Goals

• Metric

• Key Performance Indicators

• Process Key Goal Indicators

• IT Key Goal Indicators

April 18, 2023 21

Sample Goals and Metrics for the COBIT Process PO1 Define a Strategic IT Plan

Activity Goals Process Goals I T Goals Engaging with business and senior

management in aligning IT strategic planning with current and future business needs

Understanding current IT capabilities Translating IT strategic planning into

tactical plans Providing for a prioritisation scheme for

the business objectives that quantifies the business requirements

Define how business requirements are translated in service offerings.

Define the strategy to deliver service offerings.

Contribute to the management of the portfolio of IT-enabled business investments.

Establish clarity of business impact of risks to IT objectives and resources.

Provide transparency and understanding of IT costs, benefits, strategy, policies and service levels.

Respond to business requirements in alignment with the business strategy.

Respond to governance requirements in line with board direction.

Key Performance Indicators Process Key Goal I ndicators I T Key Goal Indicators Delay between updates of business

strategic/tactical plan and updates of IT strategic/tactical plan

% of strategic/tactical IT plan meetings where business representatives have actively participated

Delay between updates of IT strategic plan and updates of IT tactical plans

% of tactical IT plans complying with the Predefined structure/contents of those

plans % of IT initiatives/projects championed

by business owners

% of IT objectives in the IT strategic plan that support the strategic business plan

% of IT initiatives in the IT tactical plan that support the tactical business plan

% of IT projects in the IT project portfolio that can be directly traced back to the IT tactical plan

Degree of approval of business owners of the IT strategic/tactical plans

Degree of compliance with business and governance requirements

Level of satisfaction of the business with the current state (number, scope, etc.) of the project and applications portfolio

April 18, 2023 22

COBIT Generic Process Controls

• In addition to the process-specific control objectives, COBIT includes a set of generic process controls that are applied to all processes− PC1 Process Owner - Assign an owner for each COBIT process such

that responsibility is clear− PC2 Repeatability - Define each COBIT process such that it is

repeatable− PC3 Goals and Objectives - Establish clear goals and objectives for

each COBIT process for effective execution− PC4 Roles and Responsibilities - Define unambiguous roles,

activities and responsibilities for each COBIT process for efficient execution

− PC5 Process Performance - Measure the performance of each COBIT process against its goals

− PC6 Policy, Plans and Procedures - Document, review, keep up to date, sign off on and communicate to all involved parties any policy, plan or procedure that drives a COBIT process

April 18, 2023 23

COBIT Generic Application Controls

• As with the generic process controls, COBIT includes a set of generic application controls that are applied to all processes

− Data Origination/Authorisation Controls• AC1 Data Preparation Procedures• AC2 Source Document Authorisation Procedures• AC3 Source Document Data Collection• AC4 Source Document Error Handling• AC5 Source Document Retention

− Data Input Controls• AC6 Data Input Authorisation Procedures• AC7 Accuracy, Completeness and Authorisation Checks• AC8 Data Input Error Handling• Data Processing Controls• AC9 Data Processing Integrity• AC10 Data Processing Validation and Editing• AC11 Data Processing Error Handling

− Data Output Controls• AC12 Output Handling and Retention• AC13 Output Distribution• AC14 Output Balancing and Reconciliation• AC15 Output Review and Error Handling• AC16 Security Provision for Output Reports

− Boundary Controls• AC17 Authenticity and Integrity• AC18 Protection of Sensitive Information During Transmission and Transport

April 18, 2023 24

Current Situation

• As CMMI came first (published in 1991), many organisations have implemented CMMI and have developed processes and standards to support this framework

• With the later arrival of OPM3, many organisations are trying to establish where it fits, and whether and how a software engineering maturity model works in conjunction with a project management maturity model

April 18, 2023 25

Benefits of Implementing IT Control Framework

• Better IT to business alignment built on a business focus

• Management view of what IT does• Clear ownership and responsibilities, based on process

orientation• General acceptability with third parties and regulators• Shared understanding amongst all stakeholders, based

on a common language• Fulfillment of the governance requirements for the IT

control environment

April 18, 2023 26

Approach

AnalyseAssess and

Identify Gaps

Recommend and Quantify Next Steps

Step 1 Step 2 Step 3

April 18, 2023 27

Step 1: Analyse

•Establish scope of assessment within Customer using COBIT framework and domains

• Identify overlaps, differences and gaps between the two frameworks using COBIT’s domains within this scope

April 18, 2023 28

Example Comparison of CMMI and OMP3

Domain AssessmentPO Processes are moderately addressed by both ITIL

and PMBOK and rarely addressed or none at all by CMMI

AI Processes are frequently addressed by CMMI, moderately addressed by ITIL and none at all by PMBOK

DS Processes are frequently addressed by ITIL and rarely addressed or none at all by OPM3 and CMMI

ME Processes are moderately addressed by CMMI and rarely addressed or none at all by ITIL and PMBOK. Keep in mind a domain ranking for the three compared frameworks is a summary of rankings for each process in the domain

April 18, 2023 29

Step 2: Assess and Identify Gaps

• What is the impact of gaps in CMMI coverage in Customer’s environment?

• Will OPM3 bridge these gaps?• Can the gap closure requirement be clearly

stated in a specific recommendation?• What benefit would be derived from closing the

gap?

April 18, 2023 30

Step 3: Recommend and Quantify Next Steps

• Are the benefits of the recommendations clearly quantified?

• Can they be delivered within a realistic timetable?

April 18, 2023 31

Conclusions

• OPM3 and CMMI are not exclusive standards, and can be used together

• A practical, benefits-driven approach is required to assess the benefit of combining OPM3 with CMMI

• This must be considered within an overall framework (COBIT) if the two maturity models are not to be seen to compete

• To do this successfully, the following factors also need to be assessed− The level of compliance the business is currently subject to− The amount of software engineering and project based activity

being undertaken− The Project management skills and experience currently within

the organisation

April 18, 2023 32

More Information

Alan McSweeneyalan@alanmcsweeney.com