Defence Resource Management Information System –

Integrated Support Services and

SIGMA Integrated Support Services

Request for Proposals

Fairness Monitor Final Report

Project: Defence Resource Management Information System

Date of submission: April 22, 2016

Submitted to: Director, Fairness Monitoring

PPI Consulting Limited

Centrepointe Chambers

86 Centrepointe Drive

Ottawa ON K2G 6B1

613-567-0000

www.ppiconsultinglimited.com

Ottawa | Toronto | Atlantic Canada

2

Table of Contents 1 Introduction .......................................................................................................................................... 3

2 Project requirement .............................................................................................................................. 4

3 Attestation of assurance ....................................................................................................................... 5

4 FM engagement and observations ....................................................................................................... 6

4.1 Request for Information process ...................................................................................................... 6

4.1.1 FM Specialist observations ........................................................................................................... 6

4.2 Request for Proposals process .......................................................................................................... 7

4.2.1 Bid Open Period ............................................................................................................................ 7

4.2.2 FM Specialist observations ........................................................................................................... 7

4.2.3 Technical bid evaluation ............................................................................................................... 8

4.2.4 Financial proposal assessment.................................................................................................... 10

4.2.5 FM Specialist observations – RFP process .................................................................................. 10

5 Reference documents ......................................................................................................................... 11

3

1 Introduction PPI Consulting Limited (PPI) was engaged by Public Services and Procurement Canada (PSPC) on October

12, 2012 as the Fairness Monitor (FM) contractor to observe the procurement process related to PSPC's

acquisition of Support Services for the Department of National Defence (DND)'s Defence Resource

Management Information System (DRMIS) and PSPC's SIGMA System.

PPI is an independent third party with respect to this activity.

We reviewed all of the information provided and observed all relevant activities.

We hereby submit our Final Report, covering the activities of the Fairness Monitor, commencing with the

Letter of Interest/Request for Information process, Solicitation No: W8474-126279/A issued on

November 16, 2012 continuing through an initial Request for Proposals, Solicitation number W8474-

126279/D dated June 12, 2014, which was subsequently cancelled, and the re-issued Request for

Proposals process, Solicitation No: W8474-126279/F issued on December 10, 2015 and subsequent

evaluation consensus meetings.

This report includes our attestation of assurance, a summary of the scope and objectives of our

assignment, the methodologies applied and relevant observations from the activities undertaken.

4

2 Project requirement The Department of National Defence (DND) has an SAP-based integrated information system for the

support of Materiel Acquisition and Support and Financial and Managerial Accounting business processes,

as well as a number of other related processes. Implementation of this integrated information system

involved the replacement of a large number of Materiel Acquisition and Support Information System

(MASIS) and the Financial and Managerial Accounting System (FMAS). In April 2010, these two systems

were merged to establish a single core SAP ECC 6.0 based solution, known as DRMIS. Further

consolidation of business processes has occurred with the implementation of an SAP solution for supply

chain functionality in DRMIS, as well as the incorporation of other business processes including those

related to real property management.

As the Department continues to leverage DRMIS to meet evolving requirements, and as functionality is

increased and organizational coverage is widened (for example, the integration into DRMIS of supply chain

functionality and other business processes such as those related to real property management) it is

expected that DRMIS will grow to over 30,000 users in the coming years. These future opportunities will

allow the organization to achieve maximum benefit from its Enterprise Resource Planning (ERP) platforms.

PSPC has implemented an integrated system that supports its financial, procurement and real property

business lines using SAP ECC, known as SIGMA which includes SAP Business Intelligence (BI, SAP

Productivity Pak and other SAP tools and applications).

SIGMA is a complex SAP-based system. It was initially implemented in 2008 to meet departmental

financial and material management requirements and is currently being used by over 6,200 users located

within PSPC and in client departments.

The Department continues to leverage SIGMA to meet evolving requirements in order to achieve savings

and operational efficiencies. With the integration of real property functionality into SIGMA, it is expected

that SIGMA will grow to over 8,700 users in the coming years.

PSPC continues to identify opportunities that would allow the Department and the Government of Canada

(GC) to realize maximum benefit from its ERP platform.

5

3 Attestation of assurance It is our professional opinion that the solicitation process for the procurement of Support Services for

DND’s Defence Resource Management Information System and PSPC’s SIGMA System, that we observed,

was carried out in a fair, open and transparent manner.

Original signed by

_______________________________________

John Davis, CEO, PPI Consulting Limited

Original signed by

_______________________________________

Ian Brennan, CSCMP, FM Team Leader

Original signed by

_______________________________________

Bob Tibbo, FM Specialist

6

4 FM engagement and observations

4.1 Request for Information process

During the RFI Phase, the FM Specialist attended the following meetings:

Commercially Confidential Meetings with 10 interested suppliers (held July 16, 17, and 20, 2015)

Project team meetings/discussions to review status, evaluation approach, issues resolution,

evaluation criteria, specifications etc.

During the RFI Phase, the FM Specialist was provided with the following documents for review:

Notices of Proposed Procurement

Draft RFP documents

Correspondence with industry e.g. containing clarifications to questions raised during the

Commercially Confidential Meetings

Draft Evaluation Plan

Industry Engagement Summary

Interested suppliers provided their responses to the draft RFP. Participation in the Industry consultation

process was not a mandatory requirement and suppliers that did not participate in the consultation

process were advised that they would remain eligible to submit a proposal in response to any future RFP

relating to the DRMIS procurement.

The documents and activities completed in this phase complied with the fairness principles below:

All potential proponents were provided with the same opportunity and the same information at

the same time

Potential proponents were afforded the same opportunity to provide feedback with respect to

the requirements and proposed procurement approach

Potential proponents’ feedback received due consideration by the client based on an

assessment against the client’s operational and business requirements.

Requests for further explanation or clarification were provided to all participants.

4.1.1 FM Specialist observations The FM Specialist reviewed the draft solicitation documents and attended the Commercially Confidential

Meetings with interested suppliers.

Any requests for clarification by the suppliers during the Commercially Confidential Meetings were

addressed by the DND / PSPC project team and by the Contracting Authority either during the meetings

or provided following the meetings. The same information was provided to all participants.

7

Any observations or requests for clarification made by the FM Specialist were addressed by the DND /

PSPC project team and by the Contracting Authority.

Both the DND / PSPC project team and the Contracting Authority were responsive in providing clarification

when requested by the FM Specialist.

4.2 Request for Proposals process An RFP for the DRMIS & SIGMA In-Service Support Services was initially posted on the buyandsell website

as solicitation number W8474-126279/D on June 12, 2014 with a bid closing date of August 26, 2014. This

RFP process was subsequently cancelled. The FM attests that all activities observed by the FM for this

cancelled process were carried out in a fair, open and transparent manner.

An RFP for DRMIS and SIGMA Integrated Support Services was subsequently re-issued on the buyandsell

website on December 10, 2015 as bid solicitation number W8474-126279/F, with a bid closing date of

February 29, 2016. The re-issued RFP document replaced the previous version entirely.

During the re-issued RFP Phase, the FM Specialist attended the following meetings:

Project team meetings/discussions to review status, evaluation approach, issues resolution,

evaluation criteria, specifications etc.

During the RFP Phase, the FM Specialist was provided with the following documents for review:

Draft and final RFP documents

Correspondence with industry e.g. containing clarifications to questions raised during the

Commercially Confidential Meetings

Evaluation Plan

Evaluator Briefing / Instructions to Evaluators

4.2.1 Bid Open Period The RFP for DRMIS was issued on the buyandsell website as Solicitation No. W8474-126279/F, on

December 10, 2015 with an original closing date of January 21, 2016. During the Bid Open Period 15

amendments were posted on the buyandsell website, responding to 74 questions from industry. In

amendment 12 dated February 17, 2016, the closing date was extended to February 29, 2016.

Four proposals were received by the submission closing date and time.

4.2.2 FM Specialist observations The FM Specialist reviewed the draft and final solicitation documents.

Any requests for clarification during the Bid Open Period were addressed by the DND / PSPC project team

and by the Contracting Authority. The same information was provided to all Proponents by means of

8

addenda posted on the buyandsell website. Both the DND / PSPC project team and the Contracting

Authority were responsive in providing clarification when requested by the FM Specialist.

Any observations or requests for clarification made by the FM Specialist were addressed by the DND /

PSPC project team and by the Contracting Authority. Both the DND / PSPC project team and the

Contracting Authority were responsive in providing clarification when requested by the FM Specialist.

4.2.3 Technical bid evaluation

The FM Specialist attended the following meetings:

Evaluator Briefing / Kick-off meeting held March 2, 2016

Evaluation consensus meetings held March 16 -23, 2016;

Evaluation Peer Review meeting held April 5, 2016;

Evaluation consensus follow-up meeting held April 6, 2016;

The following process was undertaken to evaluate the bids:

Each evaluator conducted his/her own independent review and scoring of the proposal in his/her

evaluator workbook.

Once the independent reviews were completed, the evaluators participated in meetings to agree

on the consensus assessment for both mandatory and rated requirements.

o Technical mandatory requirements were reviewed for compliance with the information

provided in both the compliance matrix and in the substantiating information in the proposal.

A record of compliance or non-compliance was recorded in the master record along with the

rationale.

o Each Bid’s responses to the rated requirements were reviewed separately, on an item-by-

item basis against the identified evaluation criteria. For each requirement, the individual

evaluator scores and the rationale for the scores were reviewed and discussed by the team.

The evaluators used a consensus approach to determine a single score and rationale for each

rated requirement that all evaluators agreed to. The consensus score and associated

rationale was recorded in the master record.

Evaluators recorded and initialled any changes agreed to during consensus in their individual

workbooks.

The FM confirmed with each member of the evaluation team that they were in agreement with

the final consensus scores and rationale.

All evaluators confirmed their agreement with the recorded scores and rationale.

All evaluators confirmed that they had treated all four Bidders equitably and fairly.

The evaluation team signed the consensus record.

9

The FM’s review of documents and monitoring of activities in this phase considered the principles of

fairness, openness and transparency. This included the following:

Evaluation Plan was consistent with the published procurement documents;

Evaluation team members were chosen and confirmed prior to the receipt of proposals;

Evaluation training was provided to all evaluators. This included informing evaluators of the

following:

o identity of the proposals received and requesting evaluators to declare any conflict of

interest;

o confidentiality protocols;

o document control;

o clarification process;

o overview of scoring workbooks and method for individual assessment;

o explanation of the concept of hidden criteria;

o explanation of fairness and the need for objectivity, consistency and equitable treatment of

all proposals;

o guidance with respect to only assessing the information that is in the proposal – not

information from previous contracts, personal knowledge, or any other information.

The scoring criteria and assessment tools were established prior to the receipt of proposals and

were consistent with the RFP. The scoring criteria contained no hidden criteria;

The submissions were logged and recorded upon receipt, clearly identifying that these were

submitted on time;

The pricing was contained in a separate envelope (as applicable);

The proposals complied with the mandatory submission requirements;

The process for establishing one score from a team of evaluators was established prior to the

receipt of proposals (consensus scoring was used);

The same team of evaluators evaluated all proposals;

The scoring assessment was applied consistently and equitably by the evaluation team with no

evidence of bias;

A secure location for the evaluation exercise was established for the period of the evaluation;

Proposal documents were physically secured within a secure location.

All four proposals were deemed to have met the technical bid requirements of the RFP. The FM had no

fairness concerns related to the technical bid evaluation.

10

4.2.4 Financial proposal assessment PSPC had assigned a separate individual to complete the assessment of financial proposals. This

individual had no contact with the technical evaluation team, and had no knowledge of the technical

evaluation nor its results. Similarly, the PSPC Contracting Authority who attended the technical

evaluation team meetings had no knowledge of the financial proposal assessment, nor its results.

The PSPC individual assigned to complete the assessment of financial proposals reviewed and verified the

financial proposal of each bid. The individual verified that each financial proposal was complete and

complied with the requirements of the RFP.

The PSPC individual completed the assessment of all financial proposals using the process defined in the

RFP.

The FM met with the PSPC individual on April 4, 2016 to review and verify the financial proposal process

and results. The FM was shown all financial spreadsheets and was able to verify that the process defined

in the RFP had been used. The FM verified that the financial assessment results were consistent with the

proper application of the process defined in the RFP.

The FM had no fairness concerns related to the assessment of financial proposals.

4.2.5 FM Specialist observations – RFP process The FM witnessed the technical bid evaluation meetings and confirms that the process was conducted

appropriately in accordance with the process established in the RFP and in a fair, open and transparent

manner.

The FM reviewed the assessment of financial proposals and confirms that the process was conducted

appropriately in accordance with the process established in the RFP and in a fair, open and transparent

manner.

The DND / PSPC team and the Contracting Authority were responsive in providing clarification when

requested by the FM Specialist.

The evaluation teams were knowledgeable and diligent in their evaluation of proposals. All four proposals

were evaluated consistently and fairly.

11

5 Reference documents Documents pertaining to the Request for Proposals process for Support Services for DND’s DRMIS and

PSPC’s SIGMA (Solicitation # W8474-126279/F) can be found on the buyandsell website.

Defence Resource Management Information System – Integrated Support

Services and SIGMA Integrated Support Services

Addendum to the Final Report

September 6, 2016

Addendum to Fairness Monitor final report dated April 22, 2016.

This Addendum to the Fairness Monitor final report covers the period following the conclusion of the

evaluation phase up to and including contract award.

Contract award

As a result of solicitation # W8474-126279/F for Defence Resource Management Information System –

Integrated Support Services and SIGMA Integrated Support Services, of the four bidders, three successful

bidders were selected and contracts were awarded on June 21, 2016. Of the four bidders, three requested

and received a debriefing meeting including the one unsuccessful bidder. The fourth bidder requested

and was provided debriefing comments via email. The FM reviewed all debriefing comments and

attended the three debriefing meetings that were held in person.

It is our professional opinion that the Defence Resource Management Information System – Integrated

Support Services and SIGMA Integrated Support Services procurement process that we observed, was

carried out in a fair, open and transparent manner.

Original signed by Original signed by

John Davis, CEO, PPI Consulting Limited Ian Brennan, CSCMP, FM Team Leader

Original signed by

Original signed by

Bob Tibbo, FM Specialist (Primary) Mairi Curran, FM Specialist (Backup)