Defence Resource Management Information System ... · Defence Resource Management Information...
Transcript of Defence Resource Management Information System ... · Defence Resource Management Information...
Defence Resource Management Information System –
Integrated Support Services and
SIGMA Integrated Support Services
Request for Proposals
Fairness Monitor Final Report
Project: Defence Resource Management Information System
Date of submission: April 22, 2016
Submitted to: Director, Fairness Monitoring
PPI Consulting Limited
Centrepointe Chambers
86 Centrepointe Drive
Ottawa ON K2G 6B1
613-567-0000
www.ppiconsultinglimited.com
Ottawa | Toronto | Atlantic Canada
2
Table of Contents 1 Introduction .......................................................................................................................................... 3
2 Project requirement .............................................................................................................................. 4
3 Attestation of assurance ....................................................................................................................... 5
4 FM engagement and observations ....................................................................................................... 6
4.1 Request for Information process ...................................................................................................... 6
4.1.1 FM Specialist observations ........................................................................................................... 6
4.2 Request for Proposals process .......................................................................................................... 7
4.2.1 Bid Open Period ............................................................................................................................ 7
4.2.2 FM Specialist observations ........................................................................................................... 7
4.2.3 Technical bid evaluation ............................................................................................................... 8
4.2.4 Financial proposal assessment.................................................................................................... 10
4.2.5 FM Specialist observations – RFP process .................................................................................. 10
5 Reference documents ......................................................................................................................... 11
3
1 Introduction PPI Consulting Limited (PPI) was engaged by Public Services and Procurement Canada (PSPC) on October
12, 2012 as the Fairness Monitor (FM) contractor to observe the procurement process related to PSPC's
acquisition of Support Services for the Department of National Defence (DND)'s Defence Resource
Management Information System (DRMIS) and PSPC's SIGMA System.
PPI is an independent third party with respect to this activity.
We reviewed all of the information provided and observed all relevant activities.
We hereby submit our Final Report, covering the activities of the Fairness Monitor, commencing with the
Letter of Interest/Request for Information process, Solicitation No: W8474-126279/A issued on
November 16, 2012 continuing through an initial Request for Proposals, Solicitation number W8474-
126279/D dated June 12, 2014, which was subsequently cancelled, and the re-issued Request for
Proposals process, Solicitation No: W8474-126279/F issued on December 10, 2015 and subsequent
evaluation consensus meetings.
This report includes our attestation of assurance, a summary of the scope and objectives of our
assignment, the methodologies applied and relevant observations from the activities undertaken.
4
2 Project requirement The Department of National Defence (DND) has an SAP-based integrated information system for the
support of Materiel Acquisition and Support and Financial and Managerial Accounting business processes,
as well as a number of other related processes. Implementation of this integrated information system
involved the replacement of a large number of Materiel Acquisition and Support Information System
(MASIS) and the Financial and Managerial Accounting System (FMAS). In April 2010, these two systems
were merged to establish a single core SAP ECC 6.0 based solution, known as DRMIS. Further
consolidation of business processes has occurred with the implementation of an SAP solution for supply
chain functionality in DRMIS, as well as the incorporation of other business processes including those
related to real property management.
As the Department continues to leverage DRMIS to meet evolving requirements, and as functionality is
increased and organizational coverage is widened (for example, the integration into DRMIS of supply chain
functionality and other business processes such as those related to real property management) it is
expected that DRMIS will grow to over 30,000 users in the coming years. These future opportunities will
allow the organization to achieve maximum benefit from its Enterprise Resource Planning (ERP) platforms.
PSPC has implemented an integrated system that supports its financial, procurement and real property
business lines using SAP ECC, known as SIGMA which includes SAP Business Intelligence (BI, SAP
Productivity Pak and other SAP tools and applications).
SIGMA is a complex SAP-based system. It was initially implemented in 2008 to meet departmental
financial and material management requirements and is currently being used by over 6,200 users located
within PSPC and in client departments.
The Department continues to leverage SIGMA to meet evolving requirements in order to achieve savings
and operational efficiencies. With the integration of real property functionality into SIGMA, it is expected
that SIGMA will grow to over 8,700 users in the coming years.
PSPC continues to identify opportunities that would allow the Department and the Government of Canada
(GC) to realize maximum benefit from its ERP platform.
5
3 Attestation of assurance It is our professional opinion that the solicitation process for the procurement of Support Services for
DND’s Defence Resource Management Information System and PSPC’s SIGMA System, that we observed,
was carried out in a fair, open and transparent manner.
Original signed by
_______________________________________
John Davis, CEO, PPI Consulting Limited
Original signed by
_______________________________________
Ian Brennan, CSCMP, FM Team Leader
Original signed by
_______________________________________
Bob Tibbo, FM Specialist
6
4 FM engagement and observations
4.1 Request for Information process
During the RFI Phase, the FM Specialist attended the following meetings:
Commercially Confidential Meetings with 10 interested suppliers (held July 16, 17, and 20, 2015)
Project team meetings/discussions to review status, evaluation approach, issues resolution,
evaluation criteria, specifications etc.
During the RFI Phase, the FM Specialist was provided with the following documents for review:
Notices of Proposed Procurement
Draft RFP documents
Correspondence with industry e.g. containing clarifications to questions raised during the
Commercially Confidential Meetings
Draft Evaluation Plan
Industry Engagement Summary
Interested suppliers provided their responses to the draft RFP. Participation in the Industry consultation
process was not a mandatory requirement and suppliers that did not participate in the consultation
process were advised that they would remain eligible to submit a proposal in response to any future RFP
relating to the DRMIS procurement.
The documents and activities completed in this phase complied with the fairness principles below:
All potential proponents were provided with the same opportunity and the same information at
the same time
Potential proponents were afforded the same opportunity to provide feedback with respect to
the requirements and proposed procurement approach
Potential proponents’ feedback received due consideration by the client based on an
assessment against the client’s operational and business requirements.
Requests for further explanation or clarification were provided to all participants.
4.1.1 FM Specialist observations The FM Specialist reviewed the draft solicitation documents and attended the Commercially Confidential
Meetings with interested suppliers.
Any requests for clarification by the suppliers during the Commercially Confidential Meetings were
addressed by the DND / PSPC project team and by the Contracting Authority either during the meetings
or provided following the meetings. The same information was provided to all participants.
7
Any observations or requests for clarification made by the FM Specialist were addressed by the DND /
PSPC project team and by the Contracting Authority.
Both the DND / PSPC project team and the Contracting Authority were responsive in providing clarification
when requested by the FM Specialist.
4.2 Request for Proposals process An RFP for the DRMIS & SIGMA In-Service Support Services was initially posted on the buyandsell website
as solicitation number W8474-126279/D on June 12, 2014 with a bid closing date of August 26, 2014. This
RFP process was subsequently cancelled. The FM attests that all activities observed by the FM for this
cancelled process were carried out in a fair, open and transparent manner.
An RFP for DRMIS and SIGMA Integrated Support Services was subsequently re-issued on the buyandsell
website on December 10, 2015 as bid solicitation number W8474-126279/F, with a bid closing date of
February 29, 2016. The re-issued RFP document replaced the previous version entirely.
During the re-issued RFP Phase, the FM Specialist attended the following meetings:
Project team meetings/discussions to review status, evaluation approach, issues resolution,
evaluation criteria, specifications etc.
During the RFP Phase, the FM Specialist was provided with the following documents for review:
Draft and final RFP documents
Correspondence with industry e.g. containing clarifications to questions raised during the
Commercially Confidential Meetings
Evaluation Plan
Evaluator Briefing / Instructions to Evaluators
4.2.1 Bid Open Period The RFP for DRMIS was issued on the buyandsell website as Solicitation No. W8474-126279/F, on
December 10, 2015 with an original closing date of January 21, 2016. During the Bid Open Period 15
amendments were posted on the buyandsell website, responding to 74 questions from industry. In
amendment 12 dated February 17, 2016, the closing date was extended to February 29, 2016.
Four proposals were received by the submission closing date and time.
4.2.2 FM Specialist observations The FM Specialist reviewed the draft and final solicitation documents.
Any requests for clarification during the Bid Open Period were addressed by the DND / PSPC project team
and by the Contracting Authority. The same information was provided to all Proponents by means of
8
addenda posted on the buyandsell website. Both the DND / PSPC project team and the Contracting
Authority were responsive in providing clarification when requested by the FM Specialist.
Any observations or requests for clarification made by the FM Specialist were addressed by the DND /
PSPC project team and by the Contracting Authority. Both the DND / PSPC project team and the
Contracting Authority were responsive in providing clarification when requested by the FM Specialist.
4.2.3 Technical bid evaluation
The FM Specialist attended the following meetings:
Evaluator Briefing / Kick-off meeting held March 2, 2016
Evaluation consensus meetings held March 16 -23, 2016;
Evaluation Peer Review meeting held April 5, 2016;
Evaluation consensus follow-up meeting held April 6, 2016;
The following process was undertaken to evaluate the bids:
Each evaluator conducted his/her own independent review and scoring of the proposal in his/her
evaluator workbook.
Once the independent reviews were completed, the evaluators participated in meetings to agree
on the consensus assessment for both mandatory and rated requirements.
o Technical mandatory requirements were reviewed for compliance with the information
provided in both the compliance matrix and in the substantiating information in the proposal.
A record of compliance or non-compliance was recorded in the master record along with the
rationale.
o Each Bid’s responses to the rated requirements were reviewed separately, on an item-by-
item basis against the identified evaluation criteria. For each requirement, the individual
evaluator scores and the rationale for the scores were reviewed and discussed by the team.
The evaluators used a consensus approach to determine a single score and rationale for each
rated requirement that all evaluators agreed to. The consensus score and associated
rationale was recorded in the master record.
Evaluators recorded and initialled any changes agreed to during consensus in their individual
workbooks.
The FM confirmed with each member of the evaluation team that they were in agreement with
the final consensus scores and rationale.
All evaluators confirmed their agreement with the recorded scores and rationale.
All evaluators confirmed that they had treated all four Bidders equitably and fairly.
The evaluation team signed the consensus record.
9
The FM’s review of documents and monitoring of activities in this phase considered the principles of
fairness, openness and transparency. This included the following:
Evaluation Plan was consistent with the published procurement documents;
Evaluation team members were chosen and confirmed prior to the receipt of proposals;
Evaluation training was provided to all evaluators. This included informing evaluators of the
following:
o identity of the proposals received and requesting evaluators to declare any conflict of
interest;
o confidentiality protocols;
o document control;
o clarification process;
o overview of scoring workbooks and method for individual assessment;
o explanation of the concept of hidden criteria;
o explanation of fairness and the need for objectivity, consistency and equitable treatment of
all proposals;
o guidance with respect to only assessing the information that is in the proposal – not
information from previous contracts, personal knowledge, or any other information.
The scoring criteria and assessment tools were established prior to the receipt of proposals and
were consistent with the RFP. The scoring criteria contained no hidden criteria;
The submissions were logged and recorded upon receipt, clearly identifying that these were
submitted on time;
The pricing was contained in a separate envelope (as applicable);
The proposals complied with the mandatory submission requirements;
The process for establishing one score from a team of evaluators was established prior to the
receipt of proposals (consensus scoring was used);
The same team of evaluators evaluated all proposals;
The scoring assessment was applied consistently and equitably by the evaluation team with no
evidence of bias;
A secure location for the evaluation exercise was established for the period of the evaluation;
Proposal documents were physically secured within a secure location.
All four proposals were deemed to have met the technical bid requirements of the RFP. The FM had no
fairness concerns related to the technical bid evaluation.
10
4.2.4 Financial proposal assessment PSPC had assigned a separate individual to complete the assessment of financial proposals. This
individual had no contact with the technical evaluation team, and had no knowledge of the technical
evaluation nor its results. Similarly, the PSPC Contracting Authority who attended the technical
evaluation team meetings had no knowledge of the financial proposal assessment, nor its results.
The PSPC individual assigned to complete the assessment of financial proposals reviewed and verified the
financial proposal of each bid. The individual verified that each financial proposal was complete and
complied with the requirements of the RFP.
The PSPC individual completed the assessment of all financial proposals using the process defined in the
RFP.
The FM met with the PSPC individual on April 4, 2016 to review and verify the financial proposal process
and results. The FM was shown all financial spreadsheets and was able to verify that the process defined
in the RFP had been used. The FM verified that the financial assessment results were consistent with the
proper application of the process defined in the RFP.
The FM had no fairness concerns related to the assessment of financial proposals.
4.2.5 FM Specialist observations – RFP process The FM witnessed the technical bid evaluation meetings and confirms that the process was conducted
appropriately in accordance with the process established in the RFP and in a fair, open and transparent
manner.
The FM reviewed the assessment of financial proposals and confirms that the process was conducted
appropriately in accordance with the process established in the RFP and in a fair, open and transparent
manner.
The DND / PSPC team and the Contracting Authority were responsive in providing clarification when
requested by the FM Specialist.
The evaluation teams were knowledgeable and diligent in their evaluation of proposals. All four proposals
were evaluated consistently and fairly.
11
5 Reference documents Documents pertaining to the Request for Proposals process for Support Services for DND’s DRMIS and
PSPC’s SIGMA (Solicitation # W8474-126279/F) can be found on the buyandsell website.
Defence Resource Management Information System – Integrated Support
Services and SIGMA Integrated Support Services
Addendum to the Final Report
September 6, 2016
Addendum to Fairness Monitor final report dated April 22, 2016.
This Addendum to the Fairness Monitor final report covers the period following the conclusion of the
evaluation phase up to and including contract award.
Contract award
As a result of solicitation # W8474-126279/F for Defence Resource Management Information System –
Integrated Support Services and SIGMA Integrated Support Services, of the four bidders, three successful
bidders were selected and contracts were awarded on June 21, 2016. Of the four bidders, three requested
and received a debriefing meeting including the one unsuccessful bidder. The fourth bidder requested
and was provided debriefing comments via email. The FM reviewed all debriefing comments and
attended the three debriefing meetings that were held in person.
It is our professional opinion that the Defence Resource Management Information System – Integrated
Support Services and SIGMA Integrated Support Services procurement process that we observed, was
carried out in a fair, open and transparent manner.
Original signed by Original signed by
John Davis, CEO, PPI Consulting Limited Ian Brennan, CSCMP, FM Team Leader
Original signed by
Original signed by
Bob Tibbo, FM Specialist (Primary) Mairi Curran, FM Specialist (Backup)