1 Commercial in Confidence – Programme Assurance Report /15 th September 2004 /Version 2.1 FINAL...

Commercial in Confidence – Programme Assurance Report /15th September 2004 /Version 2.1 FINAL

1

Independent Programme AssuranceIndependent Programme Assurance

1515thth September 2004 September 2004

Version 2.1 – FINALVersion 2.1 – FINAL

Programme EngineerProgramme Engineer


2

IntroductionIntroduction

Background

Programme Engineer represents a significant investment for The Law Society and is designed to improve customer service and communications with its membership. The principal drivers for the programme are:

• To provide a common information system for The Law Society;

• To provide a set of consistent ways of working by The Law Society; and

• To provide a robust, flexible and maintainable technology base for The Law Society.

Scope of this review

This review has been structured to assess the ongoing performance of Programme Engineer with the focus on whether:

1. The programme plans, controls, processes and procedures upon which Programme Engineer is based are commensurate with the delivery of the programme objectives;

2. The processes and controls designed to identify and mitigate current and future risks that threaten the success of the programme appear reasonable; and

3. Previous recommendations are being progressed and implemented.

The review has been undertaken through interviews with key Programme staff & contractors, a review of programme documentation, and through peer review within Ernst & Young LLP.


3

Introduction (Cont).Introduction (Cont).

Structure

This report sets out our key findings from our independent review. It identifies strengths and areas for improvement within the ongoing delivery of the programme.

Within the findings, commentary is also provided on progress made against the recommendations from previous assurance reviews of the Programme.

In keeping with previous reporting structures, the areas of “Strategy”, “Structural”, and “Delivery” issues are used as a framework for the findings. Against each of these, a review of strengths and weaknesses is provided.


4

Executive Summary

Through attendance at the Programme Boards, receipt of update reports and reviews of programme management controls, we are reasonably assured that the project is on track to deliver it’s anticipated outputs within budget and in accordance with agreed timescales.

There is clear evidence that the awareness and application of sound programme management principles and discipline has continued to improve within the Programme environment over the period of our review. In general where areas of potential improvement have been identified in previous reviews, management resource has been allocated to address these and there has been further visible refinement of the programme and project processes.

Clearly, the programme management capability has matured since the last report. However, there still remain areas and opportunities for continued improvement. As with all “point in time” reviews, actions may already be underway to address weaknesses and within this report we have highlighted such instances.

Within the report there are a number of issues noted, none of which are considered fundamental barriers to the success of Engineer.

Executive SummaryExecutive Summary


5

Approach

In respect of stage 1 of the Programme we have independently and objectively reviewed and reported on:

• The programme controls, processes and procedures upon which Programme Engineer is constructed;

• The current and likely future risks that threaten the success of the Programme, and the effectiveness of the processes and controls designed to mitigate them;

• The Programme’s progress and plans in comparison with proportionate leading practice project management principles as advocated by the Prince2 methodology; and

• The effectiveness of the implementation of improvement recommendations made in previous programme reviews.

This has been undertaken through:

Structured Interviews

Interviews were conducted with key Programme Engineer staff and stakeholders. A full list of interview participants is provided as an appendix.

Documentation Review

Project documentation was reviewed in light of programme management leading practice and appropriate Prince2 project management methodology components.

Our ApproachOur Approach


6

Overview of FindingsOverview of FindingsThe findings below represent those which are considered as major strengths and areas for improvement within Programme Engineer.

What’s going well

• Programme Management Reporting and Control - The role of a dedicated programme management office (PMO) with commensurate programme management processes aligns with the Prince2 standard. This function is being well performed within the delivery team of Engineer.

• Redesign of the programme into stages with stage reviews – The re-planning of Programme Engineer into stages gives additional independent decision points. This also represents a good alignment to the industry standard of Prince2.

• Delivery against the business case – The greater effort placed on planning the stage, applying lessons previously learned and delivery through a disciplined PMO has resulted in solid progress against plan.

• Visibility of Executive Sponsorship – There continues to be clear visible leadership from senior management who are fully engaged and willing to discuss strategic issues and there is involvement through clear sponsorship support of the project execution.

• Cultural Awareness and Sensitivity - There is a strong collegiate Law Society working environment, where consultation is used that is inclusive of decision-making.

Areas of opportunity

• Drivers for change within the organisation – Programme Engineer is enabling clear change within the Society. The implementation of Siebel along with the Technical Design Authority (TDA)/ Business Design Authority (BDA) creates a solid platform for continued progress. The clear engagement of the Senior Management Team (SMT) and the business modelling work currently being undertaken by Deloitte, offers further opportunity to capture cross fertilisation of innovative ideas which the Programme will need to respond to.

• Direction & Guiding Principles – In the business case for stage 2, it would be useful to include clear re-statement “picture of the success” for the Programme.


7

Areas of opportunity (Continued)

• Benefits Tracking & Realisation – Embedding “Change for the Better” is important for the Society. This would better be supported through the early identification of key performance indicators which are designed to encourage behaviour commensurate with the realisation of anticipated programme benefits.

•Resource – The ability of the society to recruit and retain appropriately skilled staff for the programme is being constrained by its salary structure which is making it difficult to offer packages sufficient to attract the calibre of staff that the Society now understands that it needs.

Overview of Findings Overview of Findings …continued…continued


8

Findings and RecommendationsFindings and RecommendationsBasis of the information

In presenting our “point in time” assessment of the current state and outlook for Programme Engineer it is important to bear the following factors in mind:

Our assessment has been developed in two ways. Firstly, we conducted a desk based review of the project set up, controls and reporting mechanisms. Secondly, it has been developed from the views and assertions of those we have interviewed during the process. These views and assertions have occasionally been contrasting and, in arriving at a balanced overall conclusion, we have necessarily applied a degree of professional judgement.

This review has not allowed, for substantial testing of the above assertions and judgements. However, reference back to previous work has often helped place comments into context.

This was a point-in-time assessment, (effectively at the start of the review period), and some issues or areas for consideration have been acted on during the course of the review. As far as possible, this is noted in this report. However, we have erred on the side of prudence and have not identified these issues as resolved. This will enable The Law Society to track the clearance of issues over time.

Format of the following pages

On the following pages, we outline, at a high level: Areas of strength identified by the review; A summary of the opportunities for improvement identified; and Specific improvement recommendations.

As in previous reports, opportunities for improvement and areas of strength have been categorised as “Strategic”, “Structural” and “Delivery”.


9

Areas of StrengthAreas of StrengthProgramme Engineer has, in the opinion of all those interviewed, made significant headway under new management. There are a number of strengths that our review identified as probable catalysts for this change, and these are identified below.

Visibility of Executive Sponsorship – The visibility of the Programme to senior executive levels within the Law Society offers a level of comfort that consideration has been given to the direction of the Programme and that issues can be escalated for resolution. An example of this is the separate engagement of the Law Society Business Modelling undertaken by Deloitte.

Cultural Awareness and Sensitivity – There was strong evidence (e.g. the role and function of user groups) to support the notion of a collegiate Law Society working environment. This collegiate style of working is likely to engender a willingness to accept change (buy-in of scope), and this approach greatly improves the likelihood of success through knowledge transfer and Programme delivery.

Specification of Work – Significant elements of the scope of Stage 1 focuses on consultation & specification of requirements and definition of scope for later elements of the Programme (eg, Learning Zone). This collaborative development will help to define a more appropriate scope to suit the existing needs of the Law Society, and embed the ownership of the agreed scope (and its delivery) into the organisation.

Awareness of Strategic Issues – With external influences such as the Clementi review, and other reviews of the role and purpose of the Law Society, there is evidence of a very high level of strategic awareness throughout the Programme and organisation. This represents an opportunity to ensure changes are implemented into the organisation. The continued engagement of Council, Main Board and SMT in specific relation to likely implications of the Clementi Review clearly adds confidence to the project teams and user groups that their efforts are not misdirected.

Str

ate

gy


10

Areas of Strength… Areas of Strength… continuedcontinued

Programme Management Reporting & Control - The central management of reporting, information and monitoring through the Programme Office represents alignment to Prince2, an industry recognised standard. The evidence from interviews and documentation review indicates that this role is being undertaken well through a dedicated and committed team.

Quality of Project Management Resources - The skill gaps identified (both technical and experience) in previous reviews have been successfully addressed by supplementing the core team with third party consultants and contract staff. There is now a high level of competence, capability and experience amongst the core project teams albeit that the balance of skills sits outside Society core personnel. In addition, there are no material dual ‘business as usual’ conflicts for those driving project implementation.

Commitment of Project Management Resources – As identified in the previous report there continues to be a tremendous commitment and dedication to the Programme amongst the whole team with an overt feeling of achievement and drive towards a single common goal. Furthermore, the Society is managing its supplier relationships more stringently as evidenced by its requirement of dimension data to work 24/7 to correct its under-delivery. It will be critical to ensure that effective, forward looking and proactive management of key personal is undertaken as the new Stages of Programme Engineer are considered.

PRINCE2 Process Controls – Evidence from interviews and documentation supports the view that Prince2 Project Management process has been used well. Prince2 based management, especially in the public sector, is a widely accepted control process that is used by many organisations to bench-mark themselves against. A specific example where The Law Society aligns with Prince2 is the re-planning of Programme Engineer into stages, each with separate business case and independent decision making points (stop/go for the Programme).

Str

uc

ture


11


Programme Board – The programme board is well constituted and attendance of key personnel indicates commitment. Papers are available in advance and issue updates are comprehensive. Disciplined chairing of the Board ensures measured progress and attendance by External Quality Assurance is evidence of commitment to leading practices.

Recruitment of Project Personnel – It has been fully taken onboard that the Society needs to better balance in house versus bought in resource. Significant effort has been devoted to specifying roles and engaging HR in the resourcing process. Continuing to press on in this area remains important if the Society is to leverage its investment in Engineer through knowledge transfer.

Technical Design Authority (TDA) & Business Design Authority (BDA) – The Society has recently implemented both TDA and BDA as well as developing clear “guiding principles” for IT systems development. This has helped ensure that ad hoc changes are not implemented without the wider implications and interdependencies first being assessed. This protocol protects the future IT architecture integrity and ensures that system development benefits are assessed at a Society wide level.

Str

uc

ture


12

Lessons Learnt – Evidence, principally from programme team meetings, supports the view that the delivery team is effectively sharing experience and seeking to avoid repeat mistakes.

Third-Party Working Relationships – The evidence suggests that the control of and support from contracted third-parties has been significantly improved in recent months of the Programme. This includes the development and use of a common language (eg at the technical level) between the Programme team and the organisations engaged.

Success of Web Project - The recent launch of the web Programme sits outside this review. However, this success has had a positive influence on the future success of Engineer. The successful delivery of this project lends further credibility to the Programme.

Deloitte Consulting - Deloitte engagement, both Engineer and in other areas of the Society, is significant. This is understood at all levels within the Society. Strong on-site presence of senior (Partner and Director) Deloitte personnel is clearly proving beneficial and the relevant practical experience of the consultants is complimenting the skills of the “in-house” team.

De

liv

ery



13

Areas for ImprovementAreas for ImprovementThe Law Society is facing a period of externally generated uncertainty (eg Clementi review findings), and this adds a level of additional challenge to the successful delivery of a change programme. The areas of improvement are highlighted below:

Direction & Guiding Principles – The Programme, the business case for Stage 1, and the mandate for the Programme Board and governance does have historically well defined guiding principles. However, in the environment of greater uncertainly created by the Clementi review, the Society could benefit from ongoing updates to the definition of the overall direction or “picture of success” for the Programme. The absence of this could represent a risk as complex issues arise, and conflicting user requirements need to be resolved going forward.

IT Project v Organisational Change – The Senior Management Team are leading work in to relation to organisational change and the Clementi review. There is evidence that this is linking with the developments in Programme Engineer. It is important that these links are constantly maintained to ensure that Engineer truly delivers the required change and doesn’t become an IT project.

The recent development of the Business and Technical Design Authorities will be an important part of keeping organisational and technical change in step over the longer term.

Benefits Tracking & Realisation – Benefits identification and tracking is an essential part of being able to identify value for money, to decide on changes, and to resolve conflicts. It also offers quantitative evidence for decision making.

It would be of significant benefit for Law Society to focus on identification and tracking of benefits. We suggest that future Stages of Engineer identify the benefits, their tracking, and measures (eg KPIs) in the Business Case for that Stage. Benefits Management can then be owned and actively managed by The Law Society.

Str

ate

gy


14

Sign-off capability – There is evidence to support the view that The Law Society has not always made timely sign-off commitments to the Programme eg. missed sign-off deadlines. In complex and evolving projects it is not always easy to know exactly when sign-off is required by key persons. This may be due to slippage of timelines or scope changes/ re-planning. However persons required to sign-off at various stages will require as much forewarning as possible to enable them to plan their own personal time, to be in a position to review and sign-off documents. As the Programme proceeds and the complexity/scale of issues grow, it would be useful to have a robust process in place in this area. As a minimum, it is suggested that time estimates or likely time periods when review and sign-off is likely to be required are communicated by the programme to the persons required for review and sign-off, on a regular basis.

Organisational Resilience – There were concerns expressed with regard to the resilience of some areas of the Programme organisation to respond to sudden key staff departures. This issue has been noted by the Society and there is now evidence of much greater cross-working. However, further cross-training of roles/ skills transfer, cascading management & reporting structures would mitigate this risk exposure.

Str

uc

ture

Areas for Improvement… Areas for Improvement… continuedcontinued


15

Areas for Improvement… Areas for Improvement… continuedcontinued

Resourcing & Ability to Obtain Necessary Staff – The evidence supports the view that key delivery staff (eg. Testing Manager) have been difficult to identify and recruit. Also within the delivery team, the number of Law Society permanent staff is relatively low. At a higher-level, busy day-jobs and extensive wider responsibilities can , on occasion impair the ability of senior management to make the time to take informed decisions within the Programme.

It is understood that The Law Society is endeavouring to recruit (permanent and contract staff) to fill skills gaps in the delivery team, and that progress has been made.

Lessons Learnt – There is evidence of sharing experience through the existing delivery team, and it would be desirable to further formalise this for future use. There is less evidence that lessons are being tracked and learnt at the management level and at the interface between the Programme and the Organisation. Formal review at these higher levels may generate extra work. However, the investment of time in establishing these systems is likely to pay dividends in the longer term.

De

liv

ery


16

RecommendationsRecommendationsOn the basis of the strengths identified and issues uncovered, we suggest a number of key action points for the Law Society and the programme delivery team to consider. These are grouped under “Strategic”, “Structural” and “Delivery” level headings.

1. Overarching Direction & Guiding Principles – It would be beneficial for the overall direction and the detail of what the Programme is tasked with achieving to be revisited, clarified and articulated on a regular basis. It is suggested that this is used as a lever to further build Engineer as a change programme, and shift the lingering perception, amongst some, that the programme is solely an IT implementation programme. This could feed into and support The Law Society’s continuing development of strategy, direction and measures of success.

2. Benefits Tracking & Realisation – The clear identification of tangible outputs, KPIs to monitor them, and the high-level monitoring regime would help enhance credibility in both delivery and strategic rationale of the Programme. Additionally this could provide a quantitative framework for decision making, and a means of evaluating the value of current and future Stages & their business cases. As a past recommendation this has not been fully acted on.

Str

ate

gy


17

Recommendations… Recommendations… continuedcontinued

3. Resourcing the Delivery Team - The technical and experiential skill gaps identified in previous reviews have been largely addressed by supplementing the core team with third party consultants and contract staff. Due to the criticality of this issue, it is however, recommended that measures continue to be taken to ensure that key areas are covered by Law Society permanent staff within the Delivery team.

Str

uc

ture


18

4. Lessons Learnt – There is clear evidence that lessons learnt are being transferred systematically through the delivery team. However, this degree of formality does not exist at the business level. We suggest that The Law Society may wish to achieve this as part of a formal Stagegate review at the end of Stage 1.

5. Celebrate Success - Particularly in light of past failures of the Programme, and the likely desire to motivate the Programme team to perform for future Stage delivery, we recommend that successes and leading practices are identified (probably as part of a Stagegate review), and that these are celebrated, and the individuals involved are congratulated (and possibly rewarded).

Recommendations… Recommendations… continuedcontinuedD

eli

ve

ry


19

Interviewees… Interviewees… Page 1 of 2Page 1 of 2

Interviewee August 2004 Review December 2003 Review April 2003 Review November 2002 Review

Rob Adams Yes Yes No No

Howard Allen Yes Yes No No

Daryl Booth Yes Yes No No

Kate Boyle Yes Yes No No

Stuart Bushell Yes No No No

Darren Curley Yes No No No

Katherine Darley No Yes No No

Pip Dexter Yes No No No

Matthew Fellows No Yes No No

Richard Field No Yes No No

Paula Finn No Yes Yes Yes

Arron Gilbert Yes No No No

Evlynne Gilvarry No Yes Yes Yes

Judith Hetherington Smith Yes Yes Yes Yes

Stephen Holme Yes No No No

Simon Horrocks No Yes No No

Hilary Lloyd No Yes No No

Frances Low Yes No No No


20

Interviewees… Interviewees… Page 2 of 2Page 2 of 2

Interviewee August 2004 Review December 2003 Review April 2003 Review November 2002 Review

Debbie McCarthy No Yes No No

Jayne Maguire No Yes No No

Wendy Mills No Yes Yes No

Bruce Minty Yes Yes Yes Yes

Janet Paraskeva Yes Yes Yes Yes

Angela Rossiter No Yes No No

Nicola Sartorious No Yes Yes Yes

Julian Skelly No Yes No No

Julie Swan Yes No No No

John Tarry Yes No No No

Steve Thompson No Yes Yes Yes

Stephen Ward Yes No No No

1 Commercial in Confidence – Programme Assurance Report /15 th September 2004 /Version 2.1 FINAL...

Documents

Transcript of 1 Commercial in Confidence – Programme Assurance Report /15 th September 2004 /Version 2.1 FINAL...