ORF 1 Performance Management Presentation Team Members: Ronald Wilson, Team Leader Team Members:...

1

ORF

Performance Management Presentation

Team Members:

Ronald Wilson, Team LeaderTeam Members: Gerald Hines, Fred Khoshbin, Cyrena Simons

ORF National Institutes of Health

January 15, 2004

2

ORF

Table of Contents

Main Presentation

PMP Template ……………………………….…………………………………..

Customer Perspective……………………….……………………………….

Internal Business Process Perspective……………………………….

Learning and Growth Perspective………………………………………

Financial Perspective………………………………………………………….

Conclusions and Recommendations……………………………………

Appendix…………………………………………………………………………….

3

ORF

Team Members

Ronald Wilson

Gerald Hines, Fred Khoshbin, Cyrena Simons

Date: 5 January 2004

Team Leader

The Master and Facilities Planning Service Group will partner with NIH Leadership, Institutes, and Centers to:(1) Predict facility needs;(2) Balance competing needs and organizational considerations;(3) Develop plan and strategies to meet facility and space requirements within budget and on schedule; and (4) Ensure that implementation is congruent with NIH plans.

Strategy Description

Service Strategy

The Master and Facilities Planning Service Group pro-actively provides NIH Leadership, Institutes, and Centers with reliable, impartial, and informed site master plans and strategic facility and environmental planning services, and facilitates space utilization decisions to support the timely delivery of owned and leased facilities to meet NIH's research mission.

Value PropositionDS6: Manage Space Justification Document process

DS5: Manage Director's Reserve

DS4: Develop Strategic Facilities Plan

Master and Facilities Planning

Service Group

Performance Management Plan (PMP)

DS3: Coordinate community and agency planning input

DS2: Perform environmental planning

DS1: Develop Master Plans for NIH facilities

Discrete Services

Division Approval/Date: Associate Director Approval/Date:

Operational Excellence

Customer Intimacy

Product Leadership

Growth

Sustain

Harvest

4

ORFRelationship Among Performance Objectives

5

ORF

Customer Perspective

6

ORF

Customer Perspective (cont.)

Objective Measure FY 03 Target FY04 TargetFY05

TargetInitiative Owner

C1: Increase customer satisfaction

C2: Respond consistently and reliably to customers

C3: Equitably and impartially serve all ICs

C4: Anticipate customers' needs

C1: Overall average rating for Customized ORS Customer Scorecard

C2: % of Customized Scorecard survey respondents indicating satisfaction with the consistency and reliability of service received from the service group

C3: % of Customized Scorecard survey respondents indicating satisfaction with access to the planning process regardless of outcome of their individual request

C4: % of Customized Scorecard survey respondents indicating service group anticipated their needs and assisted them in incorporating respondents facilities issues into the planning process

Average rating of 6.75 (on a scale of 1.0 to 10.00)




Equal to or greater than FY03 result








Repeat customer survey regularly

7

ORFCustomer Survey Results

In the winter of 2002-2003, a customer survey of InstituteDirectors and Executive Officers was undertaken to gaugecustomer satisfaction with the services of the then Office ofFacilities Planning. • Customers were asked to rate services provided by Office

of Facilities Planning according to a planning continuum: long-, medium-, and short-range.

• Customer service targets for FY03-FY05 illustrated in slides that follow have been derived from scores attained on this customer survey.

• Complete survey results exclusive of full text comments can be found in the Appendix.

8

ORF

Summary FY03 Customer Satisfaction Ratings of Facilities Planning Services

N = 21

Unsatisfactory

Outstanding

6.57

6.72

7

6.74

6.63

6.58

6.88

7.25

7.19

7.38

6.95

7.05

6.63

7.37

7

7

6.75

7.15

1 2 3 4 5 6 7 8 9 10

Handling of Problems

Competence

Responsiveness

Availability

Timliness

Quality

Long Range Mid Range Short Range

Timeliness

9

ORF

Long-Range Planning Service Ratings by FY

N = 21

Unsatisfactory

Outstanding

Mean Ratings

N = 12

M = 7.14

M = 7.13

M = 7.57

6.57

6.72

7.00

6.74

6.63

6.58

8.56

8.00

8.13

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02

Timeliness

10

ORF

Mid-Range Planning Service Ratings by FY

Unsatisfactory

Outstanding

N = 21

N = 12

M = 7.56

M = 7.47

M = 7.53

Mean Ratings

6.88

7.25

7.19

7.38

6.95

7.05

8.22

8.38

8.38

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02

Timeliness

11

ORF

Short-Range Planning Service Ratings by FY

6.63

7.37

7.00

7.00

6.75

7.15

7.75

7.18

7.91

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02Unsatisfactory

Outstanding

Mean Ratings

M = 7.43

M = 6.91

M = 7.27

N = 21

N = 12

Timeliness

12

ORF

FY03 Long-Range Planning Service Ratings by Position

N = 6

Unsatisfactory

Outstanding

Mean Ratings

N = 11

Note: Differences are not statistically significant.

6.59

6.17

7.20

6.50

6.33

6.00

6.44

6.70

6.73

6.64

6.45

6.55

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EO

Timeliness

13

ORF

FY03 Mid-Range Planning Service Ratings by Position

N = 6

Unsatisfactory

Outstanding

Mean Ratings

N = 11


6.40

6.50

6.50

6.67

6.00

6.50

6.56

6.9

7

7.09

7

6.82

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EO

Timeliness

14

ORF

FY03 Short-Range Planning Service Ratings by Position

Unsatisfactory

Outstanding

Mean Ratings

N = 6 N = 11


6.00

6.17

6.17

6.17

6.00

6.33

7

7.7

7.36

7.55

7

7.36

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EO

Timeliness

15

ORFSummary

• Three-fourths of respondents were from the Bethesda campus– The majority of the respondents were EOs

• Highest Long-Range Planning ratings of satisfaction for responsiveness, availability, and competence– Lowest ratings for handling of problems and quality

• Highest Mid-Range Planning ratings of satisfaction for availability, competence, and responsiveness– Lowest ratings for handling of problems and timeliness

• Highest Short-Range Planning ratings of satisfaction for competence, quality, responsiveness and availability– Lowest ratings for handling of problems and timeliness

16

ORFSummary

• Top three FY03 ratings of satisfaction for all phases of planning were competence, responsiveness, and availability

• Lowest FY03 ratings of satisfaction for all phases of planning were in the area of handling of problems

• FY03 Long-Range Planning ratings lower than the other phases

• Comparison of ratings between ICDs/SDs and EOs did not indicate huge differences in perceptions– EO perceptions more positive than ICDs/SDs– Differences are NOT statistically significant

17

ORFSummary

• Comments indicate:– Need clearer definition of role responsibilities in

ORS/ORF• More coordination among DFP/DES/Leasing• Seamless process from planning to actual space

acquisition– There are perceptions of inequity among ICs– ORS/ORF needs to develop a better understanding of

billing procedures and rules within the ICs

18

ORFCustomer PerspectivePlanned Actions

The Division of Facilities Planning is currently surveying its customers for FY04 as part of the annual Buildings and Space Planning process. It has modified last year’s customer survey to incorporate the appropriate discrete services for which it is responsible and revised the survey questions to better align with customer objectives.

19

ORF

Internal Business Process Perspective

20

ORF

Internal Business Process Perspective

Objective Measure FY 03 TargetFY04

TargetFY05


IB1: Improve efficiency and effectiveness of planning services

IB2: Proactively engage customers and stakeholders and improve their awareness

IB1: A. Project Reviews--% of submittal deadlines met, such as:- Program of Requirements- Pre-Programming Documents- Design Drawings- Environmental ReviewsB. DFP Planning Studies--% of planning milestones met- Master Plans- Strategic Facilities Plan- Site Feasibility StudiesC. Short-term planning requests- SJDs--% of planning milestones met- Site Selection Requests--% of planning milestones met

Identify categories of work process outputs for planning services provide and develop a computer-based system for monitoring outputs on a quarterly basis

5 contacts per year per customer (I.e., ICs, OD, EOs, etc.) by type (0=not useful; 1=useful, collected data; 2=identified issues/developed strategies to proactively support customers; 3=educated customers on process and limitations

Establish baseline work process outputs and performance measures by planning service

TBD

TBD

TBD

Construct a planning milestone chart

Develop a planning and programming tool to assist NIH in assessing project feasibility at the early stages of project development

Develop a tracking mechanism in the SJD log to track the progress of SJDs from the original request to the initial presentation of the request to the SRB

IB2: # of meetings attended and number of issues identified and strategies that resulted from proactive contacts with customers, by source and type of meetings.

21

ORF

Internal Business Process PerspectivePlanned Actions

In the coming year, the Division will:

• Construct a planning milestone chart to track its processes

• Develop a planning and programming tool to assist NIH in assessing project feasibility at the early stages of project development

• Develop a tracking mechanism in the SJD log to track the progress of SJDs from the original request to the initial presentation of the request to the SRB

22

ORF

Learning and Growth Perspective

23

ORF


Objective Measure FY 03 Target FY04 Target FY05 Target InitiativeOwn

er

LG1: Renew technical and non-technical skills

LG2: Complete benchmarking plan to become expert planners

LG3: Enhance the structure of planning information for easy reference and retrieval and support the adoption of "best practices"

LG4: Improve collaborative knowledge sharing

L1a: % of employees with current and pertinent training plansL1b: % of training plans that are fully executed

L2: % of benchmarking plan completed

L3: % of planning communication tools in place and "best practices" adopted

L4: Number of information exchanges attended by 50% or more of staff

25% of employees have completed planning-related training (Actual--27% of employees took coursework)

20% of benchmarking plan complete

20% of planning information technology plan complete

4 Information Exchanges held during the year

80% of employees have training plans and implementation is in process

100% of benchmarking plan completed

100% of planning information technology plan completed

100% of employees have completed all mandatory training plan requirements

Implement selected "best practices"

TBD

Complete an office-wide training needs assessment. Each staff member to develop a training plan

Assemble benchmarking group and survey group members

Dedicate time in the monthly Information Exchange to discuss ongoing technology needs

Assemble benchmarking group and survey group members

24

ORF

Learning and Growth PerspectiveActions Planned and UnderwayObjective LG 1: A training needs assessment has been conducted and

each staff member is developing a training plan.

Objective LG2: A benchmarking survey is underway. In order to identify best practices, the Division has invited 9 federal installations within the Capital Region and/or DHHS, as well as 8 research universities and one hospital system to participate in a benchmarking project. To date, from among this group, 9 planning officers have confirmed their willingness to participate. Survey responses are expected by the end of January 2004. Follow up plans include interviews with some of the participants, and another survey next year to (1) include other types of organizations, and (2) expand the topics for benchmarking.

Objective LG3, Part 1: Working with IT, the Division is developing a tracking tool for its customers to measure the frequency and effectiveness of its meetings and the success of its outreach efforts.

Objective LG3, Part 2: Actions the same as for Objective LG2.

Objective LG4: As part of its monthly Information Exchange, the Division is dedicating time to identifying its technology needs. Four Information Exchanges held last year.

25

ORF

Financial Perspective

26

ORF

Financial Perspective (cont.)

Mandatory to Report on Unit Cost Objective

Objective Measure FY 03 Target FY04 TargetFY05


F1: Minimize planning unit costs

F1: DFP unit cost for master planning services measured per planning activity

F2: DFP unit cost for preparing/reviewing environmental documents measured per environmental document

F3: DFP unit cost of strategic facilities planning measured per capita (census population)

F4: DFP unit cost to coordinate community input measured per hours spent in coordination

F5: DFP unit cost of B&S planning services per B&S Plan report

F6: DFP unit cost to manage SJD process per square foot of space requested

Establish baseline master planning unit costs

Establish baseline planning activity unit costs

Establish baseline environmental planning unit costs

Establish baseline community coordination costs

Establish baseline B&S Planning report unit costs

Establish baseline short-range planning costs per square foot of SJD space requested

TBDfor all measures

TBDfor all measures

For all measures, develop a computer-based system for tracking planning costs on a quarterly basis after implementation of the CAN system

27

ORF

Financial PerspectiveActual costs for discrete services

FY02 FY03 FY04 FY02 FY03 FY04

Master and Facilities Planning Develop Master Plans for NIH facilities # of hours 1,560 1,560 1,560 2,381,485 1,437,000 1,289,025 Perform environmental planning # of projects 16 16 16 221,026 158,000 139,239 Coordinate community and agency planning input # of hours 630 630 630 165,477 109,000 129,516 Develop Strategic Facilities Plan # in Census 28,256 28,821 29,398 452,898 258,000 371,808 Manage Director's Reserve sq. ft. of Dir. Reserve 82,620 109,272 110,958 86,903 253,000 264,866 Manage Space Justification Document process # of SJDs 165 168 171 Provide construction site coordination # of requests Not Avail. 29 30 Not Avail. 3,000 16,000

Develop Transportation Management Plan # in Census 28,256 28,821 29,398 Not Avail. 44,000 135,293 Formulate B&F budget # of reports 1 1 1 607,224 352,000 Not Avail.

ACTUAL UNITS BUDGETED COSTUNIT MEASUREDISCRETE SERVICE

28

ORF

Financial PerspectivePlanned Actions

• The Division will reexamine its unit cost measures and establish new baseline unit costs from FY02 and FY03 actual cost data. FY04 costs will be monitored quarterly when the new CAN system is implemented.

29

ORF

Conclusions

30

ORF

Conclusions from PMP• The Balanced Scorecard approach has helped DFP better relate our day-to-

day planning activities to long-term ORF and DFP planning goals.

• Also, the process has helped highlight how important satisfying customer requirements (some of which customers may be unaware of) is to the work of the Division. Getting in front of customer’s needs, and thus guiding them through options before they are caught short and left with no alternatives has gained more appreciation. The process has resulted in a greater recognition of the value of working with our customers early in the facility and space planning processes enabling us to spot problems well in advance, and help customers plan through to a solution(s) that meets their needs and those of NIH.

• Working through the Learning and Growth perspective pushed our thinking outward to begin to focus on essential technical support and training needs. It’s doubtful we would have pushed as far without the PMP process.

• In summary, the process has helped sharpen our insights, deepened our understanding of what we are all about, and focused us on what we need to do to achieve the Division’s and ORF’s strategic goals.

• The major initiatives for FY04 will be to revise the Customer Survey to track the Division’s progress in addressing prior year customer concerns and completing the benchmarking survey.

31

ORF

Appendix: Measures Customer Perspective

C1: Overall average rating for Customized ORS Customer Scorecard

C2: Percent of Customized Scorecard survey respondents indicating satisfaction with the consistency and reliability of service received from the service group

C3: Percent of Customized Scorecard survey respondents indicating satisfaction with access to the planning process regardless of outcome of their individual request

C4: Percent of Customized Scorecard survey respondents indicating service group anticipated their needs and assisted them in incorporating respondents facilities issues into the planning process

Internal Business Process PerspectiveIB1: A. Project Reviews--% of submittal deadlines met, such as:- Program of Requirements- Pre-Programming Documents- Design Drawings- Environmental Reviews

B. DFP Planning Studies--% of planning milestones met- Master Plans- Strategic Facilities Plan- Site Feasibility Studies

C. Short-term planning requests- SJDs--% of planning milestones met- Site Selection Requests--% of planning milestones met

32

ORFAppendix (cont’d.)


L1a: % of employees with current and pertinent training plans

L1b: % of training plans that are fully executed L2: % of benchmarking plan completed L3: % of planning communication tools in place and "best practices" adopted

L4: Number of information exchanges attended by 50% or more of staff

Financial Perspective

F1: DFP unit cost for master planning services per planning activity

F2: DFP unit cost of B&S planning services per B&S Plan report

F3: DFP unit cost to manage SJD process per square foot of space requested

33

ORFAppendix

Results from the FY03 ORS Customer Scorecard for NIH facilities Planning

Prepared byAmy Culbertson and Joe WolskiOffice of Quality Management

16 April 2003

34

ORF

Methodology

• OQM contacted by OFP early in FY03 to discuss customer assessment methodology– Desire to establish system that is integral component of annual

Building and Space planning meetings with ICs– Cycle of meetings typically occur early in the FY– Discussed concern that had just completed customer survey in

Sept FY02• Designed new FY03 survey to address three components of

planning process– Long-range– Medium range– Short range

• Modified the administration process based on learnings from Sept FY02 process– Carefully tracked who responded to survey– Sent follow-up emails to increase response rate

35

ORFMethodology

• Surveys administered via email in December 2002 - January 2003 time frame– Sent to IC Directors (ICDs), Scientific Directors (SDs),

and Executive Officers (EOs)– Comments received back from some EOs indicated

displeasure at survey going to ICD/SD

• Reminder sent in late January – Received substantial number of surveys after reminder

was sent

• Gathered, tracked, entered, and analyzed data– Integrated responses from Sept 02 survey as

appropriate

36

ORF

Survey Distribution

FY03 AdministrationNumber of surveys distributed 70Number of respondents 21Response Rate 30%

Number of IC’s receiving survey 27Number of IC’s with at least one response 14IC Response Rate 52%

FY02 AdministrationNumber of surveys distributed 85Number of respondents 12Response Rate 18%

37

ORF

FY03 Respondents by Location

15

7

4

0 5 10 15 20

Outside BethesdaArea

Bethesda Off-Campus

Bethesda Campus

N = 20

Note: Multiple responses allowed. 1 respondent skipped this question.

38

ORF

FY03 Respondents by Primary Mission

N = 20

Note: 1 respondent skipped this question.

13

3

4

0 5 10 15

Intramural

Extramural

Both Intramural &Extramural

39

ORF

FY03 Respondents by Position

3

11

3

4

0 5 10 15

Other

Scientific Director

Executive Officer

IC Director

Note: Feedback from respondents indicated that 6 surveys were filled out by the EO “on behalf of” the IC Director.

N = 21

40

ORF

Summary FY03 Customer Satisfaction Ratings of Facilities Planning Services

6.57

6.72

7

6.74

6.63

6.58

6.88

7.25

7.19

7.38

6.95

7.05

6.63

7.37

7

7

6.75

7.15

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

Long Range Mid Range Short Range

N = 21

Unsatisfactory Outstanding

41

ORF

Long-Range Planning Service Ratings by FY

6.57

6.72

7.00

6.74

6.63

6.58

8.56

8.00

8.13

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02

N = 21


Mean Ratings

N = 12

M = 7.14

M = 7.13

M = 7.57

42

ORF

Mid-Range Planning Service Ratings by FY

6.88

7.25

7.19

7.38

6.95

7.05

8.22

8.38

8.38

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02

N = 21


Mean Ratings

N = 12

M = 7.53

M = 7.47

M = 7.56

43

ORF

Short-Range Planning Service Ratings by FY

6.63

7.37

7.00

7.00

6.75

7.15

7.75

7.18

7.91

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

FY03 FY02Unsatisfactory Outstanding

Mean Ratings

M = 7.43

M = 6.91

M = 7.27

N = 21

N = 12

44

ORF

FY03 Long-Range Planning Service Ratings by Position

6.59

6.17

7.20

6.50

6.33

6.00

6.44

6.70

6.73

6.64

6.45

6.55

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EO

N = 6


Mean Ratings

N = 11


45

ORF

FY03 Mid-Range Planning Service Ratings by Position

6.40

6.50

6.50

6.67

6.00

6.50

6.56

6.9

7

7.09

7

6.82

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EO

N = 6


Mean Ratings

N = 11


46

ORF

FY03 Short-Range Planning Service Ratings by Position

6.00

6.17

6.17

6.17

6.00

6.33

7

7.7

7.36

7.55

7

7.36

1 2 3 4 5 6 7 8 9 10


Competence

Responsiveness

Availability

Timliness

Quality

ICD/SD EOUnsatisfactory Outstanding

Mean Ratings

N = 6 N = 11


47

ORF

Do the facilities planning services in ORS support your Institute’s mission planning efforts?

9%

0%

91%

95%

5%

0%

0% 20% 40% 60% 80% 100%

Not Applicable

Don't Know

No

Yes

FY03 FY02

N = 21

N = 12

48

ORF

Do you understand how to get your Institute’s needs into the strategic facilities planning process?

8%

92%

100%

0%

0% 20% 40% 60% 80% 100%

Not Applicable

Don't Know

No

Yes

FY03 FY02

N = 21

N = 12

49

ORF

Does the current process work effectively for your Institute to acquire the space you need?

0%

0%

18%

82%

65%

15%

5%

15%

0% 20% 40% 60% 80% 100%

Not Applicable

Don't Know

No

Yes

FY03 FY02

N = 21

N = 12

50

ORF

Themes from Comments on Suggestions to Improve the Facilities Planning Process

• Need clear definition of role responsibilities in ORS and seamless service from planning to acquiring the space

– Among planners, DES, and real-estate leasing– Lack of communication between DES, Real Estate Leasing, and OBSF– Planning and SJD work well, but leasing needs more people– Space decision-making process does not consider the implications of leasing

widely dispersed off campus sites that NIH corporate infrastructure must support

• There are perceptions of inequity among IC’s for space– Method of providing space to “newer” IC’s needs to be improved– Central Service facilities are given a low priority – needs are not met.

• ORS needs to have a better understanding of billing procedures and rules– Get clarification from OBSF regarding procedures/rules for TIA– IC’s do not have full knowledge of associated costs of leased space– Inequities in the way IC’s are billed for space by ORS that are not being corrected

by ORS– IC was billed, however space was unusable– ICs would like more information on the B&F budget

• ICs would like more independence in managing and renovating their leased space

Note: Include themes from both FY02and FY03 comments.

51

ORFSummary

• Three-fourth of respondents were from the Bethesda campus– The majority of the respondents were EO’s

• Highest Long-Range Planning ratings of satisfaction for responsiveness, availability, and competence– Lowest ratings for handling of problems and quality

• Highest Mid-Range Planning ratings of satisfaction for availability, competence, and responsiveness– Lowest ratings for handling of problems and timeliness

• Highest Short-Range Planning ratings of satisfaction for competence, quality, responsiveness and availability– Lowest ratings for handling of problems and timeliness

52

ORFSummary

• Top three FY03 ratings of satisfaction for all phases of planning were competence, responsiveness, and availability

• Lowest FY03 ratings of satisfaction for all phases of planning were handling of problems

• FY03 long-range planning ratings lower than the other phases

• Comparison of ratings between ICDs/SDs and EOs did not indicate huge differences in perceptions– EO perceptions more positive than ICDs/SDs– Differences are NOT statistically significant

53

ORFSummary

• Comments indicate:– Need clearer definition of role responsibilities in ORS

• More coordination among OFP/DES/Leasing• Seamless process from planning to actual space

acquisition– There are perceptions of inequity among IC’s– ORS needs to have a better understanding of billing

procedures and rules

ORF 1 Performance Management Presentation Team Members: Ronald Wilson, Team Leader Team Members:...

Documents

Transcript of ORF 1 Performance Management Presentation Team Members: Ronald Wilson, Team Leader Team Members:...