A point system approach to program ranking
-
Upload
hilario-martinez -
Category
Government & Nonprofit
-
view
113 -
download
0
description
Transcript of A point system approach to program ranking
Hilario P. Martinez 1
ISSUES IN AN OBJECTIVE RANKING OF PROGRAMS AND PROJECTS
Difficulties in Finalizing the Annual Workplan
Hilario P. Martinez 2
The Selection Dilemma
Hilario P. Martinez 3
Which programs and projects do we need to consider for next year?
What criteria do we need to employ to select the most appropriate programs and projects for development and implementation?
Traditional Criteria in Prioritizing Programs and Projects
Financing Requirement
Schedule of Implementation
Sponsoring Institution
Hilario P. Martinez 4
What influences the prioritization of social services?
Hilario P. Martinez 5
The survey says …
Cashflow and
Funding issues
Concept issues
Collaboration issues
What influences the prioritization of social services?
Hilario P. Martinez 6
A political promise?
A political
response?
A political gimmick?
What influences the prioritization of social services?
Hilario P. Martinez 7
Hmmm … Who? What? How?
Why should I?
What influences the prioritization of social services?
Hilario P. Martinez 8
Can we make
it in time?
Who takes what?
Who gives what?
How do we put
things together?
A DATA-BASED SYSTEM OF PRIORITY RANKING OF PROGRAM/PROJECT IMPLEMENTATION
Hilario P. Martinez 9
Criteria for Ranking Programs/Projects
Do-ability
Importance
Magnitude
Urgency
Relevance
Hilario P. Martinez 10
The Criteria …
• relates to the capability and capacity of an organization to accomplish its planned targets given the limitation of its resources
D o - a b i l i t y
• refers to the qualitative degree of significance attributed to certain factors to contribute to mission critical
I m p o r t a n c e
• refers to the relative sizes and trends of a particular set of information
M a g n i t u d e
• refers to the degree of exigency of addressing a given situation U r g e n c y
• refers to the degree of congruence to the vision, mission and objectives of a service area
R e l e v a n c e
Hilario P. Martinez 11
Weight per Criterion
Do-ability 5 pts
Importance 5 pts
Magnitude 5 pts
Urgency 5 pts
Relevance 5 pts
Hilario P. Martinez 12
The Rating Scale …
5 points
Highest
4 points
2nd
Highest
3 points
Median
2 points
2nd
Lowest
1 point
Lowest
Hilario P. Martinez 13
The Rating per PPA*
Hilario P. Martinez 14
D
O
-
A
B
I
L
I
T
Y
I
M
P
O
R
T
A
N
C
E
M
A
G
N
I
T
U
D
E
U
R
G
E
N
C
Y
R
E
L
E
V
A
N
C
E
S
C
O
R
E
≤5 ≤5 ≤5 ≤5 ≤5 ≤25
* Program/Project/Activity
Agency Program/Project Assessment Form
Hilario P. Martinez 15
Page 1 Page 2
Partner Agency Assessment Form
Engaging the active participation of partner agencies in the development and implementation of public sector programs and projects
Hilario P. Martinez 16
DETAILS OF THE CRITERIA
The 5 Cs . . .
DOABILITY,
IMPORTANCE,
MAGNITUDE,
URGENCY and
RELEVANCE . . .
Hilario P. Martinez 17
Do-ability • PPAs that can be
implemented given the limits of resources and circumstances affecting it
• Must be responded to
by proponent agency
• 90%+ of required resources are available 5 pts
• 80-89% of required resources are available 4 pts
• 70-79% of required resources are available 3 pts
• 60-69% of required resources are available 2 pts
• 50-59% of required resources are available 1 pt
Hilario P. Martinez 18
Rating Level:
Doability – Rating Reference
Hilario P. Martinez 19
Doability Sample Rating Worksheet (Proponent Agency)
Hilario P. Martinez 20
Doability Sample Rating Worksheet (Partner-Agency*)
Hilario P. Martinez 21
* - Assuming 1 Partner-Organization per PPA. If more than 1, apply the same process and average rates obtained
Doability Sample Rating Worksheet (Average of Proponent and Partner-Agency)
Hilario P. Martinez 22
Importance
The qualitative degree of significance attributed to certain factors to contribute to mission critical programs
Each will determine, according to its mandate and perspective, the degree of importance a joint engagement entails to them
•Highest imperative & value to the agency’s mandate 5 pts
•2nd highest degree of significance 4 pts
•3rd or moderate degree of significance 3 pts
•4th or minor degree of significance 2 pts
•least degree of significance 1 pt
Hilario P. Martinez 23
Priority Ranking:
How would you
rate the level of importance of this
PPA to your agency?
Importance – Rating Reference
Hilario P. Martinez 24
81% - 100%
61% - 80%
41% - 60%
21% - 40%
01% - 20%
Degree of Importance
How does your Partner-agency rate the level of importance of this PPA to them?
C – Partner-Agency
P – Proponent Agency
Proponent Agency
P5
P4
P3
P2
P1
C5
C4
C3
C2
C1
Partner-Agency
Importance – Rating Matrix
5.0 - C5 & P5
4.5 - C5 & P4, C4 & P5
4.0 - C5 & P3, C4 & P4, C3 & P5
3.5 - C5 & P2, C4 & P3, C3 & P4, C2 & P5
3.0 - C5 & P1, C4 & P2, C3 & P3, C2 & P4, C1 & P5
2.5 - C4 & P1, C3 & P2, C2 & P3, C1 & P4
2.0 - C3 & P1, C2 & P2, C1 & P2
1.5 - C2 & P1, C1 & P2
1.0 - C1 & P1
Hilario P. Martinez 25
H I G H
L O W
COMBINATION OF POSSIBLE RESPONSES PER RATE
C – Client-Partner or
Target Sector
P – Proponent Agency
Importance – Sample Rating Worksheet
Hilario P. Martinez 26
Magnitude - the relative sizes and trends of
affected beneficiary sector
Hilario P. Martinez 27
Determining whether a factor has an increasing or decreasing trend is a function of time series
Determining whether a factor is increasing or decreasing in size compared to another is a function of quantity
The combined ratings for trend and size comprises the rating for “Magnitude"
Priority Ranking:
5 points Very Significant
4 points Significant
3 points Acceptable
2 points Slightly Acceptable
1 point Not Acceptable
What tendency does the analysis of the data disclose? Is it increasing, same as before or decreasing?
Magnitude – Rating Reference
Hilario P. Martinez 28
How does the latest data set compare with previous year, bigger, the same or smaller?
• 51% to 100% greater S5=5
• 11% to 50% greater S4=4
• -10% to 10% deviation S3=3
• -11% to 50% lesser S2=2
• - 51% to -100% lesser S1=1
Decreasing T1=1
Increasing T2=2
Size (Quantity) Trend (Time Series) (at least 3 years data set)
Magnitude – Rating Matrix
Hilario P. Martinez 29
S – Size
T – Trend
(S x T) / 2
H I G H
L O W
CO
MB
INA
TION
OF P
OSSIB
LE R
ESPO
NSES P
ER P
OIN
T SCO
RE
5 = T2 & S5 4 = T2 & S4 3 = T2 & S3
2.5 = T1 & S5
2 = T2 & S2, T1 & S2
1.5 = T1 & S3
1 = T2 & S1, T1 & S2
0.5 = T1 & S1
Magnitude – Sample Rating Worksheet
Hilario P. Martinez 30
Urgency - time consideration or response that must be
made to a given problem or issue over time
5
Most urgent
Within the next
2 months
4
Urgent
Within the next
3 to 4 months
3
Moderately urgent
Within the next
5 to 6 months
2
Least urgent
Within the next
7 to 8 months
1
Not urgent
Within the next 9 to 10 months
Hilario P. Martinez 31
This criterion should be responded to by the proponent organization, but preferably in consultation with the client-organization or beneficiary sector
Urgency – Sample Rating Worksheet
Hilario P. Martinez 32
Relevance - how a project relates to the vision, mission
and objectives of the service area
5
Most relevant
Beneficiary impact
estimated at 81-100%
4
Relevant
Beneficiary impact
estimated at 61-80%
3
Moderately Relevant
Beneficiary impact
estimated at 41-60%
2
Less Relevant
Beneficiary impact
estimated at 21-40%
1
Not Relevant
Beneficiary impact
estimated at 01-20%
Hilario P. Martinez 33
This criterion should be responded to by the proponent organization, but in consultation with the client-organization or beneficiary sector
Relevance – Sample Rating Worksheet
Hilario P. Martinez 34
DETERMINING THE PRIORITY RANKING
Hilario P. Martinez 35
Putting the Scores together
Hilario P. Martinez 36
Priority Rating
Doability
Importance
Magnitude
Urgency
Relevance
Ranking Summary Worksheet
Hilario P. Martinez 37
+ + + + +
S O R T
D E S C E N D I N G
Sample Summary Rating Worksheet
Hilario P. Martinez 38
Determining the qualified PPAs for prioritization
Hilario P. Martinez 39
PPA1 16.5 4
PPA2 14 7
PPA3 19 2
PPA4 11.5 9
PPA5 18 3
PPA6 15 6
PPA7 12.5 8
PPA8 11 10
PPA9 16 5
PPA10 20 1
PPAsPoints
EarnedRanking
PPA10 20 1
PPA3 19 2
PPA5 18 3
PPA1 16.5 4
PPA9 16 5
PPA6 15 6
PPA2 14 7
PPA7 12.5 8
PPA4 11.5 9
PPA8 11 10
PPAsPoints
EarnedRanking
Median = 15.5
Arranged by PPA # Arranged by Points Earned
QU
ALIFIED
PPA
s
What if Magnitude (Size & Trend) could be a factor for additional point/s?
Hilario P. Martinez 40
NOTE: The values for Mag(Size) per PPA are for illustrative purposes only, but the ratings for Mag(size) and Mag(trend) are the same with previous data used.
‘
Total Additional Points due to Comparative Implication of Magnitude (Size and Trend)
Hilario P. Martinez 41
Values of Magnitude(Size) are ranked, only the top 5 are rated in descending order (1, 0.8, 0.6, 0.4 & 0.2)
Only Magnitude (Trend) with a score of 2 is rated 0.5 for bonus point
Final Total Points Earned per PPA
Hilario P. Martinez 42
PPAs’ Final Priority Ranking
Hilario P. Martinez 43
Set Median = 15.5
Top
mo
st eligib
le P
PAs
for p
rioritizatio
n
WHAT ADVANTAGES DOES THIS CONCEPT OFFER A PROPONENT?
Hilario P. Martinez 44
Advantages of adopting this approach to program/project ranking
Hilario P. Martinez 45
The other factor
It discourages, minimizes, if not
totally prevent, the adoption of
“QUESTIONABLE” P.P.As from being considered and
funded for implementation
The
mo
st im
po
rta
nt
ad
van
tag
e o
f u
sin
g t
his
a
pp
lica
tio
n –
Hilario P. Martinez 46