Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st...
Transcript of Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st...
![Page 1: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/1.jpg)
1
Supply Chain Challenges – Deeper Analysis of EVM Paul Colrain (Consultant)
1
![Page 2: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/2.jpg)
Contents
EVM Assessment
EVM Dataset
EVM Results
Criterion Scores
Sub-indicator results
EVM Improvement
2
![Page 3: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/3.jpg)
The EVM Continuous Improvement Process
An essential component of the WHO/UNICEF strategy for strengthening in-country immunization supply chains:
“The 3 Steps to the 6 Rights” Right products
Right quantities
Right condition
Right place
Right time
Right cost
3
![Page 4: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/4.jpg)
EVM Assessment
4
![Page 5: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/5.jpg)
Primary level (PR): Vaccine stores that receive vaccine direct from an international vaccine manufacturer or distributors or a local vaccine manufacturer.
Sub-national level (SN): Vaccine stores that receive vaccine from a primary or higher level sub- national store.
Lowest distribution level (LD): Vaccine stores that receive vaccine from a primary or a sub-national store and supply vaccine to one or more health facilities.
Service point level (SP): Facilities that receive vaccine from any higher level store and supply immunization services.
1PR
1SN
2SN
1LD
1SP
EVM Assessment: What does it assess?
EVM assesses each level of the supply chain:
5
![Page 6: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/6.jpg)
E8 Vaccine
management
E7 Distribution
E3 Capacity
E2 Storage
temperature
E4 Buildings,
equipment, transport
E6 Stock
management
E5 Maintenance
E1 Pre-shipment
& arrival
E9 Information systems &
supportive management
EVM Assessment: What does it assess?
EVM assesses 9 areas of vaccine management – the 9 EVM Criteria:
6
![Page 7: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/7.jpg)
Criteria are assessed by inspection and interview
Inspection of cold chain equipment, transport vehicles and buildings
Inspection of records (temperature, stock, wastage, …) for the previous 12 months
Interview of responsible staff to assess knowledge, understanding and practice
EVM Assessment: How does it assess?
The EVM questionnaire has 400 questions (sub-indicators)
400 at PR level, 300 at SN, 200 at LD, 150 at SP
75% of the questions are binary [Y, N], 25% numerical.
Sub-indicator responses are combined to form 150 indicator scores:
150 at PR level, 100 at SN, 75 at LD, 60 at SP
Indicators can be turned off if not relevant (ex: no cold room questions if no cold rooms)
The 9 Criteria scores are weighted averages of relevant indicator scores
Indicators considered critical are given a weight of 5.
Target criteria score set at 80% - an effective incentive 7
![Page 8: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/8.jpg)
EVM Assessment: Sample Selection
EVM Assessment is used to identify areas for improvement (systemic weaknesses)
A sample of sites are selected at each level of the supply chain
International vaccine mfr/UNICEF
National store
(Level PR)
8 provincial stores (Level SN)
40 district stores (Level LD)
600 Health facilities
(Level SP)
Sampling@ LD level
Country choses precision and confidence level for Binary sub-indicator scores (% Yes) at the Lowest Distribution level (LD)
Sample indicator and sub-indicator scores are used to identify areas for improvement and prioritize improvement activities
Given a limited assessment budget, how to be confident that sample scores represent the national picture?
LD sample size determined by chosen precision and confidence level (hypergeometric statistics)
One or two Service Point (SP) facilities are randomly selected for each of the selected LD stores
Supply chain followed back to primary level to determine sample of sub-national (SN) stores
LD sample randomly selected from all LD stores
±5% ±10% ±15%
5 5 5 4
10 10 8 6
15 14 10 7
20 17 12 8
25 21 14 9
40 30 17 10
50 35 18 10
75 44 20 11
100 52 21 11
125 58 22 11
150 62 23 11
Number of LD stores
for sample selection
Option 1: 80% Confidence level and
Precision options in %
8
![Page 9: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/9.jpg)
EVM Dataset
9
![Page 10: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/10.jpg)
EVM Dataset
10
![Page 11: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/11.jpg)
EVM Dataset
2009: 3 1st assessments
11
![Page 12: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/12.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments
12
![Page 13: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/13.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments
13
![Page 14: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/14.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments
14
![Page 15: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/15.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments + 3 2nd assessments
15
![Page 16: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/16.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments + 3 2nd assessments 2013: 13 1st assessments
16
![Page 17: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/17.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments + 3 2nd assessments 2013: 13 1st assessments + 3 2nd assessments
17
![Page 18: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/18.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments + 3 2nd assessments 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments 18
![Page 19: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/19.jpg)
EVM Dataset
2009: 3 1st assessments 2010: 10 1st assessments 2011: 35 1st assessments 2012: 19 1st assessments + 3 2nd assessments 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19
![Page 20: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/20.jpg)
EVM Dataset
EVM Dataset
• 73 assessment reports Criterion scores for 73 assessments
• 48 assessment datasets Sub-indicator responses for 50 PR stores, 337 SN, 729 LD, 864 SP
20
![Page 21: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/21.jpg)
EVM Criterion Scores
21
![Page 22: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/22.jpg)
0% 20% 40% 60% 80% 100%
Vaccine arrivals E1
Temperature monitoring E2
Storage capacity E3
Infrastructure E4
Maintenance E5
Stock management E6
Distribution E7
Vaccine management E8
Information systems E9
PR
SN
LD
SP
0
5
10
15
20
25
30
35
40
45
% countries reaching 80%
Mean Score (68 countries)
National
Sub-national
Lowest distrib
Service delivery
EVM Criterion Scores
22
![Page 23: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/23.jpg)
EVM Sub-indicator results
23
![Page 24: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/24.jpg)
400 sub-indicators are grouped according to their potential impact on:
Availability of vaccines at the service delivery level
• Right quantities
• Right products
• Right place
• Right time
Quality of immunization supply chain (to safeguard vaccine potency)
• Right condition
Efficiency of immunization supply chain
• Right cost
EVM Sub-indicator results
24
![Page 25: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/25.jpg)
Availability
78%
63%
49%
63%
17%
70%
64%
0% 20% 40% 60% 80% 100%
PR
SN
LD
SP
Is a computerized stock control system in use?
Was the facility always (in previous 12 months)
able to supply vaccine to lower level stores?
National
Sub-national
Lowest distrib
Service delivery
EVM Sub-indicator results
*50 National stores, 337 Sub-National stores, 729 Lowest Distribution stores, 864 Service Points 25
![Page 26: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/26.jpg)
Quality
Do all cold rooms and freezer rooms have continuous temperature recorders?
Do all refrigerators have continuous temperature
recorders or freeze indicators?
Are freeze indicators used when transporting freeze-
sensitive vaccines with conditioned ice-packs?
Where absorption refrigerators are used, is kerosene/gas
always available?
35%
29%
9%
50%
50%
5%
32%
41%
5%
59%
40%
70%
0% 20% 40% 60% 80% 100%
PR
SN
LD
SP
National
Sub-national
Lowest distrib
Service delivery
EVM Sub-indicator results
26 *50 National stores, 337 Sub-National stores, 729 Lowest Distribution stores, 864 Service Points
![Page 27: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/27.jpg)
Efficiency
Are wastage rates calculated for each
vaccine?
64%
53%
65%
57%
60%
61%
58%
53%
68%
42%
69%
0% 20% 40% 60% 80% 100%
PR
SN
LD
SP
National
Sub-national
Lowest distrib
Service delivery
Is there a Standard Operating Procedures
(SOP) manual?
Does the facility receive regular supportive
supervision?
EVM Sub-indicator results
27 *50 National stores, 337 Sub-National stores, 729 Lowest Distribution stores, 864 Service Points
![Page 28: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/28.jpg)
EVM Improvement
28
![Page 29: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/29.jpg)
29
0%
20%
40%
60%
80%
100%
E1 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9
Senegal 2009
Senegal 2012
0%
20%
40%
60%
80%
100%
E1 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9
Viet Nam 2009
Viet Nam 2012
EVM Improvements
29
![Page 30: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/30.jpg)
30
0%
20%
40%
60%
80%
100%
E1 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9
Cameroon 2010
Cameroon 2013
0%
20%
40%
60%
80%
100%
E1 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9 E2 E3 E4 E5 E6 E7 E8 E9
Sierra Leone 2010
Sierra Leone 2013
EVM Improvements
30
![Page 31: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/31.jpg)
Conclusion
EVM Assessments
Many countries have now conducted EVM assessments, including many non-GAVI eligible countries
Many countries are scheduled for their 2nd assessments in 2014-2016
EVM Data
Quantifiable evidence of significant immunization supply chain challenges faced by countries
EVM Improvement
Challenges can be met with:
• strong country ownership of improvement process
• coordinated partner support
31
![Page 32: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/32.jpg)
EVM Criterion Scores versus DTP3 coverage
DT
P3 c
overa
ge
Lowest Distribution E3 Score (Storage Capacity)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
32
![Page 33: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/33.jpg)
EVM Criterion Scores versus DTP3 coverage
𝑫𝑻𝑷𝟑 = 𝟏 − 𝑨𝟏· 𝒆𝜷𝟏·𝑬𝟔𝑺𝑷; 𝜷𝟏= −𝟑. 𝟎 ± 𝟎. 𝟔; 𝑹𝟐 = 𝟎. 𝟐𝟗
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Service Point E6 Score (Stock Management)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
DT
P3 c
overa
ge
33
![Page 34: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/34.jpg)
EVM Criterion Scores versus DTP3 coverage
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
DT
P3 c
overa
ge
Service Point E6 Score (Stock Management)
● poor infrastructure ( I ) ● good infrastructure
𝑫𝑻𝑷𝟑 = 𝟏 − 𝐴1 · 𝐴2𝐼 · 𝒆 𝜷𝟏+𝜷𝟐·𝑰 ·𝑬𝟔𝑺𝑷; 𝜷𝟏 = −𝟐. 𝟎 ± 𝟏. 𝟏; 𝑹𝟐 = 𝟎. 𝟑𝟖
34
![Page 35: Supply Chain Challenges Deeper Analysis of EVM€¦ · Supply Chain Challenges ... 2013: 13 1st assessments + 3 2nd assessments 2014: 2 1st assessments + ? 2nd assessments 19 . EVM](https://reader033.fdocuments.in/reader033/viewer/2022053013/5f101def7e708231d4478782/html5/thumbnails/35.jpg)
Thank you
35