8 th Annual Track 1.0 ART Program Meeting Maputo, Mozambique August 10, 2010.

Strategic Guidance for a Standards-based Approach to National ART Program Supervision

to Assure Quality Care and Sustainability of U.S. Government Support to National ART Program

8th Annual Track 1.0 ART Program MeetingMaputo, Mozambique

August 10, 2010

PEPFAR IIOn July 30, 2008, the United States Global

Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act was signed into lawAuthorized $48 billion over the next 5 years to

combat HIV/AIDs, tuberculosis, and malaria$39 billion (81%) were authorized for PEPFAR

bilateral HIV/AIDS programs and U.S. contributions to multilateral partnerships To support treatment for at least 3 million people To support prevention of 12 million new infections To care for 12 million people by Sept. 20, 2014

Assuring the treatment goals of PEPFAR II

USG agencies and funded partners must demonstrate strong stewardship of the USG funds and programsReporting how funds are being usedEnsuring the quality of servicesAccurately reporting the number of persons on ART

To enhance the long-term sustainability of PEPFAR programs, there is a process to transition the administrative and fiscal responsibilities of USG-supported ART programs to the MOH and other local partners

A strong supervision system for the ART program is fundamental for fiscal reporting, quality services, and accurate results reporting.

The current state of ART program supervision and monitoring

Key areas of supportInfrastructure improvement and maintenanceHuman resources TrainingMentoringLaboratoryPharmacyData and information systemsCommodities availabilityCommunity linkages

The current state of ART program supervision and monitoring

Partners expected to conduct visits to facilities and districts on a regular basisAssess overall functioning of HIV clinicsProvide technical assistanceEnsure quality of care

Current state of supervision varies from country to countryImplementing partners have worked with the MOH to

develop and pilot supervisory programsPartners themselves are providing supervision and

mentoringSupervisory visits may be rare due to insufficient staff,

shortage of skills, authority, and transportation

The current state of quality management in ART programs

In many countries, USG funds support quality management (QM) and performance improvement and ( PI) initiatives Some use an experienced QM partner to assist them with their

QM/PI initiatives Implementing partners integrate QM with clinical guidelines and

standards of care (SOC)

An effective QM program continuously validates the reliability of performance measurement data, and uses this data to improve the quality of care services

Supportive site supervision around quality may focus on one or more of the following:data quality assessmentclinical consultative back-up, education, and mentoring facilitation of additional resources to improve care and services

The current state of ART monitoring and evaluation Implementation partners expected to collaborate with MOH staff

to develop, implement, and maintain patient and program monitoring systems at their supported facilities

Health facilities expected to follow MOH or National AIDS Control Program minimum standards for patient monitoring which include: a minimum standard data set a unique national HIV program ID an HIV care and treatment patient record pre-ART and/or ART registers (unique patients organized

chronologically by month of enrollment/initiation) standard indicators Standard monthly and/or quarterly reports on cross-sectional and

cohortsMinimum standards are not always metSome USG country teams fund external partners to provide

M&E support to specific levels of the systemRecent data quality assessments suggest that the quality of data

collected in the ART monitoring systems needs improvement

StandardsStandard 1: There is a regular schedule of

joint (USG-MOH) supportive supervisory site visits.

Standard 2: There is a reliable results monitoring system for oversight of partners and programs.

Standard 3: All ART sites have a functional and sustainable patient monitoring system.

Standard 4: There is a regular schedule and reporting of ART data quality assessment activities.

Standard 5: All ART sites have QM/PI processes in place.

Standard 1 A regular schedule of joint (USG-MOH) supportive supervisory site visitsRationale:To provide adequate and informative oversight, the USG

PEPFAR staff and the MOH should be familiar with the treatment programs at individual sites. Experience during the first 5 years of PEPFAR suggests that this is best achieved by a schedule of routine supportive supervisory site visits.

Supportive supervision is a practical and collaborative approach to supervision that aims to improve the quality of clinical care, efficiency of services, and satisfaction of staff. It focuses on ongoing identification and resolution of problems through joint observations, team discussions, and direct site involvement and emphasizes dissemination of results and the use of data for decision-making.

Standard 1 A regular schedule of joint (USG-MOH) supportive supervisory site visits Ideally, the supportive supervision framework should comprise

at least two supervisory levels in each country. District level team including the district MOH office and the USG

partner; this team conducts routine and systematic visits to every site in their area in a predefined period.

Central supervisory team including national MOH staff and a USG member will: monitor overall activities support the mid-level supervisors modify guidelines and policy as indicated by supervisory data and reports. accompany the district teams on visits to pre-selected sites on a routine basis

(e.g., every 3 months a percentage of clinics will be visited)

Site visits will be scheduled and standardized so that staff can plan for and fully participate in the visit.

A national supervisory tool is required to support standardization and supervision.

Standard 1 Action Steps to Achieve Standard

Action Steps Performance Measures Responsibility Timeline

Establish a schedule of joint USG-MOH site visits so that each USG site is visited at least once during FY 2010-FY 2011.

Detailed reliable schedule available for review Documentation that sites are aware of schedule

Develop a National Supportive Supervision Tool that outlines a standardized approach to site visits.

Document available either electronically or via hard copy

Identify staff at the central, district, and partner levels that will be stable members of the supervisory team. Clearly delineate roles and responsibilities of each member with formal allocation of time for supervisory activities

Supervisory team rosters available with responsibilities and relationships between members defined

Standard 1 Action Steps to Achieve Standard

Action Steps Performance Measures Responsibility TimelineConduct training on supportive supervision concepts for supervisory team with refresher courses as necessary

Training scheduled, implemented, and attended by teams

Establish annual budget to support supervisory team and their activities

Budget includes money for supervisory activities, including team members’ time for visits, writing of reports, and analysis of reports at central level to inform program development

Identify practical and logistical impediments (e.g., reliable transportation) to routine site visits and a plan to address them

Plan available for review

Identify technical assistance (internal or external to USG) that might be needed to actualize the schedule of site visits.

Technical assistance needs documented

Standard 2 Results monitoring system to improve oversight of partners and programs

Rationale: In the absence of fully functional MOH systems that

can deliver needed information to the USG, the USG agencies and their funded partners must support systems to monitor the quantity and quality of services provided at supported sites.

Standard 3 All ART sites have a functional and sustainable patient monitoring system Rationale: The facility-based system should have the tools and staff needed to

support patient care and to report on a timely basis on the required indicators - national, USG, and otherwise. Thus the system must include the following: Tools: Standard records, registers, reports, and databases are in use at

site. Electronic databases are likely needed for sites with more than 500 active ART patients.

Patient ID: Patients can be uniquely identified at the site (preferably across sites with unique national health or HIV patient ID if possible.

Organization: Patient records are stored in a way that allows for systematic management, retrieval and update of records.

Interoperability of Clinic, Lab, and Pharmacy Systems: Data within clinic, lab, and pharmacy systems can be exchanged and compared among those systems via common data elements, forms, processes, or electronic programs.

Confidentiality/Security: Records, registers and databases are kept in secured space with restricted access.

Staff: Sufficient and qualified staff to maintain patient monitor system. Reporting: Timely production and transmittal of all monthly and

quarterly reports

Standard 4 The regular conduct and reporting of ART data quality assessment activities

Rationale: Recent multi-country rapid data quality assessments

showed that the quality of data collected in ART monitoring systems needs improvement. Substantial over-reporting of persons on ART was discovered at selected large USG-funded ART sites in several countries, due largely to the failure of systems to subtract from the “active” or current roster of patients those persons who had not had a visit or drug pick-up in the recent period. These findings reveal challenges to other indicators of ART program performance, including the number ever on ART and the number retained in ART programs.

Standard 4 Action Steps to Achieve Standard ACTION STEPS PERFORMANCE

MEASURESRESPONSIBILITY TIMELINE

Review, develop, and disseminate additional data quality assessment protocols and tools, as needed.

Data quality assessment protocol and tools exist.

Provide technical support (with external partner) to country teams, partners, and sites for thorough data quality assessments and data quality improvement plans.

Technical support has been identified and is available to country team, partners, and sites for data quality improvement.

Require all USG care and treatment partners to conduct specified data quality assessments on all directly supported sites in FY 2010-FY2011 and report back site-specific findings with action plans for addressing challenges of underperforming sites.

Partners have conducted data quality assessments on all directly supported sites in FY2010-FY2011. Site specific findings with action plans have been reported.



Incorporate into contracts/cooperative agreements the expectation to incorporate standard operating procedures (SOPs) to check quality of data at every step in data flow AND to periodically conduct thorough data quality assessments on key indicators (e.g. current on ART and retention in ART program) and report the results of those assessments to the funding agency.

Existing contracts/cooperative agreements include requirement for periodic data quality assessments with reporting of results to the funding agency.

Standard 5 All USG-funded partners and ART sites have formal QM/PI plans and reporting mechanisms

Rationale: QM plans and PI activities promote the local review of the

quality of data, the local use and understanding of data, and the collaboration of data and clinical staff on PI activities

Each national ministry or department of health should have written standards of care (SOC) and a national QM/PI plan, which are disseminated throughout the system and demonstrable at the site level. The SOCs should be used to guide the development of specific quality indicators for each plan. The language of the national plan should be reflected in the plans of all USG-funded partners.



Identify and describe existing country standards of ART care and treatment

Country SOCs exist and are available for review.

Identify and describe existing QM/PI plans

QM/PI plan exists and is available for review. Plan has been updated within 2 years.

Engage an external partner(s) to work in collaboration with the MOH and other USG-funded partners to facilitate adoption and implementation of a national QM/PI plan, if needed

An assessment has been conducted to determine the need for an external partner to work with the MOH and other partners to facilitate adoption and implementation of a national QM/PI plan and documentd.An external partner has been engaged, if needed.



Work with USG-funded partners to evaluate QM/PI plans throughout directly supported facilities

The QM/PI plan contains the elements described above, that larger (above PHC level) sites have a QM/PI committee, that at least one staff member at each site has received QM/PI training, and that recent PI activities have been reviewed by higher level staff

Incorporate the expectation of updated QM/PI plans into contracts and cooperative agreements

Contract/cooperative agreements contain a quality section. These contain a requirement for quarterly and annual reports of QM/QI activities.



Request information in the quarterly and annual reports from the partner to the agency on the number of participating sites and current PI projects based upon national indicators

At least one PI activity has been conducted and documented, based on national indicators, in each quarter of the past year, with evidence of an improved outcome.

Request information in the quarterly and annual reports from the partner to the agency on current PI projects based upon a site-specific indicator

At least one PI activity has been conducted, based upon a site-specific indicator, in the past 6 months, with documented evidence of an improved outcome

External Assessment of ART Program Supervisory Systems

In order to support and sustain the provision of high quality clinical services and accurate program and fiscal reporting, USG headquarters proposes a regular schedule of site assessments to evaluate the progress of implementation and/or maintenance of the country-specific ART supervision plan. Each review would be conducted over the course of 2-3 weeks, potentially including staff from: the Ministry of Health USG country HIV/AIDS Adult Care and Treatment and M&E Teams USG headquarters (CDC, USAID, DOD, etc.) Adult Treatment and

Monitoring and Evaluation (M&E) Technical Work Groups implementing partner (e.g., Columbia, EGPAF) US headquarters

offices implementing partner (e.g., Columbia, EGPAF) country or regional

offices

External Assessment of ART Program Supervisory SystemsReview Process: Each country visit would begin with a preliminary planning

meeting of the review team in the country capital. The review team would then be divided into several smaller site review teams, which would be deployed to the sites. In general, each review team would review one to two sites per day, depending upon the size of the site and distance between sites, using standardized procedures.

We propose a site visit methodology in which sites would be chosen using a random sampling method of at least 5 larger sites and 5 smaller sites (> 50 patients on ART), which would be equally distributed between implementing partners. Logistics of site visits would be developed by in-country staff prior to the review period.

External Assessment of ART Program Supervisory SystemsReview Process: An existing supervisory tool (such as the one described

in the Standard 1 sample action plan), or one developed for the purpose, would be used by all teams and address each of the [five] proposed standards.

Each site review team would provide the site with a verbal review report and draft a site report with recommendations (following a standardized outline).

The review teams would participate in a group debrief at the end of the review period with all stakeholders.

A final report would be drafted by the review team for submission to the in-country stakeholders and USG headquarters.

6 month to 1 year planPilot standards-based approach in 3 countries

Hold meetings with USG country staff at Maputo meetingSummarize feedback on approach and share with relevant

TWGsMeet with MOH and other country stakeholders to develop an

implementation planConduct external assessment visits at randomly selected sitesEvaluate pilot project results and modify process based on

these resultsExpand Approach to other PEPFAR countries

Enlist and train assessors for review teamsSchedule preliminary meetings in countriesConduct assessmentsReport findings of assessments to relevant stakeholders and

USG headquarters

Thanks

To the team members who have participated in this project to date and who have greatly contributed to it. Laura Porter ([email protected]) Carla Johnson ([email protected]) Laura Broyles ([email protected]) Joseph Barker ([email protected])

To all of you who will need to work with us on this project

Please give any comments that you may have to members of the team during or after the meeting

8 th Annual Track 1.0 ART Program Meeting Maputo, Mozambique August 10, 2010.

Documents

Transcript of 8 th Annual Track 1.0 ART Program Meeting Maputo, Mozambique August 10, 2010.