Mind your Ps and Qs: Conducting QI in PCBH using the PPAQ and PDSA

• Gregory P. Beehler, Ph.D., M.A.; Associate Director of Research, VA Center for Integrated Healthcare

• Katherine M. Dollar, Ph.D.; Associate Director of Implementation, VACenter for Integrated Healthcare

• Paul R. King, Ph.D.; Clinical Research Psychologist, VA Center for Integrated Healthcare

Session # H8

CFHA 19th Annual ConferenceOctober 19-21, 2017 • Houston, Texas

1

Faculty Disclosure

The presenters of this session have NOT had any relevant financial relationships during the past 12 months.

The information provided in this presentation does not represent the views of the Department of Veterans Affairs or the United States Government.

2

Conference ResourcesSlides and handouts shared in advance by our Conference Presenters are available on the CFHA website at http://www.cfha.net/?page=Resources_2017

Slides and handouts are also available on the mobile app.

3

Learning Objectives

At the conclusion of this session, the participant will be able to:

1. Discuss the purpose and development of the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ) Toolkit

2. Describe how the PPAQ Toolkit can be used as a diagnostic self-assessment of usual PCBH practice behaviors

3. Implement Plan-Do-Study-Act (PDSA) on an individual or team basis to engage in quality improvement (QI) in local clinic settings

4

1. Beehler, G. P., Funderburk, J. S., Possemato, K., & Vair, C. (2013). Developing a measure of provider adherence to improve the implementation of behavioral health services in primary care: A Delphi study. Implementation Science, 8, 19.

2. Beehler, G. P., Funderburk, J. S., Possemato, K., & Dollar, K. Psychometric assessment of the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ). (2013). Translational Behavioral Medicine, 3, 379-391.

3. Beehler, G. P., & Lilienthal, K. R. (2017). Provider perceptions of an integrated primary care quality improvement strategy: The PPAQ Toolkit. Psychological Services, 14, 50-56.

4. McDaniel, S. H., Grus, C. L., Cubic, B. A., Hunter, C. L., Kearney, L. K., Schuman, C. C., …Johnson, S. B. (2014). Competencies for psychology practice in primary care. American Psychologist, 69, 409-429.

5. Taylor, et al. (2014). Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Quality & Safety, 23, 290-298.

Bibliography / Reference

5

Learning Assessment

A learning assessment is required for

CE credit.

A question and answer period will be

conducted at the end of this

presentation.

6

Agenda

Background: PCBH, provider behavior, and fidelity

Development of the PPAQ as an instrument and toolkit

Plan-Do-Study-Act

•Measurement-Based Care example•Team huddles example

Summary

Discussion

7

Primary Care Behavioral Health • Primary Care Behavioral Health (PCBH) is a model in which

behavioral health providers embedded in primary care deliver brief assessment and intervention for patients’ mental and behavioral health concerns as well as provide consultation to primary care providers (PCPs)

• PCBH treatment is brief (e.g.,15- to 30-min appointments) and time limited (e.g., up to four to six appointments scheduled on a biweekly to monthly interval)

• A lower frequency, intensity, and duration than traditional psychotherapy

• There is no gold standard for PCBH implementation

• Local formulations of PCBH vary based on organizational culture, resources, infrastructure, etc.

8

PCBH Providers• The role of behavioral health providers in PCBH is complex

• PCBH provider competencies are quite broad (e.g., McDaniel, et al., 2014)

• PCBH providers may have limited background and training related to integrated care (Serrano, et al., 2017)

• PCBH providers may face a number of potential practice and system barriers:

• Limited or inconsistent space in the primary care clinic• PCPs who are not fully receptive to integrated care

• Clinic or system leadership may not understand PCBH

• The above factors influence the day-to-day practice behaviors of providers and lead to variability in how PCBH is enacted locally

9

PCBH Literature Review• Narrative review of empirical studies (1990-2016) of PCBH provider behavior

• Key findings:

• Adherence to 30-min sessions was mixed

•PCBH providers may struggle with appointment length

• PCBH providers primarily addressed common mental health concerns (e.g., depression)

•Behavioral medicine concerns were less frequently addressed

• PCBH providers engaged in multidimensional functional assessments

•Use of standardized measures was low

• PCBH providers develop strong therapeutic relationships with patients

10

Beehler, G. P., Lilienthal, K. R., Possemato, K. P., Johnson, E. M., King, P. R., Shepardson, R. L., et al. (in press). Narrative review of provider behavior in Primary Care Behavioral Health: How process data can inform quality improvement. Families, Systems, & Health.

PCBH Provider Behavior and Fidelity• Provider behavior is an indicator of fidelity

• Fidelity refers to the degree to which an intervention is delivered as intended

• Is a provider utilizing certain procedures and engaging in specific tasks and activities? (Carroll et al., 2012)

• Are patients receiving the intervention’s active ingredients from the provider?

• Fidelity is the most commonly reported measure of implementation and is anindicator of quality of care (Proctor et al., 2011)

• Monitoring processes of care can identify targets for QI initiatives (Bilimoria, 2015)

• High fidelity practice DOES NOT mean providing the same service to all patients every single time

• No need to be a robot!

• Purposeful modification to provide patient-centered care is expected

11

Background: The PPAQThe PPAQ is a 48-item self-report measure of integrated care provider fidelity

• Designed to reflect the PCBH model of integrated care

• Items reflect provider behaviors/practices

Items represent provider behavior across several practice domains:

• Clinical scope and interventions

• Practice and session management

• Referral management and care continuity

• Consultation, collaboration, and interprofessional communication

Questionnaire items are classified based on their relevance for integrated care:

• Essential (consistent with the PCBH model; required for good practice)

• Prohibited (inconsistent with the PCBH model; should be avoided)

12

Developing The PPAQStudy 1: A modified Delphi study using a larger panel of experts to reach consensus on critical behaviors of PCBH providers

• Improves content validity for instrument development

Study 2: Psychometric evaluation with 173 VA PCBH providers

• Refined the PPAQ and ensured sufficient psychometric quality (reliability and validity)

Study 3: Latent class analysis

• Demonstrated that the PPAQ had utility in identifying high/low-scoring providers

13

Beehler, G. P., Funderburk, J. S., Possemato, K., & Vair, C. (2013). Developing a measure of provider adherence to improve the implementation ofbehavioral health services in primary care: A Delphi study. Implementation Science, 8, 19.Beehler, G. P., Funderburk, J. S., Possemato, K., & Dollar, K.M. (2013). Psychometric assessment of the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ). Translational Behavioral Medicine, 1, 13.Beehler, G. P., Funderburk, J. S., King, P., Wade, M., & Possemato, K. (2015). Using the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ) to identify practice patterns. Translational Behavioral Medicine, 5, 384-392.

From Measure to Toolkit•We aimed to make the PPAQ more useable for frontline PCBH providers

• Can the PPAQ be used as a self-assessment toolkit?

What’s a toolkit?

• High quality information source that organizes and summarizes information (Wandersman, et al, 2012)

• Materials and methods used for improving health care quality

•With VA quality improvement funding, we developed and pilot tested an Excel-based toolkit for providers based on the PPAQ

14

Beehler, G. P., & Lilienthal, K. R. (2017). Provider perceptions of an integrated primary care quality improvement strategy: The PPAQ Toolkit. Psychological Services, 14, 50-56.

15

16

17

18

19

20

21

Moving Forward with your PPAQ ResultsStep 1: Prioritize

• Most providers will find that they have several areas for improvement

Step 2: Partner with advocates and peers

• Reach out to your supervisor or integrated care champion at your site

Step 3: Access educational and clinical resources

• VA Center for Integrated Care website

• https://www.mirecc.va.gov/cih-visn2/clinical_resources.asp

Step 4: Setting goals and making change

• Plan-Do-Study-Act

22

From PPAQ to PDSA• PDSA originated in industry

• The goal was develop an easy-to-use process for QI

• Designed to be used by virtually any individual or team

• No extensive training required!

• The stages of PDSA mirror the scientific inquiry process of hypothesis generation, data collection, analysis, and drawing conclusions

• PDSA is different from traditional research or evaluation in its purpose

• PDSA is action-oriented

• PDSA improves quality by testing small scale changes over multiple iterations

• PDSA is highly flexible; tests of change range from days to months

23

PDSA Components All PDSA starts with a target or goal

• What do we want to change/improve?• See PPAQ Toolkit results

Plan: • What will we try to change and what predictions do we have?• What data will we collect?• Address logistics: Need materials/resources?

Do:• Test the improvement/change• Collect data, including observations

Study:• Analyze your data/observations• Compare your results to your predications

Act:• Standardize/implement a successful change• Modify less successful strategies

24

PDSA Cycles

String together multiple small PDSA cycles for the best results

• Modify/improve your change strategy

• Expand or reduce your measures

• Identify and overcome barriers encountered

• Apply unplanned strategies for success you discover

http://www.theclic.org.uk/cumbria-production-system/toolkit-2/appendices/tools-in-detail/a-module-1/a-1-1-plan-do-study-act

25

PDSA Worksheets

• PDSA can be facilitated through the use of templated worksheets

• Worksheets provide a user-friendly format that encourages documentation

• Many are freely available online• http://www.ihi.org/resources/Pages/Tools

/PlanDoStudyActWorksheet.aspx

https://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/literacy-toolkit/healthlittoolkit2-tool2b.html

26

PDSA Worksheets

https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/downloads/PDSACycledebedits.pdf

https://www.med.unc.edu

27

PDSA for Measurement-Based Care• A PCBH provider at a small community clinic self-administered the PPAQ toolkit

• In reviewing her results, scores on items 15 and 30 were both endorsed as “rarely”

• She valued the use of brief self-report measures in theory, but used them infrequently

• She identified several barriers to use (e.g., lack of time, uncertain how to apply results, etc.)

• Nonetheless, she hoped use of standardized measure would help meet her larger goal of improving her speed and accuracy of diagnosis

15 I administer one or more brief validated measures (e.g., Patient Health Questionnaire-9, or PHQ-

9) for an initial screening of symptoms of interest, or I review these findings if measures were

administered by other primary care staff.

30 I administer one or more brief validated measures (e.g., Patient Health Questionnaire-9, or PHQ-

9) for follow up screening of symptoms of interest, or I review these findings if measures were

administered by other primary care staff.

28

Cycle 1 PDSA with MBC: Plan Objective for this cycle:

Measure baseline depression score using PHQ-9 for all relevant patients on Wednesday

Comments:

• Note that the plan identified a feasible step of using the PHQ-9 on a single clinic day.

• Not tested on all presenting concerns

• Identifying small, specific practice change goals are the best first step for conducting PDSA.

29

• I will use the PHQ-9 with patients with suspected mood disorders using the screening application in the EMR.

• As part of the usual Assess phase of the baseline 5-A’s functional assessment (i.e., Assess, Advise, Agree, Assist, Arrange), I will complete the PHQ-9 just prior to completing a suicide risk assessment.

• I will use the PHQ-9 score to assist in developing a working diagnosis during this encounter.

• I will document the score in my progress note in the EMR to support my diagnosis and inform the primary care provider.

PDSA Worksheet:

Cycle 1 PDSA with MBC: Do

Comments:

• With a plan in place, this provider was ready to “Do”, or conduct a small test of practice change.

• Documenting observations is key to informing your analysis in the next step (Study).

• Be certain to record barriers encountered as well as successes.

30

• It was easier to use the PHQ-9 than I expected. However, it did take some time to get used to asking the questions while reading from the EMR screen.

• Some patients took a while to answer the questions which slowed me down. But everyone seemed to understand the questions I asked.

• For one patient who had complex concerns, my functional assessment was running long, so I was not able to fit in the PHQ-9.

PDSA Worksheet:

Cycle 1: Do (con’t)

Comments:

• A key advantage of documenting the Do of PDSA is that it can identify unplanned benefits as well.

• Note the role of PHQ-9 in assisting this provider with diagnosis and treatment planning.

31

• Having the EMR automatically score the PHQ-9 was very helpful. The score helped me feel more confident making a diagnosis for 2 patients whose verbal report of symptoms were less clear.

• Reviewing responses to the PHQ-9 with one of the patients also helped me develop an action plan that was really patient-centered and targeted his most distressing symptoms.

• I’m still not sure of the best way to document the PHQ-9 using my progress note template.

PDSA Worksheet:

Cycle 1 PDSA with MBC: Study

Comments:

• Based on your observations, summarize your results from your test day in just a few sentences.

• Quantification of your results can assist with decision making about success or next steps.

32

• Out of 4 patients referred for depression on my “test” day, I was able to screen 2 with the PHQ-9.

• For those screened, the information was very helpful for providing feedback to the patient and identifying a treatment plan.

• Overall, I partially met my goal.

PDSA Worksheet:

Cycle 1 PDSA with MBC: Act

Comments:

• Draw conclusions and identify next steps.

• Usually, the next step is testing out another (or improved) strategy

• The provider learned quite a bit about administering standardized measures and has a plan of what to try next in cycle 2.

33

• Administering the PHQ-9 through the EMR worked most of the time, but I think that having the patient complete it using pen and paper would be faster.

• I will get paper copies of the PHQ-9 and ask patients to complete the form during our session.

• I decided it was easiest to include the patient’s score in the “Assessment/ Diagnosis” section of my note.

• This way, whoever is reading my note will see that I used the score to help inform my diagnosis and plan.

PDSA Worksheet:

Cycle 2 PDSA with MBC: PlanChange to test:

Use paper copies of the Patient Health Questionnaire-9 (PHQ-9) with all patients referred for depression during Wednesday clinic

Comments:

• Informed by the first cycle, the provider chooses to change the type of administration from the EMR to paper-and-pencil.

• The provider also chooses to implement a lasting change.

• She will include the PHQ-9 score in the “Assessment/Diagnosis” section of the progress note

34

• I will ask the clerk to photocopy the blank PHQ-9 form and gather extra pens to have on hand for patients to use.

• I will continue to ask patients to complete the PHQ-9 as part of the Assess phase of the baseline 5-A’s functional assessment, just prior to completing a suicide risk assessment.

• Based on cycle 1, I will document the score in my progress note in the Assessment/Diagnosis section.

PDSA Worksheet:

Cycle 2 PDSA with MBC: Do

Comments:

• In this cycle, the provider learned about some advantages and disadvantages to using the paper copies of the PHQ-9.

• She also found that incorporating PHQ-9 results into her progress notes was becoming more routine.

35

• Using paper copies of the PHQ-9 was easier to administer at first, because I was able to begin my progress note while the patient completed the form on paper.

• However, I had to quickly score the form using a calculator while the patient waited. It didn’t take too long, but it felt awkward and required more steps.

• One patient did not have his glasses, so he was unable to self-administer the measure. I went back to using the EMR for that patient.

• I’m getting used to including the score of the PHQ-9 in my assessment section and find I’m getting to a useful diagnosis more quickly.

PDSA Worksheet:

Cycle 2 PDSA with MBC: Study

Comments:

• The provider determined that electronic administration of the PHQ-9 had a few unexpected benefits, like automatic scoring.

• Other providers may have concluded that paper administration was preferred.

• It offered the provider a few moments to begin their progress note documentation while the patient completed the form.

36

• Out of 6 patients referred for depression on my test day, I was able to screen 5 with paper copies of the PHQ-9 – one was administered verbally.

• Using paper copies of the PHQ-9 was helpful and saved administration time, but ultimately required more steps than I expected.

• I partially met my goal.

PDSA Worksheet:

Cycle 2 PDSA with MBC: ActComment:

• The provider and has a goal for incorporating the PHQ-9, primarily via the EMR, during routine clinical practice.

• Her next step, when she is ready to move forward, is to add another measure for baseline assessment of patients referred for other mental health concerns.

• Alternatively, she could choose to implement this process for follow-up sessions to better track changes over time.

37

• I plan to use the PHQ-9 routinely with patients who are referred for depression because the time spent administering the measure helps me come to a diagnostic impression and treatment plan more efficiently.

• I will use the PHQ-9 routinely starting on Monday.

• I will then consider trying other measures, like the GAD-7 for anxiety.

PDSA Worksheet:

PDSA to Initiate Team Huddles• A clinic manager asked the PCBH provider at a large community clinic to complete the

self-assessment and identify areas for clinic program improvement.

• In reviewing his results, his scores on items 11 and 13 were both endorsed as “rarely”.

• Although the primary care team members have good relationships, and value team-based care, they were not meeting in a routine, structured format.

• Huddles have not been implemented.

• To enhance implementation of team-based care, the clinic manager asked the PCBH provider to develop a process to initiate huddles, using PDSA.

13 I consult with various members of the primary care team (pharmacist, dietician),

in addition to the Primary Care Provider about behavioral aspects of medical

conditions (e.g., medications that cause nightmares)

11 I meet briefly with primary care staff as a team to provide both a

behavioral health perspective and behavioral data.

38

Cycle 1 PDSA for Team Huddles: Plan Change to Test:

Team A will hold team huddles for 10 minutes at 8:00AM every Wednesday

Comments:

• Note that the plan identified a feasible step by starting huddles with one team once a week, rather than initially implementing clinic wide on a daily basis.

• As part of planning, the team realized that training may be necessary.

• Remember: Identifying small, specific practice change goals are the best first step for conducting PDSA.

39

• Clinic Team A will meet (Huddle) for 10 minutes at 8:00 AM weekly on Wednesdays for 1 month.

• All team members will attend.

• During the huddle, the team will review the patient list for the day.

• Team members will review huddle training materials prior to the first team meeting.

• The administrative team member will prepare the patient list for the huddle on Tuesday afternoons.

PDSA Worksheet:

Cycle 1 PDSA Team Huddles: DoDo:

Carry out the plan and document observations. What did you learn?

Comments:

• With a plan in place, clinic team A was ready to “Do,” or conduct a small test of practice change.

• Documenting observations is key to informing your analysis in the next step (Study).

• Be certain to record barriers encountered as well as successes.

40

•Initially team members weren’t sure how to best use huddle time.

•It helpful to have the list prepared the night before.

•Dr. Smith was late to 3 of the 4 huddles, also the team found that 10 minutes was not enough to review the entire patient list, with only 75% reviewed.

•Warm hand-offs to the BHP increased.

•The PCP found that reviewing patients with the team helped her to be more prepared, and less overwhelmed,

PDSA Worksheet:

Cycle 1 PDSA Team Huddles: Study

Study:

Analyze your data

Comments:

Based on your observations, summarize your results from your test in just a few sentences.

Try to quantify your results of your test of change.

41

• Overall Team A partially meet their goal: they were able to meet for 10 minutes as a team, once a week, increasing team communication and collaboration.

• They rarely started at 8:00AM, though, and found that 10 minutes wasn’t enough time to review all patients.

• They initially had confusion about process and expectations.

PDSA Worksheet:

Cycle 1 PDSA Team Huddles: ActAct:

Draw conclusions; Design the next cycle.

Comments:

Finally, draw conclusions from your test and identify next steps.

Usually, the next step is testing out another (or improved) strategy based on the findings of the first cycle.

By testing a small change, the team in this example learned about team scheduling and time required to review all daily patients and developed a plan of what to try next in cycle 2.

42

• The team realized that meeting at 8:00AM for 10 minutes worked for some, but not all. They decided to identify a time that would work better for everyone.

• The team also wanted to ensure that there was enough time to briefly review all patients for the day.

• They decided to move the huddle to 8:30 and extend to 15 minutes.

PDSA Worksheet:

Cycle 2 PDSA Team Huddles: PlanChange to Test:

Team A will hold team huddles for 15 minutes at 8:30AM every Wednesday

Comments:

Informed by the first cycle, the team decided to alter the start time and duration of the huddles. They also decided to document their progress based on challenges that were previously identified and areas in which they hoped to improve in the 2nd cycle.

43

• Clinic Team A will huddle for 15 minutes at 8:30 AM on Wednesdays for 2 weeks.

• All team members will attend.

• The administrative team member will continue to prepare the patient list on Tuesday afternoons.

• Based on cycle 1, we will document our progress by noting start time, prompt attendance, % of patients discussed, and # of warm hand-offs to the BHP

PDSA Worksheet:

Cycle 2 PDSA Team Huddles: Do

• All team members were on time for each huddle.

• Extending the huddle by 5 minutes allowed time to review 100% of scheduled patients.

• Team members reported continued communication, as well as an increase in warm hand-offs to the BHP on Wednesdays.

• Team members agreed they had improved team-functioning, but wondered if these benefits could be expanded to a larger percentage of their patients. .

PDSA Worksheet

Comments:

In this cycle, the team learned about the advantages of changing their start time and duration of meeting.

44

Cycle 2 PDSA Team Huddles: Study

• Out of 2 weekly huddles, all team members attended, were on time, and all patients for the day were discussed.

• Overall, the team found the huddles to be helpful and reported improved collaboration.

• Aware of the significant effort involved in planning and the overall time required, the team began to wonder if they could huddle more than once a week to improve team-based care for a greater percentage of their patients.

PDSA Worksheet

Comments:

In reflecting on what they learned, the team determined that meeting at 8:30 for 15 minutes improved their huddle process.

45

Cycle 2 PDSA Team Huddles: Act

• For Clinic Team A, 15-minute huddles every Wednesday at 8:30 were helpful.

• However, it is unclear if this process could become part of routine care, and extend beyond 1 day a week for this team.

• Clinic Team A plans to increase their frequency of huddles to 3 times a week (Mon., Wed., and Fri.), beginning at 8:30 and lasting for 15 minutes.

PDSA Worksheet

Comments:

At this stage, the second cycle of the PDSA is complete. The Clinic Team A has implemented a 15 minute huddle once a week.

Their next step is to increase the frequency of their huddles to extend the reach of their team-based improvement process.

46

In Conclusion

Behavioral Health Providers working in support of PCBH have a complex professional role that requires a broad set of competencies.

While core features of PCBH services are generally agreed-upon (e.g., brief sessions, time-limited episodes of care), there is wide variability in provider behavior and program implementation.

While evaluation and ongoing monitoring of fidelity can facilitate efforts to improve model adherence, purposeful action is required in order to effect lasting change.

47

In Conclusion (con’t)PDSA is an approach to QI that is action-oriented, data-based, and does not require specific expertise in QI methodology in order to develop, test, evaluate, and act on a specific plan to alter practice

We outlined two examples of how PDSA can be used to modify current practices with regard to: a) an individual provider’s use of MBC, and b) an interprofessional team’s use of morning huddles to improve communication and collaboration

The PPAQ Toolkit is a resource that can be used before engaging in PDSA to identify and prioritize areas of practice that may benefit from modification, and offer suggestions of how to go about making feasible and sustainable changes.

48

Acknowledgments

• VA Mental Health Quality Enhancement Research Initiative (QLP 55-020)

• VA Office of Academic Affiliations Advanced Fellowship Program in Mental Illness Research and Treatment

• VA Center for Integrated Healthcare at the VA Western New York Healthcare System

49

CIH Postdoctoral Fellowship2-year research and clinical training program in Primary Care-Mental Health Integration

• Research training is individualized; Fellows work with mentorship team on grants, manuscripts, and building a program of research

• At minimum, 25% protected time for clinical rotations and supervision

Positions available in Buffalo, NY (APA-accredited) and Syracuse, NY (non-accredited)

Flexible start dates (Jul 1-Oct 1) and competitive salary and benefits package

For more information, contact Paul.King2@va.gov, or visit our website at http://www.mirecc.va.gov/cih-visn2/fellowship2.asp

50

Contact Information

Gregory P. Beehler, Ph.D., M.A.

Associate Director of Research

VA Center for Integrated Healthcare

716-862-7934

gregory.beehler@va.gov

Paul R. King, Ph.D.

Clinical Research Psychologist

VA Center for Integrated Healthcare

716-862-6038

Paul.king2@va.gov

Katherine M. Dollar, Ph.D.

Associate Director – Implementation

VA Center for Integrated Healthcare

203-836-7052

katherine.dollar@va.gov

51

Session Evaluation

Use the CFHA mobile app to

complete the evaluation for this

session.

Thank you!

52

Please complete the session evaluation form!

Session #: H8

Primary Presenter: Beehler