Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here?

Maria Fernandez, PhDon behalf of the CPCRN FQHC WG

Investigators

CPCRN Steering Committee - Virtual Spring WebinarApril 16, 2014

CPCRN CHC Survey

FQHC Workgroup

Partnership Committee

Qualitative Inquiry

Subgroup

Data Subgroup

CHC Survey

Subgroup

Align with CHCs’ missions

Guided by real world

health policy

&health

care delivery

landscapes Health Care Reform

Meaningful Use of EHRPatient-Centered Medical Home

National Association of Community Health Centers (NACHC)

Primary Care Associations

Community Health Centers (CHCs)

Developing Measures of Constructs Associated with EBI Implementation

Purposes

• Describe the survey development process and rationale

• Present initial psychometric validation of CFIR Inner Setting measures

• Discuss findings and what is next in our work assessing CFIR constructs (and PAR) related to EBI implementation

Value

• One of the first studies to measure several constructs from CFIR

• Developed and tested measures that describe factors influencing implementation of EBAs in FQHCs

• Sets the foundation for establishing causal pathways and developing interventions that target these factors

Consolidated Framework for Implementation Research (CFIR)

Source: Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 2009; 4:50.

Note: Authors from the VA and University of Michigan, SPH, Department of Health Management and Policy

CFIR Constructs

Patient Needs & Resources,

External Policies & Incentives

Available Resources, Leadership

Engagement, Culture,

Implementation Climate,

Learning Climate

Knowledge & Beliefs

about EBAs

Executing,Reflecting & Evaluating

Relative Advantage, Complexity

Outer Setting

ProcessCharacteristics of Individuals

Inner Setting

Characteristics of Intervention

Multiple Recruitment Strategies

• Sites partnered with their state’s PCAs; PCAs emailed their CHCs

• Sites recruited CHCs via email, telephone calls, or in-person meetings

• Introductory email with online survey link; 4 reminder emails; in-person meeting (one site)

• January - May 2013• IRB approval at each site and coordinating center

Inner Setting Constructs

• Available Resources• Culture Overall• Culture Effort• Culture Stress• Implementation Climate• Learning Climate• Readiness for Change

Analysis plan

• We conducted separate CFAs for each of the seven inner setting constructs.

• Multiple fit indices were used to evaluate CFA model fit (CFI >.90, TLI >.90, SRMR <.08, and RMSEA <.08).

• Cronbach’s alpha to evaluate internal consistency reliability.

• Correlation coefficients of each pair of scales to examine discriminant validity.

• Inter-rater reliability (ICC(1) & ICC(2)) and inter-rater agreement (rWG(J)) statistics were computed to assess the

reliability and validity of computing clinic- and systems-level means from the individual-level data.

Available Resources

A35A

A35B

A35C

C20A

C20B

C20C

C20D

.419

.546

.580

.708

.766

.626

.741

A11

A12

A13

A14

Readiness for change

.824

.896

.847

.872

C11

C12

C13

C05

Implementation climate

.546

.830

.759.462

A01

A06

A02

A15

A19

Learning climate

.694

.646

.768.727.735

A03

A05

A16Rev

A07

A22

A08

A09

A10

A21

A36

A37

A38

A39

A40

A21

A42Rev

A43

A44Rev

Culture

.663

.723.556.642.742.674.719.731.775

.332.438

.387.594

.480

.612.440

.581.387

Inner Setting

0.627

0.945

0.980

0.514

0.968

Combined CFA results and fit statistics

CFA Standardized Factor Loadingsa

1. Available Resources 0.627 (0.068)2. Culture 0.945 (0.022)3. Implementation Climate 0.514 (0.068)

4. Learning Climate 0.980 (0.024)

5. Readiness for Change 0.968 (0.017)

Robust Std. Errors in parenthesis CFI = 0.848 TLI = 0.835; SRMR = 0.079; RMSEA = 0.065

Discriminant validity of the five dimensions of inner setting

Scale Culture Available resources

Learning climate

Implementation climate

Readiness for change

Culture 1.00

Available resources

0.3641 1.00

Learning climate 0.6757 0.4555 1.00

Implementation climate

0.2464 0.6254 0.3201 1.00

Readiness for change

0.6978 0.5203 0.8114 0.3758 1.00

Correlations using average score for each scale*p < .05

Clinic Level Inter-rater reliability and inter-rater agreement statistics

Scale ICC(1) ICC(2) rWG(J)

Culture .182* .479 .929

Available resources .116* .321 .821

Learning climate .178* .476 .785

Implementation climate

.219* .504 .828

Readiness for change .169* .462 .634

Using average score for each scale*p < .05

System level Inter-rater reliability and inter-rater agreement statistics

Scale ICC(1) ICC(2) rWG(J)

Culture .181* .725 .925

Available resources .114* .542 .867

Learning climate .150* .679 .774

Implementation climate

.292* .792 .845

Readiness for change .106* .558 .592

Using average score for each scale*p < .05

Conclusions

• Our measures have structural validity• Our measures have reliability• Our measures have discriminant validity• Our measures can be aggregated to the clinic

level• Clinics within systems have more similar values

for inner-setting domain constructs than do clinics in different systems

• We fit a second order “inner setting” factor

PRACTICE ADAPTIVE RESERVE AND PATIENT CENTERED MEDICAL HOME BEST PRACTICES AT COMMUNITY HEALTH CENTER CLINICS IN SEVEN STATES

Adjusted Logistic Regression Frequency of PCMH Best Practices and PAR

Scores

PAR Frequency of PCMH Best Practices (6-8 vs. 0-5)

OR 95% CI

0.08 – 1.00 5.49 2.31,13.06

0.60 - <0.80 2.23 1.11,4.47

0.00 - <0.60 Referent

PCMH CRC Screening Best Practices Dichotomized Score (6-8 vs. 0-5)Respondent reported performing PCMH best practices “usually” or “always”

Adjusted for state, age, job type, years worked at the clinic, hours worked each week

National Demonstration Project - Highly-motivated practices w/ significant capability for change • Mean baseline PAR score 0.69 (s.d. 0.35)• Post intervention PAR score increased to 0.74

Discussion

• What have we learned about some of the measurement challenges in FQHCs?

• Where do we go from here in terms of measurement?

• Which constructs need more conceptual and measurement work?

• What additional work is needed in terms of reliability and validity assessment?

Discussion Continued

• Where do we go from here in EBI implementation in FHQCs?

• What opportunities exist?

• How can the measurement work be used in EBI implementation in FQHCs?

How FQHC WG can move forward?

• How could this group move forward on what’s next?

• What resources, partners, and preliminary work is needed to pursue what’s next?