Expanding Advocacy Capacity: Findings from the Evaluation of The California Endowment Clinic...
-
Upload
gabrielle-brunswick -
Category
Documents
-
view
219 -
download
2
Transcript of Expanding Advocacy Capacity: Findings from the Evaluation of The California Endowment Clinic...
Expanding Advocacy Capacity: Findings from the Evaluation of
The California Endowment Clinic Consortia Policy and Advocacy Program
Prepared by
Annette L. Gardner, PhD, MPH, Sara Geierstanger, MPH and
Claire Brindis, Dr PH
The Philip R. Lee Institute for Health Policy Studies
University of California, San Francisco
October 28, 2008
Workshop Objectives
• Describe approaches to assessing policy and advocacy activities funded under TCE Program
• Share findings for 4 of 6 Program outcomes
• Discuss theoretical and practical challenges to assessing policy and advocacy
Background:Clinic Consortia Policy and Advocacy Program
• Launched in 2001 and refunded two more times in 2004 and 2007 ($9M per Round)
• 14 local/regional clinic consortia and 4 statewide clinic associations representing 700+ primary care clinics funded by The California Endowment
• Goals:
– Increase services to underserved populations through consortia-initiated policy and advocacy activities
– Increase access through improved efficiencies within clinic organizations
Program Logic Model
Increased grantee capacity in policy advocacy
Increased policymaker awareness of safety net and clinic policy
issues
Increased policymaker support for clinic funding
Strengthened clinic operations
Increased services for the underserved and uninsured
Improved health outcomes for targeted communities and
populations
Increased grantee capacity in policy advocacy
Increased policymaker awareness of safety net and clinic policy
issues
Increased policymaker support for clinic funding
Strengthened clinic operations
Increased services for the underserved and uninsured
Improved health outcomes for targeted communities and
populations
Evaluation Data Collection Activities: 2002-2009
Round Areas of Investigation
Round 1 (2001-2003)
Funding secured (2001 - 2009)Policies targeted (2001 - 2009)Effectiveness of advocacy activities (2001 - 2006)Partnerships with clinics and consortia (2002 - 2003)Activities to strengthen clinic operations (2002, 2003)Media perceptions of media advocacy (2003)Clinic perceptions of consortia activities (2003)Grantee Implementation and Sustainability Survey (2003)Policymaker/Stakeholder awareness of consortia advocacy (2003, 2004)
Round 2 (2004 - 2006)
Newspaper coverage secured (2004 and 2005)Individual grantee indicator data (2004 and 2005)Partnerships with non-health partners (2004 - 2009)Perceptions of consortia policy advocacy by non-member clinics (2005)Media perceptions of media advocacy (2005)Clinic perceptions of grantee advocacy (2005)Three policy advocacy case studies (2006)Grantee Best Practices - achievement of Program outcomes (2006)
Round 3 (2007 - 2009)
Tracking policy achievements - comparison of 3 key policies (2007-2009)Case studies to assess longer-term Program outcomes (2009)
Evaluation Data Collection Activities by Outcome
Outcome 1: Increased grantee
advocacy capacity
Outcome 2: Increased
policymaker awareness
Outcome 3:
Increased policymaker
support for clinic funding
Outcome 4 Strengthened
clinic operations
Grantee Implementation and Sustainability Survey
2003 Policymaker and 2004 Stakeholder Surveys
Annual Funding Secured Worksheet
2004 and 2006 Clinic Focus Groups
Annual Advocacy Activities Worksheet
2003 and 2005 Interviews with Media and Media Consultants
Annual Policy Outcomes Worksheet
16 Grantee Best Practices
Partnerships Among Clinics, Among Consortia, and with Non-Health Allies
Content Analysis of Newspaper Coverage Secured
3 Policy Case Studies
Outcome 1: Expanded Grantee Advocacy Capacity
Data Collection Activity: Annual Advocacy Activities Worksheet
• Grantee self-report (n=15 grantees)• Previous year’s activities and their outcomes (n=13)• Source: Scholzman and Tierney (1986)
Activity Federal
(Y/N)
State (Y/N)
Local (Y/N)
Effectiveness (1 to 4, 4=
“very effective”
Outcome 1: Increased
clinic funding(Y/N)
Outcome 2: Achieve a
policy change(Y/N)
Outcome 3: Increased
policymaker awareness
(Y/N)
Media advocacy
Contact policymakers directly
Analysis:
1) Determine percent activities at the federal, state and local levels, and overall
2) Calculate effectiveness overall and by activity
3) Determine percent of activities that achieved each Outcome
4) Sort by grantee age (pre/post 1990 launch)
Informal contacts with elected officials
Hosting clinic tours for policymakers
Helping draft regulations, rules or guidelines
Results: Expanding and Innovating• Grantees engaged in more grant-funded activities over
time--increase from 73% to 86% of surveyed activities, between 2002 and 2006.
• Grantees consistently targeted most activities (80%) at the local and state levels and less (60%) at the federal level.
• Grantees perceived some activities to be more effective than others, e.g., helping draft rules, regulations and guidelines
• Newer (post-1990) grantees were equally effective in achieving advocacy goals by 2006.
Increase in Number of Grantee Advocacy Activities, by Level of Membership Focus, 2002-2006
0
20
40
60
80
100
120
140
160
Statewideconsortia (n=3)
Regional consortia(n= 4)
Local consortia(n= 8)
Statewide consortia(n=3)
18 19 27 24 23
Regional consortia (n= 4) 33 34 32 35 36
Local consortia (n= 8) 69 75 78 77 83
2002 2003 2004 2005 2006
Changes in Perceived Effectiveness of Key Activities Since 2002 (4=“very effective”)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Drafting rules,regulations
Engaging in informalcontacts withpolicymakers
Media advocacy
Drafting rules,regulations
3.14 3.60 3.60 3.70 3.50
Engaging ininformal contactswith policymakers
3.58 3.23 3.23 3.00 3.00
Media advocacy 3.31 3.25 2.57 2.57 2.75
2002 2003 2004 2005 2006
Percent of Activities That Resulted in a Policy Change, 2002-2006
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
2002 2003 2004 2005 2006
Pre-1990 Grantees(n=6)
Post-1990 Grantees(n=9)
Percent of Grantee Advocacy Activities That Achieved 3 Desired Outcomes, 2002-2006
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2002 2003 2004 2005 2006
Policymaker Awareness
Policy Change
Clinic Funding
Outcome 2:Increased policymaker awareness of
safety net and clinic policy issues
Data Collection Activities:2003 Policymaker and 2004 Stakeholder Surveys
• 2003 Policymaker Survey
– N = 86 decision makers familiar with consortia and their activities and policy issues
• 2004 Stakeholder Survey
– N = 43 stakeholders less familiar with consortia and their activities
Not at all familiar
A little familiar Somewhat familiar
Very familiar
How familiar are you with
The consortium?
Activity 1?
Yes No Not sure
Since 2001…
Has the consortium increased you awareness of…
Analysis:
1) Determine percents
Do you recall seeing….
Results:Policymaker/Stakeholder Awareness
• 2003 policymakers well-known to grantees considered policy events “very effective” (74%) compared to 2004 stakeholders less known to grantees (61%)
• Consortia media events rated “very effective” in increasing 2003 policymaker (60%) and 2004 stakeholder awareness (42%)
• Grantees reported that nearly all advocacy activities (95%) contributed to high policymaker awareness
Outcome 3:Increased Policymaker Support
for Clinic Funding
Data Collection Activity: Funding Secured Worksheet
• Grantee self-report (n=16-18 grantees)– Requires written instructions and follow-up phone calls
• Previous year’s funding secured attributable to TCE grant– Staff time (%) used to determine esimate– Use calendar year
Funding Source
$ Amount
How secured?
Write a grant?
New or Maintained funding?
Estimate the % due
to TCE grant
How much went to
consortium?
How much
went to clinics?
Federal
1. Clinic 330 grants
Analysis:
1. Calculate total funding, funding attributable to the grant, funding allocated to consortia and clinics
2. Code and county funding type - federal, state, local, private
3. Code and categorize data on how funding was secured and what funding was spent on by consortium and clinics
4. Determine whether new or maintained (carryover) funding
State
1. Tobacco Settlement $
Local
1. County contracts
Private
Results:Increased Clinic Financial Stability, 2001-2007
• Total funding secured was $2.9B, with $979M (35%) attributable to grant
• Funding due to TCE grant increased from $104M in 2001 to $226M in 2007
• Modest increase in “new” funding (29% in 2007), e.g., Information technology
• Public funding dominates:
– Federal: $344M
– County: $324M
– State: $263M
– Private: $45M
Total Funds Attributed to Grant-funded Advocacy Activities, 2001-2007
$0
$50,000,000
$100,000,000
$150,000,000
$200,000,000
$250,000,000
Total
Consortia
Total $103,993,752 $81,203,585 $84,988,466 $105,047,411 $195,028,986 $182,416,735 $226,671,718
Consortia $4,880,660 $4,721,917 $6,281,764 $9,838,452 $6,784,201 $9,219,742 $9,856,642
2001 2002 2003 2004 2005 2006 2007
Annual Funding Secured by Type, 2001-2007
$0
$10,000,000
$20,000,000
$30,000,000
$40,000,000
$50,000,000
$60,000,000
$70,000,000
$80,000,000
$90,000,000
$100,000,000
2001 2002 2003 2004 2005 2006 2007
Federal
State
Local
Private
Total Funding by Type, 2001-2007
$344,754,414
$324,438,677
$263,286,750
$45,204,640
$0
$50,000,000
$100,000,000
$150,000,000
$200,000,000
$250,000,000
$300,000,000
$350,000,000
$400,000,000
Total Funding
FederalLocalStatePrivate
Data Collection Activity: Policy Outcomes Worksheet
• Grantee self-report (n=18 grantees)• Previous year’s activities
Policy targeted
Bill Number and/or title?
Grant-funded policy advocacy activities undertaken in 2007. Choose from list: a) media advocacy; b) in-person meetings with decision makers; c) drafting of rules, regulations; d) materials development; e) grass roots activities; f) Other: list
Target populations to be affected by policy. Choose from list: a) clinics; b) uninsured; c) low-income populations; d). clinic consortia; e) other special populations; f) Other: list
Outcomes as of 12/07. Choose from list: a) Passed - signed in to law; b) Vetoed; c) Pending - still being considered by decision makers; d) funding secured; e) Other: list.
Federal
Analysis:
1. Code and categorize all policies
2. Count number of grantees that mentioned each policy
3. Count federal and state legislation
4. Code and categorize advocacy activities, target populations, and outcomes
State
Local
Results:Legislation Passed, 2001-2006
• Grantees achieved fewer wins but were involved with more legislation over time:
– 2001-2003: Involved in 58 pieces of legislation. Five (30%) federal policies passed and 26 (60%) state bills passed
– 2004-2006: Involved in 72 pieces of legislation. Six (18%) federal policies passed and 20 (55%) state bills passed.
Federal and State Legislation Targeted by Grantees: Round 1 (2001-03) vs. Round 2 (2004-06)
5 6
26
20
0
5
10
15
20
25
30
2001-03 (N=58) 2004-06 (N=72)
Fed - Passed
State- Passed
Major Policy Wins & Losses in 2006
• Federal:
– Wins: HR 5647--Health Center Appropriations
– Losses: HR 4241--Deficit Reduction Act
• State:
– Wins: SB 437--Children’s Health Coverage Expansion
– Wins: SB 2282--Safe Harbor
– Losses:SB 1427--Four Walls
– Losses:Prop 86--Tobacco Initiative
• Local:
– Wins: Mental Health Services Act (MHSA) funding, County Contracts
Methods:Developing Policy Case Studies
• Criteria for inclusion:– 1 state and 1 local policy
– Single grantee vs. multiple grantee involvement
• Policies studied:– Measure A (Alameda County)
• Local sales tax to support health care safety net
– Mental Health Services Act (MHSA)• Local funding secured for clinics in Alameda and San
Diego Counties
– State Prospective Payment System (PPS) legislation
• Move from cost-based payment to prospective payment of FQHC clinic services
Methods:Developing Policy Case Studies (cont.)
• Data collection, development and analysis:
– 30-60 minute phone interviews with:
• Grantee staff
• 1-2 decision makers per policy
• 1 clinic staff member per policy
– UCSF developed 5-7 page description of policy, advocacy activities undertaken and outcomes
– Grantees reviewed and commented on summary
– Comparison of 3 policies by advocacy activities
Analysis: Comparison of Policy Case Studies
• Variation among cases:
– Not all policies are “media-friendly”
– Challenges vary, e.g., measures requires 2/3 majority
• Common factors for success:
– Staff expertise (technical, advocacy, networking)
– Early and often involvement
– Mobilizing clinic staff and patients and building coalitions
– Leveraging partnerships with clinics
Outcome 4: Strengthened Clinic Operations
Methods:Developing Grantee Best Practices
• Criteria for inclusion:
– Exemplary activities funded under TCE Program (2004-2006)
• Types of practices:
– A particular bill or measure
– An advocacy activity or strategy
– Programs to expand access to health care
• Data collection, development and analysis:
– UCSF reviewed background materials and interviewed grantee staff
– UCSF wrote a 1-2 page description
– Grantees reviewed and commented on description
– UCSF mapped Best Practices on Program logic model
Community Clinic Association of L.A. County (CCALAC)Preserving Public-Private Partnership (PPP) Program Funding
• The Challenge:– Sustain funding for program that supports links between public and
private primary care clinics that serves 1.3M people
• The Solution:– Multi-faceted advocacy campaign to preserve and evaluate Program
• Success:– Preservation of funding - 3 year contract ($51M in 2005)
• Lessons Learned:– Work with local data
– Partnerships key
– Garner county support
• Looking Forward: – Focus on maintaining county support
Achievement of Program Outcomes:Round 2 Grantee Best Practices
Increased grantee capacity in policy and advocacy• SCCC: Creating a Voice for Change--Launching a Clinic Consortium•NSRHN: Use of Economic Data to Inform Local Decision Making
Increased grantee capacity in policy and advocacy• SCCC: Creating a Voice for Change--Launching a Clinic Consortium•NSRHN: Use of Economic Data to Inform Local Decision Making
Increased awareness of policymakers of safety net and clinic policy issues•CCC: “What’s Going Around?” on CBS Local Channel 8•COCCC: 6th Annual Bridges Conference
Increased awareness of policymakers of safety net and clinic policy issues•CCC: “What’s Going Around?” on CBS Local Channel 8•COCCC: 6th Annual Bridges Conference
Increased policymaker support for clinic funding•CFHC: Averting Cuts to Title X Funding•CPPEF: Training and Coordinating Affiliates in Grassroots Policy Advocacy•CRIHB: Develop Research Infrastructure to Inform Policymaking
Increased policymaker support for clinic funding•CFHC: Averting Cuts to Title X Funding•CPPEF: Training and Coordinating Affiliates in Grassroots Policy Advocacy•CRIHB: Develop Research Infrastructure to Inform Policymaking
Strengthened clinic operations•CPCA: Clinic Emergency Preparedness Project•CHPSCC: Patient Advocacy Training Program
Strengthened clinic operations•CPCA: Clinic Emergency Preparedness Project•CHPSCC: Patient Advocacy Training Program
Increased services for the underserved/uninsured•NCCN: Securing Health Professional Shortage Area Designations•RCHC: Expanding Enrollment in Health Care Insurance Programs
Increased services for the underserved/uninsured•NCCN: Securing Health Professional Shortage Area Designations•RCHC: Expanding Enrollment in Health Care Insurance Programs
Improved health outcomes for targeted communities and populations - TBD
Improved health outcomes for targeted communities and populations - TBD
Summary of Findings
• Achievement of first 4 Program Outcomes:
– Sustained advocacy capacity
– Increased policymaker/stakeholder awareness
– Increased funding secured
– Important policy “wins” benefiting clinics
• Early evidence of longer-term outcomes:
– Clinic services expansions
– Increased utilization by uninsured
– Consortia as agents of systems change in their communities
Challenges and Potential Solutions
• Aggregating diverse grantee achievements– Use of longitudinal tools and individual case studies
• Issue of lobbying– Identify what is funded under the grant
– Analyze legislation as measure of capacity
– Acknowledge role of lobbying
• Relying on grantee self-report– Interview member/non-member clinics
– Interview policymakers/stakeholders
– Interview partner organizations
• Attribution– Focus on what can be identified as attributable
For More Information:
Annette Gardner, PhD, MPH
Evaluation Director
Philip R. Lee Institute for Health Studies
University of California, San Francisco
3333 California Street, Suite 265
San Francisco, CA 94121
Phone: (415) 514-1543
Email: [email protected]