SPF PLANNINGSPF PLANNING

1

2

SAMHSA’s SAMHSA’s Strategic Prevention FrameworkStrategic Prevention FrameworkSupports Accountability, Capacity, and EffectivenessSupports Accountability, Capacity, and Effectiveness

AssessmentProfile population needs, resources, and readiness to address needs and gaps

EvaluationMonitor, evaluate, sustain, and improve or replace those that fail

ImplementationImplement evidence-based preventionprograms and activities

PlanningDevelop a Comprehensive Strategic Plan

CapacityMobilize and/or build capacity to address needs

3

SAMHSA’s SAMHSA’s Strategic Prevention FrameworkStrategic Prevention FrameworkSupports Accountability, Capacity, and EffectivenessSupports Accountability, Capacity, and Effectiveness

AssessmentProfile population needs, resources, and readiness to address needs and gaps

EvaluationMonitor, evaluate, sustain, and improve or replace those that fail

ImplementationImplement evidence-based preventionprograms and activities

PlanningDevelop a Comprehensive Strategic Plan

CapacityMobilize and/or build capacity to address needs

Training ObjectivesTraining Objectives

Explain Pre-Requisites to Successful PlanningDefine Purpose of PlanningProvide Guidance for Writing a Strategic PlanDistinguish Logic Models & Action PlanningOverview Environmental StrategiesLink Steps to Cultural Competency & SustainabilityExplain Evidence-Based Selection/ImplementationProvide Guidance for Monitoring

44

Key Pre-Requisites to Key Pre-Requisites to PlanningPlanning

Public Health Approach

Population Based Change

Data Informs Decisions

Outcome-BasedAssessment of Problems, Resources, Gaps, Readiness

Stakeholders Engagement

5

Pre-Requisite Pre-Requisite QuestionsQuestionsWhat is going on in my community?

How big & what kinds of problems?

Is there consequence data?How useful is data?What resources exist?How ready is community?

6

7

But Why? ProblemBut WhyHere?

7

Why Strategic Why Strategic Planning?Planning?

Communities are expected to develop and implement strategies that have a good chance of reducing county-level indicators of ATOD and attendant problems.

8

What is a Strategic What is a Strategic Plan?Plan?

Roadmap that Guides:

Data collection/sharingCapacity ExpansionLogical identification of

goals/objectivesSelection of evidence-based

programs, policiesEvaluation plan

99

Failure to PlanFailure to PlanLack of understanding of problemLimited understanding of solutions

Overlooking important detailsFailure to create shared responsibility

Poor evaluationNon Sustained procedures/programs/policies

10

The Planning TeamThe Planning TeamStart with Team Comprised from 12 Sectors

Team Collaborates Around Logic Model

Group Determines Need for Other Partners

Note: Convene team before you start writing plan

1111

Logic Model: Logic Model: Outcomes-Based PreventionOutcomes-Based Prevention

12

Substance-related

problems

Intervening Variables

Strategies/Programs

12

The Logic ModelThe Logic Model

Clearly States the Problem

Surfaces the Root Causes

Identifies Why These Root Causes Exist in Your Community

1313

Why Use a Logic Why Use a Logic Model?Model?

Insures clarity on precisely how the group intends to make a difference

Establishes a common language for how things will improve

Keeps the focus on outcomes

Integrates planning, implementation and evaluation

1414

15

High rate of alcohol-

related crash mortality

Among 15 to 24 year olds

Low or discount PRICING of alcohol

Easy RETAIL ACCESS to Alcohol for youth

Easy SOCIAL ACCESS to Alcohol

Media Advocacy to Increase Community

Concern about Underage Drinking

Restrictions on alcohol advertising in

youth markets

SOCIAL NORMS accepting and/or encouraging

youth drinking

PROMOTION of alcohol use (advertising, movies,

music, etc)

Low ENFORCEMENT of alcohol laws

Young Adult DRINKING AND

DRIVING

Social Event Monitoring and

Enforcement

Bans on alcohol price promotions and

happy hours

Enforce underage retail sales laws

Causal Factors

(Intervening Variables)

Strategies(Examples)

Substance-Related

Consequences

SubstanceUse

(consumption)

Logic Model Components:Reducing Alcohol-Related Youth Traffic Fatalities

Low PERCEIVED RISK of alcohol use

Underage BINGE

DRINKING

Young Adult

BINGE DRINKING

Underage

DRINKING AND DRIVING

16

The Care and The Care and Feeding of a Feeding of a Winning PlanWinning Plan

16

Components of Strategic Components of Strategic PlanPlan

Vision

Mission

Objectives

Strategies

Action Plan

17

Planning – Key QuestionsPlanning – Key Questions

What do we want to do about the identified problem?

What programs and practices have other communities used to solve this problem?

What will work in our community?

1818

Implementation – Key Implementation – Key QuestionsQuestions

1. How will we know if we are successful in solving the problem (what measures)?

2. What are some of the potential road blocks to successful implementation?

3. How can we improve existing strategies and/or processes to solve this problem?

1919

Substance-related Substance-related Consumption PatternsConsumption Patterns

Overall consumptionAcute heavy consumptionConsumption in risky situations

◦Drinking and driving ◦Smoking around young children

Consumption by populations/groups◦Youth, college students, older adults

◦Pregnant women◦Other?

2020

Vision StatementVision Statement

A vision statement is a statement of what a community is trying to become; the desired end state or the ultimate goal.

2121

It answers the question “why do it.”It should read in the present tense.If you were to accomplish what you want, what would it look like.

Sample Vision Sample Vision StatementsStatements“Alcohol and drug free community”

“Healthier, safer community”

Other

http://www.cadca.org/files/PlanningPrimer-06-2009.pdf

22

Mission StatementMission Statement

A mission statement expresses

HOW the coalition will work to

achieve this shared vision

23

Sample Mission Sample Mission StatementStatement

“To develop an alcohol and drug-free community through consensus planning, community action and policy advocacy.”

Other?

24

Not So Fast . . . Not So Fast . . .

Though succinct, a vision for prevention is based on:◦ Documented needs◦ Identified resources and strengths◦ Measurable objectives and

performance measures◦ Baseline data

Include a long-term strategy to sustain policies, programs, and practices

Adjust plans as the result of ongoing needs assessment and monitoring

2525

Defining and Writing Defining and Writing GoalsGoals Think of goals as targets that are to

be reached or “hit”.

Goals are broad, general statements

describing what the project or group

wants to accomplish and/or achieve

They are not specific activities or action steps!

 

2626

GoalsGoals Should . . .Should . . .

Specify end result or desired ultimate accomplishment Reflect perceived present and future needs

Be capable of being effectively pursued

Source: http://www.for.gov.bc.ca/hfd/library/documents/glossary/G.htm

2727

Examples of UAD Goals Examples of UAD Goals • Reduce underage drinking by enforcing underage drinking laws and regulations

• Educate youth and adults on the serious consequences of underage drinking

• Improve communication and collaboration among organizations involved in UAD prevention

2828

Guiding Points About Guiding Points About ObjectivesObjectivesFoundation for program

development and evaluation

Must be clear and “actionable”

Objectives are not action steps (Who/What not How)

They should be realistic . . .Unrealistic expectations may lead to failure

In most cases, it is impossible to achieve a goal of 100%.

2929

Measurable ObjectivesMeasurable ObjectivesArticulate what program is intended to

do

Should be measurable to assess progress toward the goal

Specific, attainable, timely, and lead to observable behaviors

Tools you use to make sure you reach your goals

Should not restrict or constrain vision; but ensure that actions are clearly focused and communicated so that all parties know what is going on

You can shoot your arrows many ways, but they reach and score the bulls eye!!

3030

Tips for Writing Tips for Writing ObjectivesObjectives

1. Audience – Who is this being aimed at?

2. Behavior – What do you expect them to be able to do?

3. Condition – How? Under what circumstances will the impact or learning occur?

4. Degree – How much? Must a specific set of criteria be met? Do you want total mastery (100%) of the task(s), 80%, 50%, etc.?

3131

Examples of ObjectivesExamples of ObjectivesIf the goal is to reduce underage drinking by enforcing

underage drinking laws and regulations, objectives might include:

By October 2014 increase by 10% the number of citations given to youth that violate the state’s liquor laws.

By October 2014 , decrease by 10% the number of retailers that sell alcohol to minors (as determined by compliance checks. See Community How To Guide on Underage Drinking Enforcement).

Hint: When assigning percentages to an objective, go back to the

needs assessment and determine how much improvement can be

realistically achieved over a given period of time.

3232

Prioritizing Intervening Prioritizing Intervening VariablesVariablesBefore the strategic plan can be

developed, Intervening Variables must be prioritized.

Prioritization will be based on:◦ Severity◦ Capacity◦ Ability to Implement Strategies

3333

34

Promotion Social Norms Social Access

Easy Retail Access

Low Perceived Risk

Low Enforcement

Low Prices

Substance Use

15-24 year olds involved in alcohol-related vehicle crashes and crash fatalities 34

35

Low Enforcement

Lack of Sobriety Check Points

Sales to Intoxicated People

DWI convictions result inMinimum consequence

Youth are able To Purchase Alcohol

At various stores

No Law EnforcementCurrently Involved

No Judicial officialsExperts involved

Low Priority for smallPolice Force

Low Priority for smallPolice Force

35

36

Strategizing how Strategizing how things will be things will be accomplishedaccomplished

Strategic Planning Strategic Planning ObjectivesObjectives

Prioritize Intervening Variables according to severity and existing capacity.

Define priorities and objectives and organize those objectives into a strategy

Address the needs that have been documented

3737

Merchant Education

Low Enforcement

Retailers

ID compliance

checks

Sobriety Check Points

Rewards for not Selling to intoxicated

patrons

Media Campaign

Behavior Changes

Behavior Changes

Decrease15-24 year oldsAlcohol related

classes

Community Action group

Sheriff's office

Inputs Outputs Short TermOutcomes

Long TermOutcomes

Community Change

Strategic Planning Strategic Planning ObjectivesObjectivesBuild on identified resources,

strengths and capacity

Address multiple causes of identified problems in multiple community sectors

Identify benchmarks, recruitment strategies, and action steps for capacity in each Intervening Variable

39

40

Strategic Planning Strategic Planning ObjectivesObjectives

Include measurable objectives, the performance measures and baseline data against which progress will be monitored.

Identify goal(s) for Intervening Variable(s) and determine objectives that address contributing factors

Identify strategies that are population appropriate 40

Strategic Planning Strategic Planning ObjectivesObjectives

• Consider community readiness

• Identify evidence-based programs, practices, and policies (strategies) for each contributing factor

• Identify action steps for continual assessment of each Intervening Variable

4141

Reducing Risks & Reducing Risks & Increasing Protective Increasing Protective FactorsFactors

42

Identify evidence-based programs, practices, and policies (strategies).

Strategies should meet one of the following:1. Evidence-based2. Limited evidence3. Logic Driven

42

43

What We Know So Far (Planning)What We Know So Far (Planning)

Goal: Create an environment where alcohol is not illegally or inappropriately available to adolescents and young adults in social settings.

Objective: Reduce the percentage of parents reported to provide alcohol to their minor children by 20% by May 30, 2012.

Strategy: Increase parental knowledge of underage drinking laws by disseminating “Parents Who Host Lose the Most” information cards to 4,000 APS High School Parents. 43

The Devil is in DetailsThe Devil is in Details• Identify action steps for

continual assessment

• Develop action and capacity plan

• Ensure Cultural Competence and Sustainability

44

Strategic Plan DetailsStrategic Plan Details

Plans will continuously need to be monitored and reviewed.

Why?◦ Evaluate progress◦ Accountability◦ Make necessary changes◦ Account for new data◦ Management

4545

Strategic Plan DetailsStrategic Plan Details

Ensure cultural competence and sustainability throughout the strategic planning process.

Identify a process to review progress towards benchmarks and action steps included in the strategic plan.

4646

Other 5-Step Other 5-Step ConsiderationsConsiderationsHow will Data Driven Decision-

making continue?

Mutually Reinforcing Strategies: When and how should strategies work together to impact consequence/consumption identified (ecological model)

47

To ConsiderTo ConsiderTarget multiple domains or contexts

across the lifespan

Consider questions of reach when selecting strategies:

• How many people will your interventions impact?

• Which sectors of the community will be impacted by your efforts?

• What dosage of the interventions will target audience experience?

48

Ecological ModelEcological Model

49

50

Action PlanningAction Planning

Why create a plan for action?

An action plan allows us to create an objective profile of community, identify how to focus resources and efforts, and to implement more effective strategies.

Moves us from strategizing to Implementation

51

Action PlanAction PlanBefore and After QuestionsBefore and After Questions

How will we know if we are successful in solving the problem (what measures)?

What are some of the potential road blocks to successful implementation?

How can we improve existing strategies and/or processes to solve this problem?

The Action Plan Describes: The Action Plan Describes: What specific community/system

changes to be sought in accomplishment of goals

Which objectives will be employed (EBPs/Activities)

Who is going to do what

When measurable results will occur (in conjunction with evaluation)

52

Evaluate Appropriateness Evaluate Appropriateness of Action Planof Action Plan

Completeness

Clarity

Sufficiency (can it realistically effect change?)

Resources

Currency

Flexibility

53

Double Check Your ProcessDouble Check Your Process

Is plan truly comprehensive?

Have you looked at all appropriate

data?

Were all appropriate people involved?

Built upon theory of change?

Have you considered all the angles?

Does plan, as written, make sense tosomeone who had no input?

54

Monitor the Strategic Monitor the Strategic Plan Plan

Develop a protocol – What do you want to know? Who should be interviewed?

What does action plan say you will do?

Develop a schedule – How often should you review?

Adjust plans as the result of ongoing needs assessment and monitoring

55

Phases of Phases of ImplementationImplementation

Pre-Conditions – Need, target decisions, possible interventions and trouble-shooting

Pre-Implementation - Community input, internal brainstorming, pilot test, prep for implementation

Implementation – Deliver best fit EBP, staff and key leader training, fidelity, evaluation

Maintenance - Organizational or financial

changes to sustain, customizing, TA

56

Cultural Competency Cultural Competency ConsiderationsConsiderations

Is the strategy culturally appropriate and responsive?

Will the strategy be delivered in a culturally appropriate manner?

What other thoughts might be added to this conversation?

57

Next StepNext StepAfter careful planning, it is time

to consider what evidence –based prevention will be used.

58

THANK YOU FOR THANK YOU FOR PARTICIPATING IN PARTICIPATING IN THIS TRAINING THIS TRAINING MODULE.MODULE.

59

60

PART 2PART 2

Evidence-Based Prevention

61

State LevelImplementation

Assessment MobilizingCapacity

State and Community Implementation

Planning Implement Evaluate

Implement infrastructure development activitiesstate and local level

Community Level

Implement evidence-based prevention strategies and infrastructure development activities

Identifying and Selecting Identifying and Selecting Evidence-Based Evidence-Based Interventions (EBIs)Interventions (EBIs)

62

6363

Effective intervention planning should address both risk and protective factors.

Evidence Based Evidence Based PreventionPreventionDefinition: A prevention service (program,

policy or practice) that has been proven to positively change the problem you are trying to impact.

Examples:◦Program: Strengthening Families

Program◦Policy: Increased tax on cigarettes◦Practice: Consistent enforcement of YTA

Using Research Using Research to Make Decisionsto Make DecisionsPutting the scientific method to work for

prevention: ◦ Start with an observation

◦ Make a hypothesis (a guess),

◦ Conduct trials and tests to test the hypothesis

◦ Examine the data, develop newer ideas which leads to more tests and refinement of hypotheses.

Research is used to test out theories

Often the theory being examined is whether an intervention is effective

66

Why be concerned about Why be concerned about Evidenced Based Evidenced Based Prevention?Prevention?

Federal and state agencies, as well as private funders are asking for it

Public accountabilityThe desire to improve programsWe want to be effectiveLimited resources used wisely

History of TermsHistory of TermsThen:

◦ Best Practice Principles - When certain principles are incorporated into prevention programming the effort is more likely to be effective.

◦ Research/Science Based - A pre-existing validated model program (eg. Botvin’s life skills) OR

based on a validated theory relevant to substance abuse (eg. risk/protective factor, Bandura’s social learning theory)

Now:◦ Evidence-Based Prevention - There is evidence

that the activity has shown to achieve the desired outcomes.

68

Paradigm ShiftParadigm ShiftAnecdotal bean counting trend

profilesFrom picking off lists to thinking

critically about needsFrom categorical labels to ratings along

a continuumFrom relying on strength of evidence

alone to assessing the relative importance of strength of evidence in a broader context

From stand-alone intervention selections to comprehensive community plans

69

Comprehensive Approach:Comprehensive Approach:Any priority problem will have multiple

contributing conditions

Each contributing condition will require multiple interventions

An effective community plan requires a comprehensive approach ◦ made up of multiple strategies/interventions,

and◦ each strategy/intervention requires evidence

Comprehensive PlansComprehensive Plans

Prevention interventions require‘evidence’ throughout the SPF steps:

◦ Evidence that your addressed priority is a need

◦ Evidence that you’re addressing the stuff that makes the bad thing happen (intervening variables)

◦ Evidence that your selected strategies work to fix the stuff that makes the bad thing happen

◦ Evidence that the intervention is working (evaluate)

70

Consequence and

consumption:

High-risk drinking

18–25 year olds

Low-price alcohol

specials in bars

Servers not checking IDs

Policy to limit drink specials

Responsible server

education

Evidencerequired that this is really a problem for

your community

Evidence required that these are contributing

conditions in your community

Evidence required that these strategies reduce low-price

alcohol specials and increase servers

checking ID’s

Evidence-Based InterventionsEvidence-Based Interventions

Strength of Evidence ≠ strength of effect

Strength of evidence is determined by the answers to the following: ◦ How thorough and rigorous was the research/evaluation?◦ How well were the evaluation results documented, disseminated? ◦ Is there consensus among the experts?

Evidence-Based InterventionsEvidence-Based InterventionsStrength of EvidenceStrength of Evidence

The nature of evidence is continuousWill fall along a continuum from

weak to strong

The strength of evidence is assessed using scientific standards and criteria for applying these standards

““Three Bears” EB RigorThree Bears” EB Rigor1. Logic driven - have not been widely

researched, but derived from logic or theory

2. Limited evidence - supported by evidence or individual studies, but not as rigorous as meta-analysis studies

3. Evidence-based - sufficient research and evidence to demonstrate effectiveness as identified by a meta-analysis or expert peer panel

74

Current EBP Categories Current EBP Categories Identified by CSAP Identified by CSAP

No evidence

Strongevidence

Innovative program

unlike any other effort previously

evaluated for effectiveness

Documented effectiveness supported by other sources

of info and expert

consensus

Reported with positive

results in peer

reviewed journal

Listed on a Federal registry of effective programs

Continuum

Evidence-Based interventionNot Evidence-based

Evaluated but was found to

be ineffective

76

CSAP Categories of CSAP Categories of Evidence-BasedEvidence-Based

SPF Evidence-Based CategoriesSPF Evidence-Based Categories

According to latest CSAP guidance published inthe document, “Identifying and SelectingEvidence-Based interventions” there are threecategories of EBP: 1. Included on federal registries of evidence-based

interventions

2. Reported (with positive effects on the primary targeted outcome) in peer-reviewed journals

3. Documented effectiveness supported by other sources of information, the consensus judgment of informed experts, and meeting 4 specific guidelines

Federal RegistriesFederal Registries Many federal agencies have developed a review

process to look at the evaluation of specific programs

Typically review programs that: ◦ Are discrete in scope◦ Use a curricula or manual◦ Defined and tangible settings ◦ Focus on primarily individuals or families

Level of evidence required varies considerably Still have to think critically and make reasoned

judgments

Federal RegistriesFederal RegistriesAdvantages: Easy to find Easy to replicate Federal agency reviews the research One-stop shopping Funders Challenges: Limited number of interventions Very few population-based interventions Misleading ‘global effectiveness labels’ Often over generalize outcomes

Federal RegistriesFederal Registries

OJJDP Model Programs Guide http://www.dsgonline.com/mpg2.5/mpg_index.htmExemplary and Promising Safe, Disciplined and Drug-Free Schools Programs Sponsored by the U.S. Department of Education http://www.ed.gov/admins/lead/safety/exemplary01/exemplary01.pdf

Guide to Clinical Preventive Services Sponsored by the Agency for Healthcare Research and Quality [AHRQ] http://www.ahrq.gov/clinic/cps3dix.htm

Guide to Community Preventive Services Sponsored by the Centers for Disease Control and Prevention [CDC] http://www.thecommunityguide.org

A list of other registries may be found at SAMHSA’S website: http://www.samhsa.gov/ebpWebguide/appendixB.asp.

SAMHSA National Registry of Evidence-Based Programs and Practices (NREPP) http://www.nrepp.samhsa.gov

Federal RegistriesFederal Registries

National Registry of Evidence-Based Programsand Practices (NREPP) History: ‘Model’ program endorsement Current:

◦ Inclusion does not constitute endorsement of an intervention by SAMHSA.

◦ NREPP inclusion does indicate effectiveness ◦ Provides information to support a critical review of research within the

broader context of your community. quality of the research supporting outcomes Strength of the outcomes quality and availability of training and implementation materials

Peer-Reviewed Journals Peer-Reviewed Journals (with positive effects on the primary targeted (with positive effects on the primary targeted outcome) outcome)

Peer review is the process of subjecting research to the scrutiny of other experts in the same field.

The purpose is very different than of a federal registry.

For scholarly purposes and to further a field of research.

To be published an article must:◦ meet the accepted standards for research

methodology (rigors)AND

◦ Ensure appropriate interpretations of the research conducted.

83

Glossy, often color pictures

Target = general public

General interest or current events

Many ads, often in color

Writers typically journalists or reporters

Author sometimes “unsigned”

Chosen by editor or edit board

No formal citations

Newsstand, subscription

Corporation/popular press

Popular

84

Often bland, with few colors

Targets field professionals

Research reports or professional concerns

Usually few or no ads

Writers = Practitioners/Scholars/ researchers

Author "signed articles"

Selected by field experts “Peer-Reviewed"

Article bibliographies

Subscription

Professional association or scholarly publisher

Scholarly

85

Sample Journal Titles Sample Journal Titles Available from LibrariesAvailable from Libraries

Addictive Behaviors

Alcohol Health and Research World

Contemporary Drug Problems

Journal of Ethnicity in Substance Abuse

Journal of Child & Adolescent Substance Abuse

Journal of Drug Education

Natl. Institute on Alcohol Abuse and Alcoholismwww.niaaa.nih.gov

Western North Carolina Library Network (WNCLN)

Using Peer-Reviewed Journal Using Peer-Reviewed Journal ResearchResearch

Advantages: Detailed findings and analysis that

document whether the intervention works. Contact info Meta-analysis articles

Challenges: Often don’t give enough detail to replicate Onus of determination of SOE on the reader Access to journals is limited

Where to find: University libraries Michigan E-journals

Peer Reviewed JournalsPeer Reviewed JournalsStructure of a PRJ: Abstract: a summary of the key points in the article

and the hypothesis being tested

Introduction: context for the study. What prompted the research, what research does study build on

Methods: Explains how the researchers set about testing their hypothesis

Results: detailed findings of the research conducted

Discussion: a summary of the results and what they mean

Bibliography: a listing of all the sources cited in the article, as well as relevant articles or books that were not cited 87

Peer-Reviewed Journal: Critical Peer-Reviewed Journal: Critical ReviewReview

Quality of PRJ information: How rigorous was the evaluation? Are the findings and outcomes clearly

described? Is there enough info about the intervention to

replicate it? Are there multiple articles with consistently

positive results for the intervention?

Local Fit: Does the intervention address your identified

problem and relating contributing conditions? Has the intervention been effective in a

community like yours? Has the intervention been effective with a

population similar to yours?

Documented Effectiveness…Documented Effectiveness…

Documented effectiveness supported by other sources of information and the consensus judgment of informed experts.

Guidelines: (Must meet all 4 guidelines to qualify)

1. Based on solid theory of change that is documented in a clear logic or conceptual model;

2. Similar in content and structure to interventions that appear in registries and/or peer-reviewed literature;

3. Supported by documentation that it has been effectively implemented in the past, multiple times, in a manner attentive to scientific standards of evidence and with results that show a consistent pattern of credible and positive results.

4. Reviewed and deemed appropriate by a panel of informed prevention experts

Documented Effectiveness…Documented Effectiveness…

It may be necessary to rely on this weaker strength of

evidence when no appropriate interventions areavailable in categories with stronger evidence.

Keep in mind that your intervention must beappropriate for the:

◦ Identified problem and contributing condition,

◦ population to be served/reached, and ◦ cultural and community context in which

the intervention will be implemented

Documented Effectiveness…Documented Effectiveness…

Advantages

Increases the possible strategies to use as a part of a comprehensive plan;

Flexibility for those making programming decisions;

Empowers planners to select or develop innovative, complex interventions to meet the needs of individual communities;

Creates the ability to include culturally based evidence as well as traditional evidence to support local decisions; and

Authorizes planners to exercise professional judgment in deciding the potential contribution of unique intervention components in the comprehensive plan.

Documented Documented Effectiveness…Effectiveness…

Challenges: Places substantial responsibility on prevention

planners for intervention selection decisions

The burden of proof for documented effectiveness lies with the program planners and practitioners making the selection decisions

Require prevention planners to think critically about the evidence to support the inclusion of a particular intervention in the community’s comprehensive plan.

Must prove all four guidelinesMust prove all four guidelines1. Based on solid theory of change that is

documented in a clear logic or conceptual model

2. Similar in content and structure to interventions that appear in registries and/or peer-reviewed literature

3. Supported by documentation that it has been effectively implemented in the past, multiple times, in a manner attentive to scientific standards of evidence and with results that show a consistent pattern of credible and positive results.

4. Reviewed and deemed appropriate by a panel of informed prevention experts

Selecting Best-Fit Selecting Best-Fit InterventionsInterventions

95

Selecting Best-Fit Selecting Best-Fit Prevention InterventionsPrevention Interventions

Select Specific Programs,Practices, & Policies

Ensure Effectiveness

IdentifyTypes of Strategies

Best-FitPrevention

Interventions

Demonstrate Conceptual Fit

Demonstrate Evidence of Effectiveness

Demonstrate Practical Fit

Relevant? Practical? Effective?

Questions to ask yourself?Does it fit into community logic model? Does it address the identified

contributing conditions? Has it been tested for the target

population? Does it respond to the specific needs of

your targeted population?Does the intervention contribute to a

comprehensive community plan?

Conceptual FitConceptual FitDoes it do what you need to accomplish?Does it do what you need to accomplish?

Practical Fit for CommunityPractical Fit for Community Can you actually do it?Can you actually do it?

Questions to ask yourself and your coalition:

Do you have the manpower and funding needed?Do you have the necessary community contacts

needed (police, leaders, etc.)?Will the community support this strategy? Does this strategy reflect your community’s culture?Could this strategy be sustained?Could this strategy be evaluated?

EffectivenessEffectiveness

Is there evidence that the strategy works?

Questions to ask: How effective is the strategy at impacting

desired outcome?How strong is the evidence?Would the reach be large enough to impact the

desired outcome?

99

Cultural ConsiderationsCultural ConsiderationsCan the intervention be adapted to serve

diverse groups?Has targeted community been consulted

regarding design, etc.Is the intervention responsive to targeted

group? How do you know?Do implementers understand the group

culture?Do organization(s) have the resources to

deliver culturally appropriate strategies?Are there any language/literacy barriers

Fidelity and AdaptationFidelity and Adaptation

Fidelity and Adaptation Fidelity and Adaptation ConsiderationsConsiderations Fidelity: the degree of fit between the developer-

defined components of a substance abuse prevention program and its actual implementation in a given organizational or community setting.

Program Adaptation: deliberate or accidental modification of the program, including the following:◦ deletions or additions;◦ modifications in the nature of the components that

are included;◦ changes in the manner or intensity of administration

of program components called for in the program manual, curriculum, or core components analysis; or

◦ cultural and other modifications required by local circumstances.

Fidelity/Adaptation Fidelity/Adaptation BalanceBalanceA dynamic process, often evolving over time, by which

those involved with implementing a evidence-based interventions address both the need for fidelity to the original intervention and the need for local adaptation.

◦ There is evidence that many evidence-based prevention programs still produce positive results despite significant adaptation. Some adaptations are, in fact, necessary for program success, given widely varying circumstances in different organizations or communities.

◦ There is significant evidence that the greatest impact from evidence-based programs occurs when there is program fidelity with respect to certain key elements. And some adaptations are undesirable, whether deliberate or accidental.

Fidelity/Adaptation – Fidelity/Adaptation – Be StrategicBe Strategic1. What are the theoretical underpinnings of the program?

2. Identify the core components: Those elements of a program that fundamentally define its nature, and that analysis shows are most likely to account for its main effects

3. Identify fidelity/adaptation concerns for local implementation

4. If possible, consult as needed with the program developer to review proposed adaptations

5. Consult with the organization and/or community in which the implementation will take place

6. Develop an overall implementation plan based on these results

7. Continually assess fidelity and adaptations during implementation

104

QUESTIONS?

If you have any questions or desire technical assistance on this topic please contact:

MDCH Bureau of Substance Abuse & Addiction SvcsLarry Scott, Prevention Section Manager

320 S. Walnut StreetLansing, MI 48913517-335-0174scottl11@michigan.gov

Thank you for participating in Thank you for participating in this training module!this training module!

105