2006-07 System of Care Start-up Webinar Series Making it Last… The Marriage of Social Marketing...

2006-07 System of Care Start-up Webinar Series

Making it Last…The Marriage of Social Marketing

and Evaluation

Social Marketing and Evaluation Webinar for the 2006 Newly Funded Communities

July 10, 2007


Social Marketers Are

From Mars

Evaluators Are From

Venus

Social Marketers Are Social Marketers Are From Mars…From Mars…

Like simplicityLike simplicity Comfortable with generalitiesComfortable with generalities Goal-orientedGoal-oriented Like to see quick resultsLike to see quick results Prefer short, straightforwardPrefer short, straightforward presentationspresentations Risk-takersRisk-takers

Evaluators Are From Evaluators Are From VenusVenus

Like complexityLike complexity Deal in absolutesDeal in absolutes Process-orientedProcess-oriented Like to take time, explore Like to take time, explore

all anglesall angles Prefer lengthy, abstract reportsPrefer lengthy, abstract reports Risk-averseRisk-averse


Mars and Venus Are Both Planets in the Same UniverseWant to help children: Stay in school Stay at home Become more productive members of

society at a cost that helps sustain programs in communities


Challenges and Barriers to a Happy, Productive RelationshipSocial Marketers Want interim

data to report Want easy to

communicate info

Have difficulty interpreting/ identifying alone

Evaluators Want to focus on

long-term outcomes

Want data communicated accurately

Uncomfortable passing along early results


Why Cooperate At All? Presentation of outcomes is key to sustainability Increase the public’s knowledge and awareness of an issue,

problem or solution Influence perceptions, beliefs and attitudes that may

change social norms Increase your organization’s visibility as a resource on

children’s mental health Enhance your organization’s leadership position Strengthen relationships Form a united front for projecting and reinforcing messages Create policy change Increase demand or support for community-based,

youth/family-driven, and culturally competent mental health services

Truly make a difference in the lives of children and families in your community


Why Cooperate At All? Evaluators have the information:

Descriptive/Outcome Data

Local Wraparound Fidelity Index

Service Experience Data

System of Care Assessment

Data Profile Reports

Continuous Quality Improvement Reports


Why Cooperate At All? Social marketers know how to present it:

Audience Research

Communication Strategies and Tactics

Packaging Data


Can This Marriage Be Saved?


Understanding Is the Key What are roles?

What is the language?

What are the desired outcomes?


Role of the Evaluator Understand program goals Analyze ongoing/post-program needs Design evaluation that captures pertinent

data Conduct evaluation; refine where

necessary Monitor other data sources; compare and

contrast Communicate regularly with staff across

program, particularly social marketers Report interim and final results Empower families and youth to be

persuasive advocates


Role of the Social Marketer Understand program goals Devise strategic communications plan Identify and profile key audiences Monitor changes and trends in attitudes

and behavior Share challenges and successes with

audiences Revisit and refine plan Communicate regularly with staff across

program, particularly evaluators


Role of Family and Youth Distinguish if the data reflects their

community Help social marketers package the data,

making it accessible and relevant Inform message development Complement data products with

descriptive testimonials Prevent dissemination of potentially

misleading data Provide the bridge from numbers to lives Promote retention and recruitment for

studies


What Can You Offer Each Other?Social Marketers Understanding of

audience needs Suggestions for

simpler language Clearer

communication of statistical info

Evaluators Accurate, timely

data on varied topics

Interpretation of results

Monitoring and analysis of data from outside sources


If We’re From Different Planets, Do We Need an Interpreter?

Social Marketers Audience

segmentation Editorial board Actuality

Evaluators Informed consent Program theory

model Longitudinal

statistical analysis


Breaking Down the Language Barrier Eliminate jargon where possible

Define terms

Ask questions


Making Beautiful Music Together


National Example Collaboration between the National

Evaluation Team and the Children’s Campaign has resulted in many products including: Fact sheets

PowerPoint presentations

Awareness Day data


Challenges To balance audience-friendly language

with accurate data


Lessons Learned Dialogue can start from either team

Begin working together at the very beginning of development

Collaborate on the development of a prototype

Create a timetable which includes time for multiple revisions

Learn about the other team's development process

Pose evaluation questions several different ways for better understanding

Keep the dialogue going!


Questions?

Please press *7 on your phone to unmute your line. When you are finished, please press

*6 to remute.


Local Example: Family Voices Network of Erie County We are in our 3rd Year – grant funded in

October 2004. We have an informal process for

communicating with evaluation– we’re located in the same building and maintain regular contact with the families. Sometimes this is a dual role between social marketing and evaluation staff.

We share data with our parents and youth.


Family Voices Network of Erie County: Social Marketing & Evaluation Team

Many times the social marketer and the evaluator wear dual hats.

We review data with our parents and youth and listen to their suggestions on how to make data family-friendly.

We inform the policy-makers of our findings based on the data collected bi-weekly at our Management Team meetings.


Family Voices Network of Erie County: Project Samples Community Assessment for Family

Involvement:

Knowledge & Effectiveness Survey

Children’s Mental Health Awareness Day:

NCMH Awareness Survey

Wraparound Bulletin:

Our quarterly newsletter


The 3 purposes of our Community Survey: The purpose of this survey is to identify community knowledge and attitudes

and to provide marketing assessment and direction including stigma evaluation.

The social marketing team presented the concept to the evaluation team and they worked together to develop a survey that would serve as a baseline of where our community stands today. By doing so, we have a measuring point to see improvements over time.

Between the evaluation team and the social marketing team, we identified our audience to be surveyed and worked together on identifying what we wanted to measure.

Family Voices Network of Erie County: Project #1: Knowledge and Effectiveness Survey

Category(as defined by survey code)

Total N sent N Returned Rate of Return

Care Coordinator 38 23 61%

Child Psychiatrist 2 1 50%

Family Medicine MD / DO 153 16 10%

Pediatrician MD / DO 152 29 19%

Judge 5 0 0%

Juvenile Justice 3 1 33%

Psychologist 81 18 22%

Supervisor CC 7 5 71%

Vendor 466 82 18%

Psychiatrist 19 1 5%

TOTAL 926 176 19%




We then rolled-up the data into four categories:

1. Families

2. Care Coordinators

3. Supervisors

4. Service Providers (vendor network)


We asked our families to tell us what was important to them from the survey categories. They identified:

1. Wraparound process/care coordination

2. The family role

3. Moving from system to natural supports

4. Making a referral

5. Child & Family Team (CFT) process



Next Steps:

• We will work with the evaluation team to create communications plans that address the needs of the family in the priority order they have identified.

• The evaluation team will provide us with other data from care management and other survey to assist in our plan development.

You can view the survey form on our Web site:http://www.familyvoicesnetwork.org//en/files/

Forms/Knowledge-Effectiveness-Survey.pdf


http://www.familyvoicesnetwork.org//en/files/Forms/Knowledge-Effectiveness-Survey.pdf

http://www.familyvoicesnetwork.org//en/files/Forms/Knowledge-Effectiveness-Survey.pdf


Local Example: Monterey County’s System of Care Mostly agricultural county, in many areas

60-75% Hispanic

Lead Agency: Monterey County Health Department, Children’s Behavioral Health (about 125 staff)

Social marketing and evaluation are led by outside consultants

To achieve optimal outcomes from social marketing & evaluation efforts both need to be recognized as integral parts of all other SOC activities.

And ensures that

programs have achievable outcome measures & a clear

identity & communication

plan.

Ensures that social marketing

activities have a built-in

evaluation component;

This allows the social

marketer to influence data collection from a creative and

audience perspective;

Program staff, social marketing, evaluation, families, youth & collaborating agencies work together from the very beginning – starting with the development of the social marketing plan, program logic models and design of outcome measures.


Monterey County’s System of Care:Strategy One: Bring Everyone to the Table

Family & Youth

Behavioral Health

Probation

Data Management

Cultural Competency

Social Marketing

EvaluationSocial Services

The Local Evaluation Team

Members of the Local

Evaluation Teambring

diversity & expertise

to the monthly and

“as needed” work sessions.


Monterey County’s System of Care:Strategy Two: Develop a Local Evaluation Framework

Monterey County utilizes a Comparison of Results Approach. Seeks to answer the questions:

How are we doing? Have outcomes for families and youth improved this year in comparison to last year?

13 local evaluation measures Developed local survey with

families and youth State and national data Program specific outcomes Data from partnering agencies

How do we bring partners together to discuss shared outcomes?

Monterey County’s System of Care:Strategy Three: Develop a “Content Matrix” to Share Evaluation Findings

Target Audience

Characteristics of Content

Information of Interest

Format

Families Bilingual, user-friendly. Personal stories and information. Avoid acronyms and terms that need further explaining (e.g., 200% poverty level).

Not interested in studies and data collection details. Data relevant to their child, e.g. “Last year in Monterey County, 80% of children diagnosed with a behavioral disturbance received a high school diploma.”

Fact sheet or newsletter with limited text, simple visuals, key facts without details, contact for more information.

Policy Makers, Dept. Directors

Limited time, prefer “bottom line” facts and visuals.

Outcomes, data sources, analysis “What does this mean to me and my constituents?” What is working, what isn’t?

Presentation, briefing sheet, Score Card.


Monterey County’s System of Care:Example: Bilingual Newsletter Reaching Families

Monterey County’s System of Care:Strategy Four: Develop a Distribution Plan and Implement it!

Target Audience

Format Distribution Who is responsible?

Families Newsletter Mailed to 350 families; 25 copies each dropped off at 8 clinic locations.

Family Partnership Program

Policy Maker, Dept Directors, Media, Public

Briefing

Score Card

Briefing scheduled for May 8th as part of Children’s Council meeting; Press release

Mailing (50 copies)

Project Director

Social Marketer

Staff Staff Memo

Presentation at all staff meeting.Article in staff newsletter

Staff mailboxes (135 copies) and email.Scheduled for March staff meeting.Staff mailboxes and email (article will go into February issue).

Admin

Project Director

Social Marketer


Monterey County’s System of Care:Example: Score Card


Monterey County’s System of Care:Score Card 2005- At a Glance

Purpose: To highlight system-wide (shared) accomplishments of Monterey County’s System of Care in a user-friendly way. To show why systems of care are important to our community and why it is important to continue to fund them.

Primary Target Audience: Policy-makers, Board of Supervisors, Children’s Council, Governance Council, Community Stakeholders and Staff (Language: Eng)

Distribution: Briefing to Children’s Council and media on Children’s Mental Health Summit Day; Mailing


Strategy One: Bring everyone to the table

Strategy Two: Develop a local evaluation framework

Strategy Three: Develop a content matrix to share evaluation findings

Strategy Four: Develop a distribution plan to share findings and implement it!

Strategy Five: Ask for feedback and actively involve new and existing partners in the process

Monterey County’s System of Care: Summary of Strategies


Monterey County’s System of Care:Lessons Learned & Accomplishments

Scorecard is a visually appealing and useful tool to explain SOC values and principles. Highlighting of shared programs and accomplishments generated goodwill – and momentum (many new partners this year!) Also- triggered more collaboration in other areas! Ongoing struggle – who is our target audience? Translate into Spanish? Weighing cost against level of distribution. Need to make the process youth- and family friendly – training. Goal in Monterey County: Bring more youth to the table.

Key lesson learned to new communities (from a social marketing perspective): Have the social marketer involved early on in the development stages of the evaluation framework and survey tool. The social marketer can provide input into how questions and outcomes can be translated into meaningful data to varies target audiences.

Making It LastMaking It Last Have early conversations about

goals/objectives Inform each other of

progress/challenges Be a resource for trends, new

developments, community news

Present information together for clarity and accuracy


Resources Caring for Every Child's Mental Health Campaign

www.mentalhealth.samhsa.gov/child/default.asp(Features an order form for Children's Campaign products such as Children’s Coloring Books, Annual Reports to Congress, etc.)

Subscribe to the listserve 4-THE-CHILD [email protected]

Family Leadership in Systems Evaluation Tip Sheets Engaging New Families in Evaluation & Language

Qualifying Family Preference in Hiring Federation of Families for Children’s Mental Health

The World of Evaluation: Three Short Courses


Questions?

Please press *7 on your phone to unmute your line. When you are finished, please press

*6 to remute.


Presenter email addresses: Deborah Porter,

[email protected] Lisa Rubenstein,

[email protected] Jana Sczersputowski,

[email protected] Tisha Tucker,

[email protected]

2006-07 System of Care Start-up Webinar Series Making it Last… The Marriage of Social Marketing...

Documents

Transcript of 2006-07 System of Care Start-up Webinar Series Making it Last… The Marriage of Social Marketing...