GUAM’S LOGIC MODEL: A COMMUNITY ISLAND

EXPERIENCEBy

Project I’Famagu’on-ta Evaluation Team:Velma Sablan, Ph.D., Mary Fegurgur, Psy.D.

Mariles Benavente, A.C.S.W., Keith Villaluna, B.A.

GUAM’S EXPERIENCEan overview

I. INTRODUCTION: GUAM, A PACIFIC ISLAND COMMUNITY

II. GUAM’S LOGIC MODEL: THE ELEMENTS

III. THE PROCESS USED TO DEVELOP THE MODEL

IV. USING THE MODELV. Q&A

Guam, a Pacific Island Community

• U.S. territory in the Western Pacific• Largest & Southernmost Mariana Islands• 32 miles long, 4-12 miles wide; 212 sq miles• 7,500 miles from Hawaii (12 hrs. flight time)• Closer to Asia (Tokyo, Taipei, Manila and Hong

Kong are within 2-3 hrs. flight time)• Population: 154,805 (37% Chamorros, 63% Other)• Military: ≈ 30,000• Tourists: ≈ 1.2 m annually

SIGHTSEEING

II. GUAM’S LOGIC MODEL: The Framework and the Pieces

• Guam used the Hernandez, Mario & Hodges, Sharon (2003). Making Children’s Mental Health Services Successful: Crafting Logic Models for Systems of Care: Ideas Into Action

• The Logic Model Framework was adapted for use as Guam’s frameworks

On the Logic Model Framework…

• Aspects of Hernandez & Hodges Framework helped Guam to facilitate discussions with community members who held diverse expectations and supported consensus decision making on key areas/issues (i.e., defining target population, vision of potential child/family outcomes and agreement on gaps of service components);*

Key Elements of the Framework

• Mission Statement• Population Context• Guiding Principles• System of Care Strategies• Outcomes• Evaluation and Feedback Cycles

Developing Project I’Famagu’on-ta Theory of Change-Questions & Directions

WHAT ARE THE SERVICES AND INFRASTRUCTURES NEEDED TO ACHIEVE ENDURING CHANGES FOR CHILDREN AND

FAMILIES, PROGRAMS, AGENCIES, AND THE GUAM COMMUNITY ?

WHAT ARE STRENGTHS AND ISSUES IN SERVING CHILDREN AND FAMILES?

TARGET POPULATION

STRATEGIES

OUTCOMES

Home, School, Agencies and communityA theory of change defines the relationship

between the target population, the expected outcomes, and the strategies to achieve the desired outcomes.

A Key Element

Definition of the TARGET POPULATIONwas a critical piece to accomplish!*Everyone had a different idea of who should

be served!

Here is what finally emerged…

Top 5% Children Permanent Placed Out of Home or Off Island

Top 10% of SED Children temporarily in an Out of Home Placement

25% of Target Population for Prevention of an Out of Home Placement

60% of Target Population At Risk for Out of Home Placement.

Systematically working using the Framework:

• Phase I: Pre-Planning (3 steps)• Phase II: Theory of Change Development (6

steps)• Phase III: Implementation (3 steps)

An example of Phase II, Stage 8-Defining Strategies and Outcomes….from community input to Logic Model Development

STRATEGIES TO ACHIEVE OUTCOMESPROGRAM LEVELHow does child enter program?What are the eligibility guidelines, policy and procedures?How will care of the child/family be monitored?How will services be identified, planned, and delivered for child?How does the program encourage and create individualized services and supports?How does the program encourage partnership and planning among and between agencies?How does the program encourage full participation and coordinated case management? SYSTEM LEVELWhat is the relationship between the SOCC and Project I’Famagu’on-ta?How will SOCC manage, monitor the program and array of services for policy development and sustainability?

CHILD AND FAMILY LEVEL

24 Hour crises intervention

Intensive case management

WRAP Coordinator with low case load

Child/family centered WRAP Meetings

WRAP Plan

Flex funds

Array of Services

More on the framework and pieces….

• Each level: Child & Family, Programs/ Practice, and Service system/Community level) is addressed in the model

Another example: Phase II, Stage 4-Developing the Population Context….from community input to Logic Model Development

POPULATION CONTEXT

Interconnected small town, connection to culture, warm environment, faith based communities readily available. Multi-cultural setting, multi-linguistic Society, Low SES status, high unemployment and acculturation issues

SYSTEM AND COMMUNITY CONTEXT

CHILD FAMILY

CONTEXTPROGRAMS & PRACTICE

CONTEXT

Reliance on non traditional and natural support systems, religiosity,and child could have multiple caregivers/placements. Sometimes, inability of parents to handle child, resulting in child abuse. Child acting out in home, school community settings. High rates of suicidal, self mutilating behaviors, low school attendance, low test scores, H.S drop out rates & beyond control.

Children could enter system through the education, child protective services, police or judicial system. Services are fragmented and resources are not linked across agencies.

TARGET POPULATION

Children between the ages of 5-18 years old, with Serious Emotional Disturbances. Services for 18 –21 years old, if existing agency plan

Making the Logic Model come alive…

• Color-coded chart depicts, at a glance, the connection between contextual information, vision, outcomes, strategy and indicators for each of the 3 system levels to facilitate logical connection to strategies/activities.

III. The Process Used in Developing Guam’s Logic Model

Phase I: Pre-PlanningQuestions that were faced:

• Who should initiate the process?Evaluation Team took lead role in the development and provided technical assistance to the community (families, agencies and project staff). The Guam System of Care Council/ governance board was not yet established.

Phase I: Pre-PlanningMore questions!

• Which stakeholders to include in the development process? How to recruit participants? Evaluation Team identified family members of children with SED, six child serving agency representatives, community representatives (village mayors, faith-based organization, advocacy office), and project staff for participation in this workgroup. Project Staff were asked to also identify families involved in the Project to be members of the workgroup.

What Did It Take? Hooking Everyone In!

• Small groups, charged with developing the logic model, met twice monthly for 2 months, then increased to weekly meetings for 4 subsequent months;

• All members were provided copies of the Hernandez & Hodges text;

• Initial sessions involved review of key elements with a focus on adapting the model for Guam;

• Next sessions focused on using suggested steps in articulating the mission, identifying goals and guiding principles, identifying the population context, mapping resources, assessing the system flow, identifying outcomes, defining strategies, creating and fine tuning the framework

What Did It Take? Hooking Everyone In!

The draft model was presented and discussed at several venues to generate additional community input:- Community Evaluation Advisory Team (CEAT) Meetings- Executive Commission of Directors (Youth Serving Agencies)- PROCEED (Parents Reaching Out for Children Experiencing

Emotional Disturbances- Agency Ombudsmen Meeting (six child-serving agencies)- Project Staff Meetings- Guam System of Care Council (scheduled for August 29, 2005)*

Phase III: Implementation

Stage III: Using the Framework to Inform Planning, Evaluation, & Technical Assistance

The Evaluation Team documented the input and discussions from the different community meetings and used them to refine the draft model as it is reflected to date.

This is a working model that will be updated periodically as progress is made is System of Care development

B. KeyTurning Points in the Development Process:

1. Reaching agreement on the definition of the target population

Factor that helped achieve consensus:Each youth service agency brought data oneligible youth serviced by their agency. Datareflected consistent concerns on “out-of-homeplacements” which in turn helped in clarifi-cation of the target population.

Another turning point….

2. Reaching agreement on the gaps in services that needed to be addressed via the project

Factor that helped achieve this turning point:Using case scenarios, the group traced the various ways a child with SED enters Guam’s system and understanding what happens to the child which resulted in clear identification of service gaps.

ASSESSMENT SYSTEM FLOW

PUBLIC HEALTH/CPS

DYA

SANCTUARY

FRIEND OR RELATIVE

MENTAL HEALTH

CASE MANAGEMENT

COURTS AND PROBATION OFFICERS

DOE EH PROGRAM

PSYCHOLOGICAL EVALUATION ON OR OFF

ISLAND

PRIVATE PRACTICE DOCTOR

POLICE

FOSTER HOME

CHILD OUT OF HOME PLACEMENT

HOME

HOSPITAL

CLEARANCE FOR POSSIBLE

INPT AT MENTAL HEALTH

Yet another turning point….

3. Reviewing the progress of the group’s work by depicting everything in a draft display board and observing how the group expressed understanding of the interconnectedness of the subsystemsFactor that helped achieve this turning point:Visual display of the progress and color coding of materials on the Model Display board to tie each subsystem to the vision, the outcomes/ strategies and indicators.

Factors which helped the process of developing a local model:

• Involvement by stakeholders, especially families, who had some level of continuity and history of Guam’s System of Care Initiative, and hence, understood the vision and the need for systems change;

• Involvement of a Clinical Director who was actively involved in setting Policies and Practices for the project and, hence, actively involved in assuring the project’s perspective in the logic model development;

More factors that helped the process…

• Involvement of combined representation of project staff and family members including those with adversarial relationships with DMHSA to obtain their input and support.

• Clarification of the target population for the project as this was an area of contention with various stakeholders.

More factors that helped the process…

• Continuous presentation of evolving working logic model in many forums to ensure widespread input and support from the community.

Factors that hindered the process:

• Initial confusion as to who should take the lead in development of the Logic Model;

• Turnover of key Project Staff;• Lack of a governance board;• Initial confusion on defined target

population resulting in conflicting stakeholders' expectations;

Other factors that hindered the process…

• Early sessions showed poor attendance by some members necessitating clarification of purpose for the Logic Model and value to the systems change initiative as well as value to project staff and in guiding project activities;

Status of the model development process (What next?):

• Presentation due to the Guam System of Care Council/Governance Body on Aug. 29, 2005;

• Orientation to the new Project Director and Clinical Director upon coming on board with Project I’Famagu’on-ta;

• Orientation for new project staff as they are hired by Project I’Famagu’on-ta;

• Active use of the logic model in strategic planning sessions by project/governance board;

• Active use of the logic model by project staff in program development activities;

What did we learn?

• Logic model development, when done in the recommended collaborative approach, takes long hours of planning and involvement but, IT WORKS!

• QUESTIONS & ANSWERS!