Post on 12-Jan-2016
Maryland Early Childhood Mental Health Consultation (ECMHC) Evaluation
Webinar Presentation by Deborah F. Perry Ph.D.Georgetown UniversityOctober 27, 2010
Overview of Presentation Overview of
statewide ECMHC evaluation in Maryland
Maryland’s approach Logic model and
components Levels of data
collection and other features
Future directions
Maryland ECMHC Evaluation
Funding: Maryland State Department of Education (MSDE),
Division of Early Childhood Involves 12 sites across Maryland Three year evaluation (2008-2011), with two option years Principal Investigators:
Sharon Stephan: University of Maryland Bruno Anthony, PhD & Deborah Perry, PhD:
Georgetown University Minority Business partner: CKD Communications, LLC
Early Childhood Mental Health Consultation (ECMHC)
Collaborative, capacity building services provided to early care and education (ECE) providers and family members
Can be initiated in response to concerns about an individual child’s social-emotional or behavioral development
The impact can be measured in the domains of child, parent, staff, and family
Approach to Evaluation
Collaborative Participatory Comprehensive Sensitive to respondent burden No site comparisons Technical assistance related to
evaluation and data collection provided
Funded Projects and Evaluation Liaisons
Evaluation Logic Model
Skills and Qualifications
Satisfaction
Relationship
Early Childhood Mental Health Consultation
Early Care and Education
- Attitudes, Beliefs, and Stress
- Classroom Environment
Parenting
- Behavior and Attitudes
- Stress
Young Children
↓Challenging Behavior
↑Social Skills
Expulsions
Evaluation Components
ECE ProvidersSDQTOSRQS
DECA
ParentsPSIPBI
DECA
ConsultantsClimate
RQS
ECE DirectorsGAS
Implementation Factors
Core Outcomes
Program DirectorsModel
Description,Quarterly Interview
ConsultantsKnowledge
& Skills,Service Log,
Exiting Child Form,Roster, Preschool
Climate Scale
Exiting Child Study
Consultants, ECE Directors, ECE Providers,
and Parents
QualitativeInterviews
Note: instruments in pink will only be collected for child-specific referrals
Levels of Data Collection
Program-level outcomes Measures assess changes in: ECE provider behaviors and
beliefs; job stress; structural and interpersonal characteristics of the ECE setting; and the quality of the relationship between the MHC and ECE providers.
Child-/Family level outcomes For child-specific referrals:
Measures assess changes in the child’s behavior (rated by teacher and parent), parenting stress and behaviors; and the quality of the relationship between the MHC and ECE providers.
ECMHC Program Directors
Tools Construct Frequency
Model Description Program Structure
Baseline, Annual
Quarterly Site Interviews Program Changes
Quarterly
Mental Health Consultants
Tools Construct Frequency
Knowledge and Skills Inventory Consultant Characteristics
Baseline, Annual
Service Log ECMHC Services Ongoing
Classroom Roster Administrative Tool
Ongoing
Preschool/Infant MH Climate Scale Classroom Environment
Baseline, every 4 months (ongoing)
Consultant Final Perceptions Form Quality of Relationship
4 months, or at Discharge
Exit Study (Qualitative Interviews)Perceptions and predictors of expulsion
Following an exit due to behavior issues
Preschool/Infant Climate Scale
These tools are completed by the mental health consultants for every child-specific and programmatic consultation they initiate. Preschool Mental Health Climate Scale (Gilliam, undated) Infant Climate Scale (FSU, 2009)
The instruments assess multiple dimensions of the classroom environment and ECE providers’ behaviors.
Consultants complete these scales at baseline, at 4 months or at discharge; and if ongoing, every 4 months thereafter.
Child Care Directors
Tools Construct Frequency
Goal Achievement Scale Opinions and Attitudes
Baseline, 4 months or at
discharge
Exit Study Interviews Perceptions of Expulsion
When child exits due to
behavior
Early Care and Education Providers
Tools Construct Frequency
Strengths and Difficulties Questionnaire Child Functioning
Baseline, 4 months or at discharge
Teacher Opinion Survey Teacher Attitudes
Baseline, 4 months or at discharge
Provider Job Stress Index Job Stress Baseline, 4 months or at discharge
Teacher Final Perceptions Form Quality of Relationship
Baseline, 4 months or at discharge
Exit Study Interviews Perceptions of Expulsions
When child exits due to behavior
Parents
* For those who consent to long-term follow along
Tools Construct Frequency
Parenting Stress Index-Short Form Parent stress Baseline, 4 months or at discharge
Parent Behavior Inventory Parenting practices
Baseline, 4 months or at discharge
Parent Final Perceptions Form Quality of relationship
Baseline, 4 months or at discharge
Devereux Early Childhood Assessment Child functioning
Baseline, 4 months, 12* & 24* months
Exit Study Interview Perceptions of expulsions
If their child is expelled
Exit Study Interviews
Motivated by MSDE’s interest in this policy-relevant (but infrequent) outcome
Always interview the consultant; seek the additional perspectives of ECE Director, provider and parent
Includes open-ended questions such as: What behaviors led to the child’s exit? What was done to try to maintain the child in the
placement? Did you expect this child to be expelled? Why or why not?
Professional Activity Units
Working with MSDE, established process for ECE Directors and Providers to get professional activity units when complete baseline and follow-up measures for evaluation
Increase participation and help with follow-up data
Current Status and Future Directions Data collection in the field: good
participation from all sites Exit interviews completed with
consultants, and a few ECE providers Comparison group Addition of CLASS as objective measure
of change Paper on Preschool Mental Health
Climate Scale with Walter Gilliam