Slide 1

Connecticut State Department of EducationBureau of Special Education Annual Back to School MeetingSeptember 17, 2014

Breakout Session: Supporting Student Success Through Positive School Climate(Actionable, Student Level, Social-Emotional Data)

Alvin F. Larson, Ph.D.Meriden Public SchoolsMeriden, Connecticut

Please complete your Confidentiality Sharing Exercise1CSDEs Student Success Plan Core Components

Academic: attitudes needed to be an effective learner in school;

Career: student will investigate their own interests and/or career goals;

and todays primary topic:

3.Social-Emotional: maintaining positive interpersonal relationships, managing feelings and emotions.

Todays Purpose

Demonstrate two tools that provide actionable student level data that initiate and guide differentiated social-emotional intervention.

2If we add comprehensive Social-Emotional Support tools such as

#1 Confidential School Climate Survey MPS Cares#2 Confidential Getting to Know You survey

Social-Emotional, Perseverance, Mindset and Motivation Actionable Data for Monitoring and InterventionPro-Social outputs butless behavior problemsAbsenteeismBullyingDropoutsCrimeSuicide3Pro-Social outputs butsome behavior problemsAbsenteeismBullyingDropoutsCrimeSuicideTaking Action with Social-Emotional Data (Student Success Plan)

Current Education Data Use AcademicandOvert Behavioral ABCDF Scale ScoresReferralsSuspensionsToolsPurpose (more pro-social outputs and less behavior problems)

A timely response system to identify students who perceive they are experiencing a social-emotional problem;

New insights of any student's perceptions, especially those students receiving counseling services (Special Education and Non-Special Education);

Estimate the proportion of latent, social-emotional at-risk students that remain unidentified (~5% of your total student population); and

Provide measures of "school climate" to meet state legislative requirementsMSCS-SV was approved by CSDE. 4#1 Meriden School Climate Survey for Students (MSCS-SV) based on National School Climate Standards and designed for school psychologists5Current Thinking on Bullying and Other Mean BehaviorsWhole school interventions recommend victims tell an adult, students rarely do1

Relational aggression is a bullying problem for adolescent girls. Social exclusion and friendship manipulation is associated with depression and suicidal ideation 2

To avoid many academic, social and emotional problems, we must become more adept at identifying possible victims and bullies 3

Developmentally sensitive strategies for avoiding and responding to bullying-type situations are needed as well as seeking input from adolescents bullying is a relationship problem requiring improved relationship and coping skills. 4Bauman, S., (2010). Cyberbullying in a rural intermediate school: An exploratory study. The Journal of Early Adolescence, 30(6): 803-833.Raskauskas, J., & Stoltz, A.D., (2004). Identifying and intervening in relational aggression. The Journal of School Nursing, Aug;20(4): 209-215. Lyznicki, J.M., McCaffree, M.A., & Robinowitz, CB., (2004). Childhood bullying: implications for physicians. American Family Physician, Nov 1;70(9): 1723-1728. Bostic, J.Q., & Burnt, C.C., (2011). Cornered: An approach to school bullying and cyber bullying, and forensic implications.Child and Adolescent Psychiatric Clinics of North America, 20 (3), 447-465.The online MSCS-SV is considered a needed complement to whole school character programs (such as PBIS)

Confidential, we know who logs on (student identification number); andbefore taking the survey, students are informed All of your answers are confidential. That means we cannot share your answers with anyone, unless you tell us about danger or someone is going to be hurt. We must keep all students safe.

Now, look at your Confidentiality Sharing ExerciseHow many told other people about your confidential conversations?Scenario #1 (General Confidentiality)Scenario #2 (Possible Bullying)Scenario #3 (No Friends)Scenario #4 (No Home Support)

This is also my problem; currently, teachers do not have access to MSCS-SV data (only counselors). What data can be ethically shared?6MSCS-SV: A Developmentally Sensitive Method (customized software and database)

Trigger emails produce actionable data

Students in Perceived Social-Emotional Crisis7

The following student quotes are samples of student trigger emails. They deal with suicide, harm, threats, relational and racial issues. The underlined segments, added by the presenter, align with national qualitative research findings. Case 1 Hispanic, male, grade 7, Free Lunch, not special education, not ELL, Language exposed to Spanish but English Dominant, Basic Math, Basic Reading

there are some people who call me names and people start rumors about me that i dont like sometimes i want to die and kill myself before the next day. i feel like everyone hates me and for no reason. ive sometimes been nice but when they bring up the rumor i say realy bad things to keep them away one day i was thrown a note saying im ###. and it was my by bestfriend ... i lost the only thing that i can trust. i was alone for a long time. if this is un healthy then pleace help me Case 2White, male, grade 5, Pay Lunch, not special education, not ELL, Language is English, Goal Math, Proficient Reading.

At school some people dont like me so they bully me and pick on me. I feel horrible and I feel like I want to stay at home doing nothing than get bullyed at school.Case 3 Hispanic, female, grade 9 (retained), Free Lunch, Special education, has ELL services, Language is Spanish, Below Basic Math, Basic Readingthey realy dont say anything but they just say im ugly nd that im by-sexual nd really its getting really annoying people are just roud and they dont have a life but just to talk to people and i trust this school nd i know this school can help me with this situationConfidential and Responsive Online School Climate Survey8Responsive Online School Climate Survey for StudentsCase 4 Hispanic, female, grade 8, Free Lunch, special education, not ELL, Language exposed to Spanish but English Dominant, Basic Math, Basic Readingthey make threats about how there going to fight me after school then i always have to keep looking overmy shoulder. the make fun of my red hair , and my race also they say mean thing like they spread rumer about me that are not true and most of the kids belive that they are true and that make me mad when i have no one to talk to about the things happening to me

Case 5White, female, grade 11, Free Lunch, not special education, not ELL, Language is English, Advanced Math, Goal Reading

People have spread lies about me sleeping around and being pregnant. People have spread rumors that I slept with teachers to get good grades

Case 6White, female, grade 9, Pay Lunch, not special education, not ELL, Language is English, unknown Math, Goal Reading (transferred from non-public) There are many people who call me a fat whore that should go to hell and the no bullying policy really needs to start working because it is affecting my lifeCase 7White, male, grade 8, Free Lunch, not special education, not ELL, Language is English, Proficient Math, Goal Readingi get pushed around because people think its funny. also i get called the ginger alot because i have red hair, i have also been called gay for no reasonCase 8Black, female, grade 3, Pay Lunch, not special education, not ELL, Language is English, Goal Math, Goal ReadingThe kids at school hit me because of my skin color9Matched Scores on Factor 2 Safety (hits/threats or mean rumors)October to June - Of the 1,959 students who took both climate surveys:New student application: MPS Cares open all year

1,84211729881,727115117No Longer Reporting Mean Behavior

75%Still Reporting Mean Behavior

25%No Mean Behavior

94%Students Reporting Mean Behaviors in June, but not October

6%JuneNo Mean Behavior

94%Others Mean to Me

6%October1,84210

11Meriden Cares Sample Email text to StaffStaff email alert from Noreply @ Meriden BOE MPS Cares

The following student submitted an Meriden Public Schools Cares Entry

StudentStudent ID: xxxxxName: STUDENT NAMESchool: SCHOOL

Question Responses:

Problem with friendsYesPeople are meanYesOther problemNo

Comments:(student comment is optional)

Trigger emails are automatically sent if any yes is checked.

This site is only available to students and is only accessible on Meriden Public Schools computers.

12(Student View)Meridens School Climate Survey for StudentsNeverRarelySometimesVery OftenAlways7.At my school, I have a friend who I can really trust.12345When a student responds with a 1 or 2 on this item, a trigger email is immediately sent to:

School PsychologistsSchool Social WorkersSchool AdministratorsOffice of Research and Evaluation13At the request of administrators and counselors, we added an additional trigger email alert related to no friendsScientific Validation of MSCS-SV

National School Climate Standards recommend we build instruments that are valid, reliable and measure latent psychological constructs such as: engagement in school, respect, caring, fairness and pro-social behavior. Each latent factor (attitudes, beliefs and values) influences how each student responds to a sub-set of questions. The Factor loadings are correlations with each attitude/belief/value.14Factor 4FriendsItem 19Item 9Item 14.719Item 37Item 36Item 30Factor 1Caring.711.694.767.706.563Scientific Validation of MSCS-SV

Validity and Reliability StandardsInstrument Development Standards (APA/AERA)For validity and reliability correlations(computed by computernot judgmental)CorrelationCognitive DomainAffective Domain.90 - .99Very StrongGreat.80 - .89StrongStrong.70 - .79Not AcceptableGood.69 or lessVery Low/RejectNot Acceptable/Reject15Exploratory Factor AnalysisLEA Level (2011, 2012)

Independent Confirmation Factor AnalysisUniversity Level (2013) comparative fit index (CFI) .90 for Seven Factor Model

Teachers Motivate & Care(r = .914)Safety (r = .758)Respect Differences (r = .759)Parent Values School (r = .746)Parental Support (r = .721)Caring Friends (r = .749)Aggression (r = .720)16Scientific Validation of MSCS-SV

FIGURE 1Cross-Sectional Non-Matched Average Factor 1 Scores: Teachers Motivate and Careby Grade Level

Most positive perceived climate (Factor-based scores: 1-5)Lowest perceived climate17Teachers Motivate and Care, is composed of 11 items (r = .914) Students perceive teachers as helpful, fair, caring and motivating; teachers listen to them, tell them when they do a good job and students are happy to be in this school.

FIGURE 2Cross-Sectional Non-Matched Average Factor 3 Scores: Respect Differencesby Grade Level

Most positive perceived climate (Factor-based scores: 1-5)Lowest perceived climate18Factor 3: Respect Differences is composed of 5 items (r = .759) Students respect each other's differences, they see skin color as a potential problem; other problems are fighting and students being mean to others.

FIGURE 3Cross-Sectional Non-Matched Average Factor 5 Scores: Perceived Parental Supportby Grade Level

Most positive perceived climate (Factor-based scores: 1-5)Lowest perceived climate19Factor 5: Parental Home Support is composed of 4 items (r = .721)Student perceives s/he has a place and time to do his/her homework and there is a parent who listens and will help with homework if needed.

FIGURE 4Cross-Sectional Non-Matched Average Scores of Stable Factors across grade levels: Aggression, Parents Value School, Friendship and Safetyby Grade Level

Most positive perceived climate (Factor-based scores: 1-5)Lowest perceived climateAggression Towards OthersParents Value SchoolCaring FriendsSafetyFactor 7: Aggression Towards Others is composed of 3 items (r = .720); in school, an unacceptable coping skill. Student claims to have hit, pushed or spread mean rumors about others during the past few months, sometimes in anger.

Factor 4: Parent Values School is composed of 4 items (r = .746) Students perceives s/he has a parent or other adult at home who cares about his/her school work, and wants the student to do his/her best and follow school rules; the student also feels s/he will be successful in life.

Factor 6: Caring Friends is composed of 4 items (r = .749) Students perceive they have a friend their own age who cares about them and they trust and can talk about their problems.

Factor 2: Safety is composed of 7 items (r = .758) Students perceive other students as hurting their feelings, saying mean things, possibly hitting and threatening; they worry about their safety.20

TABLE 1Average Survey Factor differences for Special Education versus non-Special Education Students by Climate Factor and Grade Level(significant > .2)

Elementary SchoolMiddle SchoolHigh SchoolGradeFactor34567891011121 Teachers care-.3-.3-.3-.10-.1+.1+.2+.102 Safety-.3-.4-.3-.1-.20000-.13 Respect Others-.4-.5-.3-.2-.1+.10+.1-.1-.14 Parent Values School-.4-.4-.4-.2-.2-.2-.3-.10-.35 Home Support-.1-.3-.2-.1-.2-.2-.1-.10-.46 Friends-.2-.3-.30-.3-.4-.1-.1-.3-.47 Aggression-.7-.8-.5-.4-.2-.2-.2-.2-.3-.321Seasonal Variations

During the school year (solid lines) all decrease except one grade 4 cohort and one grade 10 cohort.Summer (dotted lines) all increase; optimism that this school year will be better.Matched factor-based scores could be used to evaluate interventions.22FIGURE 5Matched Factor-Based Scores for Teachers Care and Motivate from September 2011 through September 2013: Eight Grade-Level Cohorts

Gr 3 to 5Gr 4 to 6Gr 5 to 7Gr 6 to 8Gr 7 to 9Gr 8 to 10Gr 9 to 11Gr 10 to 12AERA 2014 Annual MeetingMost matched cohorts show increasing matched scores Decreasing (cohort grade 8 to 10);About the same (cohorts: grade 5 to 7; grade 7 to 9; and grade 9 to 11);Increasing (the other four cohorts, despite non-matched decreases).Matched factor-based scores could be used to evaluate interventions.

23FIGURE 6Matched Factor-Based Scores for Respect Differencesfrom September 2011 through September 2013: Eight Grade-Level Cohorts

Gr 3 to 5Gr 4 to 6Gr 5 to 7Gr 6 to 8Gr 7 to 9Gr 8 to 10Gr 9 to 11Gr 10 to 12AERA 2014 Annual Meeting

Individual Student ProfilesMSCS-SV historical dataIdentify changes in student perceptionsIdentify unknown students who are latently at-risk, and Provide additional student perceptions of each counselors current caseload24SafetyAERA 2014 Annual Meeting

Log in to the School Climate Survey Website:Email: Your Meriden email addressPassword: Supplied 25School Psychologists Log in Screen for MSCS-SV

26School Psychologists Ethic Statement for MSCS-SV

Staff will need to agree to the Protocol for Sharing Confidential Information27School Psychologists View of MSCS-SV Database

Staff can search results by student name, survey administration or by Factor28School Psychologists View of MSCS-SV Database

Select student name to view Historical Profile29School Psychologists View of MSCS-SV Database

Historical Profile for student in grade 930School Psychologists View for MSCS-SV Database

Staff can select Trends to view one year changes31

School Psychologist View of MSCS-SV DatabaseStaff selects NOTIFICATIONStaff highlights student name if accepting responsibility for follow-up32

MPS Cares NotificationAccept NotificationPercent of Social-Emotional At-Risk Students

7 % perceived hit/threat or mean rumors in fall (6% matched scores)10% perceived hit/threat or mean rumors in spring (6% matched scores)4.8 % unknown, latent, students self-report social-emotional at-risk9.8% with low scores (-2 below grade level mean)______

15.1% unduplicated count of all students base onat-risk by trigger email,low score (-2 ) or substantial change in factor-based scores A working group of school psychologists and social workers have developed protocols to respond to student trigger emails and historical database reports.

3334To: Administrators, School Psychologists, Counselors and Social Workers Date: June 2014

Re: Protocol for Trigger Emails for Meriden Student Climate Survey

The following expectations serve as our protocol for responding to the trigger emails in Meriden Public Schools.

Protocol for Trigger Emails Students that are the subject of the trigger email will be met with within two school days.

The attached Response Interview Forms will be used.

Trigger email responses will be the collective responsibility of all school psychologists, counselors, and social workers in the building.

Student Support Staff shall share the active Response Interview Forms with colleagues in other Meriden schools if a child transfers during the school year.

After two years of inactive files, the Response Forms and emails should be destroyed.

The Meriden Student Climate Survey shall be administered in October and March in order to ensure adequate time for staff to support the student.

Meriden Student Climate Survey Response Interview FormOn the survey, you answered a question that met the criteria to look further into. The question was (read from survey)

Can you give me an example of what that looks/ sounds like?

Who has said/ done this?

When has it happened? How often?

Was anyone else there?

Does it continue to happen? Any other information you would like to share?

Overall do you feel safe at school? Yes or No____________________________________________________________________________Response/ Solution by interviewer Check what is applicable:____Parent contacted: Date___________ ____ Periodic check-in ____Administrator notified ____ Follow up counseling____Teacher notified ____ Follow up with perpetrator____No Follow up required at this time____Other35

Case 1 there are some people who call me names and people start rumors about me that i dont like sometimes i want to die and kill myself before the next day. i feel like everyone hates me and for no reason. ive sometimes been nice but when they bring up the rumor i say realy bad thingsto keep them away one day i was thrown a note saying im ###. and it was my by bestfriend ... i lost the only thing that i can trust. i was alone for a long time. if this is unhealthy then pleace help me

Student was in counseling within 30 minutes of receiving this trigger email_______________________________________________________________________

How many student suicides ?How many violent student assaults ?

Ex post facto analysis: Were the signs there?

How could we have known/prevented? Just ask students via confidential survey.LEAs can be aware/preventative seek developmentally appropriate input from adolescents.36AERA 2014 Annual MeetingSignificance of MSCS-SV a chance to be pro-active and preventativeIdentifying latent at-risk students (for counseling services; 4.8%)

Identifying aggressive students (potential bullies also need counseling; 2% to 3%)

Ability to follow students over time (matched scores)School climate has seasonal variations Identifying disengaging students (dropouts) being proactiveHistorical data/profiles very helpful with current caseload

Behavioral screen of future problem behaviorsProtective and risk factors of school climate and bullying victimization(Gage, et. al. 2014; in press, School Psychology Quarterly)Other possibly criminal or violent events (including community/family)

SummaryEstablished, valid, reliable and actionable tool/systemLess behavior problems with improved social-emotional health

37AERA 2014 Annual MeetingSignificance of MSCS-SV#2 Getting to Know You Survey (customized software and database) Second Year Research Project

Purpose: Help foster teacher-student relationships (teacher care);Monitor student interests and/or career goals; andProvide guidance on how to modify instructional methods to maximize and maintain student perseverance/motivation over time;motivation is needed to be an effective learner in school.

Attributes:Teacher-student relationships and student perseverance/motivation are positively correlated with student cognitive achievement.About 12 to 15 minutes, once a year (September or as new students enter).Teachers/staff only have access to their own students data.38

Getting to Know You survey (guided by research)

Research: to increase motivation, provide students with a

Perceived caring environmentStudents want teachers to know about them personally

Non-evaluative, constructive but positive feedbackStudent can do better with effort (mindset)Safe ways to take risks and okay to make mistakesBalance the task, challenging but doableMake the topic/task something they likeUseful, important and find role models

39Getting to Know You survey

Ask students about their interests, beliefs and values:

Activities:Do you play a musical instrument ... sport sing other activity?What do you plan to do after high school?

How much do you like math reading art music?Motivation: What is important? math reading art music? How hard did you work last year? (effort) math reading art music?

40Perseverance: mistakes are OK, as long as youre learning if a person works harder, he/she can do better when class work gets hard, I just work harder

Mindset: My success is related to my effort, or people cant change.

Getting to Know You survey (Pilot Data: Factors and Scales)

PerseveranceMindsetMotivation (like + effort + perceived importance) - by subject areaMathELA Reading and WritingScienceSocial StudiesArtMusicPhysical EducationTechnology41FIGURE 8

Cross-Sectional Non-Matched Average Factor-Based Scores: Perseverance

Most positive self conceptLowest perceived self conceptPerseverance (Pilot Data 2013-2014)

42FIGURE 9

Cross-Sectional Non-Matched Average Factor-Based Scores: Mindset

Most positive self conceptLowest perceived self conceptMindset (Pilot Data 2013-2014)

43FIGURE 10

Cross-Sectional Non-Matched Average Motivation Scores: Mathematics

Most positive self conceptLowest perceived self conceptMotivation in Math (Pilot Data 2013-2014)

44FIGURE 11

Cross-Sectional Non-Matched Average Motivation Scores: English Language Arts (ELA)

Most positive self conceptLowest perceived self conceptMotivation in ELA (Pilot Data 2013-2014)

45FIGURE 12

Cross-Sectional Non-Matched Average Motivation Scores: Science

Most positive self conceptLowest perceived self conceptMotivation in Science (Pilot Data 2013-2014)

46FIGURE 13

Cross-Sectional Non-Matched Average Motivation Scores: Art

Most positive self conceptLowest perceived self conceptMotivation in Art (Pilot Data 2013-2014)

47FIGURE 14

Cross-Sectional Non-Matched Average Motivation Scores: Music

Most positive self conceptLowest perceived self conceptMotivation in Music (Pilot Data 2013-2014)

48FIGURE 15

Cross-Sectional Non-Matched Average Motivation Scores: Physical Education

Most positive self conceptLowest perceived self conceptMotivation in Physical Education (Pilot Data 2013-2014)

49Factor4567891011121 Perseverance-.4-.4-.1-.30-.20-.2-.12 Mindset00-.2-.5-.6-.5-.4-.1-.43 Music+.10-.2-.3+.2+.3+.7-.1-.44 Science-.4-.2+.20+.2+.2-.1-.4-.45 Art-.1+.1-.2-.1+.4+.4+.5-.3+.36 ELA-.4-.40-.200+.2-.5-.27 Math-.3-.4-.2-.4-.2-.5-.3-.2-.28 Physical Education-.6-.6-.3-.4+.2+.1-.1+.2-.2TABLE 2Average Survey Factor differences for Special Education versus non-Special Education Students by Getting to Know You Scale and Grade Level(significant > .2)

5051

Getting to Know You survey log on page52Getting to Know You survey Part I

White Dolphinthat i want to go to college53Teacher: How much do my students love my subject?

Student: I love coming to school. My favorite subject is _______.

Average Number of Subjects Students Like and Love by Grade Level(with percent of student who dont like any subject and percent of students who love all subjects)GRADE:456789101112Dont Like Any2%1%1%2%4%7%1%6%5%Likes All14%13%5%3%3%1%2%3%3%If students dont like any or most subjects; they are less motivated to come to school or be actively engaged. Should we accept decreasing levels of interest/motivation?

Modify Instructional Methods to Maintain Student MotivationHigh MotivationLowMotivationFall Year 1Fall Year 2Match scores over timeWhat classroom activities increase student motivation or

decrease motivation Year 1Classroom Activities by Teacherby subject area:MathELAScienceSocial StudiesArtMusicPhysical Education54LectureGroup WorkLeaning Walks Presentation Summary (MSCV-SV and Getting to Know You)55MSCS-SV tool with trained staff is developmentally appropriate and needed to supplement whole-school character programs;intervention/prevention of perceived bullying (victim, bully & bully-victims);a protective factor for behavior problems, dropout, depression, suicide;promoting the perception of caring adults, and more positive respect differences, safety and aggression.

Getting to Know You tool with trained staff is expected to promote perception of caring adults; provide data on perseverance, mindset and motivation by subject area;provide data on best instructional practices that increase motivation; andbe a protective factor for behavior problems and dropping out of school.

We can add comprehensive Social-Emotional Support tools (* Free to LEAs)

Confidential School Climate Survey MPS CaresConfidential Getting to Know You survey

Social-Emotional, Perseverance, Mindset and Motivation

Actionable Data for Monitoring and Intervention

Attitudes, beliefs and self concept: latent but measurableMore Pro-Social outputs with less behavioral symptoms:AbsenteeismBullyingDropoutsCrimeSuicide56Pro-Social outputs butsome behavior problemsAbsenteeismBullyingDropoutsCrimeSuicideTaking Action with Social-Emotional Data (Student Success Plan)

Future Education Data Use AcademicandOvert Behavioral ABCDF Scale Scores (academic growth)ReferralsSuspensions* Free, except for vendor technical fees, contact: l.obrien@websolutions.com 203-235-7777ANDChart14.44.44.44.23.93.63.53.4

Sheet1F1Gr 3Gr 4Gr 5Gr 6Gr 7Gr 8Gr 9Gr 10+4.44.44.44.23.93.63.53.4

Chart14.14.24.13.73.73.63.53.23.13.1

Sheet1F1Gr 3Gr 4Gr 5Gr 6Gr 7Gr 8Gr 9Gr 10Gr 11Gr 124.14.24.13.73.73.63.53.23.13.1

Chart14.314.344.374.284.234.073.883.923.763.65

Sheet1F1Gr 3Gr 4Gr 5Gr 6Gr 7Gr 8Gr 9Gr 10Gr 11Gr 124.314.344.374.284.234.073.883.923.763.65

Chart13.94.44.14.644.544.73.94.644.73.94.644.744.64.14.73.94.544.63.94.33.94.63.84.444.73.74.444.63.74.444.7

safetyparentvalfriendsagression

Sheet1F1Gr 3Gr 4Gr 5Gr 6Gr 7Gr 8Gr 9Gr 10Gr 11Gr 12safety3.943.93.943.93.93.83.73.7parentval4.44.54.64.64.64.54.34.44.44.4friends4.14.04.04.04.14.03.94.04.04.0agression4.64.74.74.74.74.64.64.74.64.7

Chart112471

Sheet11234512471