Designing student feedback

Using Social Psychology to Li1 Achievement and

Reduce Inequality in the Classroom

Valerie Purdie-‐Vaughns Columbia University

Smart people in key places….

More effective tools for assessing change….

Broadening participation?

•  For people who are members of diverse groups, hidden and overt bias can cause added stress (stress not faced by others) that can undermine performance, mo@va@on, and health.

•  Broadening par@cipa@on requires altering the psychological climate to reduce this stress.

Take Home Message

(National Assessment of Educational Progress, 2012)

Racial Achievement Gap (Reading, 8th Grade) by State, 2011

Darker red = greater gap Grey = no data available

Field-‐based research interven@ons designed by social psychologists to reduce racial and gender opportunity gaps in

academic performance.

•  Partnering with educators to apply social psychology theories of iden@ty to educa@on.

•  Longitudinal-‐experimental interven@ons in middle schools, high schools and colleges.

NSF ADVANCE #0723909, 154685 (Closing achievement gap: A social psychological interven@on) NSF RAPID# 0918075 Tes@ng the effects of the inaugura@on of the first African American President on the affirma@on process NSF ADVANCE #1109548 (Reducing racial achievement gaps: Tes@ng Neurobiological Mechanisms)

Project ACHIEVE

STEREOTYPE THREAT

For people who are members of diverse groups, hidden and overt bias can cause added stress (stress not faced by others) that can undermine performance, mo@va@on, and health.

6

Stereotype Threat

The threat of being viewed through the lens of a nega@ve stereotype or the fear of doing something that would inadvertently confirm that stereotype.

Members of diverse groups can be wary of situa@ons in which their behavior can confirm that their GROUP lacks a valued ability. This extra pressure caused by the concern of reinforcing stereotypes can interfere with performance.

(C.M. Steele & Aronson, 1995))

“Something I deal with constantly, especially considering my educa@on taking place in the North, is how since I came from North Carolina…like I should be a red-‐neck or have a southern drawl and drop out of college. The Southern town I live in is Chapel Hill, one of my liberal college towns in the USA…”

(college student)

Stereotype Threat

Stereotype Threat

“I knew I was just as intelligent as everyone else . . . . For some reason I didn’t score well on tests. Maybe I was just nervous. There’s a lot of pressure on you,

knowing that if you fail, you fail your race.”

Rodney Ellis, State Senator (Texas), 1997

II. Hidden Stress

Female and male college students who care about math take a difficult math test (Sample Math GRE). Test instruc@ons say “no-‐ gender-‐differences” OR standard test instructions.

II. Hidden Stress

(Steele, Quinn, & Spencer, 1997)

Women in the Math and Sciences

0

5

10

15

20

25

30

Control No-Gender-Difference

Scor

e co

rrec

ted

for g

uess

ing

Men

Women

(Spencer, Steele, & Quinn, 1999)

Women in the Math and Sciences

1. Black and white undergraduates take a difficult section of Verbal GRE. 2. Test instructions:

3. Measure test performance: number of questions correct (corrected for guessing, used SAT as covariate)

Test is accurate measure of verbal ability

Test is instrument for solving problems

African-Americans

0

5

10

15

Diagnostic Nondiagnostic

Scor

e co

rrec

ted

for

gues

sing

WhitesBlacks

(Steele & Aronson, 1995)

African-Americans

Female college student test performance with two male test takers.

Female college student test

performance with two female test takers.

Other forms of stereotype threat

White male students in comparison to Asian American male student in math

Other forms of stereotype threat

0

5

10

15

Control No-Age-Difference

Scor

e on

mem

ory

task

youngerolder

(Levy, Slade, Kunkel & Kasl, 2002)

Older adults and memory

Performance: (Dananhar & Crandall; Massey & Fischer, 2005; Reardon, Adeberry,

Arshan, & Kurlander, 2009; Walton & Spencer, 2009 )

Leadership: (Schmader et al., under review)

Health: (Cook, Purdie-‐Vaughns, et al., under review; Logel & Cohen, 2012)

Stereotype threat in the real world

Stereotype threat stems from historic and contemporary threat systemic

in school structures.

“…innate differences between men and women might be one reason fewer women succeed in science and math careers.”

Star Trek poster

Coke cans Sci Fi books

Stereotypical room

Classroom Environments

Cheryan, Plaut, Davies & Steele, 2009

Nature poster

Neutral books Water

bottles

Non-stereotypical room

Classroom Environments


Interaction: F(1, 35) = 10.22, p < .01

Environment influences women’s interest in CS


Contextual

Cogni@ve

Physiological

Beyond Performance: A Multilevel Phenomenon

• Low recruitment of areas associated with learning (inf. PFC, le1 inf. parietal cort., bilateral angular gyrus), over recruitment of areas associated with self-‐regula@on of emo@ons (ventral anterior cingulate cort)

• Increased arousal (cor@sol), higher cardiovascular reac@vity, heightened immune system response (TNFα)

• Impaired execu@ve func@oning, working memory

• Decreased trust/belonging, perceived threat

Neurological

Krendl, Richeson, Kelley, & Heatherton, 2008; Schmader & Johns, 2003; Schmader, Johns, & Forbes, 2008; Blascovich, Spencer, Quinn, & Steele, 2001;

•  Stereotype threat affects any group that is seen “through the lens” of a nega@ve stereotype in a par@cular context.

•  Stereotype threat is a mul@level phenomenon that implicates psychological and biological systems in the human body.

•  Stereotype threat stems from historic and contemporary structural factors that perpetuate stereotypes in classrooms and ins@tu@ons more broadly.

Take Home Message

THE SCIENCE OF BROADENING PARTICIPATION

25

Broadening participation requires altering the psychological climate to reduce this stress.

•  Diagnosing the psychological climate. •  Tes@ng interven@ons that reduce stereotype threat.

•  Providing unambiguous clear feedback of high expecta@ons and iden@ty will not be a barrier to success. (Wise Interven@ons)

•  Providing psychological resources to reduce threat (Values-‐Affirma@on Interven@ons)

•  Capture a moment in @me when psychological climate changed in schools (Obama Effect Interven@ons)

•  Extrac@ng best prac@ces and lessons learned that can be applied na@onwide in schools and workplaces.

The Science of Broadening Participation…

WISE-‐FEEDBACK INTERVENTIONS

27

(Yeager, Purdie-Vaughns, Cohen, & Garcia, under review, Child Development)

How can you be cri@cal of a student’s behavior without undermining the

mo@va@on and self-‐confidence needed to improve?

Today’s proposal: Providing unambiguous clear feedback of high expecta@ons and assurance that student can reach those expecta@ons. This reassures them that their iden@ty will not be a barrier to success.

The Mentor’s Dilemma…

•  Convey to stereotyped individuals that they are seen in their “full humanity,” and not through the lens of a stereotype. (Goffman, 1963; Cohen & Steele, 2002)

“Wise” Interventions

STUDY 1

Original test of wise interven@on strategy on minority student performance in college.

30

(Cohen, Steele, & Ross, 1999)

•  Unbuffered cri@cism •  “Unwise”: Cri@cism + posi@ve buffer:

–  Overall, nice job. Your enthusiasm for your teacher really shows through, … You have some interes@ng ideas in your leper and make some good points. … I’ve provided …. suggested several areas that could be improved.

•  “Wise”: Cri@cism + high standards + Assurance: –  Judged by a higher standard, the one that really counts, that is, whether your leper will be publishable in our journal, I have serious reserva@ons. The comments I provide in the following pages are quite cri@cal but I hope helpful. Remember, I wouldn’t go to the trouble of giving you this feedback if I didn’t think, based on what I’ve read in your leper, that you are capable of mee@ng the higher standard I men@oned.

31

“Wise” Interventions: Laboratory study with college students.

•  Unbuffered cri@cism •  “Unwise”: Cri@cism + posi@ve buffer:

–  Overall, nice job. Your enthusiasm for your teacher really shows through, … You have some interes@ng ideas in your leper and make some good points. … I’ve provided …. suggested several areas that could be improved.

•  “Wise”: Cri+cism + high standards + Assurance: –  Judged by a higher standard, the one that really counts, that is, whether your le>er will be publishable in our journal, I have serious reserva+ons. The comments I provide in the following pages are quite cri+cal but I hope helpful. Remember, I wouldn’t go to the trouble of giving you this feedback if I didn’t think, based on what I’ve read in your le>er, that you are capable of mee+ng the higher standard I men+oned.

32


1

2

3

4

5

6

unbuffered cri@cism cri@cism + posi@ve buffer

cri@cism + high standards and assurance

task

mot

ivat

ion

Whites Blacks

(Cohen, Steele, & Ross, 1999)


STUDIES 1-‐2 Field experiment in middle school.

34

(Yeager, Purdie-‐Vaughns, Cohen, & Garcia, under review, Child Development)

•  7th graders in middle school (may be development “fork in the road”) (Eccles, Lord, & Midgley, 1991; Simmons, Black, & Zhou, 1991)

•  School is economically middle class (23% receive free lunch) and located in Northeastern U.S.

•  Student body approximately 50% African-‐American, 50% White.

Research site information “Wise” Interventions: Field experiment in middle school

•  Black and White 7th grade students wrote an essay about their hero.

•  White teachers grade and give feedback.

–  “Control”

–  “Wise” (High standards + assurance).

“Wise” Interventions: Field experiment in middle school

“Wise feedback” group: I’m giving you these comments because I have high standards and I know that you can meet them.

Control group: I’m giving you these comments so you have feedback on your essay.

62%"

17%"

87%"

72%"

0%"

10%"

20%"

30%"

40%"

50%"

60%"

70%"

80%"

90%"

100%"

White Students" Black Students"

Perc

ent R

evis

ing

Essa

y"

Control "

High Standards + Assurance "

Percentage of students resubmitting their essay at end of week.

Study 2

11.41"

9.42"

12.11" 11.95"

8"

9"

10"

11"

12"

13"

14"

White Students" Black Students"

Fina

l Sco

re o

n R

evis

ed E

ssay" Control "

High Standards + Assurance "

Quality of Final Essay (15 pt. metric)

•  Many minority students encounter or are aware of bias, stereotypes or discrimina@on. These past adversi@es have effects in part because they leave a psychological residue.

•  Raising classroom rigor may not close achievement gaps if they don’t address iden@ty-‐based ambiguity. “Wise” strategies can disabuse students of this ambiguity and unlock mo@va@on

•  However, these depend cri@cally on context: –  That teachers and administrators are trustworthy –  That students are being taught meaningful and rigorous content

Take Home Message

VALUES-‐AFFIRMATION INTERVENTIONS

43

•  Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009, Science; •  Cook, Purdie-Vaughns, Garcia, & Cohen, 2012, Journal of Personality and •  Social Psychology; •  Shnabel, Purdie-Vaughns, Cook, Garcia & Cohen, under revision, 2012, Personality •  and Social Psychology Bulletin; •  Purdie-Vaughns, Cohen, Garcia, Sumner, Cook, & Apfel, 2009, Teacher’s College Record •  Cohen, Purdie-Vaughns, & Garcia, 2012, Stereotype Threat: Theory, Process & Applications

Knowing that psychological climates can impair performance, how can we reduce threat so students can achieve in the face

of these climates?

Today’s proposal: Reduce threat by affirming students’ core sense of self.

Reducing Stereotype Threat…

I don’t drink as much sugary stuff as the person simng next to me...and anyway, I walk to my car EVERY day, which is exercise…AND…the

researchers who study diabetes are paid by medical companies...who can trust them?? Whew, this ad does

not apply to me!

“I do not consume sugary drinks. This ad does not apply to me.”

Here we protect “the self” with strategies that undermine performance (not seeking feedback, disengagement).

Academic contexts are threatening for students who belong to stereotyped groups.

self affirma@on

•  Self-‐affirma@on refers to thoughts or behaviors that bolster one’s sense of self as competent, effec@ve, and able to control important outcomes (Sherman & Cohen, 2006; Steele, 1988)

•  Self-‐affirma@on is not self-‐esteem!

50

Values-affirmation reminds people of sources of their self-worth.

The self-system

Global Self-Integrity

Roles (e.g., student,

parent)

Values (e.g., humor,

religion)

Group identities (e.g., race,

culture, nation)

Central beliefs

(e.g., ideology, political beliefs)

Goals (e.g., health,

academic success)

Relationships (e.g., family,

friends

STUDIES 1-‐3

Do values-‐affirma@on interven@ons improve academic performance for students under threat?

52

•  Structured wri@ng exercises, integrated into middle-‐school

classroom in 7th grade. Developed in conjunc@on with teachers and administrators at school site.

•  Controlled and scripted procedures developed with help of teachers and administrators at each specific site.

•  Administered at periods of high stress. Only treatment

delivered 6 weeks a1er start of middle school.

•  Double-‐blind randomized field experiments.

Operationalizing values-affirmation

[Affirmation Condition Worksheet:]

WHAT ARE YOUR PERSONAL VALUES? The most important values to me are: (circle two or three)

Athletic Ability

Being Good at Art

Creativity

Independence

Living in the Moment

Membership in A Social Group (such as your community, racial group, or school club)

Music

Politics

Relationships with Friends or Family

Religious Values

Sense of Humor

“[Art] is important to me because it makes me feel calm. When I'm very upset, like I'm going to cry I sit down and start listening to music or start drawing a picture.”

“If I didn't have creativity, I'd be bored out of

my mind.” “If I didn't have my family, I [wouldn’t] be

raised right and if I didn't have my friends I would be a boring person. If I didn't have my religion, I wouldn't know what to do, I would be lost.”

Sample excerpts from 7th graders

End-of-term course grade

2

2.5

3

3.5

EuropeanAmericans

AfricanAmericans

ControlAffirmation

Cohen, Garcia, Apfel, & Master, 2006, Science Covariates in analysis of each racial group: prior performance, teacher

End-of-quarter course grade (Social Studies)

2

2.5

3

3.5

EuropeanAmericans

AfricanAmericans

ControlAffirmation

End-of-quarter course grade

(Social Studies)


2

2.5

3

3.5

European Americans African Americans

Control

Affirmation


End-of-quarter course grade

(Social Studies)

Original Two Studies:

2

2.5

3

3.5

EuropeanAmericans

African Americans

ControlAffirmation

2

2.5

3

3.5

EuropeanAmericans

AfricanAmericans

ControlTreatment

2

2.5

3

3.5

EuropeanAmericans

African Americans

ControlAffirmation

Two Replication Studies in Math:

2

2.5

3

3.5

European Americans African Americans

ControlTreatment

Original Two Studies:

2

2.5

3

3.5

EuropeanAmericans

African Americans

ControlAffirmation

2

2.5

3

3.5

EuropeanAmericans

AfricanAmericans

ControlTreatment

2

2.5

3

3.5

EuropeanAmericans

African Americans

ControlAffirmation

Two Replication Studies in Math:

2

2.5

3

3.5

EuropeanAmericans

African Americans

ControlTreatment

0%

1%

2%

3%

4%

5%

6%

7%

8%

0%

1%

2%

3%

4%

5%

6%

7%

% going up in course difficulty

% going down in course difficulty

White, Control

White, Affirm

Minority, Control

Minority, Affirm

Intervention affects contact with academic gateway: Percentage of students moving up versus down in

math level from year 1 to year 2

Sherman & Hartson, 2011

Replication with Latino American students: Global GPA over year

1

1.5

2

2.5

3

3.5

4

Q1 Q2 (es@mate) Q3 Q4 (es@mate)

Grad

e Po

int A

verage

Time point

White -‐ Affirma@on White -‐ Control

La@no -‐ Affirma@on La@no -‐ Control

Sherman & Hartson, 2011

Replication with Latino American students: Global GPA over year

•  Value-‐affirma@ons work by securing self-‐concept under chronic condi@ons of stereotype threat, like middle school, high school and college.

Take Home Message

SYMBOLIC ROLE MODEL INTERVENTION CAN REDUCE THREAT AND IMPROVE PERFORMANCE: THE “OBAMA” EFFECT

69

•  Purdie-Vaughns, Cook, Cohen & Garcia, under review, Personality •  and Social Psychology Bulletin; •  Purdie-Vaughns & Eibach, 2012; reviewed in ”Post-Racial America?” •  Eibach & Purdie-Vaughns, 2010; Dubois Review

Did the elec@on of Obama alter the psychological climate for minority students. If so, can experimental

reminders of Obama reduce threat and improve performance?

Reducing Stereotype Threat…

Obama Effect Field Experiment

•  158 par@cipants in 6th grade

•  Middle school research site in northern United States (CT). School district is economically middle class

(~25% receiving free or reduced school lunch) •  Student body: 49% African American/ 9% La@no American,

42% European American

Purdie-Vaughns, Cook, Garcia & Cohen, in prep

Obama Effect Field Experiment

•  Experiment conducted one week a1er November 2008 elec@on. •  Two experimental condi@ons

–  Obama salient condi@on:

Students answered 12 ques@ons about the importance of the elec@on and Obama

–  Control condi@on:

Students answered 12 ques@ons about the importance of items they put in their locker

Obama Salient Condition (12 total questions. Sample questions)

•  “Is it important to you who won the United States presiden+al elec+on? If so, why?”

•  Who won this year’s United States presiden+al elec+on? (John McCain, Barack Obama, Hillary Clinton etc)

•  I am happy with who won the United States presiden+al elec+on. (1=not at all happy, 4 = very happy)

•  I care about who won the United States presiden+al elec+on (1=not at all, 4 = very much)

Dependent measures

•  Social iden@ty threat (Cohen & Garcia, 2006) e.g., “In school, I worry that people will think I am dumb if I do badly” “Some+mes in school, other people think bad thoughts about how smart my racial group is”

(1-‐strongly disagree to 6-‐strongly agree) (10 items: alpha = .87)

•  Grade point average Official school records in 2nd quarter of school

year Math, English, Social Studies, Science (7 weeks post experiment)

School Year (by quarter)

Quarter 1 Sept.-‐Oct. 2008

Quarter 2 Nov.-‐January

2008

Quarter 3 Feb.-‐April 2009

Quarter 4 April-‐June

2009 Summer

Quarter 1 Sept.-‐Oct. 2009

Social iden@ty threat and grades

Time 1 6th Grade THREAT

Time 1 Quarter 2 GRADES

TIME 2 6th Grade THREAT

TIME 3 7th Grade THREAT

Exp. Manip

.

Students’ attitudes about election and awareness of election

2.00

2.50

3.00

3.50

4.00

Respect for Obama

Following elec@on (school/

home)

*Care about poli@cs

*Na@onal Iden@fica@on

Black

White

* * ns

ns

6th grade students’ attitudes towards Obama and awareness of politics as a function of race.

* = significant racial group difference (p < .05). Means adjusted for gender. Scale ranges from 1 to 4.

Excerpts from Obama condition

Yes, it is because it will show racist stereotypers that Blacks aren't dumb but smart.”

-‐Black student, 6th grade “Yes, Barack Obama has great ideas and can help

our world.” -‐White student, 6th grade

“It's important to me because Barack Obama's win meant that we can do anything no matter what race we are.”

-‐Black student, 6th grade

Iden@ty Threat

Obama interven@on reduces threat among black 6th grade students un@l start of 7th grade.

1.70

1.90

2.10

2.30

2.50

2.70

2.90

Fall 2008 Spring 2009 Fall 2009

Thr

eat

White/Control Condition White/Obama Condition Black/Control Condition Black/Obama Condition

-

+

*

2.6

2.65

2.7

2.75

2.8

2.85

2.9

GPAQ2 with Q1

Obama

Control

Obama intervention significantly improves 2nd quarter grades for Black and White 6th grade students, 7 weeks post intervention.

B

C+

(Math, English, Social Studies, Science) Analysis controls for 1st quarter grades

•  Among African Americans, a group that contends with nega@ve intellectual stereotypes, reflec@ng about President Obama had same protec@ve effects as reflec@ng about the self. European Americans, a group not stereotyped in this context, were unaffected.

Summary of results

THE SCIENCE OF BROADENING PARTICIPATION

84

Broadening participation in STEM and beyond requires altering the psychological climate to reduce this stress.

Ø  Importance of social psychological approach to STEM mo@va@on and

achievement

Ø  Three social psychological interven@ons help li1 achievement and promote equality in the classroom

Ø  Combining social-‐psychological interven@ons with pedagogical and curricular

interven@ons may yield greater benefits

Ø  Understanding the effects of iden@ty threat help explain when and why people from “all walks of life” perform below their poten@al

Concluding Remarks

Thank you.

Valerie Purdie-‐Vaughns [email protected] hdp://www.columbia.edu/cu/

psychology/vpvaughns/people.html

Research Team, Collaborators, and Funders

Acknowledgments Teachers, administrators, and students at our school sites Faculty and student collaborators Julio Garcia; Geoffrey C. Cohen Greg Walton; Jonathan Cook Claude Steele; Lee Ross Chris@ne Logel; David Yeager Research Project Team Nancy Apfel Suzanne Taborsky-‐Barba Nick Camp Jus@n Busch Pam Brzustoski Allison Master Courtney Bearns Natalie Golaszewski Sarah Tomassem Sarah Wert Marie Scully

Research Consultants Edward Zigler Donald Green Edmund Gordon Joseph Mahoney Funding Sources National Science Foundation Spencer Foundation W.T. Grant Foundation American Psychological Association Institute for Social and Policy Studies National Institute of Mental Health Nellie Mae Education Foundation School of Arts & Sciences, Yale University Society for the Psychol. Study of Social Issues

Cogni@ve and Neural mechanisms Threat Response

Biological Marker Behavioral Outcome References

Physiological Arousal

-‐ Skin conductance response (SCR) -‐ Cardiac reac@vity

-‐ Poor performance on hard tests

-‐ Beder per. easy test

Blascovich, Spencer, Quinn, & Steele, 2001; Cassady & Johnson, 2001

Neurobiological (HPA-‐axis)

Cor@sol (hormone mobilizes energy in response to stress)

-‐ Impairs working memory -‐  Increased anxiety

(Dickerson, Gable, et al., 2009; Dickerson, et al.2004;, (Beilock, Rydell, & McConnell, 2007, Schmader & Johns, 2003; Schmader, Johns, & Forbes, 2008

Neurobiological (immune system)

Proinflammatory cytokines (TNFα) (mobilized to fight against biological threats )

Increased immune response from stress (sickness behavior = more sleep, preven@on behaviors)

Schmader, Johns, & Forbes, 2008 (theory)

Brain ac@va@on Failure to increase ac@. of inf. prefrontal cort., lex inf. parietal cort., and bilateral angular gyrus but increased vent. anterior cingulated cort.

-‐  Low recruitment of areas associated with learning

-‐  Over recruit areas associated with self-‐regula@on of emo@ons

Krendl, Richeson, Kelley, & Heatherton, 2008

Transforma@ve public figures, historic characters, and pioneers that

are symbols of special achievement widely expected to inspire others (Eibach & Purdie-‐Vaughns, 2011; Purdie-‐Vaughns, Summner & Cohen, 2010).

Symbolic Firsts

My experiences at Princeton have made me far more aware of my “Blackness” than ever before . . . no matter how liberal and open-minded some of my White professors and classmates try to be toward me, I sometimes feel like a visitor on campus; as if I really don’t belong . . . It often seems as if, to them, I will always be Black first and a student second.

- Michelle Robinson (1985)

Stereotype threat applies to a broad range of identities.

Stereotype Threat

Other research on stereotype threat (2,500+ studies): -‐ Women and math, science, logic tests -‐ Older women and driving performance -‐ White males and math performance -‐ White males and athle@c performance -‐ Older individuals and memory performance -‐ Economically disadvantaged and intelligence tests (France) -‐ Gay males and play behaviors with young children -‐ Whites and conversa@ons about race with Black conversa@on

partners

Situa@ons that trigger stereotype threat: -‐ Being a numerical minority -‐ Reminders of nega@ve stereotype (affirma@ve ac@on dust ups, hate crimes) -‐ Race/ethnicity of instructor

Obama intervention significantly improves 2nd quarter grades for Black and White 6th grade students, 7 weeks post intervention.

0.25

0.75

1.25

1.75

2.25

2.75

3.25

3.75

4.25

1.25 1.75 2.25 2.75 3.25 3.75 4.25

GPA qu

arter 2

GPA from Quarter 1 of 6th Grade

Control Obama Control Obama

Grades (Math, English, Social Studies, Science) Analysis controls for 1st quarter grades

Designing student feedback

Education

Transcript of Designing student feedback