EVALUATING STUDENT LEARNINGcrlt.umich.edu/sites/default/files/CRLT50/poster... · (4) Mixed methods...
1
WHAT IS INTERGROUP DIALOGUE? Intergroup Dialogue peer facilitation as a pedagogical tool fosters: • Critical learning through praxis • Democratic engagement • Social responsibility • Ethics in regards to civil action In Intergroup Dialogue, students who facilitate are guides in the collective learning process: • Co-facilitation creates a relationship of shared power between facilitators and within the group • The environment fosters space for multiple perspectives to co-exist • Critical facilitation teaches students how to make the connections between the personal and the structural • Peer facilitators are able to engage the implications of social identity within their co-facilitator relationship and their group dynamic METHODS FOR SURVEY VALIDATION This project began with the development of a new survey to measure processes and outcomes for undergraduate facilitators of intergroup dialogue courses. We examined previous surveys that were taken by the participants in these courses, as well as the literature on small group peer facilitation. Items were developed with input from the IGR program staff and the course instructors. Additionally, a focus group of undergraduate facilitators pre-tested the scales to insure both validity and cultural sensitivity (Rubin & Babbie, 2005). A pilot pre-test and post-test of 22 Likert scales (137 items) was administered anonymously and online in the Fall of 2010 to a convenience sample of 22 students in the Training Processes for Intergroup Dialogue Facilitation course, and 24 students in the Practicum in Intergroup Dialogue Facilitation course. Data were downloaded into SPSS, Cronbach’s Alpha was calculated for all scales, and paired t-tests were run. Additionally, students submitted final artifacts of learning and summaries of learning, and a sample of these was studied as well for qualitative analysis of outcomes of student learning. Key Demographics • Class Year: 90% juniors and seniors • Gender: 65% were women and 35% were men • Nationality: U.S. citizens 86% (Training) and 67% (Practicum) Foreign born, living permanently in U.S. 0% (Training) and 13% (Practicum) • Race: 55% identified as white/European American, 33% as African American, 15% as Asian American/Pacific Islander, 7% as Latina/o, and 7% as Arab American (students were allowed to select more than one race.) • Religiosity: 33% identified as not at all religious, 15% not very religious, 22% somewhat religious, 30% fairly religious, and 0% very religious Previous Courses: • 95% had previous IGR experience • 51% service learning credit • 93% race/ethnic studies class • 69% gender studies class KEY FINDINGS (1) Several scales evidence good reliability for future research on IGR: • Cognitive Empathy, 5 items measuring perspective-taking (α=.65 to .86 on pre-/and post-tests) • Parallel Empathy, 4 items assessing empathic understanding with different social groups (α=.67 to .80 on pre-/and post-tests) • Emotions in Intergroup Settings, 8 items rating respondents’ feelings in group dialogues (α=.85 to .86 on pre-/and post-tests) • Gender Inequality, 4 items assessing awareness of and structural attribution for social stratification by gender (α=.72 to .89 on pre-/and post-tests) • Blame for Wealth, Borrowed from the work of Joe Feagin, 3 items measuring perceived importance of political power and wealth on social dynamics (α=.70 to .90 on pre-/and post- tests) • Facilitator Effectiveness, 13 items self-assessing peer facilitator’s effectiveness at managing group discussion (α=.79 to .89 on pre-/and post-tests) • Frequency of Action, 9 items asking participants to indicate how frequently they engage in behaviors supporting a diverse society (α=.75 to .80 on pre-/ and post-tests) (2) Four NEW scales have potential for future research. For some implementations of the survey, Cronbach’s alpha was low, which may be due to small sample size. • Religious Inequality Awareness, 4 items assessing awareness of and structural attribution for religious inequality (α=.72 to .89 on pre-/and post-tests) • Identity Awareness, 4 items (α = .39 to .74 on pre-/and post-tests) • Co-facilitator Relationship, 12 items (α = .89 on post-tests) • Sexual Orientation Inequality Awareness, 4 items (α = .30 to .74 on pre-/and post-tests) (3) Several scales have potential for future research but need to be enhanced, because of challenges in assessing student social justice learning. • For example, Allyhood, which measures agreement with 4 statements about the concept, offers mixed responses because it may capture both students’ beliefs and knowledge. (4) Mixed methods are needed to fully assess student learning from courses like IGR. • Student work analysis gave us more insight into student learning. • Themes include students’ development of facilitation skills, personal identity, relationship skills, and understanding of social justice concepts. KEY THEMES FROM STUDENT FINAL PROJECTS Facilitation skills “I developed the ability to trust my co-facilitator despite our different facilitation and work styles” Identity exploration and development “My position as a facilitator enabled me to be a positive role model for my peers of color to instill in some of them a stronger racial consciousness and to connect with others through our experiences of being racial minorities.” Understanding social justice concepts “I gained new ideas about effectively communicating the reality of power and oppression to participants and challenged myself to make similarly creative contributions.” Building interpersonal and intergroup relationship skills, collaboration and action “While in the past I never challenged the perpetuation of stereotypes in public spaces, my doing so has become more commonplace in my day-to-day experience at the university and in the workplace.” REFERENCES: • Chesler, M.A., Kellman-Fritz, J., & Knife-Gould, A. (2003). Training peer facilitators for community service learning leadership. Michigan Journal of Community Service Learning, 9(2): 59-76. • Maxwell, K.E., Nagda, B.A., & Thompson, M.C., Eds. (2011). Facilitating intergroup dialogues: Building bridges, catalyzing change. Sterling, VA; Stylus. • Miles, J.R., & Kivlighan, D.M. (2008). Team cognition in group interventions: The relation between coleaders’ shared mental models and group climate. Group Dynamics, 12(3): 191-209 • Nagda, B. A., Gurin, P., Sorensen, N., & Zúñiga, X. (2009). Evaluating intergroup dialogue: Engaging diversity for personal and social responsibility. Diversity & Democracy, 12(1), 4-6. • Rubin, A., & Babbie, E. (2005). Research methods for social work. Florence, KY: Thomson/Brooks-Cole. ACKNOWLEDGMENTS: We would like to acknowledge the Investigating Student Learning (ISL) Grant Program. ABSTRACT The purpose of this study is to pilot measures that evaluate the learning process and outcomes for undergraduate student peer facilitators who facilitate intergroup dialogue courses with The Program on Intergroup Relations (IGR). We report on the reliability of the instruments designed to measure the extent to which students achieve four learning goals across a two-semester peer facilitation teaching model. We also report on key themes about student learning from students’ final projects. KEY RESEARCH QUESTIONS: To what extent do students in Training and Practicum in Intergroup Dialogue Facilitation courses: • Develop facilitation skills • Examine their social identities • Understand social justice concepts • Build interpersonal and intergroup relationship skills, collaboration and action EVALUATING STUDENT LEARNING In IGR Dialogue FacilitationTraining and Practicum Courses Adrienne Dessel, IGR, LSA and Monita Thompson, IGR, DSA Graduate Student Collaborators: Johanna Masse, School of Education; Patrice French, School of Social Work, and Elisabet Medina, School of Social Work
Transcript of EVALUATING STUDENT LEARNINGcrlt.umich.edu/sites/default/files/CRLT50/poster... · (4) Mixed methods...
WHAT IS INTERGROUP DIALOGUE?Intergroup Dialogue peer facilitation as a pedagogical tool fosters:
• Criticallearningthroughpraxis
• Democraticengagement
• Socialresponsibility
• Ethicsinregardstocivilaction
In Intergroup Dialogue, students who facilitate are guides in the collective learning process:
• Co-facilitationcreatesarelationshipof sharedpowerbetweenfacilitatorsandwithinthegroup• Theenvironmentfostersspaceformultipleperspectivestoco-exist• Criticalfacilitationteachesstudentshowtomaketheconnectionsbetweenthepersonalandthestructural
• Peerfacilitatorsareabletoengagetheimplicationsof socialidentitywithintheirco-facilitatorrelationshipandtheirgroupdynamic
METHODS FOR SURVEY VALIDATIONThisprojectbeganwiththedevelopmentof anewsurveytomeasureprocessesandoutcomesforundergraduatefacilitatorsof intergroupdialoguecourses.Weexaminedprevioussurveysthatweretakenbytheparticipantsinthesecourses,aswellastheliteratureonsmallgrouppeerfacilitation.ItemsweredevelopedwithinputfromtheIGRprogramstaff andthecourseinstructors.Additionally,afocusgroupof undergraduatefacilitatorspre-testedthescalestoinsurebothvalidityandculturalsensitivity(Rubin&Babbie,2005).A pilot pre-test and post-test of 22 Likert scales(137items)wasadministeredanonymouslyandonlineintheFallof 2010toaconveniencesampleof 22studentsintheTrainingProcessesforIntergroupDialogueFacilitationcourse,and24studentsinthePracticuminIntergroupDialogueFacilitationcourse.DataweredownloadedintoSPSS,Cronbach’sAlphawascalculatedforallscales,andpairedt-testswererun.Additionally,studentssubmittedfinalartifactsof learningandsummariesof learning,andasampleof thesewasstudiedaswellforqualitativeanalysisof outcomesof studentlearning.
Key Demographics• ClassYear:90%juniorsandseniors• Gender:65%werewomenand35%weremen• Nationality:U.S.citizens86%(Training)and67%(Practicum)
Foreignborn,livingpermanentlyinU.S.0%(Training)and13%(Practicum)
• Race:55%identifiedaswhite/EuropeanAmerican,33%asAfricanAmerican,15%asAsianAmerican/PacificIslander,7%asLatina/o,and7%asArabAmerican(studentswereallowedtoselectmorethanonerace.)
• Religiosity:33%identifiedasnotatallreligious,15%notveryreligious,22%somewhatreligious,30%fairlyreligious,and0%veryreligious
Previous Courses: • 95%hadpreviousIGRexperience• 51%servicelearningcredit• 93%race/ethnicstudiesclass• 69%genderstudiesclass
KEY FINDINGS(1) Several scales evidence good reliability for future research on IGR:
• CognitiveEmpathy,5itemsmeasuringperspective-taking(α=.65to.86onpre-/andpost-tests)• ParallelEmpathy,4itemsassessingempathicunderstandingwithdifferentsocialgroups(α=.67to.80onpre-/andpost-tests)• EmotionsinIntergroupSettings,8itemsratingrespondents’feelingsingroupdialogues(α=.85to.86onpre-/andpost-tests)• GenderInequality,4itemsassessingawarenessof andstructuralattributionforsocialstratificationbygender(α=.72to.89onpre-/andpost-tests)• BlameforWealth,Borrowedfromtheworkof JoeFeagin,3itemsmeasuringperceivedimportanceof politicalpowerandwealthonsocialdynamics
(α=.70to.90onpre-/andpost-tests)• FacilitatorEffectiveness,13itemsself-assessingpeerfacilitator’seffectivenessatmanaginggroupdiscussion(α=.79to.89onpre-/andpost-tests)• Frequencyof Action,9itemsaskingparticipantstoindicatehowfrequentlytheyengageinbehaviorssupportingadiversesociety(α=.75to.80onpre-/
andpost-tests)
(2) Four NEW scales have potential for future research. For some implementations of the survey, Cronbach’s alpha was low, which may be due to small sample size.
• ReligiousInequalityAwareness,4itemsassessingawarenessof andstructuralattributionforreligiousinequality(α=.72to.89onpre-/andpost-tests)• IdentityAwareness,4items(α=.39to.74onpre-/andpost-tests)• Co-facilitatorRelationship,12items(α=.89onpost-tests)• SexualOrientationInequalityAwareness,4items(α=.30to.74onpre-/andpost-tests)
(3) Several scales have potential for future research but need to be enhanced, because of challenges in assessing student social justice learning.• Forexample,Allyhood,whichmeasuresagreementwith4statementsabouttheconcept,offersmixedresponsesbecauseitmaycapturebothstudents’
beliefsandknowledge.
(4) Mixed methods are needed to fully assess student learning from courses like IGR.
• Studentworkanalysisgaveusmoreinsightintostudentlearning.
• Themesincludestudents’developmentof facilitationskills,personalidentity,relationshipskills,andunderstandingof socialjusticeconcepts.
KEY THEMES FROM STUDENT FINAL PROJECTSFacilitation skills“Idevelopedtheabilitytotrustmyco-facilitatordespiteourdifferentfacilitationandworkstyles”
Identity exploration and development“Mypositionasafacilitatorenabledmetobeapositiverolemodelformypeersofcolortoinstillinsomeofthemastrongerracialconsciousnessandtoconnectwithothersthroughourexperiencesofbeingracialminorities.”
Understanding social justice concepts“Igainednewideasabouteffectivelycommunicatingtherealityofpowerandoppressiontoparticipantsandchallengedmyselftomakesimilarlycreativecontributions.”
Building interpersonal and intergroup relationship skills, collaboration and action“WhileinthepastIneverchallengedtheperpetuationofstereotypesinpublicspaces,mydoingsohasbecomemorecommonplaceinmyday-to-dayexperienceattheuniversityandintheworkplace.”
REFERENCES:• Chesler,M.A.,Kellman-Fritz,J.,&Knife-Gould,A.(2003).Trainingpeerfacilitatorsforcommunityservicelearningleadership.MichiganJournalof CommunityServiceLearning,9(2):59-76.• Maxwell,K.E.,Nagda,B.A.,&Thompson,M.C.,Eds.(2011).Facilitatingintergroupdialogues:Buildingbridges,catalyzingchange.Sterling,VA;Stylus.• Miles,J.R.,&Kivlighan,D.M.(2008).Teamcognitioningroupinterventions:Therelationbetweencoleaders’sharedmentalmodelsandgroupclimate.GroupDynamics,12(3):191-209• Nagda,B.A.,Gurin,P.,Sorensen,N.,&Zúñiga,X.(2009).Evaluatingintergroupdialogue:Engagingdiversityforpersonalandsocialresponsibility.Diversity&Democracy,12(1),4-6.• Rubin,A.,&Babbie,E.(2005).Researchmethodsforsocialwork.Florence,KY:Thomson/Brooks-Cole.
ACKNOWLEDGMENTS:WewouldliketoacknowledgetheInvestigatingStudentLearning(ISL)GrantProgram.
ABSTRACTThepurposeof thisstudyistopilot measuresthatevaluatethelearningprocessandoutcomesforundergraduatestudentpeerfacilitatorswhofacilitateintergroupdialoguecourseswithTheProgramonIntergroupRelations(IGR).Wereportonthereliabilityof theinstrumentsdesignedtomeasuretheextenttowhichstudentsachievefourlearninggoalsacrossatwo-semesterpeerfacilitationteachingmodel.Wealsoreportonkeythemesaboutstudentlearningfromstudents’finalprojects.
KEY RESEARCH QUESTIONS:To what extent do students in Training and Practicum in Intergroup Dialogue Facilitation courses:
• Developfacilitationskills• Examinetheirsocialidentities• Understandsocialjusticeconcepts
• Buildinterpersonalandintergrouprelationshipskills,collaborationandaction
EVALUATING STUDENT LEARNINGInIGRDialogueFacil itationTrainingandPracticumCoursesAdrienne Dessel, IGR, LSA and Monita Thompson, IGR, DSAGraduate Student Collaborators: Johanna Masse, School of Education; Patrice French, School of Social Work, and Elisabet Medina, School of Social Work
