Development of an Assessment Strategy for Global Learning Outcomes at Koç University
description
Transcript of Development of an Assessment Strategy for Global Learning Outcomes at Koç University
Development of an Assessment Strategy for Global Learning Outcomes at Koç
University
PARTI. Become aware of the importance of
assessing global learning outcomesII. Approach to meaning and goals of global
learning and competenciesIII. Discover methodological possibilities with
SamplesIV. Get recommendations
AGENDA
Part I.Become aware of the importance
of assessing global learning outcomes
“It was a life changing experience!”
Assessment is no goal but a means to an end.
Assessment as instrument• See effectiveness of internationalization strategy
General Program Logic Model Applied to InternationalizationSource: Darla Deardorff (2004)
Assessment as instrument• See effectiveness of internationalization strategy
-> Office Assessment• Promote programs -> Marketing• Guide the learner• Evaluate the significance of program type and length• Enhance programs
Our overarching goal:
High Quality Education
Winthrop University – a pionier of assessing global learning outcomes
• Multiple methods • Key component: Questionnaire that measures changes in
global knowledge, attitudes, and engagement- Self developped - Can be administered frequently without additional cost- Students’ responses can be tracked over time through their university ID numbers
• Additional Qualitative Methods:- Student’s evaluation of global peer mentor- Global Learning cultural event reflective papers- Online presentation- Students reflective papers- Assessment of University-Level Competencies etc.
Part II.Approach to meaning and goals
of global learning and competencies
Part II.1.Determine Goals
Define Goals
By knowing where you intend to go, you increase the chances of you and
the learner ending up there.
Define Goals –
5 levels to keep in mind
Establishing Global Learning GoalsSource: Association of American Colleges and Universities (2006): Assessing Global Learning – Matching Good Intentions with Good Practice
Sample 1: Definition of Global Learning Outcomes at Webster University
Sample 2: Definition of Global Learning Outcomes at Winthrop University
Sample 3: Proposal of the Association of American Colleges and Universities
Part II.2.Define
Competences
“[...]competence can be measured. But its measurement depends first on its definition.”
Klemp, 1979
Define Competences
Use definitions that already exist Subdivide Integrate levels Define Indicators
Sample 1: Proposal of the Association of American Colleges and Universities (2009) – Definition of expected competences
Sample 1: Proposal of the Association of American Colleges and Universities (2009) – Definition of expected competences
Sense of global interconnections and interdependencies
Consider individuals intervention in a global social problem possible
Ability to describe a social problemAbility to identify some of the ethical
and moral questions that underlie a given transaction between countries
Development of greater courage to engage in social exchanges and enterprises, even when faced with radical cultural difference
Identification of obligations to people situated both inside and outside their own national borders
Sample 2: Definition of Global Learning Outcomes at Webster University
Intercultural Competence
Cultural Self-Awareness (Knowledge)Cultural Worldview Frameworks (Knowlegde)Empathy (Skills)Verbal and Nonverbal Communication (Skills)Curiosity (Attitudes)Openness (Attitudes)
Sample 2: Definition of Global Learning Outcomes at Webster University
Cultural Self-Awareness (Knowledge)Cultural Worldview Frameworks (Knowlegde)Empathy (Skills)Verbal and Nonverbal Communication (Skills)Curiosity (Attitudes)Openness (Attitudes)
BeginningDevelopingProficientExemplary
Intercultural Competence
Sample 2: Definition of Global Learning Outcomes at Webster University
Intercultural Development Continuumby M. Hammer, IDI & Milton Bennett (1993)
Part III.Methodological
Possibilities
• Developed by intercultural experts with a developmental focus
• Paper based quantitative assessment tool: pre-posttest
• Independently reviewed and tested• Strong reliability & validity• Available in several languages• Designed in 2010 and already used by over 150
Institutions and Universities
Global Perspective Inventory
Global Perspective Inventory
Abilene Christian University, Agnes Scott College, Alderson-Broaddus College, American Institute for Foreign Study(AIFS), Anderson University, Appalachian State University, Arkansas State University, Auburn University, Augstana College, Illinois, Baldwin Wallace College, Banard College, Bellarmine University, Bethany College (WV), Bethel University (MN), Boise State University, Brevard College, Brigham Young University, Brown University, Buena Vista University (IA), Campbellsville College, Carroll University (WI), Carson Newman College, CEA Global Education, Center for Cross-Cultural Study (Spanish Studies), Central College (IA), Central College Abroad, Central Lakes College (MN), College of the Mainland, College of Wooster, Colombia College (IL), Colorado State University, Concordia College (MN), Davis and Elkins College, Delta University, Des Moines University, Drake University, Duke University, Elgin Community College, Elmhurst College, Elon University, Emory University, Emory and Henry College, Fairfield University, Ferrum University, Florida International University, Florida State University, Fordham University, Global Citizen Year, Grove City College, Guilford College, Gustavus Adolphus College, Hampton University, Hartwick College, Hobart and William Smith College, Hope College, Illinois State University, Indiana University, Indiana Wesleyan University, Johnson University, Kalamazoo College, King College, LaFayette College, Lee University, Lees McRae College, Lenoir Rhyne College, Lesley University, Lincoln Memorial University, Lindenwood College, Lindsey Wilson College, Loyola University Chicago, Lynn University, Macalester College, Marquette University, Marshall University, McDaniel College, Menominee Nation College, Mercer University, Miami Regional University (OH), Michigan State University, Mississippi State University, Milligan College, Monterrey Technological University (Mexico), New York University, North Carolina State University, Northeastern University, Ohio Valley University, Oregon University System, Pacific Lutheran University, Peninsula College, Pitzer College, Portland State University, Queens University, Randolph Macon College, Regent University, Santa Barbara City College, Santa Fe College, Semester at Sea, Seminole State College, Shenandoah University, Smith College, South Central University, St Ambrose College, St Augustine College, St Bonaventure University, St Cloud State College, St Edwards University, St Lawrence University, St Louis Community College, St Mary's College, SUNY Oswego, Susquehanna University, The Chicago School of Professional Psychology, Texas A & M University, Texas Woman's University, Tulane University, Tusculum College, University of Charleston (WV), University of Cumberlands, University of Georgia, University of Illinois Chicago, University of Maryland, University of Michigan, University of North Carolina, Chapel Hill, University of Northern Iowa, University of Notre Dame, University of Pikeville, University of Pittsburgh, University of San Diego, University of South Carolina Upstate, University of Southern Indiana, University of St Thomas (MN), University of Tennessee, Knoxville, University of Texas, San Antonio, University of Texas, Tyler, University of Wisconsin, Eau Claire, University of Wisconsin, La Crosse, University of Wisconsin, Milwaukee, University of Wisconsin, Platteville, Virginia Intermont College, Viterbo University, Walsh University, Washington University, Westminster College (UT), West Virginia Wesleyan College, Wofford College, Yale University
Abilene Christian University, Agnes Scott College, Alderson-Broaddus College, American Institute for Foreign Study(AIFS), Anderson University, Appalachian State University, Arkansas State University, Auburn University, Augstana College, Illinois, Baldwin Wallace College, Banard College, Bellarmine University, Bethany College (WV), Bethel University (MN), Boise State University, Brevard College, Brigham Young University, Brown University, Buena Vista University (IA), Campbellsville College, Carroll University (WI), Carson Newman College, CEA Global Education, Center for Cross-Cultural Study (Spanish Studies), Central College (IA), Central College Abroad, Central Lakes College (MN), College of the Mainland, College of Wooster, Colombia College (IL), Colorado State University, Concordia College (MN), Davis and Elkins College, Delta University, Des Moines University, Drake University, Duke University, Elgin Community College, Elmhurst College, Elon University, Emory University, Emory and Henry College, Fairfield University, Ferrum University, Florida International University, Florida State University, Fordham University, Global Citizen Year, Grove City College, Guilford College, Gustavus Adolphus College, Hampton University, Hartwick College, Hobart and William Smith College, Hope College, Illinois State University, Indiana University, Indiana Wesleyan University, Johnson University, Kalamazoo College, King College, LaFayette College, Lee University, Lees McRae College, Lenoir Rhyne College, Lesley University, Lincoln Memorial University, Lindenwood College, Lindsey Wilson College, Loyola University Chicago, Lynn University, Macalester College, Marquette University, Marshall University, McDaniel College, Menominee Nation College, Mercer University, Miami Regional University (OH), Michigan State University, Mississippi State University, Milligan College, Monterrey Technological University (Mexico), New York University, North Carolina State University, Northeastern University, Ohio Valley University, Oregon University System, Pacific Lutheran University, Peninsula College, Pitzer College, Portland State University, Queens University, Randolph Macon College, Regent University, Santa Barbara City College, Santa Fe College, Semester at Sea, Seminole State College, Shenandoah University, Smith College, South Central University, St Ambrose College, St Augustine College, St Bonaventure University, St Cloud State College, St Edwards University, St Lawrence University, St Louis Community College, St Mary's College, SUNY Oswego, Susquehanna University, The Chicago School of Professional Psychology, Texas A & M University, Texas Woman's University, Tulane University, Tusculum College, University of Charleston (WV), University of Cumberlands, University of Georgia, University of Illinois Chicago, University of Maryland, University of Michigan, University of North Carolina, Chapel Hill, University of Northern Iowa, University of Notre Dame, University of Pikeville, University of Pittsburgh, University of San Diego, University of South Carolina Upstate, University of Southern Indiana, University of St Thomas (MN), University of Tennessee, Knoxville, University of Texas, San Antonio, University of Texas, Tyler, University of Wisconsin, Eau Claire, University of Wisconsin, La Crosse, University of Wisconsin, Milwaukee, University of Wisconsin, Platteville, Virginia Intermont College, Viterbo University, Walsh University, Washington University, Westminster College (UT), West Virginia Wesleyan College, Wofford College, Yale University.
Florida State University – GPI Pretest-Posttest
Florida State University – GPI Pretest-Posttest
Some people have a culture and others do not.
The role of student is to receive knowledge from authority figures.
Much more Quantitative Instruments to Assess Learning Outcomes
• Global Competency Inventory (GCI), Kozai Group • Intercultural Effectiveness Scale (IES), Kozai Group • The International Profiler (TIP), WorldWork • Cultural Perspectives Questionnaire (CPQ),
International Institute for Management Development
• Self-Assessment for Global Endeavors (SAGE), Caligiuri and Associates
• ...
Photo ContestCategory Description
Knowledge In what ways do you better understand the country’s people, history, religion, politics, or culture?
FlexibilityIn what ways did you integrate into local life?In what ways did you adapt yourself to be comfortable with the locals?
Curiosity In what ways were you shocked, intrigued, or inspired to learn more about the people and the culture?
SkillsIn what ways did you gain new skills, such as resourcefulness, problem-solving, leadership, communcation, and language skills?
Growth
In what ways are you better prepared to be a global citizenship because of studying abroad?In what ways are you different as a result of your experience?
: Categories at TCU
Winner in the category “knowledge”
Quantitative Methods
• Pretest – Posttests• Secondary Data Analysis• Surveys by other campus units
BUT: Only in combination with
qualitative Methods!
Qualitative Methods – Self-Assessment
• Student presentations• Student portfolios• Professor evaluations • Narrative diaries / journals• Essays• Reflection papers
Qualitative Methods – Other Reported Instruments
• In depth student interviews• Faculty observation of students• Case studies• Audio that demonstrates
foreign language competency
Qualitative Methods - Reciprocative
• Focus groups• Dialogues• Workshops• Reflection Courses
Pre-Measurements
Encouraged student awareness of the learning outcomes prior to
participation and during the program.
Even more
... thematic learning communities (or linked courses); service- and community-based learning; experiential learning; vertically integrated curricula with first-year seminars and senior capstones; teaching science through problem-based inquiry; undergraduate research and active, hands-on, collaborative learning; and inquiry-based pedagogies...
Part IV.Recommendations
Step by Step
Koç University’s Mission, Core Purpose, Core Values,
Strategic Goals, and Initiatives
Departamental Mission Statement
Goals
Objectives / Outcomes
MeasuresAnalysis / Results
Recommendations / Improvement
Resource Impact
Following: A Model for Continuous Administrative Unit Improvement of Southwestern University
Developing Methodology -Keep in Mind:
combine qualitative and quantitative
methods
financial, technical and human resource
conditions
involve whole campus community incl.
students
Assessment should be ongoing and
multiple
Students’ individualityPrograms’ duration and type
Outcomes’ components and levelstemporally
Global Learning is a multi dimensional construct!While assessing...
Differentiate!
• Astin; Banta; Cross; El-Khawas; Ewell; Hutching; et al. (1992): Nine principles of good practice for assessing student learning. American Association of Higher Education Assessment Forum. Retrieved 06/04/13 from http://www.fctel.uncc.edu/pedagogy/assessment/9Principles.html
• Braskamp; Braskamp; Merril (2005): Global Perspecives Inventory. In: International Educator May/Jun 2005, p. 26-31.• Braskamp; Braskamp; Merrill (2009): Assessing Progress in Global Learning and Development of Students with Education Abroad Experiences. In: Frontiers: The
Interdisciplinary Journal of Study Abroad, v. 18, p. 101-118. • Bennett, M. (1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.): Education for the intercultural experience, p. 21–71. • Deardorff, Darla K. (2005): A Matter of Logic? In: International Educator, v. 14, p. 26-31. • Deardorff, Darla K. (2006): The Identification and Assessment of Intercultural Competence. In: Journal of Studies in International Education, v. 10, p. 241-266. • Doyle, Dennis (2009): Holistic Assessment and the Study Abroad Experience. In: Frontiers: The Interdisciplinary Journal of Study Abroad, v. 18, p. 143-155.• Dwyer, Mary M. (2004): More Is Better: The Impact of Study Abroad Program Duration. In: Frontiers: The Interdisciplinary Journal of Study Abroad, v. 10, p. 151-163.• Florida State University: Global Learning for Global Citizenship. Florida 2010. Retrieved 06/04/13 from
http://goglobal.fiu.edu/SiteCollectionDocuments/QEP_Report-Final.pdf• Franklin, K. U. (2010). Long-term Career Impact & Professional Applicability. Frontiers: The Interdisciplinary Journal of Study Abroad, v. 19, p. 169-190.• Ingraham; Peterson (2004): Assessing the Impact of Study Abroad on Student Learning at Michigan State University. In: Frontiers: The Interdisciplinary Journal of Study
Abroad, v. 10, p. 83-100.• Kehl; Morris (2007): Differences in global-mindedness between short-term and semester-long study abroad participants at selected private universities. Frontiers: The
Interdisciplinary Journal of Study Abroad, v. 15, p. 67-80.• Klemp, George (1979): Identifying, Measuring and Integrating Competence. In: Pottinger / Goldsmith (Eds.): Defining and Measuring Competence. San Francisco.• Knight, Jane (1997): A shared vision? Stakeholders Perspectives on the Internationalzation of Higher Education in Canada. Journal of Studies of Studies in International
Education in Canada, 1 (1), p. 27-44.• Mandernach, B. J. (2003): Writing Quality Learning Objectives. Retrieved 06/25/13 from http://www.park.edu/Old/cetl2/quicktips/writinglearningobj.html.• McTighe Musil, Caryn (2009): Assessing Global Learning – Matching Good Intentions with Good Practice. Washington.• Sobania; Braskamp: Study Abroad or Study Away: It’s not merely semantics. In: Peer Review, v. 11, no. 4, p. 23-27. • Southwestern University (2011): Administrative Assessment Handbook. Retrieved 06/13/13 from
http://www.southwestern.edu/offices/planning/assessment/support.php• Texas Christian University: Photo Contest. Retrieved 06/21/13 from http://www.studyabroad.tcu.edu/default.asp?pid=572• Van de Berg; Balkcum; Scheid; Whalen (2004): The Georgetown University Consortium Project: A Report at the Halfway Mark. Frontiers: The Interdisciplinary Journal of
Study Abroad, v. 10, p. 101-116.• Williams, Tracy (2009): The reflective model of intercultural competency: a multidimensional, qualitative approach to study abroad assessment. Frontiers: The
Interdisciplinary Journal of Study Abroad, v. 18, p. 289-306.• Webster University: Global Citizenship Program – Rubrics. Retrieved 06/04/13 from http://www2.webster.edu/globalcitizenship/documents/index.shtml• Winthrop University (2011): Quality Enhancement Plan – The Global Learning Initiative. Retrieved 06/12/13 from http://www2.winthrop.edu/gli/wuPublicGli.pdf
References