Fostering Global Citizenship in Higher Education Assessment and Evaluation.

Click here to load reader

  • date post

    28-Dec-2015
  • Category

    Documents

  • view

    213
  • download

    0

Embed Size (px)

Transcript of Fostering Global Citizenship in Higher Education Assessment and Evaluation.

  • Fostering Global Citizenship in Higher EducationAssessment and Evaluation

  • Session Agenda

    TopicsSetting the contextDefining competenciesAssessing growth

    ContextsGraduateUndergraduate

  • Objectivesidentify ideas on how to implement assessment & evaluation techniques at the program and degree levelsreview how developing competencies are linked to program design and measurementunderstand the merits and challenges of different methods and timing of data collection; andimplement how to use what has been found in future program design

  • Developments in the academya number of inter-related trends at the macro and mezzo level currently are affecting program design throughout higher education in the US.. Among those trends are:

    An increase in short-term, faculty-led programs to accommodate new majority students, professional school students, and other student groups new to education abroadAn increase in required pre-departure, re-entry, and embedded programsAn increase in home campus or sponsoring agency intervention in the experience to support/direct students learningAn increase in student/parent expectations for student support services that mirror those found in the US, whether or not such services are typically provided to students in the host countryAn increase in education abroad being part of a campus-wide plan for internationalizationAn increasing emphasis on program evaluation, particularly outcomes assessmentAn increase in the emphasis on economic rationales for education abroad, consumerism, and education abroad in its job preparation contextRodman & Merrill, 2008

  • Situating the Changing Study Abroad Program ModelsClassroom-basedTour-basedExperience-basedembeddedgrand tour hybriddirect enrollservice learningservicebranch campusstudy centerworkinternshipsshort-term group excursionindependent studylanguage trainingvolunteerismadventurefield-basedmission work

  • Study Abroad Program Model Development & ICC MaturationClassroom-basedTour-basedExperience-basedembeddedgrand tour hybriddirect enrollservice learningservicebranch campusstudy centerworkinternshipsshort-term group excursionindependent studylanguage trainingvolunteerismadventurefield-based1950s2000s1960s1970s SIETARIntercultural Communications Literature Maturationmission work

  • SIT/PIM Definition of CompetencyDescribe outcomes of the learning process. action-oriented statements tell students what they will be able to do, to know, or be aware of by graduation. related to their professional and personal lives.

  • Competencies are found in the following domains:Knowledge (Cognitive)Theories, concepts, and models the student can discuss

    Skills (Behavioral)What the student can do

    Self-awareness (Affective)The awareness of personal values, attitudes, and attributes, and how they connect the individual to their larger environments.

  • Competency deliveryflow through phases of the program:

    On campus format:Prior experience + on-campus program core courses and educational activities + degree courses + reflective practice phase+ capstone + language and cultural requirement

    Low Residency format:Prior experience + summer residency + online coursework with reflective practice + summer residency + online coursework with reflective practice + capstone + language and cultural requirement

  • Program Competency AreasTheory and Practice of LearningProfessional Development Identity, Culture, and Communication Reflection and Inquiry Leadership Social Change Processes Teams and Organizations Community and Global Awareness Degree specific competencies (e.g., for International Education program design)

  • Community and Global AwarenessGlobal and community awareness is based on our respect for and appreciation of the interconnections of individuals and systems in our environment and on the planet. This awareness includes ones capacity to:

    Identify the issues and their impact on ones own and others communitiesMake informed choices regarding personal community involvement, social justice issues and leadership rolesRecognize the value of multiple perspectives in community and global lifeUnderstand macro level political and social theories on community and global awarenessIdentify the role of conflict transformation, power, and ethicsArticulate the way in which technology affects interconnections of local and global communities

  • 1=necessary2= highly desirable3-nice but not required

    Name of competencyCurrent status of delivery Review core course objectives, campus workshops & activities, etc.Value to mixed degree student group(1,2,3) *belowWhen delivered Other potential places to deliver Measurement 1-LearningTheory and PracticeLearning styles assessment ,all program phases as articulated through students learning plansEarly in Fall termOrientationworkshop2- Professional DevelopmentWorkshops career development, On-going classroom presentations, course and capstone research, Professional contacts with faculty, alumni, staff, Courses-OB1, practitioner inquiry, Training Design RPP8-Community & Global AwarenessCourses- ICC, Social Change, On-campus extra-curricular groupsRPP2Fall 2 course, Spring 1 courseYear longpracticumOrientation workshop, speakers across degrees during the academic year

  • Evaluation is any effort to use assessment evidence to improve individual, departmental, divisional or institutional effectiveness.

    Upcraft, 2003What do we mean by evaluation?

  • Assessment is any effort to gather, analyze and interpret evidence, which describes institutional, divisional, or agency effectiveness.

    Upcraft, 2003What do we mean by assessment?

  • Assessment and Evaluation are tied in with DesignFrom Trochim

  • Gardners 5 evaluation frameworks (1994)Professional judgmentMeasurementDecision-oriented evaluationGoal-free/responsive evaluationAssessment of congruence between performance and objectives

  • Assessment of congruence between performance and objectivesA Formative and Summative Process of: Identifying goals, objectives or standards of performanceIdentifying or developing tools to measure performance;Comparing measurement data collected with previously identified objectives or standards.(Gardner, 1994)

  • Assessment of congruence between performance and objectives

  • MethodsChoice depends on:Scope of assessmentValues or paradigm (you, the institution)

  • Scope of the Assessment

  • Institutional ContextWho are the decision makers inside and outside of your program or institution? What do they value?

  • Challenges of assessment and evaluationConflicting needs and valuesRange of methodological approaches Criteria ambiguity or disagreement Difficulty in attributing changeOverzealous desire to measureComplexityControversy Clarity

  • Best Practices of Assessment and Evaluation

    Involve all stakeholders at all stages.Determine individual meaning of assessment elementsClarify purpose, timing, uses and appropriate approaches Identify possible intervening factors Triangulate methods and participants. Work ethically Sort out measurability

  • Group DiscussionApplicationQuestions

    **So well focus mostly on evaluation but understanding that:The process of gathering evidence is assessment.It fits into a more holistic process called program review.IHANDOUT:NDIVIDUAL:How well am I (is the learner) meeting my (her/his) learning objectives?What kind of assessment? Criterion-based (compare specific criteria)Norm-referenced (compare other students)Ipsative (individual over time)

    TOOLS: Refer to Johnson & Prus (1994), A critical review of student assessment options. Also peer feedback, journals, other forms of narrative assessment.

    How well do learners meet course objectives?What does a course teach?What student needs are (not) being met?

    TOOLS: Interviews, observations, student evaluations.PROGRAMWhat are the program objectives and how well are they being met?What are learner objectives and how well are they being met?

    TOOLS: Cumulative of variety of course evaluations; budgets; reflections of participants; grades; outcomes (variety of methods): interviews, surveys, assessment instruments.On an international, national, regional or local level (however we define ourselves), how well do we do what we say we are doing? How do the individuals and bodies outside of the institution the ones that directly affect our collective life view us?

    TOOLS: Environmental scan, accreditation review, enrollments and budgets

    Pictuire of SITHuman vs financial needsConflicting needs and values, paradigms, approaches, meanings, and priorities among stakeholders.Range of methodological approaches, each with its proponents none clearly perfect.Ambiguity or disagreement regarding criteria to judge the merit of what has been assessed.Difficulty in attributing change as a result of educational processes.Conflicting needs, values, paradigms, approaches, meanings, and priorities among stakeholders.Range of methodological approaches, each with its proponents none clearly perfect.Ambiguity or disagreement regarding criteria to judge the merit of what has been assessed.Difficulty in attributing change as a result of educational processes.Short-term vs long-term solutiosn and approachesHandout explaining these