Kathy S. Williams, DirectorKathy A. Krentler, Associate Director
2
Michelle Lopez, Assistant DeanDivision of Undergraduate Studies
Jill Esbenshade, Associate ProfessorDepartment of Sociology
Teresa Cisneros-Donahue, Study Abroad Director College of Business Administration
Do High Impact Activities Really Make an Impact?
What are High Impact Activities?
An investment of time and energy over an extended period that has unusually positive effects on student engagement in educationally purposeful behavior.
(George Kuh, Indiana University)
3
Examples of High-Impact Activities
1st Year Seminars and Experiences Common Intellectual Experiences Learning Communities Writing-Intensive Courses Undergraduate Research International Experiences Service Learning/Community-Based
Learning Internships Capstone Courses and Project
4
The Value of High-Impact Activities (SELF-REPORTED)
On Learning
General Gains
Personal Gains
Practical Gains
Learning CommunitiesService LearningStudy AbroadUndergraduate ResearchCulminating Experiences
5
P < .001
George D. Kuh, High-Impact Educations Practices: What They Are, Who Has Access to Them, and Why They Matter; Washington , DC; AAC&U, 2008.
The Value of High-Impact Activities (As measured at SDSU by
Graduation Rate)STUDY ABROAD1st Time
Freshmen Fall 2005
Studied Abroad
No Study Abroad
4-Year Graduation Rate
32.9% 26.2%
5-Year Graduation Rate
78.0% 51.0%
6-Year Graduation Rate
90.2% 60.7%
6
The Value of High-Impact Activities (As measured at SDSU by
Graduation Rate)INTERNSHIPS1st Time
Freshmen Fall 2005
Internship No Internship
4-Year Graduation Rate
45.5% 24.6%
5-Year Graduation Rate
84.7% 50.4%
6-Year Graduation Rate
93.2% 59.4%
7
But the “next level” question is:What is the impact of HIP
on direct measures of student learning?
What student learning outcomes do we expect from High-Impact Activities?
How can we measure whether these outcomes are occurring?
8
Michelle Lopez
ACADEMIC SCHOLARSHI
PS
Academic/Merit
National Competitive Scholarships Some require endorsement by SDSU
Required 3 – 5 faculty recommendation letters (depending on type)
Essays or Proposals
Require participation in various HIPS
Office of Academic Scholarships (OAS)
Establishing SLOs for the OASFirst, asked a few key questions…
What is the Office of Academic Scholarships trying to do and why?
What is my office supposed to accomplish?
What will I do to promote the kind of learning and development that our campus seeks?
How will student learning be influenced?
What do I want students to be able to do or know as a result of completing the scholarship or fellowship process? 11
Establishing OutcomesThrough interacting with the OAS, students will be able to:
1 Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application
2 Locate and use appropriate resources, to develop scholarship ideas or projects
3 Explore long-term professional goals and connect these goals back to immediate scholarship/fellowship purpose
4 Write, evaluate, and rewrite written materials based on collaborative feedback
5 Interact with faculty, administrators, and other referrees in a timely manner
6 Demonstrate thoroughness and timeliness in completing all scholarship/fellowship applications
7 Translate overall significance of project in laymen's terms12
After drafting outcomes…
Get feedback from others
How do we know that the OAS is contributing to student learning?Where is the content delivered that
will allow the SLOs to be met?How will we measure SLO
achievement?
13
What is being done to directly/indirectly facilitate student learning?
Outcome Info Sessions
IndividualAdvising
FacultyAdviser
LibraryCourse Guide
Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application
X X X
Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application
X X X
Explore long-term professional goals and connect these goals back to immediate scholarship/fellowship purpose
X X
Write, evaluate, and rewrite written materials based on collaborative feedback
X X
14
Importance of Assessing Student LearningGreat reminder to:
Reflect on the OAS mission and goals
Check to see how student learning outcomes align with national fellowship organization or SDSU’s mission and goals
Reflect on how OAS contributes to SDSU priorities
Informs prioritization on time spent, as well as other resources
Discuss how achievement is determined 15
Lived Experience
of Immigrants and RefugeesJill EsbenshadeSociology and Honors
Course Goals To give students an understanding of and an
ability to engage in conversation and debate about immigration and the lives of immigrants . . .
For students to develop a “sociological imagination.” That is for students develop the ability to apply academic concepts and theories to individual lives . . .
For students to improve their writing skills . . . For students to develop their abilities and
desire to use their knowledge to engage in public debate and action to improve their community or the broader society.
Class Structure Background: First Section Historical and Current
Flows (migration patterns) Numbers US policy US in comparative perspective
Focus: Experience and Identity
Disruptions, Changes and Adaptation/Coping in:
Section 2: Class and Labor Section 3: Gender and Family Section 4: Race and Assimilation
“Lived Experience” Sociological Concepts
Applied to Fiction and Non-fiction Movies Internship
Internship: International Rescue Committee Students Plus Program: Crawford High School
ESL Program Homework Tutoring CASHEE Prep Peacemakers College Prep
Higher Learning Navigator
Issues with Internship Agreement with SDSU
Hours (one component of course) Reduced 30 to 20
Scheduling conflicts Civic Action Option
Grade/Evaluation (25% of course) Assessment
Assessment Before Class Discussion
Feedback Session Wrap-Up Session
Papers (4 of 5) Incorporate internship into analysis or as
separate point
Journal Connect to class material
Obstacles to Assessment Connecting internship to class material
Type of interaction at internship Hesitancy to ask Tendency to describe rather than analyze
Strengths of Experiences Clear affect on student outlook:
Open-mindedness Sympathy Judging others
Building relationship Group outings
Current Assessment: Civic Capacities From CSL document (simplified and modified) Broad ideas about growth as citizen (rather
than CSL as service) Conscious development of capacities Journals
Connections with material or capacity Development of capacity
Most Common Capacitiesin Journal Check1. Build a sense of community, locally and
globally Work with people from different
backgrounds Develop curiosity and capacity to listen Cultivate open-mindedness and
appreciation of worth of each person Enhance concern for community and the
wellbeing of others
Also, capacities 2, 4 and 5
Teresa Donahue-Cisneros
STUDY ABROAD
College of Business Administration
Cross Cultural Training for US and EU students (CROCUS)Project
• 2009, Grant by the Department of Education• Required an evaluation plan about student’s
learning • Major specific objectives:
1. Increase participants’ cultural and2. language learning
College of Business Administration
Cross Cultural Training for US and EU students (CROCUS)Project
• 2009, Grant by the Department of Education• Required an evaluation plan about student’s
learning • Major specific objectives:
1. Increase participants’ cultural and2. language learning
College of Business Administration
Student Learning Outcomes of the CROCUS Project
1. Gain knowledge of other cultures.
2. Use knowledge, diverse cultural frames of reference, and alternate perspectives to think critically and solve problems.
3. Become conscious of one’s own cultural perspective.
4. Accept cultural differences and tolerate cultural ambiguity.
5. Improve target language competency in speaking, listening, reading, and writing.
Rubrics developed by American Council on Education (ACE) and Department of Education, FIPSE
College of Business Administration
Opportunities to Learn1. Study abroad experience
2. Online intercultural communications course “Crossing the Bridge”
College of Business Administration
Assessing student learning in the CROCUS Project
1. Used assignments in the “Crossing the Bridge” Course
2. Exit interviews with students
College of Business Administration
Assignments in the CourseStudents were asked to complete written assignments
that demonstrated their accomplishment of the SLOs.
• SLO: “Gain knowledge of other cultures.”• Assignment: “Write a two-page paper (500 words or
more) about your host country. Accumulate facts about the country’s history, political situation, economy, religion, international relations etc. Ensure that you include current events in your paper (elections, economic ties etc.).”
College of Business Administration
Student Interviews• Students completed in-depth interviews
upon return from Study Abroad1. Describe what you learned about yourself?2. Describe what you learned about your host
country?
College of Business Administration
Assignments in the Course & Interviews
• Rubric developed to measure whether assignment demonstrated the SLOs
• Set benchmark – 75% of the students should be able to achieve a Moderate level (score of 3 in the rubric)
• 2 independent raters read each assignment and score it using the rubric.
• Scores reviewed for inter-rater reliability• Determination made as to whether the benchmark was
achieved.
Gain knowledge of other cultures 1 Inadequate
(descriptions
are inaccurate or
poorly developed)
2 Minimal
(Describes
basic points
accurately)
3 Moderate
(Compares and
contrasts perspectives;
uses examples to
illustrate)
4 Extensive
(Content
knowledge is extensive;
analyses are sophisticated)
1. Cultural practices (e.g., religious, secular, political, governmental, educational, family structures.)
College of Business Administration
Do They Know It?Interesting Outcome – Students scored
higher when assessed via interviews than when assessed via course assignments.
CHALLENGE: Achieving cross-measure reliability.
SLO1 SLO2 SLO3 SLO4 Average Coursework Score of Both Reviewers 2.65 2.80 2.75 2.94 Average Interviews Score of Both Reviewers 3.05 3.15 3.05 3.26
College of Business Administration
If they don’t know it, how do we fix the problem?
• Only 60% of students showed evidence of having gained cultural knowledge, below the 75% acceptable benchmark.
• Programmatic Assessment asks not “what’s wrong with these students?” but rather, “how can we strengthen the program to produce a better result?”
College of Business Administration
CLOSING THE LOOP1. Did the program deliver the content needed
to allow students to accomplish the SLOs?• Curriculum mapping can help to answer
this question.
2. How does the program need to change to improve things?
Goal Crossing the Bridge Course
Study Abroad Experience
Pre-Departure Workshop
Re-Entry Workshop
Gain Knowledge of other cultures.
Introduced, Reinforced
Reinforced, Applied Introduced Reinforced
Components of Crosus Program
40
The Cross Cultural Training for US and EU students (CROCUS) Assessment Rubricsi Knowledge Rubric The below items are designed to assess various knowledge areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.
Gain knowledge of other cultures 1 Inadequate
(descriptions
are inaccurate or
poorly developed)
2 Minimal
(Describes
basic points
accurately)
3 Moderate
(Compares
and contrasts
perspectives; uses examples
to illustrate)
4 Extensive
(Content
knowledge is extensive;
analyses are sophisticated)
1. Cultural practices (e.g., religious, secular, political, governmental, educational, family structures.) 2. Cultural beliefs and values
In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.
Become conscious of one’s own cultural perspective 1. Self in cultural context (e.g., aware of one’s own origins, history, ethnic identity, communities, etc.). 2. The history of his or her own culture in relation to the history of other cultures.
In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.
i Adapted from The ACE-FIPSE SPIF/e-Portfolio Assessment Rubric
41
Skills Rubric The below items are designed to assess various skill areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.
Uses knowledge, diverse cultural frames of reference, and alternate perspectives to think critically and solve problems.
1 Inadequate Proficiency
2 Minimal
Proficiency
3 Moderate
Proficiency
4 Extensive
Proficiency 1. Recognizing the importance and validity of others’ perspectives
2. Providing culturally-grounded evidence to make points (e.g., recognizes the cultural underpinning of evidence, opinion, and arguments). In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.
42
Attitude Rubric The below items are designed to assess various attitude areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.
Accepts cultural differences and tolerates cultural ambiguity. 1 Demonstrates
inadequate awareness
of, openness toward, or
engagement with…
2 Demonstrates
minimal awareness
of, openness toward, or
engagement with…
3 Demonstrates
moderate awareness
of, openness toward, or
engagement with…
4 Demonstrates
extensive awareness
of, openness toward, or
engagement with…
1. … the similarities and/or differences among cultures.
2. ... the process of reflecting upon his or her own thoughts and feelings toward different cultures.
3. ... the importance of interpreting cultural events and experiences "through the eyes of" individuals from different cultures. 4. ... the specific ways in which he or she has been changed and/or transformed as a result of cross-cultural experiences. 5. ... his or her own biases, prejudices, or stereotypes in relation to a different culture. In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.
Top Related