Assessment of Student Learning Faculty In-service June 5, 2006.

Assessment of Assessment of Student LearningStudent Learning

Faculty In-serviceFaculty In-service

June 5, 2006June 5, 2006

Goals for the morning:Goals for the morning:

1.1. To gain familiarity with the To gain familiarity with the available national dataavailable national data

2.2. To consider how we are doing To consider how we are doing on our learning objectiveson our learning objectives

Surveying our StudentsSurveying our Students Three surveys conducted by HERIThree surveys conducted by HERI

– Cooperative Institutional Research Project (CIRP)Cooperative Institutional Research Project (CIRP) Annual project; began in 1966Annual project; began in 1966 Nations longest and oldest empirical study of higher ed.Nations longest and oldest empirical study of higher ed. This past year surveyed 263,710 college freshman at 385 This past year surveyed 263,710 college freshman at 385

institutions.institutions.– College Student Survey (CSS)College Student Survey (CSS)

Annual surveyAnnual survey This past year surveyed 30,250 graduating students from This past year surveyed 30,250 graduating students from

116 institutions116 institutions Mirrors CIRP on many itemsMirrors CIRP on many items

– Provides the opportunity to do longitudinal work.Provides the opportunity to do longitudinal work.– FacultyFaculty

Conducted every three years starting in 1989Conducted every three years starting in 1989 421 institutions participated in 2004 for a total of 41,000 421 institutions participated in 2004 for a total of 41,000

faculty.faculty. Focuses more on faculty and less on student learningFocuses more on faculty and less on student learning

Surveying our Students, Surveying our Students, cont.cont.

National Survey of Student Engagement (NSSE)National Survey of Student Engagement (NSSE)– Began in 1999Began in 1999– 348,000 students, 437 institutions (2003)348,000 students, 437 institutions (2003)– Participation cycle – every four yearsParticipation cycle – every four years– Randomly selected sampleRandomly selected sample– Response rate 58%Response rate 58%– Uses student engagement as a proxy for learningUses student engagement as a proxy for learning

Focuses on student behavior and effective educational practices Focuses on student behavior and effective educational practices that result in desired outcomes from college.that result in desired outcomes from college.

Faculty Survey of Student Engagement (FSSE)Faculty Survey of Student Engagement (FSSE)– Began in 2003-04 – we participated – total of 14,000 facultyBegan in 2003-04 – we participated – total of 14,000 faculty– We will participate next year to be in sync with NSSEWe will participate next year to be in sync with NSSE– Measures faculty expectations for student engagementMeasures faculty expectations for student engagement– Importance faculty place on various aspects of learningImportance faculty place on various aspects of learning

Surveying our Students, Surveying our Students, cont.cont.

Collegiate Learning Assessment (CLA)Collegiate Learning Assessment (CLA)– Developed by CAE and RANDDeveloped by CAE and RAND– We participated in the trial runWe participated in the trial run– Institution is the unit of analysisInstitution is the unit of analysis– Looks at critical thinking, analytic reasoning and written Looks at critical thinking, analytic reasoning and written

communicationcommunication– First assessment tool to measure learningFirst assessment tool to measure learning– Most referenced instrument when higher ed. discusses Most referenced instrument when higher ed. discusses

“mandatory assessment”“mandatory assessment” Hardwick DayHardwick Day

– 35 year retrospective alumni survey35 year retrospective alumni survey– Conducted Fall 2005Conducted Fall 2005– For this presentation focuses on graduates since 1990For this presentation focuses on graduates since 1990

In honor of In honor of assessment, assessment,

a quiz will be part of a quiz will be part of the morning’s the morning’s

activitiesactivities

Question 1Question 1

Which do faculty believe is a Which do faculty believe is a more important goal for an more important goal for an undergraduate education……undergraduate education……

a.a. Instill a basic appreciation for Instill a basic appreciation for the liberal artsthe liberal arts

b.b. Master disciplinary knowledgeMaster disciplinary knowledge

Demonstrate knowledge in Demonstrate knowledge in disciplinary field as articulated disciplinary field as articulated

by the departments….by the departments….Will you/do you intend to do a research project with a faculty Will you/do you intend to do a research project with a faculty

member outside of coursework…..member outside of coursework….. Seniors: 23% Seniors: 23% Freshman: 30%Freshman: 30%

To what extent do you structure your courses so that To what extent do you structure your courses so that students acquire job or work related knowledge and students acquire job or work related knowledge and skills….skills….

Faculty: 55%, Quite a bit/Very MuchFaculty: 55%, Quite a bit/Very Much

To what extent has your education contributed to acquiring To what extent has your education contributed to acquiring job or work related knowledge or skills…job or work related knowledge or skills…

Seniors: 72%, Quite a bit/Very MuchSeniors: 72%, Quite a bit/Very Much

Did your major play an important role in being accepted into Did your major play an important role in being accepted into grad school or finding your first job…grad school or finding your first job…

74% of alumni say “Yes”74% of alumni say “Yes”

Integrate liberal arts and Integrate liberal arts and disciplinary knowledgedisciplinary knowledge

How often do you put together ideas/concepts from different How often do you put together ideas/concepts from different courses when completing assignments or during class courses when completing assignments or during class discussion…discussion…

Seniors: 60% Seniors: 60% Freshman: 46%, Quite a Freshman: 46%, Quite a bit/Very muchbit/Very much

How important is it that your students put together How important is it that your students put together ideas/concepts from different classes…. ideas/concepts from different classes….

Faculty: 56%, Important/very importantFaculty: 56%, Important/very important

Faculty believe an important/essential goal for Faculty believe an important/essential goal for undergraduate education is….undergraduate education is….

Master disciplinary knowledge…99%Master disciplinary knowledge…99%Instill basic appreciation for the liberal arts…67%Instill basic appreciation for the liberal arts…67%


Input data was used to predict how Input data was used to predict how students would perform on a task students would perform on a task of critical thinking. How would you of critical thinking. How would you expect our freshman students, expect our freshman students, tested at the end of their first year, tested at the end of their first year, to rank?to rank?

a.a. Out perform expectationsOut perform expectations

b.b. Perform about averagePerform about average

c.c. Perform below expectationsPerform below expectations

Apply knowledge, inquiry, and Apply knowledge, inquiry, and critical thinking skills in critical thinking skills in

problem solvingproblem solvingTo what extent has your education contributed to thinking To what extent has your education contributed to thinking

critically and analytically…critically and analytically…Seniors: 81% Seniors: 81% Freshman: 86%, Quite a bit/Very Freshman: 86%, Quite a bit/Very

muchmuch

To what extent do you structure your courses so students think To what extent do you structure your courses so students think critically and analytically…. Faculty: 96%, Quite a bit/very critically and analytically…. Faculty: 96%, Quite a bit/very muchmuch

To what extent has your education contributed to skills to solve To what extent has your education contributed to skills to solve complex real-world problems… complex real-world problems…

Seniors: 54%Seniors: 54% Freshman: 46%, Quite a bit/Very much Freshman: 46%, Quite a bit/Very much

To what extent do you structure your courses so students learn to To what extent do you structure your courses so students learn to solve complex real world problems . . . solve complex real world problems . . .

Faculty: 63%, Quite a bit/very muchFaculty: 63%, Quite a bit/very much

The CLA indicates our students outperform expectations on a task The CLA indicates our students outperform expectations on a task of critical thinking.of critical thinking.


Among freshman, what percentage Among freshman, what percentage believe it is important to keep up believe it is important to keep up on global affairs?on global affairs?

a.a. 20%20%

b.b. 40%40%

c.c. 60%60%

Demonstrate a global Demonstrate a global perspectiveperspective

Compared to when you started college, your understanding of Compared to when you started college, your understanding of global issues is . . . global issues is . . .

Seniors: 27%, much strongerSeniors: 27%, much stronger

To what extent is it important to improve your understanding of To what extent is it important to improve your understanding of other countries and cultures . . . other countries and cultures . . .

Freshman: 67%, Agree/strongly agreeFreshman: 67%, Agree/strongly agree

To what extent is it important to keep up on global affairs . . . To what extent is it important to keep up on global affairs . . . Freshman: 40%, Agree/strongly agreeFreshman: 40%, Agree/strongly agree

To what extent should racial/ethnic diversity be more strongly To what extent should racial/ethnic diversity be more strongly reflected in the curriculum . . . reflected in the curriculum . . .

Faculty: 76%, Agree/strongly agreeFaculty: 76%, Agree/strongly agree

Question 4Question 4Freshman and seniors were asked how often Freshman and seniors were asked how often

they prepared 2 or more drafts of a paper they prepared 2 or more drafts of a paper before turning the assignment in. Do you before turning the assignment in. Do you think it is . . .think it is . . . a. ~35%a. ~35% b. ~50%b. ~50% c. ~75%c. ~75%

What percentage of faculty believe it is What percentage of faculty believe it is important or very important to prepare important or very important to prepare two or more drafts of a paper before two or more drafts of a paper before turning it in?turning it in?a. ~35% a. ~35% b. ~50% b. ~50% c. ~75%c. ~75%

Communicate effectivelyCommunicate effectivelyTo what extent has your education contributed to the To what extent has your education contributed to the

development of speaking clearly and effectivelydevelopment of speaking clearly and effectivelySeniors: 65%Seniors: 65% Freshman: 54%Freshman: 54%

To what extent do you structure your course so students learn To what extent do you structure your course so students learn to speak clearly and effectivelyto speak clearly and effectively

Faculty: 43%, Quite a bit/Very muchFaculty: 43%, Quite a bit/Very much

In your experience how often have you prepared two (+) In your experience how often have you prepared two (+) drafts of an assignment . . .drafts of an assignment . . .

Seniors: 34%Seniors: 34% Freshman: 34%, Often/very oftenFreshman: 34%, Often/very often

How important is it that students prepare two (+) drafts of an How important is it that students prepare two (+) drafts of an assignment . . . assignment . . .

Faculty: 50%, Very important/importantFaculty: 50%, Very important/important


Freshman and seniors were asked to what Freshman and seniors were asked to what extent their education contributed to extent their education contributed to their ability to work effectively with their ability to work effectively with others. Was it . . . others. Was it . . . a. 30%a. 30% b. 50%b. 50% c. 70%c. 70%

What percentage of faculty structure their What percentage of faculty structure their classes so that students learn/develop classes so that students learn/develop working effectively with others?working effectively with others?a. 40%a. 40% b. 60%b. 60% c. 80%c. 80%

Demonstrate interpersonal Demonstrate interpersonal skills necessary for effective skills necessary for effective

personal and professional personal and professional relationshipsrelationships

Compared to when you started college, your interpersonal Compared to when you started college, your interpersonal skills are. . . skills are. . .

Seniors: 34%, Stronger/much strongerSeniors: 34%, Stronger/much stronger

To what extent has your education contributed to your ability To what extent has your education contributed to your ability to work effectively with others. . . .to work effectively with others. . . .

Seniors: 73%Seniors: 73% Freshman 70%, Quite a bit/very Freshman 70%, Quite a bit/very muchmuch

To what extent do you structure your classes so that students To what extent do you structure your classes so that students learn/develop working effectively with others . . . .learn/develop working effectively with others . . . .

Faculty: 58%, Quite a bit/very muchFaculty: 58%, Quite a bit/very much

Was your college effective at helping you work as part of a Was your college effective at helping you work as part of a team . . . team . . .

Alumni: 75% say “yes”Alumni: 75% say “yes”


Among seniors, what percentage Among seniors, what percentage believe their education at Seattle believe their education at Seattle Pacific contributed to their Pacific contributed to their understanding of people of other understanding of people of other racial and ethnic backgrounds?racial and ethnic backgrounds?

a.a. 35%35%

b.b. 50%50%

c.c. 75%75%

Engage with diverse othersEngage with diverse othersIndicate the importance to you personally to help promote racial Indicate the importance to you personally to help promote racial

understanding… understanding… Seniors: 40% Freshman: 26%, Important/very Seniors: 40% Freshman: 26%, Important/very

importantimportant

To what extent does your institution encourage contact among To what extent does your institution encourage contact among students from different backgrounds… students from different backgrounds…

Seniors: 35% Freshman: 47%, Quite a bit/very muchSeniors: 35% Freshman: 47%, Quite a bit/very much

To what extent do you structure your class so students learn and To what extent do you structure your class so students learn and develop an understanding of people of other racial and ethnic develop an understanding of people of other racial and ethnic backgrounds… backgrounds… Faculty: 35%, Quite a bit/very muchFaculty: 35%, Quite a bit/very much

To what extent has your SPU education contributed to your To what extent has your SPU education contributed to your understanding of other racial and ethnic backgrounds… understanding of other racial and ethnic backgrounds…


How effective was your college experience in helping you develop How effective was your college experience in helping you develop skills that enabled you to relate to people of different skills that enabled you to relate to people of different backgrounds…backgrounds…

Alumni: 59%Alumni: 59%


What percentage of our recent alumni What percentage of our recent alumni (since 1990) believe their SPU (since 1990) believe their SPU education helped them integrate education helped them integrate their faith with other aspects of their faith with other aspects of life?life?

a.a. 30%30%

b.b. 50%50%

c.c. 70%70%

Reflect upon ideas and actions Reflect upon ideas and actions through the lens of Christian through the lens of Christian

faith and ethicsfaith and ethicsTo what extent has your education helped you develop a To what extent has your education helped you develop a

personal code of values and ethics . . . .personal code of values and ethics . . . .Seniors: 72%Seniors: 72% Freshman: 77%, Quite a bit/very muchFreshman: 77%, Quite a bit/very much

To what extent do you structure your courses so that students To what extent do you structure your courses so that students develop a personal code of values and ethics . . . .develop a personal code of values and ethics . . . .

Faculty: 54%, Quite a bit/Very muchFaculty: 54%, Quite a bit/Very much

Do you agree that your college helped you integrate your faith Do you agree that your college helped you integrate your faith with other aspects of your life?with other aspects of your life?

Alumni: 50% say ‘Yes’Alumni: 50% say ‘Yes’

Is it important for you to be free to consider the moral and Is it important for you to be free to consider the moral and ethical side of decisions your make . . . . ethical side of decisions your make . . . .

Alumni: 76%, agreeAlumni: 76%, agree


To what extent has your SPU To what extent has your SPU education contributed to your education contributed to your knowledge, skills and personal knowledge, skills and personal development with regards to voting development with regards to voting in local, state or national elections in local, state or national elections (asked of seniors)?(asked of seniors)?

a.a. 10%10%

b.b. 20%20%

c.c. 30%30%

Balance interests of self, Balance interests of self, others and the community in others and the community in pursuit of the common goodpursuit of the common good

Your college encouraged you to engage the culture in order to Your college encouraged you to engage the culture in order to make a difference in society . . make a difference in society . .


It is important to you to influence the political structure . . . . It is important to you to influence the political structure . . . . Seniors: 22% Seniors: 22% Freshman: 15%, Important/very Freshman: 15%, Important/very

importantimportant

To what extend has your education contributed to your To what extend has your education contributed to your knowledge and skills in voting in local, state, or national knowledge and skills in voting in local, state, or national elections . . . .elections . . . .


Your college was effective was in developing your awareness of Your college was effective was in developing your awareness of world events . . . .world events . . . .


ConclusionConclusion

Consider each of these learning Consider each of these learning objectives. On which ones are we objectives. On which ones are we doing well?doing well?

For which learning objectives do we For which learning objectives do we see the greatest area for see the greatest area for improvement?improvement?

Assessment of Student Learning Faculty In-service June 5, 2006.

Documents

Transcript of Assessment of Student Learning Faculty In-service June 5, 2006.