Assessment of Student Affairs Initiatives for First-Year Students

National Conference on First-Year AssessmentOctober 12-14, 2008San Antonio, Texas

Jennifer R. Keup

Director, National Resource Center for

The First-Year Experience & Students in Transition

keupj@mailbox.sc.edu

Goals for Today

Clarify the relationship between FYE, Student Affairs & Assessment for you

Identify and discuss challenges to FYE assessment in Student Affairs

Review a Comprehensive Assessment Model Discuss Assessment Outcomes Assessment Audit Assessment Approaches Feedback Loops

What is Assessment? “Assessment is any effort to gather,

analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness.”

Effectiveness includes: Student learning outcomes Clientele satisfaction Compliance with professional standards Comparisons with other institutions

Assessment guides good practice

Introductions and a Question

Introduce yourself: Name Institution

Venn Diagram Exercise Identifying the relationships between

FYE, student affairs, & assessment at your institution.

Where do you fit in this model on your campus?

Venn Diagram Exercise:Identify the Relationships

Assessment

The First-Year Experience

Student Affairs

You

Venn Diagram Example:Identify the Relationships

Me

Assessment

StudentAffairs

The First-Year

Experience

What is the assessment process?

Identify expected/desired outcomes. Plan programs and services that

accomplish outcomes. Implement programs and

services. Assess the learning responsive to the

outcomes through sound data collection & interpretation

Use results to improve learning opportunities.

Why is this so hard?

Challenges of First-Year Assessment

in Student Affairs

Barriers to Assessment

Lack of Commitment & Support from Leadership

Lack of Assessment Expertise

Lack of Resources Fiscal: $$$

Human: Professional Staff Support

Fear of Results

Let’s “Talk Turkey”

What are someof the barriers that

are unique toASSESSMENT

of the First-Year Experience?

What are some of the barriers that

are unique to ASSESSMENT

in Student Affairs?

What is really going on for you?

Purposes of FYE Assessment

Backgrounds & characteristics of entering students

Entering students’ needs Entering students’ satisfaction with

orientation & FYE programs Outcomes of orientation & FYE programs Institutional climate for first-year students Comparisons to national standards Cost-effectiveness

Gain Information About:

How do I approach this?

Model of Assessment for First-Year Initiatives in

Student Affairs

Steps for Assessment

1. Define the problem2. Determine the purpose of the

study3. Determine where to get the

information needed4. Determine the best assessment

methods5. Determine whom to study6. Determine how the data will be

collected

Steps for Assessment (cont.)

7. Determine what instruments will be used

8. Determine who should collect the data

9. Determine how the data will be analyzed

10.Determine the implications of the study

11.Report the results effectively

1. Define the problem2. Determine the purpose

of the study

Identifying meaningful and measurable outcomes of Student Affairs Initiatives for First-Year

Students

Identifying Outcomes

What are the questions we need data to answer?

What do we want to know?

How do these answers relate to FYE initiatives in Student

Affairs?

Types of Outcomes Program Outcome:

What your program is accomplishing? What is the degree to which it is being

accomplished? Student Learning Outcomes:

What the students are learning as a result of programs & services?

What is the degree of learning attained? Student Development:

What the students are learning that advances their abilities to perform effectively in a life or work situation?

Examples of Outcomes Persistence Analytical & critical thinking skills Knowledge of university requirements Interpersonal/Communication skills Creation of a sense of community Identity exploration and development Develop educational career

goals/Declare a major Institutional climate

Examples of Outcomes (cont.) Cohort/generation characteristics Multicultural competence Civic engagement/Develop as

democratic citizens Ability to identify, seek, & utilize

organizational resources and student programs

Physical health & emotional wellness Leadership skills Moral and Ethical Development

Writing Outcomes

Be clear about what you are assessing Program Outcome Student Learning/Development Outcome

Make sure that your outcome is meaningful

Clearly state what you want students to know and be able to do Incremental steps, stages, points of

progress Indicate the point that represents

fulfillment/success

Outcomes Exercise

Pick an outcome of a program or service for First-Year Students on your campus that is managed by a Student Affairs unit or department.

Define the outcome in specific terms: What are you assessing? How is it meaningful? What do you want students to be able to

know and do? Key question: Is this outcome

measurable?

3. Determine where to get the information needed

Conducting an audit of current assessment activities of Student Affairs Initiatives for First-Year

Students

What to look for?

Are student data collected? What is being collected? By whom? Where are they housed? Are these data currently being

utilized? If so, how are the results communicated? If not, why not?

What future data collection efforts are planned?

Potential Sources of Data

Admissions/Registrar’s Data Institutional participation in national

surveys of first-year students Program/course evaluations Utilization statistics of SA

services/programs Satisfaction surveys for SA

services/programs

Potential Sources of Data (cont.)

Home-grown IR surveys SA departmental surveys, focus

groups, or student interviews Accreditation reports Grants Strategic planning processes Self-study processes/CAS Other sources?

“Who do I know?” Exercise

Write the name of a

“responsible party” from your campus next to as many of the “Potential

Sources of Data” items as possible.

Who do I talk to? Exercise:

How many were you able to fill in? Who were the key players?

Resources: Student Affairs information, assessment

& research specialist FYE Director (or other equivalent) IR Director VC/VP for Student Affairs Outside agencies

4. Determine the best assessment method

6. Determine how the data will be collected

Assessment methodologies

Methods to Collect Data Quantitative

Describes and analyzes “what is” Examples

Surveys Analysis of existing data

Qualitative Collect data about the meaning of events

& activities to the people involved Examples

Interviews Focus Groups Observations

Methods to Assess Outcomes Indirect

Self-reported comments that reveal a perceived increase in understanding or appreciation

The perception is usually not verified through any demonstration of knowledge acquisition or observation

Direct Demonstration of abilities, information,

knowledge, etc. as the result of participation in a program or utilization of a service

Indirect Assessment Methods Examples

Satisfaction measures Program evaluations Self-ratings of skills Self-assessment of change Agreement with statements Inventories Informal peer-to-peer conversations

Pro/Con of indirect assessment methods

Direct Assessment Methods Examples

Portfolios Capstones Performances Common assignments Exams Observations of behavior/performance Standardized tests Work samples

Pro/Cons of direct assessment methods Making indirect methods direct

10. Determine the implications of the study

11. Report the results effectively

Closing the loop

Implications

Grounded in Steps 1 & 2: Purpose & Problem Have you answered the original

assessment question? What have we learned about our

outcome of interest?

Effective Reporting: Audience Who is your client? Use it as an opportunity to connect

with your list of “responsible parties” for data.

Who should know about this information? If the information can inform practice,

whose practice will it inform? Know the climate and context of your

university Venn Diagram List of Challenges

Effective Reporting: Structure & Dissemination

Reporting formats: Paper Web Presentation

Executive summary Visual identity Leverage information channels that

already exist Responsible parties for data

Student Affairs Program Review