Planning for Assessment

Marta Colón de Toro, SPHR

College of Business Administration

UPR-Mayagüez

2

On Planning…

“Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought.” Bernard M. Baruch 1870-1965, American Financier

“To be prepared is half the victory.”Miguel De Cervantes 1547-1616, Spanish Novelist, Dramatist, Poet

“Before beginning, plan carefully.” Marcus T. Cicero c. 106-43 BC, Great Roman Orator, Politician

3

Purpose of the plan Gives us direction Establishes accountability Helps measure progress Keeps us in track Facilitates execution Increases probability of achieving goals Demonstrates commitment to institutionalize

assessment.

at least…it Should!!!

4

Levels of Assessment of Learning

Institutional Learning Goals

Course Goals

Lesson Objectives

Program Learning Goals

AACSB’s Focus

5

Writing a Program Assessment Plan: Steps 1-5

1. Revisit Vision and Mission Statements2. Express rationale and purpose3. Identify audiences you wish to

address4. Set assessable learning outcomes5. Identify (or design) enabling

experiences

6

6. Select/design assessment instruments and establish performance criteria

7. Set schedule for assessing each goal.

8. Assess learning outcomes

9. Interpret and report results

10. Introduce change strategies

The Plan: Steps 6-10

7

Must take into consideration external and internal reality

Every effort must be based on your program’s mission

Mission must be owned by faculty, students and administrators

Must be revisited periodically

Step #1: Revisit Program’s Vision and Mission

8

Is it aligned with your institution’s mission? Is it inspiring? Does it delineate the scope of academic

programs? Does it define your geographic market? Does it profile your completing students and

faculty? Is it unique?

Step #1: Revisit Program’s Vision and Mission

9

Step #2: Express Rationale and Purpose for Assessment

Rationale: Why are you writing this plan?“As a liberal arts college, we seek to enhance the

personal development of our students.”

Purpose: Who will be assessed? What will you assess? When will assessment takes place?

“The primary purpose of the assessment program is to describe the cognitive and personal development of

our students from matriculation to registration.”

10

Let’s practice Step #2: Answer the following:

Why are you doing this plan? ________________________________________________________________________________________________________________________

What main purpose will it serve?________________________________________________________________________________________________________________________

11

Step #3: Identify Audience(s) This section addresses:

To whom you plan to communicate your assessment results.The message that you want to convey.The media through which you will communicate results.

Example: Who: Prospective StudentsMessage: “We regularly assess students’

performance, and provide them feedback and guidance to improve.”

Media: Program brochure and recruiting materials.

12

Let’s Practice Step #3:Identify at least two audiences…

Audience #1- _______________________

Message - ________________________________________

________________________________________

________________________________________

Media -

__________________________________________________________________________

________________________________________

13

Step #4: Establish Program Learning Outcomes

State the broad educational expectations for each degree program.

General, not specific. general knowledge and skills management-specific learning goals

Specify the intellectual and behavioral competencies a program is intended to instill.

Clarify how we intend for graduates to be different as a result of their completion of the program.

14

Learning Outcomes…

Need to be defined operationally in order to be assessable.

Normally, four to ten learning goals will be specified for each degree program.

Express what students will be able to do when they complete your program requirements.

Must be focused on the student’s learning outcomes

Specify (or suggest) level of learning.

15

Hierarchy of Learning Outcomes

Institutional

College (Dept)

Program

Course

Session

“Apply scientific inquiry methods.”

“Think logically and critically.”

“…basic and applied research.”

“…apply data collection techniques.”

“list steps in the scientific process.”

16

The focus! The student’s learning.

Focused on Teaching Focuses on the

professor’s effort, intent, and interests

or

Focused on Learning Focuses on the student’s

achievement

Example: “Expose the student to recent

research in organizational behavior”.

Example: “After completing this course the

student should be able to evaluate recent research in organizational behavior”.

Objective #4

17

Types of Learning (According to Bloom)

What is to be learned? Cognitive: mental skills (Knowledge)

“…list three major EEO laws…” Affective: growth in feelings or emotional

areas (Attitude) “…value diversity…”

Psychomotor: manual or physical skills (Skills) “… use the keyboard…”

Objective #4

18

Levels of learning…

Know

Comprehend

Apply

Analyze

Synthesize

Evaluate

Define, describe, list, reproduce, enumerate

Classify, explain, discuss, give examples, summarize

Determine, develop, compute, utilize, conduct

Correlate, diagram, distinguish, outline, infer

Adapt, combine, compare, contrast, design, generate

Compare & contrast,

critique, justify

Objective #4

Bloom’s Taxonomy (Cognitive Domain)

19

Dealing with “understand”…

Always ask yourself… How do you know someone understands?

He/she can: Give examples of … Determine the “correct” method to use… Discuss pros and cons… Identify elements in a given case…etc.

20

Let’s Practice Step #4:Write at least 2 Program Learning Outcomes

“Graduates of the Business Administration Program at UPR-___________ will be able to:

1. (Do what?) (At which level?)

2.

21

Step #5: Identify/Design Enabling Experiences

Curriculum Focus on Core Courses

Taken by all students

Extra curriculum Student associations Seminars Guest speakers Forums Internship/coop

22

Program Goal MatrixGoal #2 - “The students will use the appropriate software packages and hardware as a working tool in the daily operations of a business”.

Course/Experience Level of Learning Application of Learning

ADMI 3007 - Intro. to Data Processing

Application

Business related problems

ESTA 3001- 3002 - Business Statistics

Application

Business related problems

CONT 3005-3006 - Elementary Accounting

Application

Accounting problems

Internship/COOP

Application

Real work environment

Computer-assisted courses

Application

Business problems

Assigned projects in several courses.

Application

Objective #4

23

Lets Practice Step #5: Choose 1 Goal and complete the following table:

Goal: ________________________________________

Courses/Experiences that help students achieve this goal

Level of Learning (Bloom’s Taxonomy)

24

Step #6: Select Assessment Instruments and Establish Performance Criteria

For each learning outcome choose an instrument

Entry Level Midpoint Exit Level Assessment Assessment Assessment

Essay included Paper written Senior Essay in admission in 2nd yr. course package

Criteria: 50% 70% 85%

Diagnostic Test Repeat Repeat

Criteria: 44% 70% 85%

25

Program Assessment Instruments

Direct Measures of Learning Diagnostic test to incoming

students on learning goals Course-embedded

assessment Capstone courses Projects Seminar Writing assignments

Internships/Coop Proficiency exams Professional exams Student Portfolio

Indirect Measures of Learning Employer Reports on Alumni Exit Surveys Focus groups Interviews Student reports on

Internships/Coop Employer performance

evaluation of interns/coop Student evaluation of courses

26

Examples of Measures of Achievement

Example 1 School A has defined a learning goal in ethical reasoning for each

of its four undergraduate majors. Student achievement on this goal is relevant to demonstrating satisfaction of Standard 16. The school’s faculty has defined the goal.

Learning Goal “Each student can recognize and analyze ethical problems and

choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing.”

Demonstration of Achievement The school uses course-embedded exercises in three required

introductory-level courses. Faculty in the three disciplines have developed different methods for instructing and assessing achievement toward this learning goal. AACSB

27

In Accounting A two-week module near the end of the introductory

course is devoted to “Ethical standards and fraud in accounting.”

A topic outline has been developed by faculty members to structure an exam on the materials of this module, and a standard set of expectations has been created for grading the exam.

In addition to this exam’s contribution to the course grade, it provides a pass/fail indication on the learning goal.

Examples of Measures of Achievement

AACSB

28

In Human Resource Management Students must provide four written analyses of problem

situations during the course. On three of these analyses (on the topics of selection, reward systems, and job design), students are asked to respond to ethical issues.

A standard scoring key on the ethical component provides evaluation toward the course grade and a pass/fail indication on the learning goal.

Examples of Measures of Achievement

AACSB

29

In Marketing Each student must compose a term paper analyzing a

current national or international marketing campaign. The analysis must include a specified set of components,

and ethical issues that have been presented in lectures are among the required components.

In addition to the overall grade of the paper, each student receives a pass/fail indicator on the ethics component.

Examples of Measures of Achievement

AACSB

30

Let’s practice Step #6:

Choose 1 learning goal and brainstorm with colleagues on assessment instruments that may be used to directly measure student achievement of the goal.

Establish criteria (“passing grade”)

31

Step #7: Create a Calendar for assessing each goal

Learning Goal Year

1

Year

2

Year

3

Year

4

Year

5

Writing X X

Speech X X

Teamwork X

Information skills X X

Ethics X X

32

Let’s Practice Step #7:

Learning Goal Year

1

Year

2

Year

3

Year

4

Year

5

1.

2.

3.

4.

5.

33

Step #8: Assess Learning Outcomes

Conduct assessment according to schedule

Remember you can use samples

34

Step #9:Interpret and Report results

Collect, process and analyze results. Envision sitting before the curriculum

committee with the assessment results in your hand.

Engage them in analyzing and reflecting on the meaning of these results and implications to them.

KISS Descriptive statistics, graphs…

35

Reporting results

Use your plan in Step #3 Audience Message Media

Change the culture!Communicate, communicate,

communicate!

36

Step #10: “Close the loop”Introduce Change to Improve

In curriculum Faculty development New learning experiences

37

Determine the gap and narrow it!

Assessment

Reveals the Gap

Intended Outcomes

1. Act ethically

2. Quantitative Analysis

3. Apply models

4. Decision making skills

5. Communication skills

Actual Outcomes

1.

2.

3.

4.

5.

Objective #1

38

Congratulations!

You did it!

Now, complete it and put it to work!

39

Remember to Assess your assessment plan!

We wish the best!

colon-m@rigel.uprm.edu