Fall Faculty WorkshopAugust 21, 2008

Assessment [CLA]

• Why Assessment• Four Approaches to Assessment• The Collegiate Learning Assessment

[CLA]• Using the CLA at Morningside

Why Measure Educational Outcomes?

• We measure what we value, we value what we measure

• Improve quality of student learning by creating a culture of assessment– Informing pedagogy and curricular design– Making assessment integral part of

teaching and learning– Create formative evaluation for the

institution, programs, major

• Enable conversations about academic expectations and standards

CAE –[CLA] slide

Why Measure Educational Outcomes?

• Given accountability pressures, if we don’t initiate, it will be done for us

CAE –[CLA] slide

Four Approaches

• Actuarial indicators (graduation rates, access)

• Quality rankings (US News & World Report)

• Student surveys (NSSE, CIRP)

• Direct measures of student learning [CLA]

CAE –[CLA] slide

Four Approaches: Actuarial Indicators

• What are the admissions test scores of entering students?

• What percent of students graduated?

• How diverse is the student body?

• For example, see U-Can at: http://www.ucan-network.org/searchInstitution.asp

CAE –[CLA] slide content

Four Approaches: Surveys

To what extent has your experience at this institution contributed to your ability to think critically and analytically?

Very Much Quite a Bit Some Very Little

For example: The National Survey of Student Engagement (NSSE) http://www.usatoday.com/news/education/2007-11-04-nsse-how-to_N.htm?loc=interstitialskip

CAE –[CLA] slide content

Four Approaches: Rankings• Question for presidents, provosts and

deans of admission: rate the academic program of other universities, where 1 = marginal and 5 = distinguished.

• Measuring value-added: what percentage of your students did you expect to graduate, and what percentage actually did?

• For example: US News http://colleges.usnews.rankingsandreviews.com/usnews/edu/college/rankings/brief/t1ccbach_mw_brief.php

CAE –[CLA] slide content

Current Approaches: Direct Measures

• Measures that assess what students are actually able to do

For example, The Collegiate Learning Assessment [CLA]

CAE –[CLA] slide

The CLA measures four sets of higher order skills

The CLA measures in a holistic manner:

– Critical thinking

– Analytic reasoning

– Problem solving

– Written communication

CAE –[CLA] slide

CLA Scoring CriteriaCritical Thinking, Analytic Reasoning,

and Problem Solving Skills

• Evaluation of Evidence

• Analysis and Synthesis of Evidence

• Drawing Conclusion

• Acknowledging Alternative Explanations/Viewpoints

CAE –[CLA] slide

CLA Scoring CriteriaWriting Skills

• Presentation

• Development

• Persuasiveness

• Mechanics

• Interest

CAE –[CLA] slide

The CLA gets at some of our educational outcomes:

1. Demonstrate analytic, synthetic, creative, evaluative, and quantitative thinking. +

2. Communicate effectively +3. Behave ethically and responsibly -4. Use knowledge of cultures to enhance

understanding of themselves and others -5. Apply knowledge and skills from multiple, diverse

disciplines and practical experiences to understand complex issues and solve problems ?

6. Exhibit a passion for life-long learning -7. Effect positive change through leadership or

active participation in communities -8. Articulate their own spirituality and values, while

understanding those of others -

There are three types of CLA measures

• Analytic Writing Task: Make-an-Argument

• Analytic Writing Task: Critique-an-Argument

• Performance Task

CAE –[CLA] slide

Analytic Writing Task: “Make-an-Argument”

Writing a persuasive, analytic essay to support a position on

an issueCAE –[CLA] slide

Make-an-Argument

“Government funding would be better spent on preventing crime than in dealing with criminals after the fact.”

Directions: 45 minutes, present your perspective on the issue, using relevant reasons and/or examples to support your views

CAE –[CLA] slide

Government Funding

• Establish a thesis

• Maintain the thesis

• Support the thesis with examples

• Anticipate and counter opposing arguments

CAE –[CLA] slide

Analytic Writing Task: “Critique-an-Argument”

Critiquing Written Arguments

CAE –[CLA] slide

Analytic Writing Task: “Critique-an Argument”

“Butter has now been replaced by margarine in Happy Pancake House restaurants throughout the southwestern United States. Only about 2 percent of customers have complained, indicating that 98 people out of 100 are happy with the change. Furthermore, many servers have reported that a number of customers who still ask for butter do not complain when they are given margarine instead. Clearly, either these customers cannot distinguish margarine from butter, or they use the term "butter" to refer to either butter or margarine. Thus, to avoid the expense of purchasing butter, the Happy Pancake House should extend this cost-saving change to its restaurants in the southeast and northeast as well.”Directions: 30 minutes, discuss what is wrong with the argument

CAE –[CLA] slide

The Happy Pancake House

• Identify logical flaws or fallacies

• How do these logical flaws influence the argument’s conclusion

CAE –[CLA] slide

Performance Task

Analyzing complex, realistic scenarios

CAE –[CLA] slide

Performance Task

Features:Open-EndedDocument-Based“Real Life” Scenarios

Components:Document Library

CAE –[CLA] slide

Crime Reduction “Pat Stone is running for reelection as

the mayor of Jefferson, a city in the state of Columbia. Mayor Stone’s opponent in this contest is Dr. Jamie Eager. Dr. Eager is a member of the Jefferson City Council. You are a consultant to Mayor Stone. Mayor Stone wants to make sure the best policy for reducing crime in Jefferson is identified, so has asked you to analyze the strengths and/or limitations of Dr. Eager’s proposal.”Directions: 90 minutes, use the evidence from the Document Library to answer the following questions.

CAE –[CLA] slide

Performance Task

• Synthesize information from multiple sources

• Recognize conflicting evidence

• Interpret data, tables, figures correctly

• Identify logical fallacies

• Develop conclusions based on available evidence

CAE –[CLA] slide

Crime Reduction

• Student is advising the mayor, who is running for re-election.

• There is an upcoming mayoral debate, for which the student must help the incumbent mayor prepare.

CAE –[CLA] slide

Crime Reduction

• There are two policy approaches for reducing crime:

– Drug education program– Increasing number of police

officers on the streets

CAE –[CLA] slide

Crime Reduction• Documents available to the student

include:– Newspaper article about crime in the

community– Research abstracts about drug education

program– Report about success of a drug education

program in another community– Police report (with table of data) about crime

and drug use in the community– Plots of the relationship between police

offers and crime– Private investigator report about possible

connection between opponent and drug education program

CAE –[CLA] slide

Crime Reduction

• Using these data, what does the student advise that the mayor should do?

CAE –[CLA] slide

Performance Tasks

• Not all quantitative

• Artwork and Technology

CAE –[CLA] slide

At Morningside College• Invited to join the CIC-CLA

consortium– approx 40 colleges– four year commitment

• Will administer the CLA to at least 100 first year students and 100 seniors each year– This year all 300 first year students

• Share data and work with the consortium

CAE –[CLA] slide

CIC-CLA ConsortiumAlaska Pacific University Allegheny College

Aurora University Averett University

Barton College Bethel University (MN)

Cabrini College Carlow University

Charleston Southern Univ. College of Notre Dame (MD)

College of St. Benedict/Saint John’s University

Dominican University Drake University

Franklin Pierce University Hilbert College

Illinois College Indiana Wesleyan Univ.

Jamestown College John Carrol Univ.

Juniata College LaGrange College

Lynchburg College Marion University (WI)

Morningside College Nebraska Wesleyan Univ.

Pace University Pacific University

Seton Hill University Southwestern University (TX)

Springfield College Stephens College

Stonehill College Texas Lutheran University

The College of St. Scholastica University of Findlay

Trinity Christian College University of Charleston

University of Evansville University of Great Falls

Upper Iowa University Ursinus College

Ursiline College Wagner College

Wartburg College Westminster College (MO)

Westminster College (UT) Williamette University

William Woods University

Questions?

Other Options

CLA in the Classroom helps address the “What Now?” question

• The key to accountability is not just transparency, but also improvement

• CLA in the Classroom connects the institution-wide CLA results to the classroom-level and faculty work on student learning

• A means to link institution-wide summative assessment with local, formative work

One component of the program supports diagnostic work

• Use of a retired (“[cla]ssic”) performance task in a classroom, with administration, scoring and advising resources

• In addition, classroom activities are suggested

• Designed to provide diagnostic feedback to students to understand why they achieved the scores they did, and what to do next to improve their skills

Another key component enables curricular efforts

• In addition, faculty walk through a process to develop their own performance tasks

• Faculty can embed course content in the tasks they create

• These faculty-developed tasks could be used as class assignments or group projects

CLA in the Classroom Academy

• Gain a deeper understanding of the CLA• Learn how to administer and score a disclosed CLA

Performance Task• Develop performance tasks where course content can

be embedded• Learn more about rubric-based assessment as it

applies to the CLA• Brainstorm strategies for using these approaches in

courses or across the institutions