1

Assessing for LearningAssessing for LearningPresented by Peggy L. Maki Presented by Peggy L. Maki

June 7, 2004June 7, 2004Seattle Pacific UniversitySeattle Pacific University

PeggyMaki@aol.comPeggyMaki@aol.com

Material from Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing and AAHE.

2

How Do You Learn?How Do You Learn?

______________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

3

Topics:Topics:

Research on Learning That Research on Learning That Informs the Relationship among Informs the Relationship among Teaching, Learning, and Teaching, Learning, and AssessmentAssessment

Curricular and Co-curricular Curricular and Co-curricular Coherence (Maps and Coherence (Maps and Inventories)Inventories)

4

Development of Learning Development of Learning Outcome StatementsOutcome Statements

Alignment of Assessment Alignment of Assessment MethodsMethods

with Learning Outcome with Learning Outcome Statements and Collective Statements and Collective Educational PracticesEducational Practices

5

Terms:Terms: Learning Outcome Statements—Learning Outcome Statements—

sentences that describe what we sentences that describe what we expect our students to expect our students to demonstrate, represent or produce demonstrate, represent or produce as a result of how and what they as a result of how and what they learn.learn.

Coherence—degree to which the Coherence—degree to which the curriculum and co-curriculum curriculum and co-curriculum intentionally offer multiple and intentionally offer multiple and varied opportunities for students to varied opportunities for students to learn what the institution and its learn what the institution and its programs and services assert they programs and services assert they teachteach

6

Maps and Inventories—strategies Maps and Inventories—strategies that enable us to ascertain how well that enable us to ascertain how well we intentionally develop what we we intentionally develop what we assert we expect students to learnassert we expect students to learn

Alignment—degree to which Alignment—degree to which learning outcome statements match learning outcome statements match how and what we teach; degree to how and what we teach; degree to which assessment methods which assessment methods methods match our collective methods match our collective educational practiceseducational practices

7

Research on Learning That Research on Learning That Informs the Relationship Informs the Relationship

among Teaching, Learning, among Teaching, Learning, and Assessmentand Assessment

Learning is a complex process of interpretation--not Learning is a complex process of interpretation--not a linear processa linear process

Learners create meaning as opposed to receive Learners create meaning as opposed to receive meaningmeaning

1.1.

8

Knowledge is socially Knowledge is socially constructed (importance of constructed (importance of

peer-to-peer interactionpeer-to-peer interaction))

People learn differently—prefer People learn differently—prefer certain ways of learning certain ways of learning (learning inventories, such as (learning inventories, such as Kolb or Vark)Kolb or Vark)

9

Deep learning occurs over time—Deep learning occurs over time—transferencetransference

Meta-cognitive processes are a Meta-cognitive processes are a significant means of reinforcing significant means of reinforcing learning (thinking about one’s learning (thinking about one’s thinking)thinking)

10

Learning involves creating Learning involves creating relationships between short-term relationships between short-term

and long-term memoryand long-term memory

Transfer of new knowledge into Transfer of new knowledge into different contexts is important to different contexts is important to deepen understandingdeepen understanding

11

Practice in various contexts Practice in various contexts creates expertisecreates expertise

Surface LearningSurface Learning

Deep LearningDeep Learning

12

Curricular and Co-Curricular Curricular and Co-Curricular CoherenceCoherence

What do you do in your classes or What do you do in your classes or in your programs or services to in your programs or services to promote the kinds of learning or promote the kinds of learning or development that the institution development that the institution seeks?seeks?

What do you expect your students What do you expect your students to know and be able to do by the to know and be able to do by the end of their education at your end of their education at your institution?institution?

13

What do the curricula and What do the curricula and other educational experiences other educational experiences “add up to?”“add up to?”

Which students benefit from Which students benefit from specific teaching strategies or specific teaching strategies or educational experiences?educational experiences?

What educational processes are What educational processes are responsible for the intended responsible for the intended student outcomes the institution student outcomes the institution seeks?seeks?

14

How can you help students make How can you help students make connections between classroom connections between classroom learning and experiences outside learning and experiences outside of the classroom?of the classroom?

What pedagogies/educational What pedagogies/educational experiences develop knowledge, experiences develop knowledge, abilities, habits of mind, ways of abilities, habits of mind, ways of knowing/problem solvingknowing/problem solving??

15

How are curricula, pedagogy, and How are curricula, pedagogy, and educational experiences designed educational experiences designed to develop knowledge, abilities, to develop knowledge, abilities, habits of mind, ways of knowing?habits of mind, ways of knowing?

How do you intentionally build How do you intentionally build upon what each of you teaches or upon what each of you teaches or fosters to achieve programmatic fosters to achieve programmatic and institutional objectivesand institutional objectives??

16

What methods of assessment What methods of assessment capture desired student capture desired student learning--methods that align learning--methods that align with pedagogy, content, and with pedagogy, content, and curricular design?curricular design?

17

When a Student Becomes aWhen a Student Becomes a Biologist, Psychologist,Biologist, Psychologist,

Engineer…..Engineer…..

Cognitive

AffectivePsychomotor

18

Maps and InventoriesMaps and Inventories

Help us determine coherence Help us determine coherence among our educational practices among our educational practices that enable us to design that enable us to design appropriate assessment methodsappropriate assessment methods

Provide a visual representation Provide a visual representation of students’ journeyof students’ journey

19

Help students make meaning of Help students make meaning of the journeythe journey

Help students develop their own Help students develop their own learning map learning map

20

Developing Learning Outcome Developing Learning Outcome StatementsStatements

Institutional Outcomes

Programmatic Outcomes

Course Outcomes

21

What Is an Outcome What Is an Outcome Statement?Statement?

A sentence that describes what A sentence that describes what students should demonstrate, students should demonstrate, represent, or produce based on represent, or produce based on how and what they learn. how and what they learn.

Emerges from what we value and Emerges from what we value and how we teach or students learn; how we teach or students learn; that is, it emerges from our that is, it emerges from our educational practices and is educational practices and is developed through consensus.developed through consensus.

22

Relies on active verbs, such as Relies on active verbs, such as create, compose, calculate, create, compose, calculate, develop, build, evaluate, develop, build, evaluate, translate, etc., that target what translate, etc., that target what we expect students to be able we expect students to be able to demonstrateto demonstrate

Is written for a course, Is written for a course, program, or institutionprogram, or institution

23

Is aligned with curriculum and co-Is aligned with curriculum and co-curriculumcurriculum

--varied and multiple ways to --varied and multiple ways to

learn that desired outcomelearn that desired outcome

Is based on intentionality Is based on intentionality (established through dialogue)(established through dialogue)

24

Helps students understand Helps students understand expectations and holds them expectations and holds them accountable for their learningaccountable for their learning

Can be assessed quantitatively Can be assessed quantitatively or qualitativelyor qualitatively

25

Example from ACRLExample from ACRL

ONE OUTCOME: Student examines ONE OUTCOME: Student examines and compares information from and compares information from various sources in order to evaluate various sources in order to evaluate reliability, validity,accuracy, reliability, validity,accuracy, timeliness, and point of view or bias.timeliness, and point of view or bias.

26

Write one outcome statement Write one outcome statement that captures what a graduate that captures what a graduate

from your institution or a from your institution or a program should be able to program should be able to demonstrate, represent, or demonstrate, represent, or

produce:produce:

27

How well does your outcome How well does your outcome statement meet statement meet

characteristics of a good characteristics of a good

statement?statement?

Describes learning desired Describes learning desired within a contextwithin a context

Relies on active verbsRelies on active verbs

28

Emerges from your collective Emerges from your collective intentions intentions

Can be mapped to curricular and Can be mapped to curricular and

co-curricular practices (multiple co-curricular practices (multiple and varied opportunities to learn and varied opportunities to learn over time)over time)

Can be assessed quantitatively Can be assessed quantitatively or qualitatively during students’ or qualitatively during students’ undergraduate and graduate undergraduate and graduate careerscareers

29

“ “The tasks to which students are The tasks to which students are

asked to respond on an assessment asked to respond on an assessment are not arbitrary. They must be are not arbitrary. They must be carefully designed to provide carefully designed to provide evidence that is linked to the evidence that is linked to the cognitive model of learning and to cognitive model of learning and to support the kinds of inferences and support the kinds of inferences and decisions that will be based on the decisions that will be based on the assessment results.”assessment results.”

National Research Council. National Research Council. Knowing what students know: The science and Knowing what students know: The science and design of educational assessmentdesign of educational assessment . Washington, D.C.: National Academy . Washington, D.C.: National Academy Press, 2001, p. 47.Press, 2001, p. 47.

30

Assumptions UnderlyingTeaching

Actual Practices

Assumptions UnderlyingAssessment Tasks

Actual Tasks

31

Development of Assessment Development of Assessment Methods Methods

Align methods with learning Align methods with learning outcome statements and outcome statements and students’ educational students’ educational experiences (verified through experiences (verified through maps or inventories)maps or inventories)

32

When Will You Seek When Will You Seek EvidenceEvidence??

Formative—along the way?Formative—along the way? For example, to ascertain For example, to ascertain progress or development progress or development

Summative—at the end?Summative—at the end? For example, to ascertain For example, to ascertain mastery level of mastery level of

achievementachievement

33

What Tasks Elicit Learning What Tasks Elicit Learning You Desire?You Desire?

Tasks that require students to Tasks that require students to select among possible answers select among possible answers (multiple choice test)?(multiple choice test)?

Tasks that require students to Tasks that require students to constructconstruct answers (students’ answers (students’ problem-solving and thinking problem-solving and thinking abilities)?abilities)?

34

Some Methods That Some Methods That Provide Direct EvidenceProvide Direct Evidence

Student work samplesStudent work samples

Collections of student work Collections of student work (e.g. Portfolios)(e.g. Portfolios)

Capstone projectsCapstone projects

35

Course-embedded assessment Course-embedded assessment (derive examples; develop (derive examples; develop agreed upon prompt and ask agreed upon prompt and ask students to respond to it in class students to respond to it in class or at a designated time)or at a designated time)

Observations of student Observations of student behaviorbehavior

Internal juried review of student Internal juried review of student projectsprojects

36

External juried review of student External juried review of student projectsprojects

Externally reviewed internshipExternally reviewed internship

Performance on a case Performance on a case study/problemstudy/problem

Performance on case study Performance on case study accompanied with students’ accompanied with students’ analysisanalysis

37

Performance on national Performance on national licensure examinations licensure examinations

Locally developed testsLocally developed tests

Standardized testsStandardized tests

Pre-and post-testsPre-and post-tests

38

Response to critical incidentResponse to critical incident

Essay tests blind scored across Essay tests blind scored across unitsunits

MappingMapping

Learning Logs or JournalsLearning Logs or Journals

Writing, Speaking, Visual Writing, Speaking, Visual RepresentationRepresentation

39

Methods That Do Not Provide Methods That Do Not Provide Direct Evidence but May Be Direct Evidence but May Be

Combined with Other Methods-Combined with Other Methods-Indirect MethodsIndirect Methods

Faculty publications (unless Faculty publications (unless students are involved)students are involved)

Courses selected or elected by Courses selected or elected by studentsstudents

Faculty/Student ratiosFaculty/Student ratios

40

Percentage of students who study Percentage of students who study abroadabroad

Enrollment trendsEnrollment trends

Percentage of students who Percentage of students who graduate within five-six yearsgraduate within five-six years

Diversity of student bodyDiversity of student body

41

Focus group (representative of Focus group (representative of the population)the population)

Interviews (representative of the Interviews (representative of the population)population)

Surveys Surveys

42

Other sources of information that Other sources of information that contribute to your inference contribute to your inference making: NSSE results, grades, making: NSSE results, grades, participation rates or persistence participation rates or persistence in support services, course-in support services, course-taking patterns, majorstaking patterns, majors

43

“What and how students learn depends to a major extent on how they think they will be assessed.”

John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.

44

Works CitedWorks Cited

Biggs, J. (1999).Biggs, J. (1999). Teaching for Quality Learning at Teaching for Quality Learning at University: What The Student DoesUniversity: What The Student Does.. Society for Research Society for Research into Higher Education & Open University Press, 1999, p. into Higher Education & Open University Press, 1999, p. 141.141.

Maki, P. (forthcoming, 2004., May). Maki, P. (forthcoming, 2004., May). Assessing for Learning: Assessing for Learning: Building a Sustainable Commitment Across the Institution. Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus Publishing, LLC, and the American Sterling, VA: Stylus Publishing, LLC, and the American Association for Higher Education.Association for Higher Education.

National Research Council. 2001. National Research Council. 2001. Knowing What Students Knowing What Students Know: The Science and Design of Educational Assessment.Know: The Science and Design of Educational Assessment. Washington, D.C.: National Academy PressWashington, D.C.: National Academy Press