Webinar Series – Session 2

Outcome -based Learning Opportunities

Michael Derntl 1, Jad Najjar 2 and Susanne Neumann 1

1 University of Vienna, Austria2 WU Vienna, Austria

Webinar overview

� Background and motivation for outcome-based education

� Formulating learning outcomes� Learning outcome based teaching methods and learning

designs

� Models and standards for learning outcomes and opportunities

� Learning outcome motivated assessment

Using Adobe Connect

Using Adobe Connect

Why outcome-oriented teaching and learning?

� The Bologna Process� quality assurance in higher education� a unified educational system (mobility, transparency and mutual

recognition of qualification)

� Revise curricula � integrate student centeredness and learning outcome orientation

� Pedagogical issues became central: Alignment of teaching and assessment methods in accordance with learning outcomes

Background and motivation

What is your background?

� Please shortly state in the chat:

1. Your background. (e.g. higher education, PhD student, vocational training, consulting, instructional design, other…)

2. Your position or relation to teaching (if any).

3. Subject(s) you teach / PhD topic.

Background and motivation

Learning outcomes

A learning outcome is a statement of what a learner is expected to know, understand or be able to do as a result of a learning process ―European Qualification Framework for Lifelong Learning (EQF)

ALIGNMENTLearning outcomes

Teaching methods

Workload (ECTS)

Assessment

Formulating learning outcomes

Types of learning outcomes

Formulating learning outcomes

KnowledgeThe outcome of the assimilation of information through learning.

SkillsThe ability to apply knowledge and use know-how to complete tasks and solve problems.

Competence (personal and professional) The proven ability to use knowledge, skills and personal, social and/or methodological abilities.

proven abilityto use

abilityto apply

Personal, social and/or methodological

abilities

Learning outcomes

Intended vs. achieved learning outcomes

The teacher defines the intended learning outcomes for a learning opportunityThe teacher assesses and learners demonstrate the achievement of intended learning outcomes

Formulating learning outcomes

enumerate the most influential politicians of the 20th century

Sentence structure of intended learning outcomes

An intended learning outcome is formulated as a sentence that follows a basic structure:

Learners are able to Verb Object

Describes intended (cognitive) process

Describes what students are expected to acquire

or construct

Formulating learning outcomes

Example:

Learners are able to

What type of learning outcome is this?

Examples of learning outcomes and their type

Students are able to …

… explain the seven layers of the ISO/OSI Reference Model

… enumerate the most influential politicians of the 20th century

… design an interactive website that complies with Web Accessibility Guidelines

… filter out relevant literature for a given research problem.

… guide their team mates in improving the architectural model of a system.

… moderate a virtual task-force meeting

Formulating learning outcomes

Knowledge

Skills

Competence

Guidelines

1. Students are taught about software design flaws

2. Students are able to identify critical design flaws in a given software system

→ Focus on the learning rather than the teaching!

3. Students complete Unit 1 on basics of academic writing

4. Students are able to explain the basic principles of academic writing

→ Focus on the outcome, not the process!

5. Students are able to give a good presentation about their semester project

6. Students are able to convey the essential outcomes of their semester project in a five-minute slideshow presentation

→ Make sure that achievement can be assessed!

Formulating learning outcomes

Activity

Formulate and write in the chat:� If you’re a (PhD) student : a learning outcome of the program or

course you‘re taking.� If you’re a teacher/tutor : a learning outcome of a course you’re

teaching.� Indicate in parentheses whether the outcome is related to:

� knowledge (K)� skills (S) or � competence (C)

Formulating learning outcomes

Example:Students are able to explain the basic principles of academic writing (K)

Problem outline

Alignment of outcomes, methods, and workload� Here: focus on aligning teaching methods with learning outcomes� Goal: facilitating learners in achieving the intended outcomes

Teaching methods and learning designs

ALIGNMENTLearning outcomes

Teaching methods

Workload (ECTS)

Assessment

Learning outcomes

Teaching methods

ICOPER concept model

Key concepts

Sharable Educational Resources

Learning Outcome

Learning Content

TEACHING METHOD

intended result of

involves

Stored inthe OICS

imple-ments

Learning Opportunity

LEARNING DESIGN

INSTRUCTIONAL MODEL

is a

is a

Assessment Method

is a

imple-ments

instantiation of

uses

definesroles

Learning Facilitator

Learner

is a

Assessment Opportunity

includes

Assessment Design

is a

instantiationof

uses

Examples of Teaching Methods 1/5

Lecture : The teacher is the primary delivery channel for instructional content – often through presentation.

Teaching methods and learning designs

Examples of Teaching Methods 2/5

One-Minute Paper : Learners quickly write thoughts to make their level of understanding explicit.

Teaching methods and learning designs

Examples of Teaching Methods 3/5

Feedback : Reinforce or correct actions performed by learners.

Teaching methods and learning designs

Examples of Teaching Methods 4/5

Think-Pair-Share : To break long presentations, learners first think about a question individually, then discuss their thoughts with a partner, and lastly report back to the teacher.

Teaching methods and learning designs

Role Play : is a “dramatized case-study”. Its purpose is to have members of a group act out roles surrounding a situation, condition, or circumstance to make different perspectives on the issue visible.

Examples of Teaching Methods 5/5

Teaching methods and learning designs

Description of instructional models

ICOPER researched description elements that practitioners require for browsing, selecting and implementing instructional models.� General description: title, summary, rationale, subject, learning

outcomes, learner characteristics, group size, duration, setting, author/licensing

� Detailed description: graphical representation, sequence of activities, roles, assessment method, resources, variations, implementations, references

� Comments: teacher reflections, student feedback, peer review

This allows for a structured human-readable description

Teaching methods and learning designs

Description of instructional models

To describe instructional models in an interoperable machine-readable format, there are several existing standards and specifications, e.g.:� IMS Learning Design (LD)� SCORM� IMS Simple Sequencing (SS)

In ICOPER: Instructional models described with IMS LD (other formats allowed), with LOM metadata record(s) on topStorage in the Open ICOPER Content Space (OICS)� for OICS/LOM introduction see the replay of Webinar 1; for IMS LD the upcoming Webinar 3; for application demos Webinar 5.

Teaching methods and learning designs

Alignment of Learning Outcomes and Assessment

Teaching methods and learning designs

ALIGNMENTLearning outcomes

Teaching methods

Workload (ECTS)

Assessment

Alignment of Learning Outcomes and Assessment

Learning outcome:

Students are able to moderate a virtual task-force meeting.

Aligned assessment: A tutor observes a task-force meeting where a student moderates, and gives feedback to the student based on pre-defined performance criteria.

Misaligned assessment:Students take a multiple-choice test on how to moderate meetings.

Teaching methods and learning designs

Alignment of Learning Outcomes and Teaching Methods: General

General alignment: What do we mean by “alignment” when talking about teaching methods?

Teaching methods and learning designs

ALIGNMENTLearning outcomes

Teaching methods

Workload (ECTS)

Assessment

Alignment of Learning Outcomes and Teaching Methods: General

Learning outcome:

Students are able to moderate a virtual task-force meeting.

Teaching methods:� Lecture� One-minute paper� Feedback� Think-pair-share� Role play

Teaching methods and learning designs

Take a vote:Which teaching method(s) do you consider a good/best fit for this learning outcome?

“Students are able to moderate a virtual task-force meeting”

Teaching methods:� Lecture� One-minute paper� Feedback� Think-pair-share� Role play

Voting Activity

Teaching methods and learning designs

Alignment of Learning Outcomes and Teaching Methods: Specific

Use taxonomy for alignment, for instance, Anderson & Krathwohl taxonomy of educational objectives

Teaching methods and learning designs

Students are able to moderate a virtual task-force meeting.

Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objec tives (Complete ed.). New York: Addison Wesley Longman.

LO: moderation

Alignment of Learning Outcomes and Teaching Methods: Specific

Teaching methods and learning designs

Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objec tives (Complete ed.). New York: Addison Wesley Longman.

LO: Moderation

TM: Lecture

TM: One-Minute-Paper

TM: Feedback

TM: Think-Pair-Share

TM: Role Play

Assessment: Moderation

Teaching methods

What to look after besides learning outcome alignment when choosing teaching methods?� Allow learners to practice and do things themselves; don’t just “talk”

about it� Plan for social interactions between learners� Support learning process (feedback, scaffolding, coaching)

Teaching methods and learning designs

Learning Design

Building a learning design: Adding content and assessment to the teaching method according to learning outcome.

Teaching methods and learning designs

Generic teaching method

Integration of a whiteboard tool to collect reasons

Specific learning design integrating the teaching method

Specifications in Focus

� Learning Outcomes (IEEE RCD)� Describe and manage Learning Outcome Definitions� ICOPER LOD AP

� Learning Opportunities (CEN MLO)� Publish metadata of learning opportunities� ICOPER MLO APP

� Personal Achievements (new spec.)� Describe and share information on learners/teacher achieved learning

outcomes and associated evidence, level and context data� Personal Achieved Learning Outcomes (ICOPER PALO)� Backbone for a new CEN specification

Models and standards

Metadata for Learning Outcomes

� Describes the characteristics of a learning outcomes� Enable storage, referencing, sharing Learning outcomes data

Models and standards

Indexed large set of learning outcomes

http://oics.icoper.org/LOD/

Sources Learning Outcomes � OpenLearn (OUUK)

� All courses have defined learning outcomes� Stored in IEEE LOM metadata classification category

� IEEE/ACM Computer Science Curricula (Task Force)� All course descriptions have learning outcomes� PDF file� Used worldwide by many universities

� 21st Century Engineer (NSF project)� 36 generic learning outcomes� Engineering domain� .ppt file

� European Computer Science Driving License (ECDL)� Full list of learning outcomes per topic� Excel sheets

� WU Vienna – Business Education study programme� Local XML into LOD XML

Models and standards

Learning Opportunities

Example:

Course catalog

Models and standards

Metadata for Learning Opportunities � Describe an opportunity a learner can follow� Enables publishing metadata across universities/countries

Models and standards

Generate MLO Instances

http://oics.icoper.org/MLO/

� Course catalogs� Local XML format� Transformation into

ICOPER MLO XML� 1000s instances� Want to produce MLO

instances from your course catalogs?…Talk to us ;-)

Metadata for Personal Achievements� Represent information on achieved/required/desired learning outcomes of a learner. Also teachers

(taught outcomes)� Share data with LMSs, recruitment systems, social applications

Models and standards

PALO ExampleModels and standards

Najjar et al. (2010) A data model for describing and exchanging Personal Achieved Learning Outcomes (PALO). International Journal of IT Standards and Standardization Research, vol. 8, pp. 87-104.

Generated PALO Data

Models and standards

� Data collected acrosssystems

� ATOM/RSS feeds� Shared in

� iGoogle� Facebook� LMSs� etc.

Evidence of achievement

� Successful completion of learning opportunity

� Evidence that a learning outcome was achieved/attained

� Each learner achievement has an assessment record

� Assessment record is a summary result of one or more learning assessments

Learning outcome motivated assessment

Approaches

� Automatic� System generates assessment records

from assessment results in LMS� Submit data to learner profile when course is closed

� Semi-automatic� Student claimes achievement of

outcomes � submit assessment records to learner profile

� Teacher approves assessment records � data submitted to learner profile

Learning outcome motivated assessment

More Resources & Feedback

ICOPER Training Infrastructure:� Learning Outcome Course:

http://training.icoper.org/moodle/mod/resource/view.php?id=269� Instructional Modeling Course:

http://training.icoper.org/moodle/mod/resource/view.php?id=270

Your feedback:What do you take home from this webinar?Please share via the chat window or raise your hand to speak.

Feedback form:http://tinyurl.com/28gjny2

Closing

Upcoming Webinars

Webinar 3: “eContent for designing and delivering outcome-based learning opportunities”February 3, 2011 (next Thursday)16.00-17.30

Webinar 4: “Learning outcome-based assessment and applications”February 10, 201116.00-17.30

Webinar 5: “Applications for outcome-based learning”February 17, 201116.00-17.30

ICOPER Webinar Series