Designing Assessment,Assessing Instructional Design

TCC Hawaii 2014Stefanie Panke

Aloha Ya’ll!

From Pedagogical Concepts to Practical Applications

Resources

http://panke.web.unc.edu/tcc2014

AACE Special Interest Group ‘Assessing, Designing, and Developing E-Learning’

http://panke.web.unc.edu/tcc2014

“Practically everybody in the academic community gets assessed these days,

and practically everybody assesses somebody else”(Astin, 2012).

“We need to get past the emulation society, where instead

of learning you spend all your energy on convincing others

what you know”

“Assessment is the fabric of education, if

you change assessment, you change everything”

Ambjorn Naeve, Tel-Map Group Expert Meeting, Lisbon 2011

Learning and assessment are two sides of the same coin, and they strongly

influence each other (Gulikers, Bastiaens & Kirschner, 2004).

Assessment = Student

Feedback

“Effective learning occurs when students receive feedback, i.e. when they receive information on what they have (and have not) already learned. The process of generating such information is assessment” (Entwistle, 2000).

Assessment = Learning

Organization Catalyst

“We view assessment as the gathering of information concerning the

functioning of students, staff and institutions of higher education. The

basic motive is to improve the functioning of the institution and its

people”(Astin, 2012).

Measure vs. Treasure

From ‘assessment of learning’ towards ‘assessment for learning’

Knowledge imparted by the instructor (input)

Competencies students can apply (output).

“To change student learning in the direction of competency development, authentic competency-based instruction aligned to authentic competency-based assessment is needed” (Gulikers, Bastiaens & Kirschner, 2004).

“For learning objectives not easily measured by multiple choice or true/false questions, assessment and evaluation can be time-consuming and difficult” (Anders, 2012).

The growth of virtual conferences has high

potential as a tool for a more authentic assessment

in higher education graduate programs (Menchaca, Ho & Hoffmann, 2013).

Five dimensions of authentic assessment: (a) the task, (b) the physical context, (c) the social context, (d) the results(e) the criteria.

Gulikers, Bastiaens & Kirschner (2004)

“The perception of what authenticity is may vary among individuals as a result of educational level, personal interest, or amount of professional experience” (Gulikers, Bastiaens & Kirschner, 2004).

Threshold Concepts (Meyer & Land 2003, 2005):• Transformative • Integrative • Oftentimes bounded • Probably irreversible• Potentially troublesome

Designing creative assessment helps students unfold emerging understanding of key concepts.

Moving on…

Examples &Tools

(1) Needs Assessment (2) Impact Assessment (3) Classroom Assessment

Needs Assessment

• Stakeholder input• Example: Website

Redesign Process• Workshop Series:– Audience– Content– Categories– Navigation

Audience

• Goal: Identify potential audiences for the online resource

• Exercise: Personas

• Structure: Participants create personas in groups (approx. 20 minutes), present and vote on most important / fitting representations of their audience.

Content

• Goal: Identify organizational output, recurring website elementsExercise: Content Toolkit

• Visual building blocks • Create representation of

different assets (books, news,user profiles, articles….)

Results

Categories

1. Review existing categories

2. Label sample content

3. Taxonomy on the fly

Examples &Tools

(1) Needs Assessment (2) Impact Assessment (3) Classroom Assessment

Impact Assessment

• Formative evaluation of material, learning environment, curriculum

• Example: Learning Analytics– MPA competence

assessment process

Data is the new oil – are learners the new sardines?

Analytics Dashboards

“Deciphering trends and patterns from educational big data” (Horizon Report 2013).

“The measurement, collection, analysis and reporting of data about learners and their contexts” (SOLAR 2014).

Competence- based

CurriculumRedesign

• Analytical rubrics facilitate the assessment of student performance in central learning outcomes.

• Collaborative effort of instructional designer, program leaders and faculty.

Rubrics

Rubrics

Examples &Tools

(1) Needs Assessment (2) Impact Assessment (3) Classroom Assessment

Classroom Assessment

• Support students’ critical thinking abilities and transfer learning

• Example: Portfolio– Infrastructure– Scaffolds– Peer Learning– Assessment– Evaluation

Portfolio Components

• WordPress Multisite• Sample Portfolio• Portfolio Guide• WordPress Training• Competence Memo Template• Peer Feedback and Milestones in PUBA 746• Portfolio Assessment Rubric• Portfolio Evaluation (Questionnaires, Focus

Groups, Interview Guides)

Benefits of e-portfolios • Fostering digital

literacy /multimedia storytelling (Wakimoto & Lewis, 2014).

• Supporting career development (Reese & Levy, 2009).

• Strengthening organizational ties (cf. Lorenzo & Ittelson, 2005; Reese & Levy, 2009).

• Encouraging reflection (Roberts, Maor & Herrington, 2013).

• Promoting self-regulation (Abrami et al, 2008, Meyer at al, 2010).

• Improving knowledge management (Chang, Tseng, Liang & Chen, 2013).

• Acknowledging diversity and transfer learning (O’Toole, 2013).

Types of Portfolios (Hewett, 2004)

• Documentation portfolios: growth toward learning goals

• Process portfolios: phases of the learning process

• Showcase portfolios: accomplishments and competences

What nuggets can you use in your work or personal environment?

Thanks you – have a good conference!