Reverse Engineering Gen ED: Using ePortfolios to Align Institutional and

Programmatic Goals- CS409:15-10:30

Kathleen Langan Pusecker, Director of the Office of Educational Assessment and Manuel R. Torres,

Senior Research AnalystUniversity of Delaware assessment@udel.edu

klp@udel.edu & mrtorres@udel.edu

Agenda

• What is reverse engineering

• What is Gen Ed at UD?

• What is a programmatic eP?

• How can an eP be used to reverse engineer assessment of GenEd?

• Activity! Using an eP assessment tool to assess student work!

Reverse Engineering Assessment

3

UD Gen Ed Goals1. Attain effective skills in (a) oral and (b) written communication, (c) quantitative reasoning,

and (d) the use of information technology.2. Learn to think critically to solve problems.3. Be able to work and learn both independently and collaboratively.4. Engage questions of ethics and recognize responsibilities to self, community, and society

at large.5. Understand the diverse ways of thinking that underlie the search for knowledge in the

arts, humanities, sciences and social sciences.6. Develop the intellectual curiosity, confidence, and engagement that will lead to lifelong

learning.7. Develop the ability to integrate academic knowledge with experiences that extend the

boundaries of the classroom.8. Expand understanding and appreciation of human creativity and diverse forms of

aesthetic and intellectual expression.9. Understand the foundations of United States society including the significance of its

cultural diversity.10. Develop an international perspective in order to live and work effectively in an

increasingly global society.

Implementation of Gen Ed

1. English 110: Homogenous experience

2. First Year Seminar which includes: Diverse experiences depending on major

3. 3 credits multicultural course: Diverse experiences

4. 3 credits Discovery Learning Experience: Diverse experiences

5. Capstone experience: Discipline Specific

OEA’s Charge and Challenges

To assess how well undergraduate students attain the UD General Education goals and to support academic departments and units assessing undergraduate and graduate student learning.

Assessment Challenges •4 diverse Gen Ed experiences

•Challenging to find evidence of students gaining Gen Ed skills

•Gen Ed goals hard to assess (abstract)

•Accreditation (for ex., Middle States visit in 2011)

•UD’s participation in the VSA

So what tool would allow us to measure

General Education?

7

What is an eP?

• Presentation eP• Assessment eP• Hybrid eP= TLA

Reverse Engineering: How to create a programmatic eP

Identify programmatic student learning goals that connect to Gen Ed

Identify where students can demonstrate these goals

Determine who provides feedback

Create developmental rubrics or adapt a preexisting rubric

Create prompts for your rubric (assessment tool)

Bad Example•Goal- Latin American Studies students will be able to demonstrate their Awareness/ understanding of Latin America's cultural diversity and challenges.•Reflective Prompts: How have your studies made you more knowledgeable of Latin America’s cultural diversity and challenges?•What do you know now that you did not know before about Latin America’s cultural diversity and challenges?•How does the attached artifact reflect your cultural awareness and understanding?•What will you do differently or continue to do as a result of the experiences documented by the artifacts?

10

Inadequate Developing Competent ProficientGoal 1Cultural knowledge: Acquisition and command of information about societies, geography, history, politics, arts and cultural norms in Latin America

Knowledge of Latin America expected of educated non-specialist. No knowledge of existing sources other than random computer searches.

Basic factual knowledge about Latin America. Knowledge of general sources for information.

Understands theories and concepts as ways to interpret knowledge. Active use of important discipline-based sources for information.

Applies knowledge to organize information and solve problems. Ability to select the most appropriate sources for information.

Good Example•Goal- Latin American Studies students will be able to demonstrate their Awareness/ understanding of Latin America's cultural diversity and challenges.•Reflective Prompts: Reflect in writing about how have your studies in HIST 135 made you more knowledgeable of Latin America’s cultural diversity and challenges?• How does the attached artifact reflect your cultural awareness and understanding?•What will you do differently or continue to do as a result of the experiences documented by the artifacts?• Please site your general sources for information understanding?•What will you do differently or continue to do as a result of the experiences documented by the artifacts?

11

Inadequate Developing

Target 100 level courses

Competent

Target 200-300 level courses

Proficient

Target 400 level courses

Goal 1Cultural knowledge: Acquisition and command of information about societies, geography, history, politics, arts and cultural norms in Latin America

Knowledge of Latin America expected of educated non-specialist. No knowledge of existing sources other than random computer searches.

Basic factual knowledge about Latin America that demonstrates an awareness/understanding of Latin America's cultural diversity and challenges. Cites knowledge of general sources for information.

Understands theories and concepts as ways to interpret knowledge.Provides examples that Compare and contrast Latin America’s cultural diversity and challenges to that of the US. Active use of important discipline-based sources for information.

Applies knowledge to organize information and solve problems. Ability to select the most appropriate sources for information.

AACU Rubrics

12

Assessment tool to examine mastery of critical thinking skills with the eP

• Select and modify -Valid Assessment of Undergraduate Education (VALUE) Rubrics

• Train Faculty/Assessment Personnel

• Assessment of Artifacts

Benefits of VALUE Rubric Assessments

• With little time, faculty can accurately evaluate large numbers of students work in a reliable and valid manner

• The institution can spend less money on assessing Gen Ed

• The information yielded from these assessments and evaluations is useful and actionable

What questions to ask before you implement eP

1.Define the learning outcomes2.Understand your learners3.Identify your stakeholders4.Design the learning activities5.Use rubrics to evaluate eP6.Anticipate external uses of evidence7.Include multiple forms of evidence8.Evaluate the impact of implementation of eP

Chen and Penny, and Ittleson’s (2012) 8 guiding questions

Hands on Component

• Read the artifact• Examine the rubric• Score the artifact

16

Resources• AACU VALUE project-http://www.aacu.org/value/• Barrett, H.C. ,Electronic Portfolio Planning Worksheet, 2000 • Chen, H. Penny Light, T., Ittleson, J.C., (2012) “Documenting Learning

With ePortfolios”. Jossey-Bass: San Francisco • Committing to Quality: Guidelines for Assessment and Accountability

in Higher education-The new leadership Alliance for student learning and Accountability (2012). http://www.newleadershipalliance.org/images/uploads/committing%20to%20quality.pdf

• National Coalition of EPortfolio Research, Cambrige, B., et al. 2009 - http://ncepr.org/

• Palomba, C., Banta, T., et al. 1999. Assessment essentials: Planning, implementing, improving.

• Suskie, L. Assessing Student Learning: A Common Sense Guide, San Francisco, Calif: Jossey-Bass; (2009).

• University of Delaware Faculty Senate Committee on General Education issued a report (http://www.udel.edu/facsen/reports/GenEdRpt.html)