The 21st Century Century Digital Learner and The 21st Century Skills
Measuring digital learning in the 21st century
description
Transcript of Measuring digital learning in the 21st century
Engaging Online Faculty and Administrators in the Assessment Process at the American Public University SystemMeasuring Digital Learning in the 21st Century - Data Warehouses, Dialects, and
Dashboards
INTRODUCTIONS
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Dave BecherDirector, Academic Information Analysis
Phil Ice, Ph.D.Director, Course Design, Research, and
Development
Frank McCluskey, Ph.D.Executive Vice President and Provost
Jennifer Stephens Helm, Ph.D.Associate Vice-President, Dean of Assessment
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
WHO WE ARE
American Public University System has been dedicated to educating those who serve since our doors first opened in 1991. The American Public University System (APUS) is a fully online system, encompassing both American Military University and American Public University. We are regionally accredited by the Higher Learning Commission (HLC) of the North Central Association.
“Educating Those Who Serve”
EXCELLENCE IN EDUCATION
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
We face challenges that are well documented in the educational literature. We are an open enrollment college, as well as a fully online institution of higher education. APUS is committed to improving our learning environment. To continue and increase the success of our students we employ a rigorous assessment plan. Excellence in education is fostered through a cycle of data collection, analysis, remediation, and then further data collection to measure the learning again.
ASSESSMENT AT APUS
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
STRATEGY #1:
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
External Review
Analysis
Program Review
Meeting
Three Year Plan
Follow-Up
Data Collection
Promote Accountability Among Faculty and Administrators – APUS
Program Review Process
EFFECTIVE DOCUMENTATION
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Promoting Accountability through the Documentation of Student Learning Outcomes Evidence
We begin with the assumption that our faculty are the experts. Our data collection assists them in translating their expertise into student learning. We want to help them free our students from fixed ideas, cloudy thinking and imprecise analysis. We want to liberate the learning already taking place.
EFFECTIVE DOCUMENTATION
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Course books
• Electronic resources
• Learning strategies
Library and Learning Resources
• Student learning outcomes
• Instructional strategies
• Evaluation procedures
• Academic rigor
Curriculum Assessment
• Expert reviewer report
• Industry Advisory Council report
External Reviewer Feedback
Data Collection and Analysis
EFFECTIVE DOCUMENTATION
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Analysis of faculty credentials and expertise to ensure breadth and diversity
Faculty
• Student demographic information
• Enrollment History
• Growth trends
Students
• Curricular Mapping
• Assessment measures
• Fact book
Learning Outcomes Assessment
Data Collection and Analysis
EFFECTIVE DOCUMENTATION
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Program benchmarking with similar programs and institutions
Program Benchmarking
• Evaluation of findings
• Program recommendations
• Three year proposed strategic plan
Program Directory Summary
• Dean’s observations
• Meeting minutes
Review and Findings
Data Collection and Analysis
STRATEGY #2
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• One of fourteen charter member institutions
• College Choices for Adults website
http://www.collegechoicesforadults.org/
Transparency by Design Initiative
http://www.apus.edu/community-scholars/learning-outcomes-assessment/index.htm
APUS Learning Outcomes Assessment Website
• Meetings
• Faculty, staff, and student newsletters
• Faculty and student lounges
Internal Communications
Inform Internal and ExternalStakeholders on the Achievement
of Student Learning Outcomes
STRATEGY #3
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Establish a Reciprocal Relationship among Stakeholders and the Institution on the Sharing of Data
• Regular dissemination of data to stakeholders
• Reciprocal relationship between data team, Deans, and Program Directors
• Assessment feedback loop
FACT BOOK DATA COLLECTION TIMELINE
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
STUDENT DEMOGRAPHICS
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
STUDENT COUNT
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
GRADES BY CORE COURSES (IN PROGRAM): %
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
STRATEGY #4
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Incorporate Assessment Data intoDecision Making Processes
APUS utilizes a number of nationally benchmarked tests and validated instrumentsto measure learner engagement, learner success, and learning outcomes.
STRATEGY #4
National Survey of Student Engagement - NSSE
• Administered at several hundred colleges and universities in the United States and Canada
• Hundreds of thousands of freshman and seniors surveyed using a validated instrument recognized in the literature and nationally
• Survey uses more than 70 questions gathering self reported student perceptions
APUS is one of few fully online, for-profit institutions of higher education participating.
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
STRATEGY #4
The Educational Testing Service
• APUS uses the Proficiency Profile (formerly the MAPP) to measure skills of college program graduates.
• Presumption for open admission and fully online institutions to score below the national average.
• 2009 APUS scores rise above the national average with the sole exception of mathematics which is less than 1% below the norm (See Table 1 – 2009 MAPP Test Data).
• ETS performs a major field test for seven of our undergraduate majors upon graduation (See Table 2 – 2009 MFT CJ Data).
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
COMMUNITY OF INQUIRY
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
APUS participates in the Community ofInquiry (COI) End of Course survey.
The COI is a validated instrument based onthe research around Social, Cognitive, andTeaching Presence.
More than 500,000 learners have used this instrument and created a strong baseline forfurther research.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105
• Promote Accountability Among Faculty and Administrators
• Inform Internal and External Stakeholders on the Achievement of Student Learning Outcomes
• Establish a Reciprocal Relationship among Stakeholders and the Institution on the Sharing of Data
• Incorporate Assessment Data into Decision Making Processes
RECAP OF 4 STRATEGIES
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE