The “Quality Agenda:” Implications for Assessment, Policy, and Professional Accreditation Peter...
-
Upload
katherine-furness -
Category
Documents
-
view
216 -
download
0
Transcript of The “Quality Agenda:” Implications for Assessment, Policy, and Professional Accreditation Peter...
The “Quality Agenda:” Implications for Assessment, Policy, and Professional Accreditation
Peter T. Ewell
National Center for Higher Education Management Systems (NCHEMS)
AACSB International Conference
New Orleans, LA
March 18, 2014
Logic of this Presentation
The “Quality Agenda:” Where Did It Come From?
The Historical Evolution of Discussions About Quality in Higher Education
An Evolving “Standards” Movement in Higher Educationo National: Degree Qualifications Frameworks with
the U.S. DQP as an Example
o Professional: Assessment in Business Programs
Into the Future: A “Changing Ecology” for Accreditation and Its Implications for Assessing Quality
A Prominent “Completion Agenda” in Higher Education Policy
Stimulated by Recognition of Link Between Economic and Social Benefits and Postsecondary Attainment in Many Countries
Widespread National Attainment Goals (e.g. The Obama Goal in the U.S.: 60% of Young Adults with a Postsecondary Credential by 2020)
But Concern that “Completion” Could Come at the Expense of Academic Quality
Hence, a Growing “Quality Agenda”
New Interest in Aligned Student Learning Outcomes on a National and International Basis
Shift of Attention from “Doing Assessment” to “Standards for Learning”
Stakeholder Concern About Graduate Quality, Particularly from the Employment Community
So the Current Challenge is to Raise Completion Rates while Not Losing Academic Quality
=
Changing Notions of “Quality” in Policy Discourse About Higher Education
Largely taken for granted until the emergence of formal systems of “quality assurance” like accreditation and national Quality Assurance Agencies
New conceptions of “quality” emerged gradually as colleges and universities diversified and became more complicated
Today’s conception of “quality” is thus a sedimentary construction with new notions of what should count “layered in” on top of old ones
First Incarnation: Reputation
Colleges for “The Quality” (who were members of a privileged elite or entering a professional class like the clergy, law, or medicine)
“College” as a reserved term: distinct from postsecondary institutions: “institutes,” “normal schools,” etc.
Alive and well on bumper stickers and big time sports teams [not to mention media rankings]
Second Incarnation: Resources
U.S. Accreditation Criteria of the 1920s (North Central Association): “The college should:
• Enroll at least 200 students
• Should comprise at least eight departments with at least one person of professorial rank
• Should maintain a live, well-distributed library of at least 8000 volumes”
Quantitative resource-based criteria like these officially a thing of the past…but stuff still matters a lot in popular views of “quality”
Third Incarnation: Selectivity
The rise of admissions selectivity in the 1950s
Reputation and exclusivity in a new guise: in the U.S., the role of the SAT was said to be to “uncover the hidden aristocracy of talent”
An implied theory of education: smart begets smart by association and osmosis
Fourth Incarnation: “Fitness for Purpose”
“Mission-based” quality review emerges as most appropriate for diverse postsecondary systems
Peer review and institutional audit become the primary “assessment instruments” under this approach.
“Purposes appropriate to an institution of higher education…”
But what happened to standards?
Fifth Incarnation: Outcomes
An “exo-skeletal” approach: outcomes assessment largely added on to the regular processes of teaching and learning.
The resulting paradigm:
• Statements of intended learning outcomes
• Various ways to gather evidence of attainment
• Use of the resulting information to improve
Embedded in U.S. regional accreditation standards by the mid-1990s and national QAA reviews by 2000
A Sixth Incarnation: Exit Proficiencies?
A common set of graduation proficiencies adopted by all providers
Assessments embedded in the regular teaching and learning process:
• Signature assignments in key classes
• Developed in common by teaching staff
• Graded or rated using standard rubrics
Re-positioned proficiency-based transcripts that show “student learning as academic currency”
National Qualifications Frameworks
Matrix of Identified Proficiencies by Degree Levels
Purpose to Align and ‘“Moderate Academic Standards at Various Degree Levels
Some Examples:
Bologna Process Common Outcomes Benchmarks
QFs in UK, Australia, Ireland, Scotland, and Many Others
“Tuning” in Many Disciplines
Background to the DQP in the U.S.
Qualifications Frameworks in Many Other Countries (as Noted)
AAC&U LEAP Outcomes Statements and Rubrics
State-Level Outcomes Frameworks in U.S. (e.g. UT, WI, CSU, ND, VA)
Some Alignment of Cross-Cutting Abilities Statements Among Institutional Accreditors
What Does the DQP Look Like?
Three Degree Levels: Associate, Bachelor’s, and Master’s
Five Learning Areas: Specialized Knowledge, Broad/Integrative Knowledge, Intellectual Skills, Applied Learning, and Civic Learning
Framed as Successively Inclusive Hierarchies of “Action Verbs” to Describe Outcomes at Each Degree Level
Intended as a “Beta” Version, for Testing, Experimentation, and Further Development
An Example: Communication Skills
Associate Level: The student presents substantially error-free prose in both argumentative and narrative forms to general and specialized audiences
Bachelor’s Level: The student constructs sustained, coherent arguments and/or narratives and/or explications of technical issues and processes, in two media, to general and specialized audiences
Master’s Level: The student creates sustained, coherent arguments or explanations and reflections on his or her work or that of collaborators (if applicable) in two or more media or languages, to both general and specialized audiences
Some Implications of the DQP
The DQP Asserts that Every Student Should Graduate with the Designated Proficiencies. This Means that:
The Typical Approach of Setting Outcomes as “Aspirations” and Conducting Assessments of “Average” Student Performance is not Enough
Assessment as an “Add-On” to the Curriculum is Not Enough
Assessment Must Be Embedded in Regular Student Assignments and Examination Questions and Certified at Multiple Levels on the Way to Degree Completion
AACSB’s “Assurance of Learning” Similar in Form and Intent to Assessment Requirements of
Other Specialized Accreditors
Outcomes Set by Program Consistent with Mission
Both Generic and Field-Specific Outcomes
Methods Selected by Program and Not Standardized
Outside Consultation (e.g. with Employers) Encouraged
Use of Results Emphasized as Much as Assessment Process
Assessment in Business: NILOA Survey
Fourth of Twelve Schools or Departments with Respect to Overall Assessment Activity
Fifth of Twelve in Citing “Specialized Accreditation” as Reason for Doing Assessment
Second of Twelve in Using Capstone Courses
Second of Twelve in Reporting High Levels of Faculty Involvement in Assessment
Top School or Department Wanting Better Instruments
A Changing Ecology for Accreditation
New Patterns of Student Participation
New Kinds of Providers
A Transformed and Contingent Faculty
New Approaches to Instructional Provision
Constrained Resources
Most of These Forcing Learning Outcomes as the Only Practicable Way to Determine Quality
What are Accreditors Doing in Response?
Look More Explicitly at:
Coherence of Student Experience Across Settings
How Faculty Roles are Constructed and Discharged
“Outsourcing” of Courses and Teaching
Response to Stakeholder Needs (e.g. Employers)
Transparency and Public Reporting
Use of Electronic Media and “Virtual Review”
Putting It All Together
New Discourse About “Quality” Centered on Standards for Learning that All Students Will Achieve
As Documented in an Identified Array of Proficiencies that Schools Adopt in Common
Reported Publicly to a Wide Array of Stakeholders
Review Techniques Transformed to Respond to New Ecology for Higher Education and New Modes of Provision