George Kuh SAIR New Orleans, LA September 26, 2010

download George Kuh SAIR New Orleans, LA September 26, 2010

of 51

  • date post

    31-Dec-2015
  • Category

    Documents

  • view

    22
  • download

    0

Embed Size (px)

description

NILOA: Tracking the Status of Outcomes Assessment in the U.S. George Kuh SAIR New Orleans, LA September 26, 2010. - PowerPoint PPT Presentation

Transcript of George Kuh SAIR New Orleans, LA September 26, 2010

  • George Kuh

    SAIR

    New Orleans, LASeptember 26, 2010NILOA: Tracking the Status of Outcomes Assessment in the U.S.

  • Colleges do so little to measure what students learn between freshman and senior years. So doubt lurks: how much does a college education the actual teaching and learning that happens on campus really matter?

    David Leonhardt, NYTimes, Sept 27, 2009

  • Advance OrganizersWhat kind of information about student learning and institutional effectiveness is compelling and useful for: (a) guiding improvement efforts?(b) responding to accountability demands? What can be done to motivate encourage faculty and staff to use assessment results to enhance student learning?

  • Advance Organizers

    What are the obstacles that must be addressed for IR to help advance the assessment agenda?What could you and/or NILOA do to overcome these challenges?

  • OverviewThe current context for assessmentThe NILOA agenda: activities and productsQuestions to ponder

  • Context

    Global Competitiveness in Degree AttainmentThe New Majority and Demographic GapsQuestionable Levels of Student PerformanceIn a Most Challenging Fiscal Environment We Need Higher Levels of Student Attainment at an Affordable Cost

  • Assessment 2010Greater emphasis on student learning outcomes and evidence that student performance measures up

  • Assessment 2010Greater emphasis on student learning outcomes and evidence that student performance measures upDemands for comparative measuresIncreased calls for transparency ---public disclosure of student and institutional performanceAssessment technology has improved markedly, but still is insufficient to document learning outcomes most institutions claim

  • Far too little is known about assessment practices on campuses around the country

  • NILOAs mission is to document SLO assessment work, identify and disseminate best practices, and support institutions in their assessment efforts www.learningoutcomesassessment.org

  • FUNDERSLumina Foundation for EducationCarnegie Corporation of New YorkThe Teagle Foundation NATIONAL ADVISORY PANEL

  • NILOA ActivitiesGo to Web sitewww.learningoutcomesassessment.org

  • NILOA ActivitiesGo to Web sitewww.learningoutcomesassessment.orgNational surveys

  • Survey asked chief academic officers at all accredited 2 & 4 year colleges and universities about their assessment practices.Survey report More Than You Think, Less Than We Need released October, 2009

  • Report Summary Perhaps more assessment underway than some acknowledge or wish to believeMore attention needed to using and reporting assessment resultsInvolving faculty is a major challengeMore investment likely needed to move from data to improvement

  • According to Provosts, what is the driving force for assessment?

    a. Institutional Commitment to Improvementb. Accreditationc. Faculty & Staff Interestd. Governing Board Mandatee. None of the abovehigh importance 85% Regional80% Specialized

  • Report Summary Perhaps more assessment underway than some acknowledge or wish to believeMore attention needed to using and reporting assessment resultsInvolving faculty is a major challengeMore investment likely needed to move from data to improvementAccreditation is a major force shaping assessment

  • Assessment driven more by accreditation and commitment to improve than external pressures from government or employers

  • Accreditation tops the list for uses of assessment data

  • Regional accreditors cite deficiencies in student learning outcomes assessment with greater frequency Middle States - 2/3 of institutions have follow-up; number one reason being assessmentNEASC - 80% of institutions asked for follow-up on student learning outcomes assessmentHLC - 7 out of 10 institutions are being monitored; the vast majority for student learning outcomes assessment.

  • Attention to transparency in student learning outcomes assessment is growing.Middle States, HLC, SACS consider transparency an integrity issueNEASC increased requests for candor by asking for student learning goals and requirements to be published so that they are readily availableWASC-ACCJC ask every school to publish their student learning outcomes as eligibility requirement

  • NILOA 2010 Program-Level SurveyAll accredited, undergraduate degree-granting 2- and 4-year public, private, and for-profit institutions in the US (n=2,678)Provosts asked to identify 3-4 program areas to respond

  • NILOA 2010 Program-Level QuestionnairePreview survey at: www.niloasurvey.org

  • NILOA ActivitiesGo to Web site www.learningoutcomesassessment.orgNational surveysWeb scans: 725 institutions

  • Exploring the Landscape: What Institutions Post on Their Websites About Student Learning Outcomes Assessment Activities Natasha Jankowski & Julia MakelaCampuses report doing more assessment than they make accessible on their institutional websites. The typical institution shows only an average of two assessment activities. When information is posted, it is more often intended for internal institutional audiences

  • Web Scan Guiding Questions

    What does the institution display on its website regarding student learning outcomes assessment? On which web pages is information about assessment of student learning outcomes located?

  • Say vs. ShowMore assessment activity was reported by chief academic officers than was available on institution websites

  • NILOA ActivitiesGo to Web sitewww.learningoutcomesassessment.orgNational surveysWeb scans: 725 institutionsOccasional papers & products

  • Occasional Paper #1 Assessment, Accountability, and Improvement Peter T. EwellAssessments of what students learn during college are typically used for either improvement or accountability, and occasionally both. Yet, since the early days of the assessment movement in the US, these two purposes of outcomes assessment have not rested comfortably together.www.learningoutcomeassessment.org/OccasionalPapers.htm9e

  • Two Paradigms of Assessment

    Ewell, Peter T. (2007). Assessment and Accountability in America Today: Background and Context. In Assessing and Accounting for Student Learning: Beyond the Spellings Commission. Victor M. H. Borden and Gary R. Pike, Eds. Jossey-Bass: San Francisco.

    Continuous Improvement AccountabilityStrategic dimensions PurposeFormative (improvement)Summative (judgment) OrientationInternalExternal MotivationEngagementComplianceImplementation InstrumentationMultiple/triangulationStandardized Nature of evidenceQuantitative and qualitativeQuantitative Reference pointsOver time, comparative, established goalComparative or fixed standard Communication of resultsMultiple internal channels Public communication, media Use of resultsMultiple feedback loopsReporting

  • Looking Back: Whats Been Accomplished?Assessment Seen as LegitimateGoals for Learning EstablishedA Semi-Profession for AssessmentMuch Better Instruments and Methods

  • Looking Back: What Remains to be Done?Authentic Faculty OwnershipAssessment Still an Add-OnUse of Information for Improvement is UnderdevelopedSincere Institutional Engagement with Accreditors in Assessment

  • Occasional Paper #2 Three Promising Alternatives for Assessing College Students Knowledge and Skills Banta, Griffin, Flateby & Kahn Of the various ways to assess student learning outcomes, many faculty members prefer what are called authentic approaches that document student performance during or at the end of a course or program of study. www.learningoutcomeassessment.org/OccasionalPapers.htm

  • Occasional Paper #3 Connecting the Dots Between Learning and Resources Jane V. WellmanWith all the talk about the need for more accountability, surprisingly little is known about what kind of resources an institution needs in order to produce a given level of student attainment. www.learningoutcomeassessment.org/OccasionalPapers.htm

  • Wellmans Conclusions Intentionality matters as much or more than money aloneSpending on instruction and student services pays off in learning, retention and graduationExcess units cost institutions money, cost students in time and money, and do not get students to the finish line

  • Occasional Paper #4 Opening Doors for Faculty Involvement in Assessment Pat HutchingsThe key to effectively using assessment data to improve student learning is to engage faculty in meaningful ways in this critical activity. This paper discusses the challenges inherent in and opportunities for doing so.www.learningoutcomeassessment.org/OccasionalPapers.htm

  • Occasional Paper #5 Valuing Assessment: Cost-Benefit ConsiderationsRandy Swing & Christopher CooganHow can a campus know when enough spending is really enough? The authors systematically unpack what needs to be taken into account when allocating resources to the assessment of student learning outcomes.www.learningoutcomeassessment.org/OccasionalPapers.htm

  • NILOA ActivitiesGo to Web site www.learningoutcomesassessment.orgNational surveysWeb scans: 725 institutionsOccasional papers & productsInterviews & focus groups with key actors: -- AAC&U -- ACE -- AIR

  • Perspectives from Campus Leaders on the Current State of Student Learning Outcomes Assessment Jillian KinzieThis paper highlights lessons from four focus group sessions with campus leaders--presidents, provosts, academic deans and directors of institutional research from a variety of tw