Leveraging Blackboard Outcomes Assessment as a Continuous Improvement Strategy at Amarillo College
Tina Babb Director of Institutional Effectiveness
Learning OutcomesParticipants will…• Identify several practices Amarillo College has built around the
Blackboard Outcomes Assessment application• Consider a continuous improvement strategy for your own
institution’s assessment and accreditation efforts • View a demonstration of artifact assessment• Examine the reporting outputs of this tool that trigger discussions
with faculty
Outcomes Assessment Then and Now…
• New streamlined approach• Manual assessment of hard-copy
artifacts vs. online in current LMS (Blackboard)
• Sampling?• Analysis time-consuming and
completed separately• Lacked structure for goal setting• Lacked “Closing the Loop”
Green Bay, Wisconsin | January 14, 2019
AC Assessment Plan Components
Green Bay, Wisconsin | January 14, 2019
• 8.2.a – Student Outcomes: Educational Programs• Program Student Learning Outcomes (PSLOs) defined by each department on a
Curriculum Map and assessed
• 8.2.b – Student Outcomes: General Education• Institutional Student Learning Outcomes (ISLOs) defined by the Texas Higher
Education Coordinating Board and assessed• Communication, Critical Thinking, Empirical & Quant Skills, Personal Responsibility, Social
Responsibility, and Teamwork
Alignment - SACSCOC Standards 8.2.a and 8.2.b
Green Bay, Wisconsin | January 14, 2019
Building Practices with Outcomes Assessment: Core Curriculum and General Education Competencies
Green Bay, Wisconsin | January 14, 2019
Building Practices with Outcomes Assessment:General Education Competencies – Teamwork Example
Green Bay, Wisconsin | January 14, 2019
Building Practices: Development of Planning Template
Green Bay, Wisconsin | January 14, 2019
Building Capacity: Leveraging Technology
Green Bay, Wisconsin | January 14, 2019
Building Capacity: Continuous Improvement Strategy
Assessment Results Provided to Faculty - Identify Improvements
Assessment Results Provided to Faculty - Identify Improvements
Assessment Results Provided to Faculty - Identify Improvements
Assessment Results Provided to Faculty - Identify Improvements
Implement Improvements &
RE-ASSESS
Personal ResponsibilityStudents Upload
Work to Bb Common Assessment Folder
Faculty Evaluate Artifacts
Implement Improvements &
RE-ASSESS
Faculty Evaluate Artifacts
Faculty Evaluate Artifacts
Empirical & Quantitative Skills
Students Upload Work to Bb Common
Assessment Folder
Faculty Evaluate Artifacts
Critical Thinking PILOTStudents Upload
Work to Bb Common Assessment Folder
Faculty Evaluate Artifacts
Implement Improvements &
RE-ASSESS
Overall Assessment Map
Amarillo College Assessment Plan
Gen Ed Competency Map
General Education Competencies
Foundational Component AreaCommunicationCritical ThinkingEmpirical & Quantitative SkillsPersonal ResponsibilitySocial ResponsibilityTeamwork
American HistoryXXXX
CommunicationXXXX
Creative ArtsXXXX
Government/Political ScienceXXXX
Language, Philosophy, & CultureXXXX
Life & Physical SciencesXXXX
MathematicsXXX
Social & Behavioral SciencesXXXX
Schedule of Gen Ed Assessment
Amarillo College 5 Year Assessment Schedule of General Education Competencies
2016-17 (Pilot)2017-182018-192019-202020-21
General Education Competency (ISLO)*FALLSPRINGFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING
Communication Skills (COM)PILOTPlanning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IE
Critical Thinking (CT)PILOTPlanning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IE
Empirical & Quantitative Skills (EQS)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESS
Personal Responsibility (PR)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify Improvements
Social Responsibility (SR)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify Improvements
Teamwork (TW)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To Depts
*BEFORE TERM BEGINS Identify course sections via the planning template and place assignment & instructions in Common Assessment folder in Bb.
Faculty will complete the assessment in the Blackboard Outcomes Assessment
*ISLO = Institutional Student Learning Outcome, IE = Institutional Effectiveness, Bb = Blackboard
Revision in Progress-Dec 2019
General Education Competencies2017-182018-192019-202020-21
FALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING
Communication SkillsPILOTStudents Upload Work to Bb Common Assessment Folder Faculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts
Critical ThinkingPILOTStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts
Empirical & Quantitative SkillsStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESS
Personal ResponsibilityStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify Improvements
Social ResponsibilityStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify Improvements
TeamworkStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts
Gen Ed Steps
AC ISLO/PSLO Assessment Cycle
*ISLO = Institutional Student Learning Outcomes (General Education Competencies)
*PSLO = Program Student Learning Outcomes (Curriculum Map)
*IE = Institutional Effectiveness
STEP 1: Prior to Semester Start: Departments Identify Courses/Artifacts on the Bb Outcomes Assessment Planning Template + Identify Faculty Evaluators
STEP 2: IE Samples Artifacts as Semester Ends and Provides Evaluation Link to the Faculty Evaluators via Bb Outcomes Assessment Tool
STEP 3: Depts/Faculty Evaluators Assess Sampled Artifacts the Following Semester
(ASSESSMENT PART 1)
STEP 4: IE Provides Analysis & Reports of Assessment Results to Program Chairs/Coordinators
STEP 5: Program Chairs/Coordinators along with Faculty Analyze Results and Identify Improvements
STEP 6: Program Chairs/Coordinators/Faculty Implement Improvements for the ISLOs/PSLOs
STEP 7: Re-Assess with Implemented Improvements (ASSESSMENT PART 2)
ASSESSMENT PART 3: CLOSING THE LOOP
STEPS 2-5 Repeated on the Improvements - Data analyzed for continuous improvement
PSLO Cycles
2017-182018-192019-202020-212021-22
Course(s) Identified on Curriculum MapFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING
AssessDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsRE-ASSESS & Implement ImprovementsDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsAssessDepts Complete Assessment-Sample Provided by IE
AssessDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsRE-ASSESS & Implement ImprovementsDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsAssessDepts Complete Assessment-Sample Provided by IE
AssessDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsRE-ASSESS & Implement ImprovementsDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsAssessDepts Complete Assessment-Sample Provided by IE
Legend:
Departments identify course assignments to be assessed via the planning template and place assignment in the Common Assessment folder in Bb.
Departments will complete the assessment in the Bb Tool.
*ISLO = Institutional Student Learning Outcome
IE = Institutional Effectiveness
Green Bay, Wisconsin | January 14, 2019
Blackboard Outcomes Assessment: Goal Sets, Goal Types and Outcomes
Goal Types can include: General Education Competencies (ISLOs), Program StudentLearning Outcomes (PSLOs), Accreditation Standards, QEP, etc.
Green Bay, Wisconsin | January 14, 2019
Blackboard Outcomes Assessment: Creating Rubrics for Alignment with Courses, Goals and Student Artifact Collection
Ok, So How Do We Perform the Actual Assessment?• An assessment session can be created when these three criteria are in place in
Blackboard Outcomes:• Students have uploaded work into ”Common Assessment” drop-box• Faculty attach instructions to the drop-box• Institutional Effectiveness attaches the alignments to the drop-box
• An email from Blackboard is generated to all faculty evaluators to begin their assessment
• Faculty have approximately 4 weeks to assess student work• Video: You Are Now a Faculty Evaluator
https://use.vg/usbDnvXwFvDc
Blackboard Outcomes Assessment Application: Results
2 reports are generated at the close of each assessment session: 1. Excel spreadsheet-contains raw data of faculty scoring2. Data visualization graphs in .pdf
Identifying Gaps in Student Learning by Measuring Extent of Achievement of Learning Outcomes
Frequency Distribution – How Are Students Doing Overall? Where Do Students Struggle?
Green Bay, Wisconsin | January 14, 2019
• “I felt that the Bb Assessment pilot worked much better than the old system of artifact collection. Being able to assess the artifacts is so much easier than doing it by hand and trying to get everyone together to meet. I have no suggestions for improvement. I thought it went smoothly and was efficient.” –Professor of Social and Behavioral Sciences
• “I think that the key factor in the whole process is to allow plenty of time for training of developing the rubrics and in learning to assess the artifacts in Bb and in doing the actual assessment. This new process of assessing was faster and more streamlined than the old process of assessing by hand.” –Professor of English
• “I found the tool for reviewing easy to use and simple to navigate.” –Instructor of Humanities
Faculty Feedback
Green Bay, Wisconsin | January 14, 2019
Closing the Loop…
Green Bay, Wisconsin | January 14, 2019
• Communication• 52 departments/135 academic programs
• Leveraging Technology• Blackboard Outcomes Assessment tool• Institutional Research data dashboards
• IE Collaboration • Center for Teaching and Learning built into assessment cycle
• Faculty Development and time to reflect• Faculty Development Model
• Incentivize (https://www.actx.edu/ctl/resources)
Building Capacity: Increasing Faculty Engagement
https://www.actx.edu/ctl/resources
Building Capacity: Data Resources for Faculty(dair.actx.edu)
Example: Criminal Justice Common Assessment
Green Bay, Wisconsin | January 14, 2019
Example: Closing the Loop on Empirical Skills for Criminal Justice Students
Green Bay, Wisconsin | January 14, 2019
A Faculty Member’s Perspective: Helpful Hints & Improvements Student Engagement Techniques
• TILT• Capstone Assignment
Design a high-quality, efficient assignment• Signature assignment
Train your faculty evaluators
Reflection – Build time in the assessment cycleCollaboration – Faculty development opportunitiesData Driven – Use of assessment data to make informed decisionsFaculty-Led Improvement strategies
Building Capacity: Faculty Development as a Focal Point of Student Success
Contact Information:Tina M. BabbDirector of Institutional Effectiveness(806) [email protected]/ie/
Questions?
mailto:[email protected]://www.actx.edu/ie/training-materials
Slide Number 1Learning OutcomesOutcomes Assessment Then and Now…��Slide Number 6Slide Number 7Slide Number 8�Building Capacity: Leveraging Technology�Slide Number 10Slide Number 11Slide Number 12 Ok, So How Do We Perform the Actual Assessment?Blackboard Outcomes Assessment Application: ResultsIdentifying Gaps in Student Learning by �Measuring Extent of Achievement of Learning OutcomesFrequency Distribution – How Are Students Doing Overall? Where Do Students Struggle?���Building Capacity: Data Resources for Faculty�(dair.actx.edu) Example: Criminal Justice Common AssessmentSlide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26
Top Related