Online Program Review:
Reflections on Lessons Learned and Paving the Way Forward
Why we went online...
• Why’d you come?– What are the outcomes you’d like to achieve?– What do you want to learn?
• Guess what...– Our colleges had the same concerns
• In addition...
Introduction and Framework
• Antelope Valley College (Aeron)– Weave Online
• Cosumnes River College (Kathy)– Homegrown
• Mesa College (Bri & Jill)– TaskStream
• Yuba College (Erik)– TracDat
INTEGRATED PL
ANNING
USING
ONLINE PR
OGRAM REVIEW
A N T E L O P E VA L L E Y C
O L L E G E
BENEFITS OF GOING ONLINE Universal access to the multiple constituents
in comparison to a document. Standard formatting and layout. Simple compliance reporting.
WEAVEONLINE Customizable to effectively accommodate
our PR questions. Easy compliance reporting. Evidence database allows users to support
statements by attaching documentation to the questions.
No internal IT support needed Streamline reporting via PDF or MS Word
formats.
END USER EXPERIENCECommon themes:Likes Universal access Simple entry process Evidence database
Challenges Data overload Unable to enter tables into document No auto saving
CUSTOMIZATION Customizable homepage which allows
for news , updates and deadlines to be posted .
Question customizations from character limitations to link integration to HTML coding capabilities.
Customizable reporting.
WORKSPACE
INTEGRATION INTO OTHER PROCESSES Same tool provides access to SLO data which
can be essential in providing support in decisions related to action plans within the program review process.
Customized reports can focus on specific questions which can be used as evidence in other planning reports (e.g., Educational Master Plan, Staffing plans, ACCJC reports, etc.).
Cosumnes River College A Homegrown Online
Program Review System (PrOF)
Why Homegrown?Faculty driven project
VisionUnderstandingProgramming expertise
Why Homegrown?Practical Considerations
Commitment to current program review model Had immediate access to reassigned timeCould create connections with other databases No budget for ongoing costsCould institutionalize and expandCould phase in the roll-out
The questions in the Midterm PrOF were clear.
The online environment was easy to navigate.
The training I received was sufficient.
The Midterm PrOF contributed to reflection about our program.
The Midterm PrOF helped us plan for the future.
The Midterm PrOF reminded us of things we had forgotten.
The Midterm PrOF reminded us of our ac-complishments.
The Direction Tab helped me.
It was helpful to access other PrOFs
0.00 0.50 1.00 1.50 2.00 2.50 3.00
Please indicate the degree to which you agree with the following statements (1 = Strongly Agree, 3 = Neutral, 5 = Strongly Disagree)
Selected CommentsI am typically the type of person that finds this
kind of activity frustrating, and a complete waste of my time. I feel PrOF has become a useful tool for real program evaluation as opposed to useless paperwork.
Filling out the sections really only took about two hours....the other hours were meeting with the fellow colleagues in the department to have a genuine program review.
Thanks for making it a smoother process for end users.
Can easily customize during implementation
Currentlyupdating and modifying PrOFproviding training to institutionalize and
fully utilize planning for development of other
applications
Customization
Information is easily extracted to inform planning and resource allocation fulfill other functions live and static extraction capabilities
System can be integrated with other systems to enhance integrity and save work
Need More Info?Dr. Katherine McLain, Dean of College
Planning and Research; [email protected]; 916-691-7144
Mark Ford, Faculty Developer and Librarian [email protected]; 916-691-7628
INTEGRATED PLANNING USING ONLINE PROGRAM REVIEW
RP ConferenceApril 1, 2013
WHY TASKSTREAM? Worked with our program review model
Could handle our complex process (e.g., multiple reviewers, collaboration with Liaisons, etc.)
Cost-effective College was already using TaskStream for SLO
assessment Can integrate with SLO component
Customization capabilities Workspaces, forms, and roles/access are created,
modified, or managed locally Reporting capabilities
Real-time analytics and Excel exporting capabilities
THE USER EXPERIENCE
Online module was tested and adjusted based on feedback from the Program Review Committee and shared governance groups
What did users like? 24/7 access Virtual spaces for collaboration One-stop shop for data, program review, resource
requests What did users find challenging?
The software learning curve Managing multiple roles (Lead Writer, Liaison, etc.) Reviewer functions
Based on preliminary feedback in fall 2012…Process evaluation in progress…
CUSTOMIZING IT FOR MESA Customized workspace template that can be
adjusted with some minor programming Workspace layouts, data, attachments, and
requirements were adjusted to fit college needs
Custom forms/fields and attachment capabilities
Created custom feedback forms for reviewers Ability to create custom rubrics in future
Exported reports to Excel for reformatting and customization
WHAT IT LOOKS LIKE: WORKSPACE LAYOUT
WHAT IT LOOKS LIKE: INSTRUCTION UPDATE FORM
WHAT IT LOOKS LIKE: NEW GOALS FORM
PULLING IT ALL TOGETHER Online module allows for a more
dynamic experience for users Information is stored and can be
viewed or exported as: A holistic report (full program review
document) OR As pieces (e.g., new goals, resource
requests, etc.) to inform planning and resource allocation recommendations
REPORTINGAggregate
Reports
Individual Program Reports
QUESTIONS? Jill Baker, Dean of Institutional
Effectiveness [email protected] 619-388-2320
Bri Hays, Campus Based Researcher [email protected] 619-388-2319
Erik [email protected]
Why TracDat? Using TracDat for SLOs
› WCC Faculty› Considered home grown
Adobe Database Survey Monkey Colleague Integration
The User Experience“TracDat: An ancient Swedish word
meaning ‘evil one’” Year 1
› Unintuitive› Unfamiliar› Unexcited› Unsure
The User Experience
Year 2› Collaborative› Time Saving› Meaningful› Useful› Still Unintuitive...but more support helps
Administration Integration with MIS
› Excel uploads...so not really Integration with Planning/Budgeting
› Standard and Ad Hoc reports› Multi-user access...except
Customization› Multi-college, similar but different
processes› Easy enough
Reporting
Reporting
Lessons Learned
• Trials, Pains, and Refinement– Implementation• Successes• Challenges
– Evolution• Cultural considerations• Next steps
Questions?
Need to contact us?
• Aeron Zentner– [email protected]
• Bri Hays– [email protected]
• Jill Baker– [email protected]
• Kathy McLain– [email protected]
• Erik Cooper– [email protected]
Top Related