The Role of Faculty Institutes in Improving Student Learning: Core Abilities ... · 2019. 7....
Transcript of The Role of Faculty Institutes in Improving Student Learning: Core Abilities ... · 2019. 7....
The Role of Faculty Institutes in Improving Student Learning: Core Abilities Assessment Report
Submitted by Minerva Holk
June 2014
1 | P a g e
Table of Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Planning for the Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 The Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Feedback from Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Appendix A Invitation to Faculty. . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Appendix B Sample Collection Materials . . . . . . . . . . . . . . . . . . . . 11 Appendix C Permission to use Student Work . . . . . . . . . . . . . . . . . 14 Appendix D Permission to use Online Student Work . . . . . . . . . . . 15 Appendix E Sample Rating Materials for Assignments . . . . . . . . . . 16 Appendix F Sample Rating Materials for Student Samples . . . . . . . 17 Appendix G Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Appendix H Faculty Evaluation Tool . . . . . . . . . . . . . . . . . . . . . . . 19 Appendix I Institute Evaluation Responses . . . . . . . . . . . . . . . . . . . . 20
Introduction
The Summer Institute was held June 17 and 18, 2014, and focused on the Thinking Core Ability.
Twenty-nine (29) faculty attended. The faculty represented all the divisions at OC and included full-time
(16) and adjunct (13). There were seasoned Institute attendees and nine who attended for the first time.
The format was slightly different from previous Institutes in that the first day included a workshop on
Quality Matters (QM) titled “Standard 2/3 Connecting Learning Objectives and Assessments.” The
workshop was presented twice and the faculty attended one session. The other half of the day was used to
evaluate sample assignments and student work. The Quality Matters sessions were held in HSS 110 and
the rating sessions were held in HSS 112. There were 16 full-time faculty (55.2%) of the participants
were and 13 (44.8%) were adjunct faculty. Nine participants (31%) had not attended a previous Institute.
An overview of the process and the results attained will be presented, followed by recommendations for
future Institutes.
Planning for the Institute
Invitations were sent using the all faculty email option. First time participants were required to
watch an on-line video prior to the Institute. Viewing the pre-Institute video was mandatory for new
Institute participants and replaced the previous use of the first hour of the Institute to orient new faculty
attendees. The video link was also available to returning participants who wanted to refresh personal
knowledge of the goals and procedures of the Institute. The video reviewed the history of Core Abilities
at Olympic College, reviewed the five Core Abilities, and explained the expectations for participants
during the two-day workshop.
All faculty were invited to submit sample assignment prompts and five samples of student work.
Faculty who were attending the Institute were requested to submit samples but few did. One possible
reason for the decrease in samples submitted was the presentations that were to be done by the faculty
who attended the Winter 2013 Institute on the second afternoon of the Institute Specific guidelines on the
requested submissions are included in Appendix B. A coversheet was attached and to be completed for
the samples submitted including the faculty assessment of the competency level for meeting the identified
Core Ability for the assignment and each student sample. Samples were accepted in hard copy or
electronically via Canvas or the Institute Coordinators mailbox. Samples were only requested from
faculty who were attending the Institute.
Submission of samples was not a prerequisite for attendance at the Institute which resulted in
fewer submissions and a more thorough evaluation of the assignments received. The limited sample
submissions resulted in a change in the structure of future assessment Institutes and will be discussed in
the recommendations. The decision was made to print enough copies of each submission for all
participants to rate them at the Institute. In general, sample collection and preparation continues to require
significant effort both in terms of time needed to follow-up with faculty, and time involved to prepare the
samples for copying. Electronic submissions decreased the work involved preparing for copying.
An outside catering company was selected to provide the lunches for the Institute. Aside from
lunch, coffee was provided each morning and light refreshments were provided in the morning before
beginning the Institute as well as in the afternoon. Name tags for participants, and packets containing
several key documents were prepared by the Office of Instruction. Lunch orders from registered faculty
were collected, and the catering order submitted, by the Office of Instruction.
The Institute
Institute participants were given Quality Matters™ packets containing pertinent information
related to learning objectives, assessment and measurement, and a Quality Matters™ Rubric Workbook
for Higher Education. The packet also included a verb wheel based on Bloom’s Taxonomy.
Information relating to the Core Abilities was also provided including agenda (See Appendix I), the
College’s Mission Statement and Guidelines for Assessing Core Abilities, Outcomes and Rubric for the
Thinking Core Ability, a Glossary of Terms for Core Abilities Rubrics. Outcomes and rubrics for the
other Core Abilities (Communication, Global Perspective, Lifelong Learning, Information Literacy and
Technology) were available for faculty. Each participant was given a name tag. Table numbers were not
assigned at this Institute due to the last minute accommodation of several attendees’ schedules and being
in separate rooms for each half of the first day. The faculty were given the option of self-selecting their
Table collaborators and to use the same group on day two of the Institute.
The self-selected table groups were a mix of experienced and new faculty, as well as adjunct and
full-time (Table 1). Working in groups of four, with one group of five, assignments and student samples
were rated using the Thinking Rubric developed by the Core Abilities’ Taskforce. There was
representation at the Institute from each division at Olympic College (Table 2). The participants engaged
in the rating process with enthusiasm and diligence.
The initial step in the process was a group norming session in which each participant read the
same assignments and the samples of student work submitted for the specific assignments. The
participants rated both the assignment and the respective student samples according to the four levels of
proficiency utilized in each Core Ability: Emerging, Developing, Competent and Strong. Individual
evaluation was followed by a collaborative discussion during which group members at each table collated
their results and created a ‘table rating’ for the assignment and the sample. The table results were then
reported to the group at large and discussion of the results followed. An important component of the
norming session is to increase consistency in rating and identify possible biases in the evaluators. Of the
seven groups, the norming assignment was rated as providing the opportunity to meet Thinking Outcomes
one and two at the developing level. The student samples provided evidence that Thinking Outcomes one
and two were met at the emerging level. There was active participation in the discussion that followed
resulting in clarification of the purpose of the Institute which is to assess student learning using the Core
Abilities. After the norming session, the individual groups applied the rating process to assignments and
samples specific to the Thinking Core Ability. The Mission Statement for Assessing Core Abilities
states: Olympic College seeks to improve teaching and learning by focusing inquiry to ensure that
students are getting ample opportunities to develop Core Abilities; and students are performing
sufficiently on Core Abilities. There was uniform agreement about the Outcomes met by the assignments
and student samples.
The tables below give the specific information according to teams arranged for the review and
discussion of assignments and samples each day.
Table 1: Institution Level Assessment of Core Abilities Summer Institute 2014 Thinking
Group 1 (AM) Carey, Diane Land, Cynthia
Westlund, Mark Bermea, Nancy Elauria, Angela Lawrence, Amy
Group 2 (AM) Rubin, Tova
Gold, Emma Kirk, Nari Latham, Colin Tuncol, Goker
Group 3 (AM) Farley, Sandra
Group 1 (PM) Bright, Kathy
Rubin, Tova *# Kerdus, Mary
Krystal, Barbara *
Group (PM) Johnson, Dawnette # Hayward, Victoria #
Parker, Barbara Mackaben, Kandace *
Group 3 (PM) Mulligan, Anne
Baldwin, Kathleen *# Stevens, Leon * Latham, Colin *
Group 4 (PM)
Blackwell, Kevin Puskarcik, Russ *# Pellock, JD DeWire, Elinor *#
Key:
*: Part time #: new
Table 2: Institutional Level Assessment of Core Abilities Summer Institute 2014 Faculty Participant List by Division
Business and Technology
Mathematics, Engineering, Sciences and
Health
Nursing Social Sciences and Humanities
Library
Bermea, Nancy Elauria, Angela Hayward, Victoria Baldwin, Kathleen Bright, Kathy
Blackwell, Kevin Lawrence, Amy Holk, Minerva DeWire, Elinor Carey, Dianne
Farley, Sandra Pellock, JD Johnson, Dawnette Gold, Emma
MacKaben, Kandace
Rubin, Tova Kerdus, Mary Hendrickson, Kimberly
Parker, Barbara Tuncol, Goker Mulligan, Anne Kirk, Nari
Puskarcik, Russ Krystal, Barbara
Stevens, Leon Land, Cynthia Westlund, Mark Colin Latham
Executive Summary
The Core Abilities Summer Institute 2014 focused on the Thinking Core Ability as identified in the
Olympic College Master Plan for Core Ability development and assessment. The outcomes for the
Thinking Core Ability are:
Outcome 1: Graduates engage in critical analysis. Outcome 2: Graduates engage in creative problem solving. Outcome 3: Graduates engage in quantitative reasoning. The Data Each participant was given a stack of ratings sheets (See Appendix E & F). Each faculty member
was asked to rate the assignment prompt and the set of student samples that accompanied the assignment
prompt as Emerging, Developing, Competent or Strong according to the Thinking Core Ability Rubric.
After each participant had rated the sample and the accompanying assignments, the tables were asked to
discuss both the assignment and the samples and to reach a consensus rating for the assignment and for
each sample. The table groups reviewed and rated the assignments individually and then as a group and
discussed the ratings before assessing the student samples. After the student samples had been evaluated,
the groups discussed the proficiency levels for each assignment and determine the leveling from their
group for each assignment and the levels attained by the student samples. The discussion also provided an
opportunity to write comments for the Institute Chair to send back to the faculty who submitted the
sample. The limited time at this Institute for evaluation of assignments and samples worked out well as
there were three assignments and 16 student samples were reviewed. Two student samples were five to
six pages and one was two pages. The assignments ranged from one page to four.
Sample one was from a nursing faculty. The assignment was rated at competent for Outcomes
one and two and not applicable for outcome number three. The assessment by the faculty who submitted
the assignment was consistent with the assessment by the evaluators at the Institute. Four student samples
were rated as Outcome One: emerging, one; developing, two, competent, one; and strong, one. For
Outcome Two the samples were rated: emerging, one; developing, one, competent, one; and strong, one.
Sample two was from an English faculty. The faculty had rated the assignment as developing and
addressed two Core Ability Outcomes. The faculty at the Institute rated the assignment as competent for
the first Outcome and did not identify the assignment as meeting either Outcome Two or Three. The
student samples had been rated by the faculty submitting as emerging or developing. The Institute
evaluation was that Outcome One was met at the competent level by three students, developing by two
and emerging by one student. There was unanimous agreement by the Institute assessment that Outcomes
Two and Three were not addressed by the assignment or the student samples.
The third sample assignment was rated by the submitting faculty as addressing Outcomes One
and Two at the Strong level. Of the five assignments submitted, two were rated by the submitting faculty
as strong and three were at developing. The Institute assessment supported that Outcomes One and Two
were addressed by the assignment and provided opportunity for the students to gain a measure of
competency at the level of competent. The student samples received ratings between less than emerging
and strong from the Institute groups. One student sample was rated as strong, two competent, one
developing and one less than emerging. The samples the faculty rated as strong were rated by the
Institute as one strong and one competent.
1. Opportunity for Student Growth
The assignment ratings and the sample ratings by the faculty submitting and the faculty at the
Institute support that the assignments provide sufficient opportunities so that students can learn,
grow, and improve their comprehension of the material being addressed in the academic unit
being taught. The assignments submitted were rated by faculty submitting as either developing,
competent, or strong. The submission rated at the developing level was from a course offered
early in the student’s college experience, likely first quarter. The assignment was also completed
early in the quarter. There were similarities between the groups and the ratings assessed.
2. Consistency
There was consistency between the raters as to whether the assignments and student samples
addressed the Thinking Core Ability. The results indicate consistency between; a) faculty at
different tables rating the same samples and b) faculty ratings of their own assignments and the
ratings given to those assignments by their peers at the Institute. The small number of
assignments and samples presented limits the interpretation of the findings.
The faculty members who attended the Institute were energized by the process and worked very
diligently. According to the evaluation forms completed at the end of the Institute, more understanding is
needed by some faculty about the Core Abilities. The increased awareness of the Core Abilities was
particularly evident among faculty attending an Institute for the first time. This reflects positively on the
sustained campus-wide efforts to increase awareness of, and buy-in for, Core Abilities at Olympic
College. Many participants expressed continued enthusiasm for integrating Core Abilities into their
classrooms and reviewing their assignments to improve their clarity and usefulness vis-à-vis the
integration of Core Abilities. Review of the Core Abilities and evaluation from the attendees is leading to
potential consolidation of the committees at Olympic College involved with similar aspects of assessing
student learning. The connection with the Quality Matters™ may prove beneficial to the group at this
Institute according to the evaluation information and many faculty are interested in increasing their
abilities to assess student learning.
The second afternoon of the Institute was designed to have faculty share updates on the projects
they worked on at the Winter Institute 2013. The presentations were representative of all disciplines of
the College and provided an opportunity for faculty to showcase some of their excellent work as well as
discussion of some challenges related to the assessment process. The workshop on Quality Matters™ had
stimulated the need for deliberateness in the assessment process and the importance of clearly defined
outcomes for the identified learning goals. Several faculty included on the evaluation form that they
would like to have more opportunities for this type of association with their colleagues. The presentations
ranged from brief updates of syllabi revision and course development to the development of a new
program and curriculum. Two faculty presented a project designed to showcase student accomplishment
of the outcomes and the collaboration between faculty teaching different sections of the same course to
identify the Core Abilities addressed and accomplished. It was an excellent example and the faculty
present were very positive in their feedback. The sharing from the faculty provided insight into the
challenges of teaching and the similarities of many situations even though the course content may be very
different.
Feedback from Participants
An Institute Evaluation Form was distributed at the end of the second day. It was completed by
26 of the 29 participants. The evaluation tool and a complete list of the comments received are included in
Appendices H and I. Post-it Notes were placed on every table and participants were encouraged to write
suggestions and comments throughout the Institute. In previous Institutes the Post-it Notes were used by
many faculty but no comments or questions were received using Post-it Notes at the Summer 2014
Institute. The overwhelming belief that the Core Ability Institutes are worthwhile was reaffirmed by 23
faculty selecting the yes option, one chose no and two did not respond. Eighteen faculty indicated they
will be changing learning outcomes or assessments based on what they learned in the Institute; one opted
for maybe; two said no; and two did not respond. As a result of attending the Institute; 23 said they would
attend a similar event in the future, 21 chose the strongly agree, two agreed, and one selected ‘no
opinion’.
The recommendations below are based on information gleaned from the Institute Evaluation
form, observations made during the Institute by the two Coordinators, and data derived from the rating
sheets.
Recommendations
1. Make explicit the link between this work and Olympic College’s accreditation by the Northwest
Commission on College and Universities (NWCCU) and what accreditation really means to
Olympic College.
Some faculty members do not see, or understand, the link between the work being done on
Outcomes and the stipulations placed on the College by our regional accrediting commission.
2. Update tutorial available in Tegrity (no longer available at Olympic College) to continue
streamlining the work we do at the Institutes and make available online using Panopto.
The Pre-Institute Workshop has been on Tegrity for two years and will be updated to make it
more pertinent to the changes we are making related to the assessment and tracking of
competencies in the Core Abilities. One attendee at the 2014 Institute commented on the
evaluation tool that “the video could be condensed”. Having new Institute participants view the
video before the Institute has streamlined the introductory period and enabled faculty to get
started with the norming and evaluation sooner increasing efficiency at the Institute. It was also
suggested that a Tegrity (Panopto would be used as Tegrity is no longer an option.) lecture on the
basics of Course Mapping would be helpful and the LAC will consider moving this forward. Key
elements of this lecture might include: how mapping is done (a demonstration of the database),
why it is done, and some common errors and considerations.
3. Revise and implement change in sample collection process.
Collection of the sample assignments and student work requires significant time on the part of the
Core Ability Chair. Submitting the samples takes time and effort on the part of the faculty
member who are contributing samples for evaluation. Changes made to allow submission of
samples electronically has been minimally adopted. Discussions are underway in the LAC to
investigate the feasibility of collecting samples via Canvas©. Collecting via Canvas© will
increase objectivity of selection of student samples and copies may be made by the LAC, tracking
will also be facilitated. Rating assignments and student work is a necessary requirement to
measure learning outcomes at Olympic College and a broad range of samples is desired,
employing other tactics to solicit samples is necessary. Another change to the collection of
samples will be implemented before the Winter 2014 Faculty Institute to specifically choose
classes on the list of approved as meeting Core Ability graduation requirements, and being taught
during fall quarter, 2014. The method implementation will be included in the Winter 2014
Institute Summary Report.
i. Detailed instructions will be provided to faculty from whom samples have been
requested. This will make the process of sample preparation and duplication more
streamlined and efficient for the facilitators. Specifically, in addition to the assignment
and grading/assessment rubric, participants will be asked to submit 10 samples which
should be single-sided, paper-clipped rather than stapled, and all names must be removed
prior to submission.
ii. The option of submission via Canvas© is anticipated for sample collection during fall
quarter, 2014. The utilization of Dropbox© may also be an option to support faculty
submissions electronically throughout the quarter. A conversation has been initiated with
Kathy Bright, Director of Distance Learning, to determine the most appropriate
implementation process – specifically which platform is utilized by the greatest number
of faculty? Canvas©, Dropbox.com, Wiggio, Googledocs, or some other site?
4. Consider having chairperson for Institute prepare anonymous listing of faculty assessment of
assignment and student samples.
Faculty are asked to rate the assignments and the samples submitted. Since the Winter 2012
Institute several faculty have expressed interest in knowing how the faculty member who
submitted an assignment had rated the assignment. A concern of the Institute chair is the
possibility that knowing how an assignment or student sample was “graded/rated” by the
instructor before rating it oneself (at the Institute) would compromise the assessment ratings at
the Institutes. The information is available in the raw data and could be included in the Institute
Summary but the faculty would like to know sooner. Further discussion amongst the LAC
members will be done before this information is shared for specific assignments and samples.
5. Continue the conversation with the campus community why stipends are necessary for the
continuation of this work.
Faculty consistently report on the feedback survey that they would not continue participating in
the Institutes if a stipend were not offered. Yet, in these tough economic times, alternatives to
stipends must be considered. However, given the fact that the Institutes occur on non-contract
days, best practices stress that those involved evaluating student outcomes must participate
voluntarily (as opposed to, say, program evaluation by IPP which is mandatory). Additionally the
time required to conduct this work is difficult to undertake during the quarter while faculty are
teaching and grading.
6. Continue curriculum development and incorporation of the Core Abilities at the Institutes.
Feedback at the Summer 2014 Institute was positive about the cross-discipline efforts at the
Institute. Some faculty members would like examples of how other instructors have implemented
Core Abilities as a division. Sharing examples of instructors’ incorporation of Core Abilities into
assignment prompts directly was also desired.
7. Feedback to those who submit samples requires increased systemization.
Faculty requests for feedback on their samples and assignments is challenging using the current
method which is tasked to the Institute chair. The tracking of the faculty’s submission and the
table ratings must be collected and an email composed and sent to the faculty. It would be
helpful if an automated system could be implemented.
8. Sharing examples of instructors’ incorporation of Core Abilities into assignment prompts directly
was also desired.
Faculty who participated in the Winter Institute 2013 shared the progress made in their courses on
the afternoon of the second day of the Summer Institute 2014. The feedback on the evaluation
form was very positive for this section of the Institute and should continue to be a part of the
Institutes.
9. The Quality Matters™ Workshop received high ratings on the evaluation forms. The faculty
requested additional similar workshops to support their development of outcomes and appropriate
assessment methods for their courses. See the Professional Development Subcommittee’s Year
End Report for Learning Assessment Council, May 2014 for additional information about Quality
Matters™.
Appendix A Application to Participate in The Core Abilities Institute
Summer 2014
I would like to participate in Institutional Level Analysis and Curriculum Development tied to Core Abilities. As part of this process, I agree to engage with other faculty in collaborative discussion and curriculum development on Tuesday, June 17th from 09 AM-4 PM and Wednesday, June 18h from 09 AM-4 PM. The location will be sent to attendees after receipt of application. Faculty attending for the first time are required to watch the Pre-Institute Workshop on Tegrity. The link will be included in the follow-up email. I understand I will receive a $400 stipend for participating in this entire two-day process.
To apply, please complete this sheet and send it as an email attachment to: ([email protected]) by Friday, May 30, 2014.
Faculty Name: __________________________ Discipline: ______________________ Division: ________________ Do you work full-time or part-time at OC: ________________
Please check this box if this is your first time attending a Core Abilities Institute: ___ Please check this box if you need to integrate Core Abilities into your syllabi: ___ Please check your first preference for the Quality Matters Workshop: AM_____ PM_______
There is limited funding for stipends so please reply early for your best chance to participate.
An effort will be made to include faculty (full and part-time) representing all of the various divisions as well as first time attendees.
All faculty are encouraged to apply.
Appendix B
DUE JUNE 10, 2014 Contact Person (faculty name, please print): _____________________________ (this name will be removed before samples are evaluated) Course Number: _____________ Number of Samples1: ________ (example: SOC 190, MATH& 264) (Please submit 5 random samples) Course Background: Please tell us a bit about your course:
• Are there any prerequisites on the course? If so, please list: _____________
• When is this assignment given? Early Middle End of Course ?
• Does this Assignment build on other areas of the course (previous assignments)? If so please tell us what previous assignments have been completed that would help understand the quality of the samples submitted: __________________________ ____________________________________________________
• Are there any other instructions/resources/preparation (e.g. library tutorial or in-class demonstration) given to students that would help contextualize the samples? If so, please list: ____________________________________________________
1 Please see the section “What NOT to Submit” on p. 4 for some guidelines on the kinds of samples we can and cannot use.
COVER SHEET SUMMER 2014 SAMPLES FOR ASSESSING CORE ABILITIES
• Please number each sample in the top right corner in pencil. Then please tell us the level you think each sample attains:
Sample 1 Emerging Developing Competent Strong Sample 2 Emerging Developing Competent Strong Sample 3 Emerging Developing Competent Strong Sample 4 Emerging Developing Competent Strong Sample 5 Emerging Developing Competent Strong
If you would like feedback about what we learned after using your samples at the institute please indicate below:
Yes, I would like feedback No, I do not need any feedback
Please select one Core Ability the work is most clearly demonstrating.
Thinking Outcome 1: Graduates engage in critical analysis. What general level do you feel this assignment addresses (check only one box for each outcome)2:
Emerging Developing
Competent Strong
2 Please see any of the rubrics for a definitions of the standards expected at each of these four levels.
Thinking Outcome 2: Graduates engage in creative problem solving. What general level do you feel this assignment addresses (check only one box for each outcome):
Emerging Developing
Competent Strong
Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level do you feel this assignment addresses (check only one box for each outcome):
Emerging Developing
Competent Strong Send submission via campus mail to: Minerva Holk: [email protected] What NOT to Submit
• When selecting the 5 samples to submit please select random samples so that we can obtain a representative cross section of the student population. In short, we do not only need or want examples of the best work.
• Do not submit graded work – only clean, unmarked copies of student work should be submitted.
• Do not submit copies of multiple choice, True-False, short answer or matching tests.
• Do not submit copies of entire student course or program portfolios or journals. Components of a portfolio or journal that demonstrate a particular Core Ability may be submitted.
• Do not submit samples in a foreign language or that cannot easily be understood by faculty outside your discipline (for example containing complex chemical or mathematical equations)
• Materials will not be returned - do not submit original student work or work that you need returned for grading or other purposes.
• Do not submit samples from developmental or ABE courses.
• Do not submit samples without the corresponding assignment and cover sheet.
Thank you for your help with these efforts!
Appendix C
PERMISSION TO USE STUDENT WORK
Quarter & Year: Instructor: As indicated by my signature below, I authorize Olympic College to use my work as part of one or more Olympic College activities that are specific to campus-wide assessment for student learning outcomes. Your name and other identifying information will be removed. PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________
Appendix D
PERMISSION TO USE STUDENT WORK: ONLINE CLASS
Hello class,
As of Summer 2012, new students are required to take coursework addressing each of five Core Abilities. The Core Abilities are: Communication, Thinking, Information Literacy and Technology, Lifelong Learning, and Global Perspective. This class (Class name, Number) should satisfy the thinking and XXX (fill in which core abilities your class addresses) core abilities, while meeting the existing quantitative reasoning requirement, too.
In preparation for this requirement there will be an institute during your break in which faculty will evaluate student work from classes that might be used to meet the requirements to determine whether students are, in fact, demonstrating that the Core Abilities are addressed.
I will be submitting X Project (fill in title of assignment) to this group. Your name and any identifying information will be removed before any other faculty see the projects, and all work will be shredded once the papers have been evaluated.
If you do not want your work to be included, please let me know before Friday, and I will not submit it.
Thank you in advance,
Your Name
Table #: _______
SAMPLE LETTER: _______
TABLE SUMMARY: _______
Please indicate outcome and the level of attainment the work is demonstrating (check all that apply):
Thinking Outcome 1: Graduates engage in critical analysis. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
Thinking Outcome 2: Graduates engage in creative problem solving. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
APPENDIX E EVALUATION TOOL-THINKING ASSIGNMENT
WINTER 2011 ASSESSING CORE ABILITIES TABLE SUMMARY:____
Suggestions for faculty:
APPENDIX F EVALUATION TOOL-THINKING STUDENT SAMPLE
WINTER 2011 ASSESSING CORE ABILITIES TABLE SUMMARY
Table #: _______
SAMPLE LETTER: _______
TABLE SUMMARY: _______
Please indicate outcome and the level of attainment the work is demonstrating (check all that apply):
Thinking Outcome 1: Graduates engage in critical analysis. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
Thinking Outcome 2: Graduates engage in creative problem solving. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):
Emerging Developing Competent Strong
Suggestions for faculty:
Appendix G
Agenda for Summer Institute 2014
Tuesday, June 17, 2014: 08:00 a.m. to 5:00 p.m. Faculty Institute/Quality Matters Workshop
Lunch Provided at 1200
Faculty Institute
08:00-12:00 and 1:00 p.m.-5:00 p.m. (Alternates with Quality Matters)
a. Core Abilities, definitions and explanations of rubrics b. Explanation and discussion of the goal(s) of the Institutes in general c. Norming in preparation of assessing assignments and student work d. Assessing assignments and samples of student work
Quality Matters: 0800 a.m.-12:00 p.m. and 1 p.m.-5:00 p.m. Alternates with Faculty Institute Wednesday, June 18, 2014: 09:00- 3:30 p.m.
09:00-11:00: Assessing assignments and samples of student work
11:00-12:00: Presentations from Winter Institute
12:00-12:45: Lunch provided
12:45-1:45: Presentations from Winter Institute 2013
1:45-2:45: work on either outcomes or course mapping of Core Abilities in prof/tech for graduation requirements
2:45-3:45: Large Group Debrief: Post-Its, Progress, and Questions
Appendix H
Faculty Institute Summer 2014 Evaluation Summary Data
Please add any additional comments on the back. Thank you for participating!
1. This Institute included a professional development workshop for the first time. Do you think professional development is an important component for an Institute?
a. Yes b. No 2. Was the topic for the Quality Matters workshop appropriate/or related to the other work being
done at the Institute? Circle one: Yes No a. List other topics that would be helpful to you
3. Was the information useful? Circle one: Yes No
4. Will you be changing learning outcomes or assessments based on what you learned? Circle one: Yes No
5. Are you interested in additional training and/or information to increase your understanding of the Core Abilities and the process set up to assess them?
a. Outcome development, either course or assignment b. Rubric development c. Grading on Canvas d. Other?
________________________________________________________________________________________________________________________________________________
6. What did you like best about this Institute?
7. Would you participate in a similar event in the future? □ Strongly agree □ Agree □ Disagree □ Strongly disagree □ No opinion
8. Do you have suggestions to improve the event? Circle one: Yes 11 No If yes, please elaborate:
9. What questions remain about Core Abilities and assessment of student learning at Olympic
College? 10. Do you see a need for more workshops on course mapping with Gloria Martin? Circle one:
a. Yes b. No 11. Do you have suggestions for the timing of the Institutes? Rank in order of preference
a. One day b. Two consecutive days c. First non-contract day after end of quarter d. Second non-contract day after end of quarter
Appendix I
Faculty Institute Summer 2014 Evaluation Summary Data
Please add any additional comments on the back. Thank you for participating!
1. This Institute included a professional development workshop for the first time. Do you think professional development is an important component for an Institute?
a. Yes 25 (one added definitely) b. No 0 2. Was the topic for the Quality Matters workshop appropriate/or related to the other work being
done at the Institute? Circle one: Yes 22 No 0 Not sure (added by evaluator): 1 a. List other topics that would be helpful to you
i. viewing other online courses ii. I would do separate workshops for each standard so that we would go in more
depth, rather than offer the same workshop twice & split the group iii. Accessibility and core abilities- what does a “strong” core ability look like for
someone with a disability? iv. Student motivation v. Designing to prevent cheating
vi. Other of the QM workshops. Also, function of Canvas. Since we are changing to Panopto, that would be good now.
vii. Not sure viii. Rubic [sic]
ix. Accessibility would be good. Repeat outcomes & assessment for those who have not done them.
x. More Quality Matters training. xi. I would like to see other standards covered every month for a half day if
possible.
3. Was the information useful? Circle one: Yes 23 No 1 I really didn’t get new info; Two (2) people gave no response, one added “I will take a look at them again”
4. Will you be changing learning outcomes or assessments based on what you learned? Circle one: Yes 18 (added comment by one: “I will be assessing now”) No 2 Maybe 1 (option not on evaluation form. No response 2; added comment “Creating a blended class and need to redo all outcomes”
5. Are you interested in additional training and/or information to increase your understanding of
the Core Abilities and the process set up to assess them? a. Outcome development, either course or assignment 9 b. Rubric development 17 c. Grading on Canvas 11 d. Other?
i. No: 1; No response 1; ii. advanced Canvas;
iii. “Is this data driven? What are the statistics showing this is a benefit to students”.
iv. All of the above. v. Anything that will help us do things faster.
6. What did you like best about this Institute?
a. Working with peers and seeing their perspectives and comments b. Learning from other faculty about their projects c. Interaction with colleagues d. Group support e. No answer f. Aligning outcomes with assessment g. Opportunity to work on my courses h. Casual atmosphere i. QM session j. Collaboration with other instructors k. Working with faculty in other disciplines l. Getting to hear what people have done with this information on 2nd afternoon.
Appreciate the venue & food. m. I liked the organization of this institute. n. QM workshop o. Looking at student samples; assessing them, really enjoyed the QM session- It helped to
clarify several things for me p. Interaction and discussion to develop usable perspective q. Working with other faculty, friendly atmosphere. r. Working with fellow faculty/see what I am doing right based on the presentations. s. Interaction and working with individuals from other areas of study/department t. The presentation of winter institute was the most informative u. Working with other disciplines & with my own to level assessment v. Networking with other faculty and discussing our challenges w. The chance to work with other profs & the QM training x. Meeting other disciplines and collaborating. Seeing how other faculty apply core
abilities.
7. Would you participate in a similar event in the future? □ Strongly agree 12 □ Agree 11 □ Disagree □ Strongly disagree □ No opinion 1
8. Do you have suggestions to improve the event? Circle one: Yes 14 No 7 NoResponse 1 If yes, please elaborate:
a. More examples with rubrics b. More planning needed. QM would make more sense if I was told to bring my
syllabus/course outline c. Give instructions to participants before 1st day on what they will be working on/should
bring to class. d. One selected yes but did not elaborate
e. Show examples from all disciplines that includes professor’s evaluation of that work/example
f. NO MORE RATING STUDENT SAMPLES. g. Overall less time is needed to complete tasks. Could have accomplished all the tasks in ¼
of the given time. h. Breaks every 2 hours; other activities besides reading, especially just after submitting
grades i. I would like to have had some of the terminology and associations with the Core
Abilities evaluations explained before doing the evaluations j. More time for interacting with staff from a variety of disciplines- and, a chance for each
staff to share what is new in their field or that they have learned in the past three months.
k. 0900 start was better. More information in advance on what would be covered. It was fine, but I did not know I’d be evaluating course/student work.
l. The exemplars & evaluation are a bit long. (May not be exact words, could not make out word after eval.)
m. It seemed less organized & structured than in the past. Participants & leaders were coming & going & it felt like a little of both were not taking it as being important
n. Better organization and pace. Maybe stricter assignments. o. (u on data sheet) work with discipline on specific event p. Agenda in advance. Structured breaks (during 2 sessions in separate rooms we had to
interrupt to get coffee.)
9. What questions remain about Core Abilities and assessment of student learning at Olympic College?
a. Getting campus involved b. ? c. What happens to our evaluations of student work? d. No response e. No response f. No response g. No response h. No response i. NA j. No response k. Will our evaluations of our courses be double checked by anyone l. No response m. No response n. No response o. No response p. Need more!! q. No response r. Are we going to be required to put it in the syllabus s. Several-will do workshops etc. t. No response
u. Other core abilities & criteria v. None w. None right now x. Was not familiar with core abilities. I have a better understanding.
10. Do you see a need for more workshops on course mapping with Gloria Martin? Circle one:
a. Yes 13 (maybe a video) b. No 8 b. Comment: I don’t know what this (course mapping) is. c. No selection but suggested: maybe a good review d. Comment: I am not familiar with course mapping & I need a class
11. Do you have suggestions for the timing of the Institutes? Rank in order of preference
a. One day 4, 1, 1, 4, 2, 1, 1, 1, 1 (Is that possible? (2 days of work zipped into a day?), 2, 4, 1
b. Two consecutive days 1, 1, 1, 1, 1, 1, 1, 3, 1, 3, 3 (circled option also), 1, 1 c. First non-contract day after end of quarter 3, 1, 3, 1, 1 or 2, 2 or 3, 4, 1, 1,1 d. Second non-contract day after end of quarter 2, 1, 1, 2, 1 or 2, 2 or 3), 1, 2, 1 (this person
also selected one day) added Thanks! , 1
Additional comments for question 11 from Evaluation Forms: -Winter institute provided opportunity to do assessment project (rather than just grading student work) -Three people selected two options without prioritizing -One, no answer -Two added and e option: after grades are due; one went on to select or c. -Additional comment: nice video…could be edited down to ½ the time -Not sure what this comment was in reference to but someone wrote: “8 standards running” -The pre-video helped to review. Thanks! -One person selected b, c, & d without prioritizing and stated open to when classes are given.
0
2
4
6
8
10
12
One day Two days 1st NC day 2nd NC day
Num
ber p
refe
rrin
g Institute Timing Preference
1st
2nd
3rd
4th