2011-2012 Annual report - University of Ottawa · SenateCommitteeonTeachingandTeachingEval...

Senate Committee on Teaching and Teaching Evaluation

Annual Report September 2011 | August 2012

Senate Committee on Teaching and Teaching Evaluation – Annual Report

2

Table of Contents

Introduction 3

Committee Members 4

Executive Summary 5

Terms of Reference 1, 2 and 3 5

Terms of Reference 4 8

Terms of Reference 5 and 6 10

Course Evaluation Statistics 17

Terms of Reference 7 and 8 20


3

Introduction

I am pleased to present the annual report of the Senate Committee on Teaching and Teaching Evaluation. The Committee is a standing advisory committee of the Senate. It reports to the Executive Committee of the Senate and advises the Senate on all aspects of teaching.

Terms of Reference:

1. To promote teaching and learning excellence in all its forms as well as the importance and the innovative character of teaching at the University.

2. To implement initiatives that influences the quality of professors’ teaching experience at the University (tools, practices, environment and policies).

3. To implement strategies to ensure appropriate recognition of excellence in teaching during an academic career.

4. To administer the Teaching and Learning Grants Program and the Special Grants Program for Innovative Pedagogical Material in French.

5. To implement strategies to ensure appropriate recognition of excellence in teaching during an academic career.

6. To oversee maintenance and development of tools that enable students to provide their input on the quality of teaching.

7. To ensure the development of evaluation tools that take into account the different types of teaching and the academic support offered at the University of Ottawa.

8. To facilitate and promote communication with the entire academic community to better meet its evaluation needs.

9. To ensure that evaluation is promoted to students and professors.

10. To submit an annual activity report to the University Senate.

As Chair of the Committee, I would like to sincerely thank all members who actively contributed to the meetings. Their expertise is very valuable and their contribution is vital. Finally, I would like to make special mention of the work done by Madeleine Boisvert, Committee Secretary.

Chair of the Senate Committee on Teaching and Teaching Evaluation

Yves Herry


4

Committee Members

The Senate Committee on Teaching and Teaching Evaluation is comprised of:

The Associate Vice-‐President, Teaching and Learning Support Ø Y v e s H e rry , C o m m itte e C h a ir

The following full-‐time faculty members of each faculty, recommended by the Vice-‐President Academic and Provost and appointed by the Executive Committee of the Senate:

Ø A m ir A tta ra n (C o m m o n L a w ) Ø K o u k y F ia n u (A rts ) Ø Jo h n L e d d y (M e d ic in e ) Ø R o b e rt L a g a n iè re (E n g in e e rin g ) Ø M a rtin M e u n ie r (S o c ia l S c ie n c e s ) – sa b b a tica l le a v e Ø C o lin M o n tp e tit (S c ie n c e ) Ø D e n is N a d e a u (D ro it c iv i l) Ø V io la P o lo m e n o (H e a lth S c ie n c e s) Ø M a u ric e T a y lo r (E d u c a tio n )

Two members of the part-‐time teaching staff, one from Humanities and one from Science, recommended by the Vice-‐President Academic and Provost and appointed by the Executive Committee of the Senate:

Ø L a w re n ce H a rris (H u m a n itie s ) Ø D a re n e T o a l S u ll iv a n (S c ie n c e )

Two undergraduate student senators, appointed by the student senators: Ø N ich o la s Z o rn , u n d e rg ra d u a te stu d e n t se n a to r Ø V a c a n t: 2 n d u n d e rg ra d u a te s tu d e n t s e n a to r

Two graduate student senators, appointed by the student senators: Ø B re n n a Q u ig le y , g ra d u a te stu d e n t se n a to r Ø Jo se p h H ic k e y , g ra d u a te stu d e n t se n a to r

The Committee may invite the following people to its meetings, for consultation purposes: Ø N a n cy V é z in a , P ro g ra m s M a n a g e r, C e n tre fo r U n iv e rsity te a ch in g

Guests: Ø Y o la in e R u e l, M a n a g e r, Senior Policy Officer, Accessibility and Diversity Ø L u cie M o rg a d o , E d u ca tio n a l D e v e lo p e r, Coordinator, Strategic Initiatives (SASS)

Secretary:

Madeleine Boisvert, coordinator, Evaluation of Teaching and Courses (Office of the Vice-‐President Academic and Provost), Committee Secretary

The Committee met six times in 2011–2012.


5

Executive Summary

This annual report presents the accomplishments of the Senate Committee on Teaching and Teaching Evaluation as well as initiatives discussed under each term of reference, below. The report covers the period from September 2010 to August 2011. Terms of Reference 1, 2 and 3

1 : To promote teaching and learning excellence in all its forms as well as the importance and the innovative character of teaching at the University.

2 : To implement initiatives that influence the quality of professors’ teaching experience at the University (tools, practices, environment and policies).

3 : To implement strategies to ensure appropriate recognition of excellence in teaching during an academic career.

In order to promote teaching excellence, innovation in instructional methods and the quality of students’ learning experiences, the Committee works hand-‐in-‐hand with the University’s Centre for University Teaching (CUT). The Director of the Centre is a Committee member by invitation. The Director provides the Committee with information on proposed activities or those being developed and solicits the comments and suggestions of Committee members related to activities designed to promote teaching excellence, innovation in instructional methods and the quality of students’ learning experiences.

The Centre for University Teaching’s mandate is to support the faculties and teaching staff by providing a range of activities related to all aspects of university teaching and pedagogy. The CUT offers professional development programs, specialized courses on teaching methods, individual or group consultations with teaching staff, departments and faculties. The Centre organizes events, presentations and activities to encourage reflection on the profession of teaching. The CUT also runs the new professor orientation program (for regular and part-‐time teaching staff as well as teaching assistants) provides teaching observation sessions aimed at offering teachers feedback on their teaching strengths and areas for growth, as well as provides support for professors applying for teaching and learning grants or in preparing nominations for the excellence prizes and awards.


6

Main activities – September 2011 to August 2012:

Activity Dates Number of participants

Workshops on university teaching (52) Fall 2011 and Winter 2012 semester

269 participants

CUT Lecture Series (6) Fall 2011 and Winter 2012 semester

112 participants

Book Club Winter 2012 semester 20 participants

Lunch for TAs (4) Winter 2012 semester 46 participants

ESG 5300 and ESG 5500 courses Fall 2011 and Winter 2012 semester

36 students

ESG 8300 and ESG 8500 practicum Fall 2011 and Winter 2012 semester

3 students

Orientation Day for Teaching Assistants September 8, 2012 445 participants (avec FSS)

Kesarwani Lecture

Panel – December 2011

Panel – February 2012

Panel – April 2012

91 participants

51 participants

56 participants

Online Forum for Part-‐Time Professors Fall 2011 and Winter 2012 semester

62 participants

Course Design Institute 2012

Institut sur la conception de cours 2012

August, 7 to 10, 2012

August, 14 to 17, 2012

22 participants

22 participants

2012 Orientation Program for New Professors

August, 21 to 23, 2012 More than 50 professors

Personalised Consultations Fall 2011 and Winter 2012 semester

Over 110 professors (more than 320 hours of consultation)

CUT Publications Fall 2011 and Winter 2012 semester

3 Short Texts, 3 Short Guides and 5 Assessment Tools for TA (French and English)


7

Percentage of participants by type

Chairs in university teaching

As part of the Destination 20/20 strategic plan, the Centre for University Teaching has created chairs in university teaching, which highlights the University of Ottawa’s commitment to excellence in teaching. These chairs aim to:

• promote innovative teaching and learning practices that will benefit the entire university community;

• recognize the value of leadership and excellence in university teaching; and • support professors who are committed to furthering research on teaching and learning.

The Centre for University Teaching (CUT) has created a website (http://www.saea.uottawa.ca/CPU/index.php?option=com_content&view=article&id=171&Itemid=657&lang=en=article&id=171&Itemid=657&lang=fr) that promotes the chairs and describes the nomination processes.

Members of the Senate Committee on Teaching and Teaching Evaluation have been asked to comment on the documents produced by the CUT and to recommend changes before these documents are posted on the website.

In April 2012, the Centre for University Teaching (CUT) granted two 3-‐year chairs in university teaching. The recipients were Jacqueline Carnegie, Faculty of Medicine, and Scot Findlay, Faculty of Science.

Regular Professors

16%

Part-‐Sme Professors

34%

Postdoctoral Fellows 11%

Graduate Students 23%

TA 6%

Others 10%

Regular Professors

Part-‐jme Professors

Postdoctoral Fellows

Graduate Students

Teaching Assistants

Others


8

Term of reference 4 To administer the Teaching and Learning Grants Program and the Special Grants Program for Innovative Pedagogical Material in French. The Senate Committee on Teaching and Teaching Evaluation, through the Teaching and Learning Support Service, administers the Teaching and Learning Grants Program and the Special Grants Program for Innovative Pedagogical Material in French. The Senate Committee on Teaching and Teaching Evaluation, in cooperation with the Centre for University Teaching, has reviewed the documents relating to these two grant programs. This included a review of the guidelines, eligibility criteria, evaluation method, funding distribution and the form used to submit a proposal for a teaching/learning grant. These revised documents can be found on the Centre for University Teaching website at (http://www.saea.uottawa.ca/cpu/index.php?option=com_k2&view=item&id=39&Itemid=245&lang=en) The following grants were awarded in June 2012:

Recipient Title of the project Amount

Ellen B. Zweibel & Adam Dodek (Faculty of Law)

Web-‐based Learning Module on Writing Predictive Legal Memos $7,500

Hélène Knoerr (Faculté des arts)

Développement de modules d’apprentissage multimédias pour les cours d’immersion en français. $8,546

Brandon Schaufele (Faculty of Social Sciences)

Crowdsourcing in the Classroom: Using Prediction Mkts to Teach Econ Fundamentals $795

Alison Flynn (Faculty of Science)

An Interactive, Online Chemistry Nomenclature Learning Tool $10,000

William Ogilvie & Alison Flynn (Faculty of Science)

Teaching for Understanding: A mechanistic approach in organic chemistry $7,000

Odette Laneuville (Faculté de médecine) *2nd year of June 2011 project

Extraction de concepts en biologie : méthode d’apprentissage et évaluation de son impact sur la réussite aux examens

$5,845


9

Review of the guidelines The mandate of this ad hoc committee will be to review the guidelines, eligibility criteria, evaluation method, funding distribution and the form used to submit a proposal for a teaching/learning grant.


10

Terms of reference 5 and 6 5. To oversee maintenance and development of tools that enable students to provide their

input on the quality of teaching. 6. To ensure the development of evaluation tools that take into account the different types

of teaching and the academic support offered at the University of Ottawa. To meet these objectives, the Senate Committee on Teaching and Evaluation relies on a classroom questionnaire, an online evaluation completed by distance education students and the possibility for professors to include their own questions in the questionnaire. . Moreover, a large part of the Committee’s work this year focussed on researching the possibility of providing online evaluations for all courses offered on campus. Official evaluation

At the University of Ottawa, all courses involving at least nine contact hours with the same professor will be subject to an evaluation by students, regardless of the number of students and the teaching method used. This evaluation must be done using the official questionnaire (13 questions). Online evaluations of distance education courses

All web-‐based courses will be evaluated by students on the Web. Evaluations should be done on the official questionnaire on the Web. The official questionnaire is located on a secure site and can be filled out only during the official evaluation period. Customized formative evaluation

The Customized Formative Evaluation component was added in 2002 allowing teachers to get feedback on specific aspects of their course. Professor's who want to participate in the customized formative evaluation can choose up to 10 questions from the bank of customized questions; this evaluation will be administered along with the official evaluation. The choice of questions and the results are stored in a separate database and only the professor can access them through InfoWeb. Online evaluations for all courses

Course evaluation and the promotion of course evaluation are important components of a system of teaching assessment, which contributes to the student experience. Course evaluation is an important feedback resource for professors. In view of increasing demands from students and professors, the Senate Committee on Teaching and Teaching Evaluation has decided to study the possibility of carrying out online course evaluation. This document is based on discussions held during meetings of the Committee; a report entitled Évaluation en ligne de l’enseignement et des cours universitaires (Online Evaluation of Teaching and University Courses) and on meetings held in a number of universities.


11

This document is divided into two parts: the first outlines the advantages, challenges and immediate risks of online evaluations, and the second describes the possibilities offered by online evaluations.

Advantages, Challenges and Immediate Risks Related to Online Evaluation

Advantages Risks and Challenges

Accessibility

1. Accessibility to evaluation forms at all times during the evaluation period:

• Students may complete forms in class or in any other place whenever they wish

• Students who cannot attend class on evaluation day may still evaluate the course.

2. Meets a need expressed by students.

3. Online evaluation may be adapted electronically for special needs (visual, auditory, learning difficulties, etc.).

4. Provides those who need it with more time to reply.

1. It is a prerequisite of online evaluation that all students have access to electronic resources and the technological knowledge to access such evaluation (Please note the University already communicates with students by electronic means).

Results

1. Studies show that students’ opinion when using written evaluations is the same as when using online evaluations. In addition, comments provided in online evaluations are generally more complete and better structured.

1. Online evaluation could introduce a bias to the detriment of the group as a whole if a sub-‐group of students had greater participation than another. There is little research on this subject, but the little that does exist seems to indicate this is not in fact the case.

2. In multiple-‐section courses, students’ online evaluations could be included in the evaluation of the wrong professor if, for whatever reason, the student attends a course of another section and is not registered for it.

Efficiency

1. Savings in paper (preservation of the environment): written evaluations represent more 600,000 pages and 20,000 envelopes.

2. Reduction in handling:

1. Online evaluation requires the implementation of a computer system and contribution from the faculties if they wish to personalize their evaluation forms, as well as


12

• Centrally: no more handling of envelopes, optical reading of evaluation forms, checking for reading mistakes, physical storage of forms (40 boxes per session)

• In faculties: no more preparation of envelopes containing the number of evaluations required for each course, no more recovery of completed forms or persons in charge of evaluations to administer them in classes (some faculties).

3. Increase in teaching time if the evaluation is not done in class.

4. Faster publishing of results.

promotion of their course evaluation.

Participation levels

1. Accessibility of evaluation forms at all times during the evaluation period may allow students who would not have had the possibility to complete the evaluation form in class to do so online.

1. If online evaluation is done outside class time, we can expect lower participation levels than for evaluation done during course time:

• Participation levels in online evaluation vary between 30 and 53% depending on the institution

• The average participation level in online evaluation of distance courses at the University of Ottawa is 35% (there is little promotion of the evaluation with students)

• The average level of participation in written evaluations for the University of Ottawa is 65% (this level is lower than that of other institutions that carry out evaluation during the course. This level varies between 75 and 80%).

• Universities have noted only a slight increase in online participation levels over the years.

2. Students may take online evaluations less seriously if the University no longer uses class time for this purpose.

3. If course evaluations are done outside class time, this represents an extra burden on


13

students. Some students will not complete the evaluation.

4. The effect that lower participation levels could have on consideration of results for obtaining tenure, promotions and contract renewals (full-‐ and part-‐time professors) must be taken into account.

Questions to consider:

What is the most positive aspect of online evaluation?

What is your main concern regarding online evaluation?

Should we consider offering online evaluations of courses and teaching?

If so, under what conditions should this be done?


14

Future Possibilities Provided by Online Evaluation

Personalization

1. Possibility for the professor, the department or the faculty to adapt evaluation questionnaires based on their needs and the characteristics of their programs, while retaining a group of compulsory questions for the University as a whole.

2. Possibility of doing a formative mid-‐session evaluation (which studies show increases students’ participation level in the final evaluation).

3. It would be possible to gather information on the course evaluation, the quality of the program, the contribution of teaching and laboratory assistants. This would make it possible to know students’ perception of these areas and also to place the course in its wider context. 1. Online evaluations require the implementation of a computer system, which could be relatively complex, based on the desire of the faculties and professors to personalize their evaluation forms.

2. We must consider an adjustment in the way committees of academic staff deal with tailor-‐made questions that departments or professors would add to personalized questionnaires and that professors would wish to pass on voluntarily to committees.

1. Online evaluations require the implementation of a computer system, which could be relatively, based on the desire of the faculties and professors to personalize their evaluation forms.

2. An adjustment must be envisaged in the way committees of teaching staff committees deal with tailor-‐made questions that departments or professors would add to personalized questionnaires and that professors would wish to pass on voluntarily to committees.

Communication

1. Possibility of producing more complete and more relevant reports for professors and administrators (e.g. by making distributions with a frequency based on certain variables, by presenting data over several years, etc.).

1. Online evaluations require the implementation of a computer system, the complexity of which is to be determined, based on the course evaluation that we want to develop (e.g. email invitations to take part, and reminders, result-‐sharing system, etc.). The cost of this type of system


15

2. Speed of transmission of results: results and comments would be available as soon as final grades are submitted by professors. This would enable professors to make adjustments for the next session if necessary and enable students to have evaluation results available rapidly.

3. Possibility of obtaining additional comments to be transmitted to different recipients (e.g. the professor, the dean, the director of each department or program) while respecting the spirit of the collective agreements.

• A number of students address comments to the dean or department head, and are unaware that comments are currently are only meant for the professor

• In some universities, comments are sent to the professor and the department head

• People sometimes think that these comments are mostly criticisms, but many of them are intended to highlight the excellent work of the professor or the quality of the initiatives he or she has taken in the class.

4. As all students would receive a personal invitation to complete course evaluations, they could develop a feeling of empowerment regarding course evaluations.

will depend on its complexity.

The faculties are being consulted in order to gather their feedback on the feasibility of online course evaluations for all courses. Even so, the greatest challenge presented by this type of evaluation is that student participation rates for online evaluations are lower than those for paper evaluations.


16

An interim solution to this problem could be to develop an online platform to conduct a formative evaluation of teaching at mid-‐semester, which would:

§ give professors an opportunity to use the online evaluation platform to gather student input on how their course is going at midpoint in the semester;

§ initiate a formative evaluation of teaching assistants; and § foster an institutional culture with respect to using online platforms for evaluations.

The evaluation of teaching (or of the learning experience) should not be limited to an end-‐of-‐course evaluation based solely on the students’ general opinion of the professor’s work. Several studies point to the importance of considering student feedback as the course progresses and of evaluating the teaching team’s interventions more globally (by gathering feedback on both the professor’s and the teaching assistant’s approaches and methods). Hence, the university could develop an online platform to allow professors who wish to do so to implement a mid-‐semester evaluation, which would have an immediate impact on the teaching methods they use to meet student needs. Moreover, this same platform could be used to produce a formative evaluation of the work performed by teaching assistants.

An online platform would be user-‐friendly. Professors or teaching assistants who wish to conduct a formative evaluation would log on to a personal account to access the courses that they teach. They could use a ready-‐made questionnaire, or build their own questionnaire from a database of questions. Students enrolled in the course would receive an email asking for their opinion of the course, which they would answer online. One section of the form would allow students to write comments. The system would quickly generate an evaluation report to be sent to the person requesting the evaluation.

The development and use of an online platform could become an effective way to foster an institutional culture around teaching evaluation because it is more user-‐friendly than the current paper-‐based evaluation method. It would allow us to assess the impact of an online evaluation process without jeopardizing the formal course evaluation process.

Other issues being examined

During 2010-‐2011, the Committee addressed a new issue, namely the mid-‐semester formative evaluation of teaching. In 2011-‐2012, the Committee, in conjunction with the Centre for University Teaching, began working to produce a guide and a website for instructors who seek to integrate formative mid-‐semester evaluations of their teaching. The formative evaluations would be voluntary and would allow instructors to obtain feedback on the quality of their teaching and insight into possible changes that might enhance student learning in the second half of the semester. Recent research on the topic maintains that conducting formative mid-‐semester evaluations often improves the quality of student learning and the level of student satisfaction. Additionally, integration of these types of formative evaluations promotes increased student participation in the final evaluation of the course. The guide and the website will be launched in 2012. The website suggests ways of implementing a mid-‐semester formative evaluation, how to create one’s own evaluation tools, sample questions and questionnaires, how to interpret student feedback and how to use this feedback to improve teaching.


17

Course evaluation statistics The following two tables represent the course evaluation statistics. The first table provides information on the number of courses evaluated from September 2011 to August 2012. The second table provides the evaluation participation rates.


18

Table 1 : Number of courses evaluated from September 2011 to August 2012

(Fall 2011 + Winter 2012 + Spring-‐Summer 2012 combined)

Faculty Number of courses evaluated

Telfer School of Management 354 Arts 1523 Civil Law 150 Common Law 321 Education 423 Engineering 309 Graduate and Postdoctoral 1108 Health Science 246 Medicine * 26 Science 419 Social Sciences 1062 TOTAL 5,941

* Given the structure of their programs, the Faculty of Medicine uses a different system of course evaluation.

Table 2 : Number of students participating in evaluations – September 2011 to August 2012 (Fall 2011 + Winter 2012 + Spring-‐Summer 2012 combined)

Faculty Number of students registered in course being evaluated*

Number of students participating in evaluation

Percentage students participating in evaluation

Telfer School of Management 24,282 14,825 61.1 Arts 72,283 47,474 65.7 Civil Law 8,206 5,788 69.9 Common Law 10,806 8,063 74.6 Education 15,586 11,934 76.6 Engineering 12,245 7,913 64.6 Graduate and Postdoctoral 17,703 14,434 82.5 Health Science 17,770 11,730 66.0 Medicine * 6,749 3,751 55.6 Science 39,110 23,133 59.1 Social Sciences 68,297 42,746 62.6 TOTAL 293,037 191,741 65.4

* The number of total students registered in all faculty courses that were evaluated (for example, 1 student registered in 4 courses = 4)


19

The following figures represent the students’ response, in percentage (%), on questions 1, 4, and 9 of the evaluation form (September 2011 to August 2012).

68.3%

23.9%

5.9% 1.3% 0.5% 0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Almost always Oven Somejmes Rarely Almost never

QuesSon 1: I find the professor well prepared for class

43.9% 34.9%

16.1% 3.6% 1.5%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Almost always Oven Somejmes Rarely Almost never

QuesSon 4: I think the professor conveys the subject macer effecSvely

42.6% 36.3%

14.9% 4.4% 1.8%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Excellent Good Acceptable Poor Very poor

QuesSon 9: I find that the professor as a teacher is


20

Terms of reference 7 and 8

7. To facilitate and promote communication with the entire academic community to better meet its evaluation needs.

8. To ensure that evaluation is promoted to students and professors. The Committee on Teaching and Teaching Evaluation would like to make the evaluation results more accessible. Results are made available on the EvaluAction website. The Committee has also been promoting the importance of course evaluations and the impact of evaluations on teaching quality and the quality of students’ learning experience.

EvaluAction website

The EvaluAction website (www.evaluaction.uottawa.ca/index-‐e.html) was created to compile and present all the information on the evaluation of teaching and courses at the University of Ottawa. The site contains information on the evaluation process, the different reports issued and the follow-‐up carried out as part of this important exercise. The FAQ section provides the answers to the questions most commonly asked by students. The following tables provide statistics on the number of times evaluation results were consulted.

Number of consultations (teaching evaluation results)*

Total

September 2011 21,584

October 2011 20,324

November 2011 23,484

December 2011 20,924

January 2012 24,174

February 2012 23,671

March 2012 24,595

April 2012 21,373

May 2012 23,835

June 2012 35,449

July 2012 32,955

August 2012 22,075

TOTAL 294,443

* Corresponds to the number of “clicks” by InfoWeb visitors once they were in the teaching and course evaluation section


21

Promoting the evaluation of teaching and courses

Promoting the evaluations is an important factor in achieving high student participation. The Committee made recommendations to the program and course evaluation coordinator about the campaign to promote course evaluations. Promotional activities during the course evaluation period involve the following elements:

Ø Ads on bulletin boards around the main campus and at Roger-‐Guindon

Ø Message on the University’s homepage and the faculties that describes the process and why it is important to participate

Ø Message posted on the EvaluAction website

Ø Message posted to encourage professors and students to read the evaluation results (S-‐Report)

Ø Two large banners at the main entries to the campus

Ø Poster board ads in various locations around campus

Ø Message to students inviting them to participate

Ø Message to students through Virtual Campus

Ø Message for students on uoZone The Committee would like to review the advertising that aims to promote course evaluations. The University of Ottawa’s Communications Directorate was consulted and the Center for University Teaching has been mandated to conduct professor and student focus groups to identify the messages that should be conveyed by such promotional activities.

2011-2012 Annual report - University of Ottawa · SenateCommitteeonTeachingandTeachingEval...

Documents

Transcript of 2011-2012 Annual report - University of Ottawa · SenateCommitteeonTeachingandTeachingEval...