With thanks to David Leyton-Brown, former Executive Director OCGS

http://ocgs.cou.on.ca/http://ocgs.cou.on.ca/

KEY ELEMENTS IN THE APPRAISAL BRIEF

Wilfrid Laurier University, Faculty of Graduate StudiesNovember, 2008

With thanks to David Leyton-Brown,former Executive Director OCGS


Types of Appraisal Briefs

• Standard Appraisal Brief: for new programs– Likely to remain the same after transition to new review

agency

• Periodical Appraisal Brief: for existing programs– Template likely to change when transition to new

review structure, but business as usual until 2010– Abbreviated brief if program less than 3 years old

• New Fields: request may be submitted any time


OCGS review process

• Brief sent to OCGS by the graduate dean on behalf of the university after all internal approvals, including Senate, obtained.

• OCGS sends brief to a multidisciplinary Appraisal Committee composed of senior faculty members experienced in graduate education (nominated by universities and approved by Council)

• Appraisal Committee identifies concerns/issues; selects consultants and OCGS informs the university


Process (cont’d)• University gathers additional material for consultants to

address these issues or provide information on new developments

• Consultants visit: usually a day or a day and a half– Investigate issues and others they’ve identified

– Meet with grad dean first, then all stakeholders

– Write and submit report to OCGS

• Report is sent to the university; we respond to all issues• Rating of Good Quality, Good Quality with Report,

Conditional Approval, Not Approved• We can offer and receive MTCU funding only for

approved programs


Unit of Analysis

• Appraisal of the Graduate Program– Not the administrative unit (i.e. department, school)

• Only three levels of analysis in the appraisal process– Program– Field– Individual (professor, student, course, etc.)


Template

• Consult the bylaws, follow the template and use my checklist – Ric Knowles’ notes are also helpful

• Not a previous brief (significant changes in bylaws in the last 2 years)


“Truth in Advertizing” Key concept in OCGS reviews

• If a student enters the program because of the promise of strength in a particular area of interest, that strength must be offered in the program

• Strength seen in:– Faculty– Courses– Fellow students– Intellectual activities


The Program

• A coherent program of study and research• Academic content and appropriate requirements• Curriculum• Show how it is different from undergraduate

program (e.g., master’s program shouldn’t look like the 5th year of an undergraduate degree)


Research Component

• A graduate level research component is essential

• Must be demonstrated• Analytical/interpretive skills• Including in:

– professional programs– coursework-only programs


Faculty

Provide intellectual leadership in the disciplinary area(s) of the program

• Research active– Peer reviewed publication– Research funding

• Graduate education involvement– Membership in Faculty of Graduate Studies– Supervision – experience and activity– Program activities (intellectual community)


Faculty Data

• CV (OCGS format or approved alternative)

• Faculty By Field Table

• Categories for each core member of the program– Do not need to include CAS instructors in the tables,

but should in the narrative


Fields: optional in master’s; required in doctoral programs

For appraisal purposes any of these are fields, whatever term is used internally:

• Streams• Subfields• Areas• Sub-areas• Concentrations• Specializations• Themes• Options

• No such thing as a subfield– If it is an area or strength in the program that the university wants

to advertise, for appraisal purposes it is a field


Faculty Complement

How many faculty per field?

• Critical mass – supervision, teaching, intellectual community– 1 is not enough– 2 is not enough– 3? 4? 5? ….

• No benefit in maximizing the number of fields


Consequences of field identification

• CAN include a faculty member whose expertise is not in one of the fields

• CAN offer a course outside the fields• CAN permit a student to write a thesis on a

subject outside the fields– But not explicitly recruit the student to do so

• CAN name each faculty member’s expertise

• CANNOT advertise other program “strengths”


Outcomes

• Learning objectives and learning outcomes• Student data

– Completion rates and times to completion– External scholarships and awards

• Outputs – productivity, quality, success– Remember to keep collecting these data:

• Publications – from thesis research• First employment or further study• Awards


Objectives (Bylaw 10.1.2)

• Degree level expectations (how demonstrated?)• Nature of graduate education experience• Intended career preparation• Learning objectives

– What do you want your students to learn or be able to do by the time they complete the program?

– How will you know if they have/can?


Undergraduate-Graduate Courses (Bylaw 10.4.4)

“The number of undergraduate courses, or combined courses in which undergraduate students predominate, should be no more than one third of the total course requirement for the degree”

• Program requirements – i.e. maximum number of combined courses

• Course enrolment data• Number of graduate courses offered


Consultants

• In all standard and periodic appraisals for the next year, at least.– Benefits - provide feedback– 2 for master’s– Normally 2 (perhaps 3) for doctoral

• Committee’s questions to consultants– Helpful to provide material to consultants (and

Committee)


Consultants’ Report

Response to consultants’ report(s)

• University response, not program response

• Respond – don’t cherry-pick or cheerlead

• Address all the consultants’ points

• Address Committee questions (if needed)


Interdisciplinarity

• Assure faculty availability– Effect on home program(s)?

• Truth in advertising– Deliver what you claim

• Financial support– Especially TAs – spillover effects?


Part-Time

• Section 10.4.3

• Principle – content and educational experience should not be inferior to that for full-time students

• Time path – mapping progress through the program


Admission Requirements

• Section 10.4.2

• Research-intensive or course master’s– Honours degree – four year

• Professional master’s– Four year degree or equivalent– Exceptional admission

• State criteria• Report on proportion admitted exceptionally


Irritants to appraisal committee

• Denial

– Concerns expressed in the previous appraisal– Major issues – elephant in the room

• Sloppiness

– Tables and text don’t match– Required data incomplete


Special Situations1. Abbreviated Brief (18.0)

• Usually no consultants unless OCGS feels there’s a problem: then one consultant

• Describe program and provide calendar material• NEW enrolments, graduations, withdrawals etc.

by year; funding, thesis topics, employment• Describe programs and fields• List core faculty by field:

– Highlight changes and the impact on the program– Include CVs of new faculty members


Special Situations2. New Fields (28.1.4 a)

• OCGS will look for viability of new fields given existing fields (i.e., enough faculty, courses)

• Describe intellectual focus of new field• Core faculty by field table (and CVs)• Courses in new field• Relationship of new field to parent program• Impact on existing fields (effect of diversion of faculty,

expected new hires, involvement of other university faculty, etc.)

• A problem to give field same name as program

With thanks to David Leyton-Brown, former Executive Director OCGS

Documents

Transcript of With thanks to David Leyton-Brown, former Executive Director OCGS