A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit...

19
A GUIDE TO ACADEMIC PROGRAM REVIEW Office of the Provost Revised March 2019

Transcript of A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit...

Page 1: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

A GUIDE TO ACADEMIC PROGRAM REVIEW Office of the Provost Revised March 2019

Page 2: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 2

Table of Contents

Preface…………………………………………………………………………………………………………….3

Purpose of Academic Program Review…………………………………………………………….3

The Self-Study………………………………………………………………………………………………….4

The External Review Site Visit…………………………………………………………………………..5

Timeline and Activities……………………………………………………………………………………..6

Appendix A: Primary Roles………………………………………………………………………………..7

Appendix B: Guidelines for Preparation of a Self-Study Report…………………………8

Appendix C: Standards and Criteria for Academic Program Review...................11

Appendix D: Request for External Reviewer Nominations Letter………………..…..14

Appendix E: Sample Itinerary………………………………………………………………………….15

Frequently Asked Questions……………………………………………………………………………17

Contacts and Additional Information………………………………………………………………19

Page 3: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 3

Preface

Academic program review is a cyclical process for evaluating and continuously enhancing the

quality and currency of academic programs. The evaluation is conducted through a process of

self-evaluation, followed by peer evaluation via reviewers external to the program or

department, usually also external to the organization. It is a comprehensive analysis of program

quality, utilizing a wide variety of data about the program. Program review operates on a

nominal eight-year cycle, meaning that each program is reviewed every eight years. Academic

Program Reviews are conducted through the Faculty Senate’s Curricular Affairs Committee

(CAC) in partnership with the Office of the Provost.

To be effective, the system of academic program review must be straightforward, objective,

and transparent. It must be carried out in a timely manner and implemented deliberately.

The result of the academic program review process is a clear picture of the program’s strengths,

challenges, and opportunities. These outcomes are used to inform strategic planning and

resource allocation at program, department, college, and university levels.

Purpose of an Academic Program Review

The purposes of academic program review are to:

Ensure that academic programs are maintained at the highest possible level of quality.

Provide a basis for continuous quality improvement of academic programs.

Help ensure the viability of academic programs.

Guide strategic planning and decision-making regarding academic programs.

Ensure that academic programs serve the mission and vision of the university.

Page 4: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 4

The Self-Study

At the commencement of an academic program review, all graduate and undergraduate

programs prepare a self-study, a succinct report (approximately 12 pages) depending on the

number of the programs being included in the review. The self-study reports are to be prepared

by the responsible faculty and department chairperson or director of the program under

review. There are six sections to the self-study report:

Section One: General Information

Section Two: Introduction/Overview

Section Three: Standards and Criteria1

o Contribution to Mission

o Program Quality

o Demand

o Societal Need

o Quality Control Mechanisms

o Effectiveness

Section Four: Analysis

Section Five: Prospective

Section Six: Appendices

Programs will be encouraged to present their indicators and metrics in order to adapt the criteria to their particular circumstances and characteristics. In addition to addressing the criteria, the report should include a narrative (1,000 words maximum) to explain the meaning and implications of the data and to clarify any important qualitative characteristics and circumstances that may not be included in the data. Specifically, the narrative should answer the questions:

What are the program’s strengths, weaknesses, and opportunities?

What characteristics of the program should be maintained?

What characteristics of the program should be modified or abandoned?

What plans do the faculty, chairperson, and dean have for the program over the next

three to five years?

Criteria definitions, format, and guidelines for self-study reports are contained in Appendix B.

1 In Section Three, the program will demonstrate the extent to which it meets each standard and criterion using relevant data supplied by the Office of Institutional Research as well as the Graduate College.

Page 5: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 5

External Review Team Site Visit

Program chairs/directors will be asked to provide a list of seven or eight names of highly

qualified persons who could provide a rigorous review of the program. (See Appendix C.) The

list should include faculty from distinguished universities with programs that embody key

elements of the program under review as well as its aspirations for the future. The names and

CVs should be provided to the Academic Program Review (APR) Coordinator, and will be ranked

by the Associate Provost for Teaching and Learning, Dean of the Graduate College, and the

Curricular Affairs Committee (CAC) chair. Two individuals will be invited, in order of rank, to

serve as external reviewers.

External reviews are carried out over two days and typically include the following:

A welcome breakfast with the Associate Provost for Teaching and Learning, the program

chair/director, and the chair of the internal review subcommittee2.

Meetings with:

The dean of the school/college and/or associate dean

The Dean of the Graduate College

The department program chair/director

Faculty groups by program, rank, or specialization

Undergraduate majors

Graduates residing in the Burlington area

Graduate students

Tour of relevant facilities

Working lunch with faculty on the first and second days

Optional program/department dinner

An exit interview at which the external review team reports preliminary findings to the

program chair/director, the Associate Provost for Teaching and Learning, and the

internal review subcommittee

The team will have an afternoon break on the second day to serve as private work time to prepare a preliminary report for the exit interview. (Please see the sample schedule in Appendix D.)

2 The Faculty Senate Curricular Affairs Committee appoints two members to serve as an internal review subcommittee. (See Appendix A for additional information.)

Page 6: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 6

Timeline and Activities

Approximate Time Frame

Responsibilities

12 months prior to external review team site visit

Provost’s Office sends notification letter to program chair/director.

Associate Provost for Teaching & Learning, APR Coordinator, Office of Institutional Research (OIR) Director, program chair/director meet to discuss the review.

APR Coordinator creates Sharepoint site for program.

11 months prior to site visit

Program provides names and CVs of 7 to 8 individuals from peer and aspirant programs to serve as potential reviewers.

Provost’s Office extends invitations and requests potential site visit dates.

Program chair/director meets with OIR Director to discuss data for the self-study.

The chair/director organizes and initiates the self-study.

3 months prior to site visit

In consultation with external reviewers and program chair/director, APR Coordinator finalizes site visit dates and coordinates travel arrangements.

2 months prior to site visit

APR Coordinator and program chair/director draft itinerary and schedule meetings.

1 month prior to site visit

Program chair/director posts final self-study report to program APR Sharepoint site. APR Coordinator notifies external reviewers and internal reviewers the self-study is available for review.

1 week prior to site visit

APR Coordinator sends final itinerary to external and internal reviewers, and dean(s), and posts to Sharepoint site.

When required, the Graduate Executive Committee provides a written assessment to the graduate program(s).

External Review Team Site Visit Within 6 weeks of site visit

The external review team sends their report to the Associate Provost for Teaching and Learning.

APR Coordinator distributes external reviewers report to program chair/director, relevant dean(s), and Internal Review Subcommittee (IRS) for factual corrections.

IRS prepares report, and submits to full Curricular Affairs Committee (CAC) for vote.

Within 6 months after CAC vote

Provost, Associate Provost, relevant dean(s), program chair/director, CAC Chair, internal review subcommittee, and APR Coordinator meet to discuss external and internal reviewers reports and develop a summary memo.

Within 1 month of summary meeting

APR Coordinator drafts the APR Summary Memo for review and signature by Provost, Associate Provost, and CAC chair. APR Coordinator distributes signed report.

Two years following summary meeting

Provost, Associate Provost, relevant dean(s), program chair/director, CAC Chair, and APR Coordinator meet to review findings and recommendations in the APR Summary Memo.

Within 1 month of 2-Yr Follow-Up Meeting

APR Coordinator drafts two-year follow-up report for review and signature by Provost, Associate Provost, and CAC Chair. APR Coordinator distributes signed memo to meeting participants.

Page 7: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 7

APPENDIX A: Primary Roles: Provost’s Office, Faculty Senate, and Program

Provost’s Office Establish and maintain a long-term schedule of program reviews

Manage the academic program review budget

Establish and maintain a Sharepoint site for each program review

Invite external reviewers

Work with external review team to arrange travel and secure accommodations

Collaborate with the program chair/director to establish a detailed site visit itinerary

Serve as the primary point of contact for correspondence with team members

Set up guest accounts for Sharepoint and WiFi

Notify team members when self-study is available for review on Sharepoint

Send final schedule

Follow-up with team members for external reviewers’ final report

Process honorarium and reimbursement for travel expenses

Move reports through the final stages of the process

Oversee summary meeting and report process

Oversee two-year follow-up review

Faculty Senate Curricular Affairs Committee (CAC) Assign committee members to serve on internal review subcommittee

Read and thoroughly understand the Program’s self-study

Participate in site visit

Receive the report of the external reviewers

Prepare written report summarizing the external reviewers’ recommendations and

present to full CAC

Receive the report from the internal review subcommittee

Vote to accept or reject the report of the internal review subcommittee

Participate in the summary and two-year follow-up meetings

Provide an annual report on APR activities

Program Chair/Director Prepare self-study and related materials according to the Guidelines. See Appendix B.

Suggest qualified persons to serve as external reviewers. See Appendix C.

Organize and participate in program-specific site visit activities

College/School Dean(s) Participate in site visit activities

Participate in summary and two-year follow-up meetings

Page 8: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 8

APPENDIX B: Guidelines for Preparation of a Self-Study Report

For Academic Program Review Introduction: The self-study report of an academic program describes the academic program using a common set of institutionally determined standards and criteria. The self-study report, together with external reviewer’s input, identifies the program’s strengths, challenges and opportunities, and provides a basis for informed decision making about future directions. The report is structured around the APR standards and criteria and agreed-upon unit-specific indicators, and should be built upon evidence that clearly indicates how the criteria are being met.

Guidelines for Writing the Self-Study Report The self-study report is prepared by the responsible faculty and department chairperson or director of the program under review. The report should include relevant data supplied by the Office of Institutional Research (enrollments, FTE ratios, performance of graduates, etc.). The report is expected to provide a review of these data, along with other information collected through program-based assessment and other review processes. The program should utilize these data to explain its status with respect to the standards and criteria included in these guidelines. Evaluation data from existing reviews of the program such as accreditation reports, and any program changes made in response to accreditation reviews, should be incorporated into the self-study report wherever appropriate. The main body of the report is divided into five sections, and should be approximately fifteen pages in total. Appropriate appendices comprise a sixth section and should be attached to the main body of the report:

Section One: General Information

Section Two: Introduction/Overview

Section Three: Standards and Criteria

Section Four: Analysis

Section Five: Summary and Prospective

Section Six: Appendices The first two sections of the report provide general information and an executive summary. Sections Two and Three review data for each of the APR standards, and are followed by an analysis of the data in Section 4. Finally, Section 5 comprises an integrative Summary and Prospective that specifically identifies program strengths, challenges and opportunities, and poses future plans and directions for improvement. Each of these sections is described more fully below.

Section One: General Information The General Information section provides factual data about the program, including name of the program, program type, college or school in which the program is located, name of the

Page 9: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 9

chairperson/director of the program, name of the dean of the academic unit, names of faculty writing the report, and date of the report. The process used to develop the report and the participation of different constituencies in its formulation should be described.

Section Two: Introduction/Overview The Introduction/Overview section establishes the background and context for the review. It should include a brief history of the program, a brief description of its present status, the goals and mission of its graduate and undergraduate programs, unique and distinguishing characteristics, and links with other units such as joint faculty appointments, cross-listed courses, shared undergraduate and graduate service courses, and research collaborations.

Section Three: Standards and Criteria In this section the program provides data for each standard and criterion. The standards are:

I) Contribution to Mission II) Program Quality III) Demand IV) Societal Need V) Quality Control Mechanisms; and VI) Efficiency

In addressing Standard I, Contribution to Mission, the program should identify courses it offers that contribute to the University’s General Education program. The assessment of student learning outcomes is one of several items under Standard V, Criterion 5c and it requires special attention. To address this part of the standards, the program needs to:

a) state its learning outcomes for students in the program and outline the methods and processes for assessing those outcomes. In addition to listing current learning outcomes and indicating the website where they are posted, all programs must provide an updated version of NECHE form E1A or, in the case of an externally accredited program, form E1B. Both forms are posted on the Assessment website https://www.uvm.edu/assessment.

b) describe its long-term, cyclical plan and processes for assessing these learning outcomes. i. Non-accredited programs should utilize the assessment plan template posted on the

Assessment website to outline their cyclical assessment plan. If the department has a current assessment plan, this can be attached; if it does not, training and consultations are available to support the program as it develops the plan.

ii. Externally accredited programs do not need to fill out an assessment plan form. NECHE form E1B should be filled out with clear reference to the indicators of program success and areas of remediation identified by the external accreditors.

The completed forms should be included as an Appendix. Note that additional consultation contacts, resources, and support services are posted on the Assessment website. All programs preparing for Academic Program Review are encouraged to consult with their school or college’s Assessment Coordinator and the Provost’s Office.

Page 10: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 10

Where possible, direct assessment of student work should be included in the evaluation of student achievement of program outcomes along with indirect assessments. Direct assessments are those that evaluate student work as evidence of achievement of learning outcomes. In most cases these evaluations will be conducted by program faculty and/or staff (where appropriate). However, some direct measures may be completed by people outside the program. These include students’ performance on the licensure exams for which a program prepares them, or direct evaluation of student/graduate performance by employers or internship supervisors using criteria supplied by the program.

In addition to direct assessment of student work, indirect indicators of program outcomes should also be presented. These indicators may include student self-evaluations; interviews, surveys or focus groups of majors; interview, survey or focus group data on alumni satisfaction with the program; interview, survey or focus group data on employer satisfaction with program graduates' performance; post-doctoral placement of graduate students; academic or professional achievements of program graduates; job placement and career progression; and creative works, publications, and grant awards by program students and graduates. Program faculty can also include other data they deem indicative of student outcomes.

Section Four: Analysis This section should present the main findings of the self-study including an analysis of the extent to which the program meets each standard. Data from direct and indirect assessment3 of student achievement of program learning outcomes must be included in this analysis, as well as any planned or in-process responses to assessment data. Other regular internal review and evaluation processes, such as departmental reports and retreats, can also provide useful data and examples to demonstrate how well the program is meetings the standards. The meaning, implications, and any departmental response to the findings should be explained. Section Five: Summary and Prospective The Summary and Prospective should present a vision for the program grounded in the program’s strategic goals. It should also present a balanced assessment of the program’s strengths, challenges and opportunities as well as directions for the future as informed by the findings. The discussion should include scholarly directions, research plans, curricular or degree program changes, and plans for maintaining and enhancing excellence and diversity of faculty and students over the next eight years. Given the persistence of budgetary constraints, the discussion should include ways in which the unit can be strengthened without receiving additional internal resources. Section Six: Appendices Supporting data and materials may be appended to the main body of the report.

3 See Standard 5c for an explanation of direct and indirect assessment.

Page 11: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 11

Appendix C: Standards and Criteria for Academic Program Review

Approved by the University of Vermont Faculty Senate January 3, 2019

Standards and Criteria. Standard I: The program has a clear and publicly stated purpose that contributes

to the mission of the University.

Criterion 1: The program contributes to the mission of the University, the College/School, and department by: a) Having an active strategic plan that is aligned with the vision, mission, and strategic plan of

the University. b) Supporting research and creative activities that generate new knowledge and

understanding and enrich the intellectual environment for students, staff, and faculty.

c) Engaging in relevant application of new knowledge to contemporary problems through teaching, scholarship, creative activities, and service and outreach.

d) Preparing students for productive, responsible, and creative lives.

e) Encouraging students to use their knowledge and skills for the benefit of society.

f) Promoting global perspective and appreciation of cultural and intellectual diversity.

g) Reflects university priorities for diversity and inclusion in the faculty and student bodies.

h) Fostering an enduring commitment to learning.

i) Fostering the qualities of respect, integrity, innovation, openness, justice, and responsibility as expressed in Our Common Ground.

j) Additional unit-specific indicators.

Standard II: The program is of high quality

Criterion 2: The program quality is evidenced by:

a) Faculty - The Program faculty are qualified to teach the curriculum, as indicated by earned academic degrees and professional certifications. The program invests in the professional and scholarly development of its faculty, including the mentoring and guidance of junior faculty members through the RPT process.

Page 12: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 12

b) Resources - The program has adequate faculty, support staff, library resources, equipment, and facilities to accomplish its purpose.

c) Reputation – The program is well regarded, as evidenced by external rankings and assessments by external reviewers of students, faculty, resources, and productivity. The program attracts and retains excellent students as evidenced by admission qualifications, performance on standardized examinations, etc.

d) Faculty performance – Faculty demonstrate effectiveness in teaching and student advising, scholarship, and service, as evidenced by evaluations, awards, honors, grants, research contributions, publications, citations, and service endeavors.

e) Student performance – The program assess student mastery of learning outcomes by means of direct and indirect assessments, performance in the field, professional achievements, and performance on professional licensure exams. Program graduates succeed in finding jobs and progress well in their chosen careers; alumni are satisfied with the program. Undergraduate and graduate students produce creative works, publications, and receive grant awards. Graduate students are awarded post-doctoral fellowships.

f) Benchmarks – The program reflects “best practices” and compares well to relevant

performance standards from comparable institutions and/or accrediting agencies and/or other authoritative sources. The program demonstrates leadership in its performances relative to appropriate external benchmarks.

g) Advising – Program faculty provide excellent academic advising, per student evaluations and

other appropriate indicators.

h) Extramural Funding (for programs where such funding is critical) – Success in attracting extramural funding that contributes to the Program’s long-term stability.

Standard III: There is demand for the program. Criterion 3. There is demand for the program as evidenced by:

a) external demand based on local, regional, national, and global trends and forecasts for persons with particular types and levels of education.

b) internal demand as reflected by both student enrollment in the program and the scope of service teaching for students from other programs.

Standard IV: The program provides graduates who contribute to social institutions.

Criterion 4: Societal need for the program is reflected by:

a) evidence for private, public and/or not-for-profit sector needs for persons with particular knowledge, skills, attitudes, and values required to make social institutions work.

Page 13: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 13

b) evidence of the need at national, state, and local levels for persons who can be informed and responsible citizens.

Standard V: The program uses an identified plan for systematic evaluation and assessment of goals and purposes.

Criterion 5: The program has quality control processes that are used:

a) to evaluate how well the program is achieving its strategic goals.

b) to monitor on an ongoing basis, the design and delivery of the curriculum/curricula as informed by student outcomes.

c) for ongoing evaluation of clearly stated student outcomes. This includes but is not

limited to direct and indirect assessments of student learning at the course level. The program has a sustainable cyclical assessment plan in place to evaluate students’ achievement of each program outcome, as well as a process for using assessment data to inform specific changes that are intended to improve student outcomes.

d) to monitor the quality of student advising.

e) to utilize data gathered in 5b-d to determine needed changes in tactics, policies, curriculum, and course contents.

f) to plan and implement the self-determined changes in a timely manner

Standard VI: The program accomplishes effectively its educational and related purposes Criterion 6: The effectiveness of the program is reflected by:

a) improvements in the design and delivery of the curriculum based on assessment of student achievement of program learning outcomes, new knowledge in the discipline, societal need, and demand for the program.

b) measures to maintain or improve high quality student advising, including career preparation advising.

c) programmatic features that foster an appreciation of cultural and intellectual diversity.

d) linkages with other programs, including articulation agreements, co-sponsored academic majors, minors, or concentrations, joint appointments of faculty members, cross-listed courses, student internships, practica, or field-based projects with organizations outside the University, resources shared with other academic units, dual degrees, and 3-2, 4-1, or other undergraduate + graduate degree arrangements.

January 2019

Page 14: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 14

APPENDIX D: Request for External Reviewer Nominations

MEMORANDUM

TO: ________________, Chair, Department of _________________

FROM: ________________, Associate Provost for Teaching and Learning

DATE:

SUBJ: Request for (Program) APR External Reviewer Nominations

The University of Vermont Faculty Senate, in cooperation with the Provost’s Office, has

developed procedures to identify and contact external persons for periodic reviews of academic

programs at UVM. I request your assistance in implementing these procedures.

As an initial step, please provide me with the following information:

(1) The names of seven or eight individuals and their CVs from which I could identify highly

qualified persons who could provide a rigorous review of your program. The list should

include faculty from distinguished universities with programs in your discipline that embody

key elements of your program and its aspirations for the future.

(2) Suggestions regarding any special areas of expertise that would be important for the review

of your programs.

(3) Preferred dates for the site visit which shall occur between __________ and

______________. Generally, the reviews begin with a working breakfast and conclude with

an exit interview on the second day. (Please refer to the attached sample site visit itinerary.)

The site visit should not conflict with national professional meetings or previous program

commitments.

Please submit this information at your earliest convenience but no later than _______________,

so that the Academic Program Review team can select potential external reviewers. I will then

forward the names of those individuals to you for your approval. Please contact me if you have

any questions.

cc: ____________________, Chair, Faculty Senate Curricular Affairs Committee

____________________, Dean of the Graduate College (if grad program)

Attachment: Sample Site Visit Itinerary

Page 15: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 15

APPENDIX E: Sample Site Visit Itinerary

AGENDA

Degree Program Names

Program Review Site Visit

Dates of Site Visit

Day One

External review team arrives

Evening Optional Working Dinner Meeting at Hotel

External Reviewers only

Day Two

8:00 a.m. Welcome Breakfast Meeting at Hotel

External Reviewers

Associate Provost for Teaching and Learning

Program Chair/Director

Faculty Senate Curricular Affairs Internal Review Subcommittee

9:30 a.m. Meeting with Program Chair/Director

10:30 a.m. – Meeting Groups (45 – 60 minute slots arranged in advance by

12 noon department/program[s])

Office of the Dean of Graduate College

School/College Dean and/or Associate Dean

Undergraduate and/or Graduate Committees

Faculty Groups by program, rank, or specialization

Undergraduate Majors

Graduates residing in the Burlington area

Graduate Students

Graduate Assistants

Tour of relevant facilities

Representatives from related programs

12:15 – 1:30 p.m. Working Lunch with Faculty

(Waterman Manor Restaurant)

External Reviewers with up to 8 Faculty

1:45 – 5:30 p.m. Meeting Groups Continued (see 8:30 a.m.)

Page 16: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 16

6:00 p.m. Optional Dinner at area restaurant

External Reviewers hosted by Program/Department

~or~

Working Dinner at Hotel

External Reviewers only

Day Three

8:00 a.m. External Reviewers picked up at hotel

8:30 a.m. – Meeting Groups Continued (See Day Two)

12 noon

12:00 – 1:15 p.m. Working Lunch

(Waterman Manor Restaurant)

External Reviewers

Faculty

1:30 p.m. External Reviewers Escorted to Private Work Space

1:30 – 3:00 p.m. External Reviewers Private Work Time

3:00 – 4:00 p.m. Exit Interview/Preliminary Report

External Reviewers

Internal Review Subcommittee

Associate Provost for Teaching and Learning

Program Representative(s)

4:00 p.m. External Reviewers Transported to Hotel/Airport

Page 17: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 17

Frequently Asked Questions:

What is an appropriate length for a self-study report?

It depends on the number of programs being included in the review. Generally, the report should be approximately 12 pages. Appendices may be attached to the report or submitted separately.

What is the process for gathering data? Contact the Director of Office of Institutional Research (see page 19) for assistance. Refer to the Guidelines for Preparation of a Self-Study Report for suggested data to be included.

What do I do with the self-study report when it’s completed? The report and accompanying documents are to be uploaded to the program’s Sharepoint site by either the program or the APR Coordinator. In either case, the APR Coordinator (see page 19) should be notified when the self-study report is completed.

What is the process for selecting external reviewers? The program will be notified to submit names and CVs of suggested reviewers (see page 14). This list will then be ranked by the Associate Provost for Teaching and Learning, the Dean of the Graduate College, and the Chair of the Faculty Senate Curricular Affairs Committee. The APR Coordinator will contact the ranked reviewers to determine availability. The program will be notified when two individuals have accepted our invitation to serve as external reviewers.

What is the process for selecting internal reviewers? The Internal Review Subcommittee consists of two committee members appointed by the Chair of the Curricular Affairs Committee. For reviews that include a significant number of programs and/or require a member of the graduate faculty, a third member may be appointed.

What is the department/program responsible for? The department/program is responsible for preparing the self-study report, providing a list of suggested external reviewers, working with the APR Coordinator in developing the site visit itinerary, hosting the external reviewers, and participating in the summary and two-year follow-up meetings.

Who pays for what? The Provost’s Office covers all hotel and meal expenses for the external reviewers except the optional* dinner on the second day and the lunch with faculty on the third day.

Page 18: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 18

How are the external reviewers paid?

The Provost’s Office handles all budget matters concerning the external reviewers.

Who makes the lunch reservations? Either the APR Coordinator or the program may make reservations.

Who attends the dinner on the first evening? An optional working dinner at the hotel is built into the schedule to allow time for the external reviewers to meet before the official start of the site visit. UVM program and administrative officials do not attend.

What happens after the external reviewers’ site visit? The external reviewers are given up to six weeks to submit their report. This report is distributed to the program and the dean(s) for factual corrections. After receiving the program’s input, the internal review subcommittee will draft a report and present their findings and recommendation to the full Curricular Affairs Committee (CAC). Following the vote of the CAC, the APR Coordinator schedules a summary meeting. The purpose of the meeting is to review and discuss the key findings of the program review. A summary is then drafted to document the main findings of the meeting, and to set expectations for follow-up in two years.

What happens with the APR Summary? The APR Summary document is signed by the Provost, the Associate Provost for Teaching and Learning, and the Chair of the Curricular Affairs Committee. Signed copies are distributed to the summary meeting participants, and uploaded to the program’s Sharepoint site.

What is the purpose of the two-year follow-up meeting? A follow-up meeting will be scheduled two years from the summary meeting. The purpose of the two-year follow-up meeting is to review the findings and recommendations in the APR Summary and to discuss the current status of the program and progress on identified goals and objectives. A second report will be drafted, signed and distributed. Upon completion and distribution of the second document, the APR will be considered closed with the next review to occur in approximately six years (eight-year cycle).

*Dinner on the second day, hosted by the program under review at its own expense, is optional. However, programs typically choose to host this dinner.

Page 19: A GUIDE TO ACADEMIC PROGRAM REVIEW...Academic Program Review Guide 5 External Review Team Site Visit Program chairs/directors will be asked to provide a list of seven or eight names

Academic Program Review Guide 19

Contacts

For matters of policy and procedure: Brian Reed Associate Provost for Teaching and Learning 352 Waterman Bldg. Phone: 656-2232 [email protected]

For questions about metrics and data: Alex Yin Director, Office of Institutional Research 440 College Street Phone: 656-4418 [email protected] For logistical/coordination matters:

Catherine Symans Academic Program Review Coordinator 348 Waterman Bldg. Phone: 656-0903 [email protected]

For Faculty Senate curricular affairs matters: Laura Almstead Chair, Curricular Affairs Committee Department of Plant Biology 307 Jeffords Hall Phone: 656-2919 Laura Almstead Academic Program Review Web site: http://www.uvm.edu/~provost/?Page=academicprogramreview.html