Program Quality Assurance Manual - SAIT SAIT/Administration/Policies and...Criteria for External...

Program Quality Assurance Manual

June 2017 Version 2.8

TABLE OF CONTENTS

Preface ................................................................................................................................................................................................. 4

Background .................................................................................................................................................................................... 4

Program Quality Assurance at SAIT ............................................................................................................................................ 6

Scope ............................................................................................................................................................................................... 6

Programs with Regulatory Bodies ......................................................................................................................................... 7

Process Overview ........................................................................................................................................................................ 7

Information Portfolio .................................................................................................................................................................. 8

The Self-Study .............................................................................................................................................................................. 9

People Involved ......................................................................................................................................................................... 11

Resources.................................................................................................................................................................................... 13

Scheduling ................................................................................................................................................................................... 13

PQA Deliverables ...................................................................................................................................................................... 14

PQA Self-Study Process ......................................................................................................................................................... 14

External Review and Site Visit.................................................................................................................................................... 16

Selecting External Reviewers ................................................................................................................................................ 17

Criteria for External Reviewers ............................................................................................................................................. 17

Approval of Reviewers ............................................................................................................................................................ 18

Contact ......................................................................................................................................................................................... 19

Timing of the External Review .............................................................................................................................................. 19

The Site Visit ............................................................................................................................................................................... 20

PQA Action Plan ........................................................................................................................................................................ 21

Academic Quality Council ....................................................................................................................................................... 22

Annual Self-Assessment ........................................................................................................................................................ 23

Glossary ............................................................................................................................................................................................ 24

References ....................................................................................................................................................................................... 26

PQA-Manual and Guide-2017-06-13-V2.8.docx 2

O:\staff\Common\data\Program Quality and Development Department\2016-17-Program Quality Assurance (PQA)\PQA-Manual and Guide-2017-06-13-V2.8.docx

REVISIONS TABLE

Version Date Description

2.7 May 30, 2017 Modifications in manual to reflect the PQA Roles and Responsibilities document.

2.8 June 13, 2017 Modifications to the external review and site visit section to reflect the External Review Nomination Form document.



PREFACE

The quality assurance manual that follows, and that also is referred to in the SAIT program quality assurance (PQA) policy, is intended to provide an overview of the requirements for cyclical review at SAIT. It is provided to assist programs participating in SAIT’s PQA process.

Background

It is the policy of the Board of Governors that SAIT maintains a PQA process that ensures programs align with SAIT’s strategic direction and with internal, external and industry requirements. The PQA process is one way SAIT assures the community that it is a leader in post-secondary education and adept at meeting industry requirements for skilled graduates. Students, alumni, government, post-secondary partners and employers must be confident that SAIT has done everything necessary to ensure that graduates have met program goals and outcomes, and has also been engaged in a high quality learning experience along the way. In 2014/15, SAIT developed, prototyped and piloted a new program cyclical review process that is more robust, is multi-year in scope and is results-oriented to replace the limited program success process. After consultation with Deans’ Council, investigation of a new process began in April 2013 on how SAIT could expand the existing program review processes to meet or exceed best practices. The cyclical review process, called program quality assurance (PQA), was developed following a comparative scan of program review processes used by polytechnics and other post-secondary institutions across Canada. The SAIT Board of Governors approved the PQA policy and procedure in May 2016 (AC.2.19, 2016; AC.2.19.1, 2016). The review process supports SAIT’s commitment to providing leading-edge education for students. The PQA approach assumes that SAIT graduates are answering a need in the employment market. Graduates fill that need by achieving program learning outcomes that make them highly successful on the job. The quality assurance lens makes certain that SAIT programs are viewed holistically. SAIT recognizes that a program goes beyond its curriculum, and that human, financial, physical and technological resources all contribute to student success and an effective program. Regular review and monitoring of the effectiveness and efficient use of these resources is foundational to quality assurance.



The theoretical framework for SAIT’s quality assurance process rests primarily within the discipline of program evaluation. Michael Quinn Patton, a recognized expert in the field, suggests that evaluation is the systematic collection and analysis of information about program activities, characteristics and outcomes to make judgments about the program, to improve program effectiveness and/or to inform decisions about future programming (Patton, 1997). Program evaluation is an emerging discipline and offers an array of approaches to evaluation. Theoretical approaches have to be practical and applicable to the environment in which they are situated. At SAIT, focused, action-oriented thinking that is a part of the institution’s system and embedded in its people will support long-term culture shift to a more evaluative mindset.



PROGRAM QUALITY ASSURANCE AT SAIT

Program quality assurance provides us with the opportunity to tell our quality story - and to make sure that we are engaging in the right activities to meet our promise to our students and partners. The process ensures that we engage the appropriate stakeholders to help us make informed adjustments to our programs based on feedback from a wide range of inputs and a multitude of data sources. Program quality assurance:

• Requires that we have foresight and vision and that we put forth effective strategic plans. • Requires us to scan the environment to ensure that we stay relevant and competitive. • Requires us to collect stakeholder feedback, regularly, for analysis and decision-making. • Follows from having well-defined parameters and key performance indicators that are measured

carefully and regularly. • Means that we are aware and mindful of our day-to-day business practices and initiatives over the

long term. • Requires that we measure ourselves regularly and respond accordingly to both challenges and

successes.

At SAIT, we aspire to a quality culture - well-defined policies/procedures/processes, data-based decision making, stakeholder engagement and feedback, effective communication, and vision – which mean that all SAIT stakeholders will rise to their optimal potential at SAIT.

Scope

Participation in the quality assurance process applies to programs as defined in the procedure (see above), and is a mandatory activity for SAIT programs in scope.

Procedure AC.2.19.1 Program Quality Assurance states:

A prescribed curriculum leading to a SAIT credential. A program is divided into a number of courses. For the purposes of this procedure, a program includes credit programs, apprenticeship programs, and non-credit programs (AC.2.19.1, 2016).



Programs with Regulatory Bodies

Effort is made to ensure that the PQA processes and external accreditation processes are complementary and redundancies are limited.

Process Overview

Program quality assurance follows a continuous cycle with key phases and associated processes:

1. Programs are assessed based on criteria during the program quality self-study process. This process includes three stages: the self-study, the external review and the report and approval.

2. The outcome of the program quality review is the PQA action plan, which identifies those actions which will result in the continuous improvement of the program.


The program quality assurance process may be modified for programs subject to external accreditation reviews, for apprenticeship programs, and for non-credit programs, in order to limit redundancies and to ensure process efficiencies. Modifications may include, but are not limited to, adapting the timelines and/or scope of the review. The Vice President Academic, in consultation with Quality Council and the program’s dean, must approve the modifications to the process (AC.2.19.1, 2016).


The program quality assurance process is a developmental process that includes a systematic collection and review of multiple program quality criteria and evidence from both a qualitative and a quantitative perspective. The program’s faculty, support staff, and management work together in this process to evaluate and report on how well the program meets the needs of students and employers, and to recommend and implement improvements to the program (AC.2.19.1, 2016).



3. Implementation of the recommendations is monitored by the program academic chair and/or dean until the next program review is initiated. An annual self-assessment of progress is conducted and updates are provided to the school dean and vice president, academic.

The process relies on input from numerous stakeholders, internal and external. In addition to the review conducted every five years, and based on the outcome of the program quality process, the program implements the goals of the PQA action plan over a four-year time frame. Progress is reviewed annually by the faculty, program support staff, academic chair and dean. Processes and templates support the PQA policy and procedure (AC.2.19, 2016; AC.2.19.1, 2016).

Information Portfolio

Program staff collect and organize the program’s information portfolio. During the self-study year, the program completes the self-study, whereby the program, in collaboration with institutional support areas, collects and analyzes feedback from individuals and groups directly and indirectly involved with the program. These individuals and groups may include, for instance, students, industry representatives, alumni, program staff, instructors and program advisory members. Existing key performance and other SAIT data is part of the information portfolio. As the cycle of the review is generally five years, the data includes five years’ worth of information. The self-study process includes the review of multiple program quality criteria from a quantitative and qualitative perspective and is intended to be systematic and developmental in nature.

Annual progress monitored

Program Action Plan Implemented

Approved by VP Academic

Review by Quality Council

Create Program Response and Action Plan

Conduct External Review (May be modified for accredited)

Conduct Program Self Study (Modified for accredited)

Program Review Initiated



• Qualitative data is descriptive. It represents what can be observed, but cannot be measured by

numbers. It also represents types or categories of information. • Quantitative data are measures of values or counts and are expressed as numbers.

Coaches from SAIT’s Program Quality and Development department work directly with faculty leads to prepare them for the self-study process. A lead from the program’s faculty works with stakeholders to understand program information. This step is usually achieved with small groups tasked with analyzing program information and drawing conclusions. Institutional resources are available to support this analysis. Programs work in a workshop environment to provide initial findings and to validate them. See Program Quality Assurance (PQA): Managing Data for detailed information about the data collection and analysis (Program Quality and Development, 2016).

The Self-Study

The self-study process asks faculty to consider how well the program is meeting its goals and SAIT’s goals. The self-study is guided by the PQA inquiry framework, which is organized into five quality standards. The inquiry framework ensures that each program is assessed against established criteria. The standards inform the review in a comprehensive way – this creates consistency within the review process. Indicators are seen as a total picture and no particular indicator dominates. The framework allows for team members to include information which is relevant to their program during the review. The purpose of the self-study is to investigate program performance in light of the five quality standards. The PQA standards guide the evaluation. They are not “laws”. Similar to technical standards with exact dimensions and tolerances for manufactured items, for example, the standards identify ways to improve quality. However, unlike technical standards, the PQA standards require those involved in the evaluation to demonstrate responsiveness and judgment specific to the evaluation context (Caruthers, Hopson, Shulha, & Yarbrough, 2011). The standards include: • Quality Management – regular quality review that involves faculty, students and industry representatives; • Program Goals – employment in a specific field with the opportunity for advancement; • Educational Design – students meeting expected learning outcomes; • Educational Experience – students’ learning experiences leading to lifelong learning and career

progression; • Program Resources – physical, technological, financial resources and time used effectively to meet

program goals.



Each PQA standard is supported by enabling questions which help participants create findings and recommendations related to each standard, as well as draw global conclusions about the program. This information is provided in the self-study report, a key output of the periodic review process. The Program Faculty Self-Study Advisory group, consisting of a faculty lead, faculty members and other school stakeholders, supports the self-study process. Using a decision-making protocol, the advisory group reviews findings in order to make recommendations. The self-study report identifies overall strengths of the program and summarizes key recommendations.

The self-study report is distributed to stakeholders, and their feedback is included with the report as an appendix. The self-study report is formatted as follows:

A. Table of Contents B. Executive Summary C. Self-Study Detailed Report (five sections aligned with PQA framework)

i. External Reviewer Comments



ii. Reflections and Conclusions iii. Recommendations iv. Evidence

D. Appendices A. Abbreviations B. Information Portfolio C. Stakeholder Feedback D. External Reviewer Comments E. Confirmation of Completion F. Site Visit Schedule

Each PQA self-study report is reviewed by an external review team that writes a report, which is incorporated into the self-study report.

People Involved

Program quality assurance relies on active engagement from program faculty, the academic chair, the school and/or SAIT stakeholders. Self-study projects are carried out with the support of the Program Quality and Development department, in consultation with the school responsible for the program. Quality assurance is enabled by the department; however, the evaluation mindset does not rest with the quality group, but is inherent in the institution and found at every desk. The focus of the review is on outputs as opposed to the process, though a rigorous, collaborative review process is essential to the quality of the report. Specific roles and responsibilities are detailed in project documents. The self-study is a collaborative, team activity. The school dean appoints the academic chair/coordinator who is responsible for the completion of the final report. A self-study team may include:

• Academic chair/program coordinator • Faculty PQA lead • Faculty member(s) (program and/or service) • Program support staff • School curriculum coordinator • Program specialist • Program administrator



Ancillary members could include: • A senior student from the program • Industry representative(s) - for example, from the Program Advisory Committee (PAC), or a graduate • Administrators associated with academic activities

Roles and Responsibilities Refer to the “PQA roles and responsibilities for self-study” document for the responsibilities of the roles listed below. The document is located at: O:\staff\Common\data\Program Quality and Development Department (Search for “roles”)

PQA Role SAIT, School or Program Title

Institutional Level

Executive Project Sponsor Vice President Academic

Institutional Contributor Manager, Program Quality and Development

Project Coordinator Coordinator, Program Quality and Development *may be resourced through alternate SAIT departments

Evaluation Process Coach Specialist, Program Quality and Development *may be resourced through alternate SAIT departments

Self-Study Contributors (SAIT stakeholders) Various SAIT managers and coordinators

School and Program Level

Project Champion Dean, or Associate Dean

Program Champion Academic Chair (AC)

School Curriculum Contributor Curriculum Coordinator

Faculty Lead Faculty

Administrative Assistant Supervisor Various

Project Support and Administrative Assistant Administrative Assistant

Self-Study Contributors Program Faculty and Staff



Resources

The self-study project may include representation from: • School leadership team • School financial analyst • School IT support team • SAIT Academic Services • SAIT Centre for Instructional Technology and Development • SAIT Institutional Planning and Analysis • SAIT Learner Services • SAIT Marketing, Insight and Analytics

Scheduling

The procedure includes provisions for scheduling programs within scope of the policy on a five-year review cycle. Ideally, programs complete phases 1 to 6 (below) in one academic year. Phases 7 and 8 require an annual update on progress toward the actions identified in the action plan and, therefore, are an on-going activity. SAIT’s institutional review schedule, as set by the Office of the Vice President Academic, determines when program quality projects are completed. Program quality assurance phases are as follows:

1. Project initiation, planning and orientation 2. Data collection 3. Self-study process 4. Report completion 5. Site visit and external review 6. Action plan review and approval 7. Action plan implementation 8. PQA annual progress updates


The program quality assurance process is a five-year cycle, unless modified as described above for accreditation requirements, apprenticeship requirements, or external requirements. This cycle starts with a review year, which is then followed by several self-assessment years (AC.2.19.1, 2016).



PQA Deliverables

Key deliverables of the PQA process include:

• Program information portfolio • Self-study report • External review report • PQA action plan

PQA Self-Study Process

Findings The findings step of the PQA process encourages faculty groups to self-manage to achieve identified outputs or deliverables. The program analyzes feedback and data. Process coaches support the review with tools and templates; self-study participants review program information and evaluate the evidence in order to draw conclusions or “findings” that answer the PQA inquiry framework’s key questions. A final review of the findings is completed before they are integrated into the self-study report and/or reviewed by stakeholders. Findings are included in the Reflections and Conclusions section of the report. Activities during this stage of the process include:

• Design of the evaluation approach • Data review and analysis • Creation of initial findings • Findings validation

Results During the results step in the process, the project lead takes on a greater role as he or she assembles the report and writes additional components. Faculty continue to be involved in the process during the results phase. Usually, the Program Faculty Self-Study Advisory group works closely with the lead to finalize the self-study report. The committee includes representation across the PQA faculty working groups and an academic chair. In programs with a small number of faculty participants, the advisory group may include all of those involved.


Self-study: A systematic, structured examination of program quality criteria in relation to the program’s internal, external and industry requirements (AC.2.19.1, 2016).



Also, central to this phase is the creation of recommendations, the executive summary and the review of the self-study report with key stakeholders. Activities during this step include:

• Creation of the self-study report, including recommendations and executive summary • Finalization of the program information portfolio • Stakeholder feedback

Stakeholder Engagement The purpose of the stakeholder engagement process is to seek feedback on the contents of the report. Feedback is gathered during a facilitated meeting or through a virtual process. Feedback informs the next phases in the process. Stakeholders are encouraged to think of providing feedback as a response to the findings and recommendations.



EXTERNAL REVIEW AND SITE VISIT

An external review is an essential stage in the quality assurance process and supports the integrity of the academic system. The review provides an independent and objective assessment of the program/unit under review, assesses the quality of the program/unit in a broader academic and industry context and provides new and valuable insights into how well the program/unit is meeting its goals. The review process supports continuous improvement. When the self-study report and program information portfolio is complete, the academic chair/coordinator notifies the Program Quality and Development department via email. The Program Quality and Development department ensures that the self-study report is complete. If it is not, the program academic chair/coordinator is notified with a request to provide the missing information and/or revise the report. A complete report is ready for the next step in the process – the external review and site visit. Once the self-study report is complete and has been distributed for review, it is made available to the dean or school and faculty members in the department or program under review. The self-study report should be provided to the reviewers at least four weeks before the external reviewers are scheduled to conduct the site visit. The academic chair/coordinator, in collaboration with the Program Quality and Development department, establishes a schedule for completing the final steps in the review process. External reviewers:

• Are required to sign a Statement of Non-Conflict and a Confidentially Agreement • Provide a current CV or resume • Review the program self-study report in detail, specifically, the findings, evidence and

recommendations • Provide high level instructive and/or constructive feedback about the program • Identify information that may have been missed during the self-study process • Identify areas where there are gaps in information, and suggest additional information where needed


External review: A program review conducted by at least two reviewers who are external to the program’s school and/or SAIT. Detailed information governing the selection of the reviewers and their roles and responsibilities are set out in the Program Quality Assurance Manual (AC.2.19.1, 2016).



• Identify areas for additional reflection and/or exploration • Comment on the sections of the report that are most relevant to their area(s) of expertise, but not feel

compelled to comment on those areas that are not • Report on the program’s self-study report using templates provided by the Program Quality and

Development department • Finalize the review within four weeks of the site visit • Are compensated for expenses and provided with a honorarium

The reviewers (“review team”) work with the External Review Site Facilitator (ERSF) to decide on the approach/process they will take to complete the external review report. The review team uses the templates provided by SAIT to structure the report findings and commentary. Following the site visit, the external report is returned to the Program Quality and Development department by the review team on the date established by the ERSF and the review team prior to/during the site visit. The Program Quality and Development department compiles the report (by adding reviewer comments as provided) and distributes the final copy to the reviewers for confirmation of completion. The review team provides written confirmation that the external review report is complete. This is included with the report.

Selecting External Reviewers

The academic chair/coordinator, in collaboration with the dean or associate dean, faculty members and the Program Quality and Development department, identifies potential members for the external review team. A review team primarily consists of at least two members: one academic reviewer and one industry reviewer (a third reviewer may be added). To maintain the integrity of the process, reviewers should be selected based on publicly-available information about their qualifications and background. The academic chair/coordinator, in collaboration with the Program Quality and Development department, establishes a schedule for completing the external review process and site visit. External reviewers are confirmed by the academic chair and approved by the dean.

Criteria for External Reviewers

All External Reviewers To maintain the integrity of the process, external reviewers should not:

• Have served as a member of any SAIT Program Advisory Committee (PAC) in the past five years. • Be or have been employed by SAIT.



• Have received a SAIT credential the same as or closely related to the program or unit under review. • Have served as a mentor, research collaborator, co-author, department member or external examiner

to faculty or the academic chair. • Be in a close family relationship with a member of the program or unit under review. • Be or have been a Master’s or Doctoral supervisor of anyone in the program or unit under review. • Usually be a resident of Canada.

External Academic Reviewers To maintain the integrity of the process, external academic reviewers should:

• Be full professors/instructors (where possible), or associate professors/instructors or adjunct faculty from another post-secondary institution.

• Have Doctoral degrees or the highest possible credential in a discipline that is the same as, or closely related to, the program or unit under review.

• Usually be from a post-secondary institution comparable in size to SAIT.

External Industry Reviewers To maintain the integrity of the process, external industry reviewers should:

• Have a recognized credential and/or certification in an industry area or discipline that is the same as, or closely related to, the program or unit under review.

• Actively employed in a relevant industry position Have academic credentials that may enhance the review

Approval of Reviewers

The academic chair/coordinator develops a list of three or more potential academic reviewers and three or more potential industry reviewers who meet the criteria, preferably ranked in order of preference and forwards the list with accompanying documentation (nomination form for each potential reviewer) to the dean. The list includes, for each individual:

• Name, position, organization, e-mail address and telephone number • Brief background information including: academic qualifications, awarding institutions, industry

certifications and designations • A curriculum vitae, if freely available online: organization website, Linkenin, etc. • Rationale for inclusion



The dean or designate reviews the list of potential external reviewers and then either:

• Approves the list—if the dean approves the list, he/she forwards it to the Program Quality and Development department and informs the academic chair/coordinator that it has been approved and the Program Quality and Development department begins contacting the external review candidates (see below).

• Amends the list and requests revisions—the dean may reject an individual(s) on the list whom he or she deems inappropriate to serve as an external reviewer and provides a brief rationale. The academic chair/coordinator will revise the list and resubmit for approval. If the dean/associate dean approves the list, he/she forwards it to the Program Quality and Development department, informs the academic chair/coordinator that it has been approved and the Program Quality and Development department begins contacting the external review candidates (see below).

Contact

A representative of the Program Quality and Development department contacts the individuals on the approved list to secure two of them (or more if required) as external reviewers. Once the required number of reviewers has been confirmed, the academic chair and program administrator work collaboratively with the Program Quality and Development department to arrange the site visit. In summary, there may be three points of contact with the reviewers by:

• The dean or designate, during the selection process. • The academic chair and/or the administrative assistant of the program or unit under review, in support

of the site visit. • The Program Quality and Development department to arrange the details of the site visit, to provide

the review documents and general expectations in advance of the site visit, to follow up on the status of the external review report, and to inform the reviewers about the outcomes of the review.

Up to the point of the site visit, no other contact with the reviewers should be made by the program.

Timing of the External Review

At least four weeks before the external reviewers’ site visit, the Program Quality and Development department provides the external reviewers with access to: the self-study report, policy AC.2.19 Program Quality Assurance and its accompanying procedure, the program information portfolio, and directions and a timeline for completing the external review report.



The Site Visit

The Program Quality and Development department works with the review team and the program to set the agenda and plan the site visit. During the visit, team members interview key personnel involved with the program. The site visit may require one or two days, depending on the needs of the program.

A site visit involves the review team meeting with and/or interviewing a variety of individuals and groups that may include:

• School leadership • Program academic chair/coordinator • Service academic chair/coordinator • Faculty self-study lead • Faculty members (in groups and/or individually) • Students and Alumni • Program administrator(s) and support staff • School support staff • Representatives from the following SAIT stakeholders: Library, Office of the Registrar, Articulation,

Learner Success Services and ARIS • Representative from the Program Quality and Development department • Industry advisors (such as members of the Program Advisory Committee (PAC), members of

accrediting bodies, etc., as appropriate) The ERSF greets the team and guides the reviewers through their site visit, addresses questions and concerns as they arise and facilitates access to people and facilities as required. For a sample site visit agenda, see the template for the site visit.

Following the site visit, the review team prepares its findings, using the template provided by the Program Quality and Development department. The external review team provides the report to the Program Quality and Development department within four weeks of the site visit. In summary, the order of events for the site visit is as follows:

• Reviewers are selected and site visit is arranged • Self-study report and accompanying documentation is provided to external reviewers • External review site visit takes place • External reviewers’ report is provided to Program Quality and Development department • External reviewers’ report is provided to school’s dean and program academic chair



PQA Action Plan

Following the site visit, the program responds by creating a PQA action plan. This is the plan developed by the program in response to the internal self-study and the external review. The plan contains specific recommendations for program improvement and enhancement, is strongly encouraged to be presented to Academic Quality Council within six months of the external review and is approved by the Vice President, Academic. It also provides an area for annual progress updates. The template for the action plan is as follows:

The development and approval of the PQA action plan involves a number of key individuals: PQA Action Plan - Roles and Responsibilities

Role Responsibility

Program Quality and Development department

• Reviews quality assurance documents and advances the quality assurance process



Academic chair, in collaboration with faculty and school support staff

• Develops the action plan in response to the self-study and the external review

• Implements the action plan once approved • Completes annual progress updates for the action plan • Follows up with the dean on an annual basis

Dean or designate/school leadership

• Supports the development and implementation of the action plan • Presents the action plan to SAIT Academic Quality Council • Supports action plan annual progress updates

SAIT Academic Quality Council

• Reviews quality assurance documents and makes recommendations to Deans’ Council and the vice president, academic

Vice president, academic • Approves the action plan

Academic Quality Council

SAIT’s Academic Quality Council reviews the program’s self-study report and resulting action plan and provides recommendations to the vice president, academic and Deans’ Council regarding implementation. The mandate of the council is to support the quality of educational experiences for SAIT learners and the communities that SAIT serves. The council works within the context of SAIT’s Strategic Plan and Education Plan. It provides a forum for open discussion to support informed decision-making in matters related to programs, curriculum acquisition (such as licensing, purchasing or brokering), academic service areas and SAIT programming quality. In addition, the council contributes to quality at an institutional level, including supporting the development and implementation of initiatives that achieve and enhance institutional quality. In terms of quality assurance, the council’s responsibility is to carry out PQA activities as per the provisions of policy AC.2.19 Program Quality Assurance (AC.2.19, 2016), its accompanying procedure and accompanying


A council that reviews the internal self-study, the external reviewers’ report, and the program’s action plan, and that reports to the Vice President Academic. Detailed information governing the Council and its roles and responsibilities is set out in the Program Quality Assurance Manual (AC.2.19.1, 2016).



processes. The Academic Quality Council Terms of Reference (Terms of Reference, 2016) provides more detail about the council’s membership and responsibilities.

Annual Self-Assessment

An annual check-in on the program and the progress towards the goals identified through the PQA process is completed during non-self-study years. SAIT’s review process emphasizes continual improvement based on reflection and assessment of educational quality. All programs conduct an annual self-assessment at the end of each academic year and produce an annual program report, which supports ongoing reflection and planning and serves as a resource for programs when they undertake comprehensive review. The annual self-assessment provides the program with the opportunity to examine curriculum, teaching and learning practices, relationships and insights gained from industry, to highlight achievements, to address issues and to set goals for the coming year. The completion of the annual program report is the responsibility of the dean; the academic chair administers the ongoing review and improvement process outlined in the action plan.


During subsequent self-assessment years, the program’s academic chair/coordinator, faculty and support staff work together to conduct an annual self-assessment and to provide an update to the program’s dean and the Office of the Vice President Academic, including an update on the progress that the program has made towards the goals identified in its action plan (AC.2.19.1, 2016).



GLOSSARY

Accreditation The process by which a (non-) governmental or private body evaluates the quality of a higher education institution as a whole or of a specific educational programme in order to formally recognize it as having met certain pre-determined minimal criteria or standards (Grunberg, Parlea, & Vlasceanu, 2007).

Champion A project champion is an individual who has the authority to use resources

within or outside an organization for completion of a given project. A project champion is chosen by the management so as to ensure supervision of a specific project right from its initiation phase to its execution phase (Khan, 2012).

Current state analysis A systematic process for gathering information about the current conditions

of the program - also called a needs assessment. Enabling questions These are the questions inside each section of the PQA inquiry framework.

They are intended to enable participants to answer the key question associated with the "standard". Not all enabling questions need to be answered.

Evaluation The systematic collection and analysis of information about program

activities, characteristics and outcomes to make judgments about the program, to improve program effectiveness and/or to inform decisions about future programming (Patton, 1997).

Evaluation design A plan for conducting evaluations. Designs may be pre-ordinate or emergent. Findings A finding uses evidence from one or more sources to allow for a factual

statement. Gap analysis The use of ‘gap’ does not necessarily imply negative; if we are trying to better

understand our policies, processes and procedures, then the gaps we find between what we ‘say’ we will do and ‘what’ we do should inspire discussion and positive change. Understanding gaps is essential to minimizing your variances in practice and feeding continuous improvement.



Key questions These are the main questions associated with the subsection(s) of the PQA

inquiry framework. Outcomes Changes or benefits that result from activities or outputs; in our case, the

evaluation is focused on understanding whether or not the program has met its intended outcomes in the areas defined by the framework.

Resource A party who provides input to the self-study process; may also have an

interest in the output or outcome. Review An assessment of the performance of a program, periodically or on an ad hoc

basis. Frequently “evaluation” is used for a more comprehensive and/or more in-depth assessment than “review”. Reviews tend to emphasize operational aspects (Development Assistance Committee, 2002).

Sponsor This is an active senior management role, responsible for identifying the

business need, problem or opportunity. The sponsor ensures the project remains a viable proposition and that benefits are realised, resolving any issues outside the control of the project manager (Project Smart, n.d.).

Stakeholder A party with an interest or concern in the output and/or outcome of the self-

study process.



REFERENCES

AC.2.19 program quality assurance policy [PDF]. (2016, April 14). Retrieved from http://www.sait.ca/Documents/About%20SAIT/Policies%20and%20Procedures/Academic%20Student/pdf/AC.2.19%20Program%20Quality%20Assurance%20Policy.pdf

AC.2.19.1 program quality assurance procedure [PDF]. (2016, April 06). Retrieved from

http://www.sait.ca/Documents/About%20SAIT/Policies%20and%20Procedures/Academic%20Student/pdf/AC.2.19.1%20Program%20Quality%20Assurance%20Procedure.pdf

Caruthers, F., Hopson, R., Shulha, L., & Yarbrough, D. (2011). The program evaluation standards: A guide for

evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc. Also retrieved from https://books.google.ca/books?id=81kXBAAAQBAJ&pg=PA305&lpg=PA305&dq=preskill+and+Lynn+evaluation+rigour&source=bl&ots=8BvQp0KER8&sig=dpLNGApjUnJgqY58uakdiJ_tHEQ&hl=en&sa=X&ved=0ahUKEwi0kaG8zZjKAhVHxWMKHeOkAAUQ6AEIMDAD#v=onepage&q=preskill%20and%20Lynn%20evaluation%20rigour&f=false

Community Solutions: Planning & Evaluation. (2015). Evaluation glossary. Retrieved from

http://communitysolutions.ca/web/evaluation-glossary-2/ Development Assistance Committee. (2002). Glossary of key terms in evaluation and results-based management.

Paris, France: OECD. Grunberg, L., Parlea, D., & Vlasceanu, L. (2007). Quality assurance and accreditation: A glossary of basic terms and

definitions (2nd ed.). Bucharest: UNESCO. Khan, V. (2012). Responsibilities of a Six Sigma project champion. Retrieved from

http://www.brighthubpm.com/resource-management/49034-responsibities-of-a-project-champion/ Patton, M. Q. (1997). Utilization-focused evaluation. Thousand Oaks, CA: Sage Publications. Program Quality and Development. (2016). Program Quality Assurance (PQA): Managing data. Project Smart. (n.d.). Great sponsor + great PM = great success. Retrieved from

https://www.projectsmart.co.uk/project-sponsorship.php Terms of reference: Academic quality council [PDF]. (2016, May 25).



http://www.sait.ca/Documents/About%20SAIT/Policies%20and%20Procedures/Academic%20Student/pdf/AC.2.19%20Program%20Quality%20Assurance%20Policy.pdf

http://www.sait.ca/Documents/About%20SAIT/Policies%20and%20Procedures/Academic%20Student/pdf/AC.2.19%20Program%20Quality%20Assurance%20Policy.pdf



https://books.google.ca/books?id=81kXBAAAQBAJ&pg=PA305&lpg=PA305&dq=preskill+and+Lynn+evaluation+rigour&source=bl&ots=8BvQp0KER8&sig=dpLNGApjUnJgqY58uakdiJ_tHEQ&hl=en&sa=X&ved=0ahUKEwi0kaG8zZjKAhVHxWMKHeOkAAUQ6AEIMDAD%23v=onepage&q=preskill%20and%20Lynn%20evaluation%20rigour&f=false




http://communitysolutions.ca/web/evaluation-glossary-2/

http://www.brighthubpm.com/resource-management/49034-responsibities-of-a-project-champion/

https://www.projectsmart.co.uk/project-sponsorship.php

© 2016, Southern Alberta Institute of Technology. All rights reserved. This publication and materials herein are protected by applicable intellectual property laws. Unauthorized reproduction and distribution of this publication in whole or part is prohibited. For more information, contact: Program Quality and Development Department Southern Alberta Institute of Technology 1301 16 Ave. N.W., Calgary, AB T2M 0L4

Program Quality Assurance Manual - SAIT SAIT/Administration/Policies and...Criteria for External...

Documents

Transcript of Program Quality Assurance Manual - SAIT SAIT/Administration/Policies and...Criteria for External...