ACCREDITATION COUNCIL FOR BUSINESS SCHOOLS AND PROGRAMS

Site Visit Team ChairsRole and Responsibilities

January 2013

AGENDA ITEMS:

Welcome/Introductions Role of the chair Resources available to site

visit team Scheduling of Consensus Calls Travel arrangements Communication with

institutional contact Site Visit Schedule

WEBINAR AGENDA, CONT.

Working with first-time evaluators

Preparing feedback report – format, editing, timetable, etc.

Submission of feedback report Submission of expense reports

and other forms Any other topics

ROLE OF THE CHAIR Receive self-study of the institution to be

visited.Review self-study for completeness and content. Identify those areas in need of intensive scrutiny by the team while on site. Coordinate with team members,

commissioner, and mentor to schedule consensus call.

Determine how the self-study should be divided among the team members for on site review.

Prepare tentative assignments for team members and communicate those assignments to each member.

ROLE OF THE CHAIR Contact team members to obtain travel

itineraries for arrival to and return from the site to be visited. Discuss tentative self-study assignments and areas of concern identified in the self-study. 

Contact institutional coordinator to confirm visit dates.

Determine transportation and lodging accommodations.

Provide the coordinator with each team member’s itinerary.  

ROLE OF THE CHAIR, CONT.

Plan and finalize schedule of visit to include meetings with president, head of business unit, faculty, and other key personnel with the institutional coordinator.

Provide institutional coordinator with a list of sites and units to be visited by the team.

Provide institutional coordinator with the requirements for the team work room, including all materials to be provided, computer and clerical support, and telephones.

Develop a schedule of activities, visits, and inspections for each day of the visit to be distributed to institutional representatives and to the visiting team.

ROLE OF THE CHAIR, CONT.

Schedule a meeting of the evaluators for the afternoon/evening prior to the initiation of the site visit to - finalize the schedule of activities, - discuss the self-study, - finalize the distribution/assignment of individual team member responsibilities, - establish the procedure to follow in summarizing observations, - determine the format of the exit conference, - discuss preparation of the team report, and finalize the time frame for completion of all activities.  

ROLE OF THE CHAIR, CONT.

Include time frames in each day’s schedule when the team can convene to determine progress, identify problems or needs, and schedule additional interviews or data gathering as needed.

Communicate to institutional coordinator the undesirability of scheduling social events unless it is agreed that such events will facilitate interaction with key personnel, i.e., board members, alumni, employers, community residents, etc., who would otherwise be unavailable.

ROLE OF THE CHAIR, CONT.

Utilize ACBSP staff or the Commissioners as resource persons to clarify any problems of interpretation, etc., which may confront the team during the course of the visit.

Assure that each team member performs his/her tasks in such a manner as to validate the content of the self-study and not to function as consultants. Do not express personal comments as Chair nor allow team members to express such comments regarding their perceptions that the institution does/does not meet standards, or how the business unit should be structured, etc., in contrast to what is observed or documented.

References to how business activities are conducted at team member’s institutions should not be used as the criterion for judging the quality of the business unit being examined.

ROLE OF THE CHAIR, CONT.

 As team Chair, you will be expected to prepare a final report of the team visit that will reflect the consensus of the team as a whole.

The Chair will be expected to prepare an Executive Summary of the visit to provide an overview of the entire visit and to express the extent to which the institution accurately described the business unit and the extent to which the team was able to validate the information contained in the self-study document.

This Summary should be concise (about 3 pages) and cover all the major standards, e.g., Leadership, Strategic Planning, etc.

RESOURCES AVAILABLE

The CD enclosed with the site visit materials contains several folders with these files:

Evaluation and Site visit handoutsACBSP Observation GuidelinesScoring GuidelinesPractical Guide for Evaluators (which

includes)Site Visit Issue Worksheet

EVALUATION AND SITE VISIT HANDOUTS, CONT.

Site Visit Questions Site Visit Issue Strategy

Evaluator Workbook Executive Summary Feedback Report

Forms to sign and return – evaluator agreement form, expense form, and evaluation form (electronically).

EVALUATOR RESOURCES ON CD, CONT.

Accreditation Process Manuals

Standards and Criteria (most current version)

Electronic copy of the self-study when provided

CHANGES IN WORKBOOK FORMAT :ASSOCIATE DEGREE WORKBOOK EXAMPLE

STANDARD 1: LEADERSHIP   Business Unit administrators and faculty should lead and be involved in creating and

sustaining values, business school or program directions, performance expectations, student focus, and a leadership system that promotes performance excellence. Values and expectations should be integrated into the business unit’s leadership system to enable the business unit to continuously learn, improve, and address its societal responsibilities and community involvement.

  Criterion 1.1 - Leadership Strategies Explain the business unit leadership strategies or systems to foster a high-performance work

environment that results in meeting the mission of the business unit.   Explain how the business unit maintains effective communications and develops participation

throughout the business unit.   Observations of Evaluator:   Is there evidence that all areas in this Criterion have been addressed? Yes No   Strengths:       OFIs:  

CHANGES IN WORKBOOK FORMAT :BAC/GRAD DEGREE WORKBOOK EXAMPLE

Criterion 1.1.c The business school or program must have processes in place for evaluating the performance of both administrators and faculty.

  Explain how the performances of administrators and faculty are evaluated.   Observations of Evaluator:   Is there evidence that all areas in this Criterion have been addressed? Yes No   Strengths:         OFIs:

CHANGES IN FORMAT – NO MORE TABLE FORMAT TO ENTER INFORMATION – NO USE OF + AND – FOR COMMENTS.

Advantage: After each team member has finalized their comments, it is a simple copy and paste into the feedback report. You do not have to remove the table formatting.

FEEDBACK REPORT TEMPLATENEW FOR ASSOCIATE DEGREE

  No changes for Baccalaureate/Graduate Degree Reports. The Executive

Summary is part of the feedback report. EXECUTIVE SUMMARY     Key summary findings from the feedback report:   The most important strengths or outstanding practices are:     The most significant opportunities for improvement are:      

FEEDBACK REPORT TEMPLATE, CONT. COPY AND PASTE FINAL COMMENTS FROM EVALUATOR WORKBOOK:

STANDARD #1. Leadership Business Unit administrators and faculty should lead and be involved in creating

and sustaining values, business school or program directions, performance expectations, student focus, and a leadership system that promotes performance excellence. Values and expectations should be integrated into the business unit’s leadership system to enable the business unit to continuously learn, improve, and address its societal responsibilities and community involvement.

  Strengths  Example:  The business unit administrators communicate program directions and

performance expectations to their faculty as part of the faculty handbook and during the mandatory training provided at the beginning of the academic year.

   Opportunities for Improvement Example: While performance expectations are clearly communicated, there is no evidence

that the business unit has a process in place to use results from administrator and faculty evaluations to make improvements.

NEW EVALUATOR CRITIQUE FORM

Excel Worksheet rather than Word Document

Process can now be evaluated by Pre-site visit performance Site visit performance

Post Site visit performanceIndividual team member performance

The new format will allow ACBSP to provide each team member with feedback from the team as well as commissioners.

NEW FORMAT FOR EVALUATOR CRITIQUE:

SAMPLE CONSENSUS CALL AGENDA: Introduction of Participants / Self-introductions (5 min)

Name, School, Location History with ACBSP / Experience as an evaluator Cell phone contact info

  Review of Planned Schedule (5 min)

Plans for arrival in destination city Review of Divide and Conquer plans (attached) Travel Plans to Satellite Campuses (if applicable) Institution Accommodation forms

  Discussion of Scoring Summary Worksheet (30 min)

Discussion led by Lead Reviewer for each Standard Goal = Preliminary score of BiC, VG/E, VG, G, IN, MIN for each

examination item (worksheet attached)  

SCHEDULING CONSENSUS CALLS

ACBSP requires that all site visit teams conduct a consensus call prior to the site visit. Once you have finalized the date and time, contact

Maria Hallerud at mhallerud@acbsp.org to schedule calls.

SAMPLE CONSENSUS CALL AGENDA:  Identification of Additional Info Needs from Institution (10 min)

Roundtable – Open Discussion / Other Topics (5 min) Complete/submit Program Evaluator Agreements prior to Site Visit Goal: Complete Draft Feedback Report with Strengths and OFI’s before

departure on Wednesday  Wrap-Up / Adjourn (2 min)

Divide & Conquer Plan Standard 1 - Chair Backup = Team Member 2 Standard 2 - Team Member 2 Backup = Team Member 3 Standard 3 - Team Member 3 Backup = Chair Standard 4 - Chair Backup = Team Member 2 Standard 5 - Team Member 2 Backup = Team Member 3 Standard 6  - Team Member 3 Backup = Chair      

SCORING BAND USED DURING CONSENSUS CALL:

Individual Assessment Dimensions (See Qualitative Scoring Band Dimensions and Assessment Guidelines for definitions)

Best in Class Very Good to Excellent Very Good Good Improvements Needed Major Improvements Needed

SCORING BANDQualitative Quantitative

Poor * no systematic approach evident; anecdotal information Poor * no results or poor results in areas reported

* beginning of a systematic approach to the primary purposes of the item

* early stages of developing trends; some improvements and/or early good performance levels in a few areas

* early stages of a transition from reacting to problems to a general improvement orientation

* major gaps exist in deployment that would inhibit prog- ress in achieving the primary purposes of the item

* a sound, systematic approach, responsive to the pri- mary purposes of the itema fact-based improvement process in place in key areas; more emphasis is placed on improvement than on reaction to problems

no pattern of adverse trends and/or poor performance levels in areas of importance to the applicant’s key

no major gaps in deployment, though some areas or business requirementswork units may be in very early stages of deployment * some trends and/or current performance levels --

evaluated against relevant comparisons and/or good to very good relative performance levels

* a sound, systematic approach, responsive to the over- all purposes of the item

* current performance is good to excellent in most areas of importance to the applicant’s key business requirements

a fact-based improvement process is a key manage- ment tool; clear evidence of refinement and improved

* most improvement trends and/or performance levels are sustained

integration as a result of improvement cycles and anal- ysis

many to most trends and/or current performance levels -- evaluated against relevant comparisons and/or bench-

* approach is well-deployed, with no major gaps; deploy- ment may vary in some areas or work units

marks -- show areas of leadership and very good relative performance levels

* a sound, systematic approach, fully responsive to all the requirements of the item

current performance is excellent in most areas of importance to the applicant’s key business require-

a very strong, fact-based improvement process is a mentskey management tool; strong refinement and integra- tion -- backed by excellent analysis

* excellent improvement trends and/or sustained excellent performance levels in most areas

* approach is fully deployed without any significant weaknesses or gaps in any areas or work units

* strong evidence of industry and benchmark leadership demonstrated in many areas

*

*

*

*Best in Class Best in Class

* results not reported for many to most areas of impor- tance to the applicant’s key business requirements

*

*

Good

Good to Excellent

Good to Excellent

improvement trends and/or good performance levels reported for many to most areas of importance to the applicant’s key business requirements

Results

*

Dimensions Approach/Deployment Dimensions

Good

Major Improvements Needed

Major Improvements Needed

*

ASSOCIATE DEGREE SCORING SUMMARY WORKSHEETEvaluator Name: School Name:

SUMMARY OF EXAMINATION ITEMS Individual Assessment Dimension1 LEADERSHIP 2 STRATEGIC PLANNING 3 STUDENT, STAKEHOLDER, AND MARKET FOCUS

4 MEASUREMENT, ANALYSIS, AND KNOWLEDGE MANAGEMENT 4.1 Student Learning Outcomes Assessment 4.2 Program Evaluation 4.3 Student Assessment 5 FACULTY AND STAFF FOCUS 5.1 Human Resource Planning 5.2 Faculty Qualifications 5.3 Faculty Composition 5.4 Faculty Deployment 5.5 Faculty Load 5.6 Faculty Evaluation

5.7 Faculty and Staff Professional Development & Scholarly Activities 5.8 Faculty Instructional Development 5.9 Faculty Operational Policies, Procedures, & Practices6 PROCESS MANAGEMENT 6.1 Curriculum 6.2 Professional Component 6.3 General Education Component 6.4 Business Major Component6.5 Off-Campus Operations and Unique Items6.6 Minimum Grade Requirement6.7 Learning and Academic Resources6.8 Support Services6.9 Educational Innovation6.10 Articulation and Transfer Relationships

Examiner Name:

School's Name:

SUMMARY OF EXAMINATION ITEMS

Individual Assessment Dimension

1 LEADERSHIP

2 STRATEGIC PLANNING

3 STUDENT AND STAKEHOLDER FOCUS

4 MEASUREMENT AND ANALYSIS OF STUDENT LEARNING AND PERFORMANCE 4.1 Selection and Use of Information and Data

4.2 Selection and Use of Information Results

4.3 Selection and Use of Comparative Information and Data

4.4 Continuous Process Improvement

5 FACULTY AND STAFF FOCUS

5.1 Human Resource Planning

5.2 Employment

5.3 Faculty Deployment

5.4 Faculty Size and Load

5.5 Faculty Evaluation

5.6 Faculty and Staff Development

5.7 Faculty Operational Procedures, Policies, and Practices

5.8 Scholarly and Professional Activities

Baccalaureate/Graduate Degree Scoring Summary Worksheet

TRAVEL ARRANGEMENTS:

Airfare – ACBSP is trying to reduce expenses for our member institutions. Contact Steve or Diana prior to booking a flight if airfare is over $500. Location will determine airfare.

Hotel arrangements – some schools will arrange for direct billing. Others will ask that each team member be responsible for payment and get reimbursed from ACBSP. Please ask your institutional contact how they want to handle this.

TRAVEL ARRANGEMENTS:

Meals – ACBSP appreciates your efforts to keep expenses down during your stay. Try to avoid the most expensive items on the menu and purchasing alcohol as some schools do not cover alcoholic beverages for meal reimbursements.

Share transportation when possible.

INSTITUTIONAL CONTACT

The chair serves as the primary contact for the team. Any questions for the school should come through you so that the contact person is not overwhelmed with multiple messages and to avoid conflicting information.

If you have difficulty getting responses from the institutional contact or other team members, please let ACBSP know.

INSTITUTIONAL CONTACT

As you know, some institutions are more prepared for a site visit than others.

You can help by: providing a list of documents you will

need in the resource room, asking where the resource room is

located, where you should park, get lunch, etc.

SITE VISIT SCHEDULE

The Practical Guide for Evaluators - good outline of how the schedule should be arranged. Your schedule may vary depending on the availability of administrators, advisory groups, etc.

Build in time for the team to regroup and discuss any additional items that have yet to be covered or need clarification.

WORKING WITH FIRST-TIME EVALUATORS Make sure they know you are available

for questions and assistance. Provide examples of how you conduct

your review. Explain that institutions provide

examples of how they meet criteria in various areas of the self-study, not necessarily where they are asked to address the specific criteria.

WORKING WITH FIRST-TIME EVALUATORS Do they understand what has been

provided or need guidance? Reinforce the importance of following

the schedule and meeting deadlines. Suggest wording of questions if they are

struggling with getting information. Review the comments early in the

process to see if the comments sufficiently address the standard.

PREPARING THE FEEDBACK REPORT

Executive Summary – no more than three pages – cover the major strengths and OFI’s.

The feedback report should Be written in complete sentences with

correct grammar and punctuation. State that “there is no evidence”

rather than “it is not clear”. Any questions regarding a standard or criteria should be clarified during the site visit.

FEEDBACK REPORT, CONT.

The feedback report should Use the same style and font throughout the

document. After comments are copied and pasted, make sure format is the same.

Be reviewed by the entire team with a final edit by the chair prior to submitting to ACBSP. Please complete the final review after returning home and submitting two to three days after the site visit, but no later than 10 days following the visit.

FEEDBACK REPORT, CONT.

The feedback report should not indicate an OFI if evidence of meeting a standard or criteria was provided on site even though not covered in the self-study. It is an OFI if the business unit does not provide evidence that the standard or criteria was met.

Example: While the business unit did not provide information about Standard Four, Criteria 4.4 in the self-study, some evidence of improvements were provided on site for accounting, but no improvements were observed in the management discipline.

FEEDBACK REPORT CONTINUED: THINGS TO REMEMBER:Comments are based on the requirements of the standards and criteria, not personal opinions or how processes are followed at other institutions.

Comments should be clear and concise and should address the situation, how the business unit does (for strengths) or does not meet (for OFI’s) the standard, and explain the significance of the situation.

Comments should not be prescriptive unless you are using language directly from the standards and criteria.

Comments should be written in complete sentences and should use proper grammar and punctuation.

Avoid the use of slang or local expressions others may not be familiar with, especially when on international visits. Example: “pig in a poke”.

FEEDBACK REPORTS - STRENGTHS:

If the institution has a strong strategic plan that includes both short and long-term goals andhas a good approach,has been deployed or implemented, demonstrates learning, and identifies improvements, your comment could state:

“The John Doe School of Business has a strategic plan that has identified both short and long-term goals. The plan is systematic with an approach that has been implemented for all disciplines, provides examples of learning and evidence of improvements the school has made. The strategic plan helps the school to plan for increasing enrollment and the replacement of faculty members as they retire.”

WRITING A STRENGTH:

If the institution has an assessment instrument and provides evidence of a systematic process and examples of how the results are used to make improvements, you could provide a comment such as: “The John Doe School of Business has administered the

MFT test for all disciplines for five years. Each department documents their review of the pre and post results each semester during department meetings and makes improvements based on the results. For example, the accounting department added additional projects for preparing financial statements in the ACCT 3300 course.”

FEEDBACK REPORTS - OPPORTUNITY FOR IMPROVEMENTS (OFI’S):

Suppose the institution has identified: - increasing enrollment and a large number of

faculty scheduled for retirement as strategic challenges.

- strategic objectives in the strategic plan are for one to three years, and faculty are not consulted in the development of the objectives.

The opportunity for improvement could read, “While the John Doe School of business has strategic objectives for up to three years, there was no evidence of planning beyond three years or involvement by faculty in the development of the objectives. Developing long term goals and involving more stakeholders such as faculty could provide resource planning to meet the increasing enrollment and replacement of faculty members scheduled to retire in the next five years.

WRITING AN OPPORTUNITY FOR IMPROVEMENT:Suppose the institution administers the Major Field Test (MFT) to all students as a pre and post test. There is no evidence, either in the self-study or

on-site, that the results are reviewed to indicate that there is learning or subsequent improvements are made. In addition, the school does not use the data for comparative purposes

Your opportunity for improvement comment could read:“The John Doe School of Business provided three years of data for the Major Field Test (MFT). However, there is no evidence that the school is using this data to measure how their students are performing in each discipline and how the results compare to other institutions with a similar mission. Analyzing the results to identify program strengths and weaknesses could allow the school of business to make improvements where needed and provide an external source of comparative data.”

FEEDBACK REPORT, CONT.

Make sure your comments are written so that the business unit understands why an OFI was provided.

There is a big difference (conditional accreditation v deferral) between providing an OFI because the business unit does not have results from an assessment and having results, but not using the results to make improvements.

FEEDBACK REPORT, CONT.

Once all team members have reviewed the feedback report in the entirety and it has been edited with a spell-check and checked for grammar,

Email the report as a Word document to:

reports@acbsp.org. Please request a read receipt so you know it was received. We do not need a hard copy.

SUBMISSION OF EXPENSES, ETC.

Submit expense form and all receipts as soon as possible following the site visit. This can be sent to Mary Riley via email at maryriley@acbsp.org or fax at 913-339-6226.

Payments are processed on the 15th & 30th each month.

Submit evaluator agreement forms and evaluation forms via email at reports@acbsp.org or fax to 913-339-6226.

BEST PRACTICES – WHAT DO YOU DO? Please share practices you have

found to be helpful on site visits.

WRAP-UP

Final Questions

Thank you for giving your time and helping ACBSP continuously improve our processes.