Company

LOGO

Understanding the Evaluator’s Responsibilities

Understanding the Evaluator’s Responsibilities

Karen Froslid JonesDirector, Office of Institutional Research and Assessment

American University

OverviewOverview

1. Accreditation and the Visit1. Accreditation and the Visit

2. The Role of the Chair and Evaluators2. The Role of the Chair and Evaluators

3. Preparing for the Visit3. Preparing for the Visit

4. What to Expect During the Visit4. What to Expect During the Visit

6. Keys to a Successful Evaluation6. Keys to a Successful Evaluation

5. What Happens After you Leave5. What Happens After you Leave

Participants in the Accreditation Process

Participants in the Accreditation Process

Commission

InstitutionMiddle States

Rep

TeamChair

Visiting Team

Public

Self-Study CommitteePresidentFaculty, administratorsStaff and Students

Putting The Visit in Context:The Road to Re-AccreditationPutting The Visit in Context:

The Road to Re-Accreditation

Before VisitBefore Visit During VisitDuring Visit After VisitAfter Visit

•Self-Study Institute•Chair and Steering Committee Selected•Self-Study Design Submitted•Visiting Team selected•Self-Study Completed•Visit to campus by MS Rep•Logistics decided•Materials Sent to Team•Additional documents requested•Travel info communicated.

2004-2007•Team meetings•Reception/Dinner• Meetings with Steering Committee, etc.•Interviews•Writing time•Report written•Recommendation on re-accreditation•Oral Report

•Chair sends report to institution•Institution corrects mistakes of fact•Chair sends final report•Institution sends response•Chair attends commission meeting•Commission acts on accreditation•Institution – and team- is notified of commission action.

Purpose of VisitPurpose of Visit

• The importance of peer review.

• The value of meeting face-to-face.

• The opportunity for institution to benefit from other’s expertise.

• The need for objective assessment.

The Value of an EvaluatorThe Value of an Evaluator• Colleague

– Similar institution?– Similar mission?– Appreciation of the challenges facing institutions

• Expertise– Particular Area(s)– Characteristics of Excellence/Middle States

Expectations• Provide Outside Perspective

– Not tied to politics/personalities/history of institution– Able to look at ‘big picture’ and put it into context– No conflict of interest

The Role of the ChairThe Role of the Chair

• Acts as conduit between visiting team and institution.– Visits Institution before team visit– Gets early copy of Self-Study– Represents team when planning logistics– Resolve problems/conflicts between team and

institution– Main communicator between team and

institution during visit– Represents team at Commission meeting

The Role of the Chair (cont.)The Role of the Chair (cont.)

• Organizes and Leads Work of Visiting Team– Sets team member assignments (primary and

secondary)– Sets up some meetings with campus groups in

advance (board of trustees, steering committee, etc.)– Sets report format– Sets work agenda/deadlines

• Is draft required?• When will team meet?• When is ‘final’ report due to chair?

– Leads discussions– Leads/Organizes Exit Interview

The Role of the EvaluatorThe Role of the Evaluator

• Assigned one or two primary areas (usually connected to the Characteristics of Excellence) to review.

• Responsible for reading the self-study, conducting interviews, and reviewing documents to determine the degree to which Characteristics are being meet.

• Responsible for writing portion of final report.• Responsible for contributing to overall report.

Before the Visit: What to do NowBefore the Visit: What to do Now

• Know Characteristics of Excellence– Be prepared on standards you consider your areas of

expertise– Know all standards thoroughly – Look for ways that standards interact

• Familiarize yourself with self-study options – be sure

you understand the type of self-study your institution is using

• Know the Report Process/Possible Actions• Begin to familiarize yourself with institution

– Check out website

Before the Visit: What to do When you Receive the Report

Before the Visit: What to do When you Receive the Report

• Read entire report several times at least a few weeks before visit.

• Clarify with chair your expected areas of coverage.

• Decide on ‘must’ interviews – inform chair

• Any documents you need right away? Let chair know.

• Provide institution/chair with travel plans.

Arriving on CampusArriving on Campus

• Accommodations– Usually stay at same location, close to

campus– Leave plenty of time to get checked in/settled

before first meeting.

• Transportation

• Meeting Room(s)- Usually a work room/ meeting room set up in

central location

Arriving:The Atmosphere on Campus

Arriving:The Atmosphere on Campus

• Keep in mind that this is a BIG deal– Welcoming– Nervous– Worried– Excited– Agendas– Protective– Defensive– Helpful

The Visit BeginsThe Visit Begins

• 1st Team Meeting– Discuss logistics/assignments/report format.– Come prepared to discuss initial impressions.– Have preliminary interview needs ready.

• Host’s Program– Chance to get to know individuals/’players’.– Convey interest/enthusiasm for visit.– Informal inquiries/ not hard line questions.– Be aware of time

Setting Up InterviewsSetting Up Interviews

• Read report (and other documents) with eye towards people you may want to meet.– Who seem to be a key players in the area you are covering?– Do certain names come up in the report?

• Don’t be afraid to ask ‘who would know about_____?’

• Prioritize and Coordinate

• Can ask interviewees if they can suggest others that may be worth meeting.

Conducting InterviewsConducting Interviews• Have a clear objective going in.

– What needs clarification?– What is missing from self-study and needs to be answered?– What has happened since self-study was written?

• Be open to where the interview may go.• Don’t assume everyone you interview knows about accreditation.• Create a friendly, open conversation. Avoid overt judgments.• Don’t make promises/commitments.• Look for evidence of a culture of ‘continuous improvement’.

– Example:– What did you learn about your institution as a result of doing the self-

study? Did any findings surprise you?– How do you plan to implement the recommendations of the self-study?

• Ask questions that give you a sense of direction/commitment/focus

Conducting Interviews (cont.)Conducting Interviews (cont.)

• Keep very good notes.– Get cards, if possible, from interviewees.

• Consider meeting them on their ‘home turf’.

• Consider how combinations of individuals together may change dynamics of interviews.

Tips for Interactions with InstitutionTips for Interactions with Institution

• Be careful of off-hand remarks– Everything you say/do will be interpreted

• Be neutral/open

• Listen

• Observe

• Reserve judgments/opinions

• Smile

Using the Document RoomUsing the Document Room

• Make sure you build in time early to view documents.• Come prepared with a list of ‘must see’ documents.• Be open to looking through a wide range of

information.– Meeting notes, school newspaper stories, survey results, annual reports, etc.

• Put things back where you found them!

• Shred – or take home - sensitive notes. Don’t forget to erase info from borrowed computer!)

Writing the ReportWriting the Report

Informal Observation

Self-Study DocumentAnalysis

IndividualInterviews TEXTTeam

Members

Institution Mission

Characteristics

FindingsSignificant Accomplishments

Suggestions, Recommendations, Requirements

Writing the Report*Writing the Report*DO• Speak directly to the accreditation standards and institution mission• Respond to/recognize the approach used in the self-study• Concentrate on the fundamentals• Cover all areas of activity, including off-cite, certificate programs, etc.• Provide rationale and evidence for the team’s recommendations

– (model effective institutional assessment)• Make clear distinctions between suggestions, recommendations and

requirements.• Emphasize the importance of outcomes and assessment of both overall

institutional effectiveness and student learning.DON’T• Be highly prescriptive.• Name individuals.• Cite specific formulas, solutions, educational theories, etc.• Reveal accreditation recommendation.

* See pp.46-49, Team Visits

Writing TipsWriting Tips

• Clarify which ‘voice’ you will use as a team.

• Begin writing early – you can always revise.

• Keep track of who you talked to/when

• Keep track of sources of factual information (keep citations)

• Integrate report as much as possible.

Team InteractionsTeam Interactions

• A friendly, collegial atmosphere

• “Where you stand depends on where you sit.”

• Ask questions, seek input from others.

• Offer insight from your perspective.

• Disagreements likely but usually resolvable.

Completing the VisitCompleting the Visit

• The final draft– Be open to last minute changes

• The oral report

• Leaving the institution

After the Visit: What Happens Next?After the Visit: What Happens Next?

• Team Chair submits report to institution for review.• Institution corrects factual errors.• Team Chair submits final team report along with a

confidential memo to the Commission.• Institution sends team members final team report.• Institution can provide formal response to report.• Chair Attends commission meeting. Commission acts on

report.• Logistics: expense reports, evaluations, etc.

Miscellaneous Tips: (What you won’t find in any official publication)

Miscellaneous Tips: (What you won’t find in any official publication)

• Come to the visit rested

• Leave home institution at home

• Where comfortable/sensible clothing/shoes

• Alert the institution to any food allergies, special needs, etc.

• Bring a flash drive.

• Expect the unexpected – be prepared to go with the flow

• Don’t try to preach/perfect.

• Have fun!