A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method...

18
A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to achieve its mission? X can equal program, service, department What evidence is available to support the determination?

Transcript of A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method...

Page 1: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

A Connoisseur, a Critic, and a Skeptic walk into a student union…

CAS provides a structured method of evaluation:

Is the “X” functioning effectively to achieve its mission?X can equal program, service, department

What evidence is available to support the determination?

Page 2: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

What is CAS?The mission of the Council for the

Advancement of Standards in Higher Education (CAS) is to promote the improvement of programs and services to enhance the quality of student learning and development.

CAS is a consortium of professional associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self-assessment.

Page 3: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Essential Elementsin the CAS ApproachCulture that values involvement of all its

membersHas identified quality indicatorsUse of standards and guidelinesWillingness/capacity:

To examine itself and assemble resultsTo report and use the results

Credibility, accountability, improvement:Program & service improvement; measures of quality

and effectiveness; measures of impact on learning

Page 4: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

PurposeIntroduce the CAS StandardsDevelop capacity to conduct CAS self-reviewRecognize value of Program & Learning

OutcomesProvide support and motivation as you continue

to progress toward becoming…

Mission Driven

Evidence-based

decision-making

Informed Practitioners

Culture of Evidence

Data drivenAble to demonstrate contributions to student learning

Page 5: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Assessment Pyramid SL

CAS

Accreditation

Page 6: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS Self-Assessment Process1. Establish and prepare the Self-

Assessment Teams2. Compile and evaluate evidence3. Determine how well Standards are met4. Where needed, determine appropriate

corrective action5. Recommend steps for program

enhancement6. Prepare an action plan

Page 7: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

The Self-Assessment TeamAppoint and Train the Team:Size of team ≈ 5Composition: Faculty, Staff, Students &

OthersLeader & admin. supportEstablish team ground rulesDiscuss meaning of each standardEstablish team’s inter-rater reliabilityEncourage team discussion; expect

disagreements; commit to consensual resolution

Page 8: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Team ActionsDecide whether to include guidelines or

other measures that go beyond the standardsGather and analyze quantitative and

qualitative dataIndividuals rate each and every criterion

measureObtain additional documentary evidence if

required to make an informed team decisionComplete the assessment, ratings & action

plan worksheets

Page 9: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Organization of CAS StandardsMissionProgramLeadershipHuman ResourcesEthicsLegal

ResponsibilitiesEquity and AccessDiversity

Organization and Management

Campus and External Relations

Financial ResourcesTechnologyFacilities and

EquipmentAssessment and

Evaluation

Page 10: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS: Standards and GuidelinesSTANDARD (BOLD TYPE; AUXILARY VERB MUST)

Financial Aid Programs (FAP) must have adequate, suitably located facilities and equipment to support its mission and goals efficiently and effectively. Facilities and equipment must be evaluated regularly and be in compliance with relevant federal, state, provincial, and local requirements to provide for access, health, safety, and security.

GUIDELINE (LIGHT-FACED TYPE; VERB SHOULD)

The program should have facilities or have access to: private office or space for confidential counseling,

interviewing, and other meetingsoffice, reception, and storage space and security sufficient to

accommodate assigned staff, supplies, equipment, library resources, and machinery

conference room or meeting space

Page 11: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Assessment Criteria ExamplePart 6: FINANCIAL RESOURCESFinancial Aid Programs (FAP) must have adequate funding to accomplish its mission and

goals. Funding priorities must be determined within the context of the stated mission, goals, objectives, and comprehensive analysis of the needs and capabilities of students and the availability of internal or external resources.

FAP must demonstrate fiscal responsibility and cost effectiveness consistent with institutional protocols.

PART 6. FINANCIAL RESOURCES (Criterion Measures) Rating ScaleNOTES

6.1 The program has adequate funding to accomplish its mission and goals. ND 1 2 3 4 NR

6.2 Funding priorities are determined within the context of program mission, student needs, and available fiscal resources. ND 1 2 3 4 NR

6.3 The program demonstrates fiscal responsibility and cost effectiveness consistent with institutional protocols. ND 1 2 3 4 NR

Part 6: Financial Resources Overview Questions

A. What is the funding strategy for the program?

B. What evidence exists to confirm fiscal responsibility and cost-effectiveness?

Page 12: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Criterion Measure Rating Scale

Using this scale, consider the evidence available and decide the extent to which each criterion measure has been met.

ND 1 2 3 4 NRNot Done Not met Minimally Well Fully Not Rated

Met Met Met

Page 13: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS: Work Form AAssessment, Ratings, and Significant Items

CASWork Form A

Assessment, Ratings, and Significant ItemsINSTRUCTIONS:This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.

Step One

Part Items Excellent Unsatisfactory

1. Mission 1.1a1.4

1.1b1.5

1.1c1.6

1.2 1.3

2. Program 2.1 2.4.3 2.4.8 2.4.132.6 2.8d2.11

2.2 2.4.4 2.4.9 2.4.142.7 2.8e2.12

2.32.4.52.4.102.4.152.8a2.8f2.13

2.4.12.4.62.4.112.4.162.8b2.9

2.4.22.4.72.4.122.52.8c2.10

Page 14: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS: Work Form A continuedStep Two: List item number(s) for each Part determined to merit follow-up and describe the practice weaknesses that require attention

1.

2.

3.

4.

5.

Page 15: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS Work Form B: Follow Up Actions

INSTRUCTIONS:The purpose of this work form is to begin the planning for action to be taken on the practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.

Step Three: Describe the current practice that requires change and actions to initiate the change

Practice Description

Corrective Action Sought

Task Assigned To Timeline Due Dates

Page 16: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

CAS Work Form C:Summary Action Plan

Step Four: This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.

Part 1: Mission

Part 2: Program

Part 3: Leadership

Part 4: Human Resources

Part 5: Ethics

Page 17: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

An iterative Cycle…1. Purpose & scope of study is defined2. Self-study team is created and trained3. Evaluative evidence is collected4. Determine how well Standards are met5. Appropriate actions are recognized6. Action plan is developed and approved7. Plan is communicated and enacted8. Repeat (i.e., step 1 = check if improvement

achieved)

Page 18: A Connoisseur, a Critic, and a Skeptic walk into a student union… CAS provides a structured method of evaluation: Is the “X” functioning effectively to.

Participant OutcomesIn order to be able to conduct a CAS self-review,

by the end of this session participants will be able to:Articulate the purpose and benefits of the reviewFeel increased self-efficacyIdentify institutional stakeholders needed for

review processIdentify and obtain appropriate sources of

data/evidenceUse data to evaluate compliance with standards &

guidelines Document and present the results of the review