Using Assessments for Program and Unit Review_Boyd_Colley

8/8/2019 Using Assessments for Program and Unit Review_Boyd_Colley

1/22

Using Assessments forProgram and Unit Review

Dr. Debra Colley, Niagara University

Dr. Kim Boyd, Oral Roberts University


2/22

How are assessments tied to theunits assessment system?

How are data from assessmentsused to inform program and unit

changes?


3/22

The unit has an assessment systemthat collects and analyzes data onapplicant qualifications, candidate and

Standard 2: Assessment System

and Unit Evaluation

graduate performance, and unitoperations to evaluate and improve theperformance of candidates, the unit, andits programs.


4/22

2a. Assessment System

Unacceptable Acceptable Target

The unit has not involved itsprofessional community in thedevelopment of its assessmentsystem. The units assessmentsystem is limited in its capacityto monitor candidateperformance, unit operations,and programs. Theassessment system does notreflect professional, state,and institutional standards.

The unit has an assessmentsystem that reflects theconceptual framework andprofessional and state standardsand is regularly evaluated by itsprofessional community. Theunits system includescomprehensive andintegrated assessment andevaluation measures to monitorcandidate performance and manage

The unit, with the involvement of itsprofessional community, isregularly evaluating the capacityand effectiveness of its assessmentsystem, which reflects the

conceptual framework andincorporates candidate proficienciesoutlined in professional and statestandards. The unit regularlyexamines the validity and utility ofthe data produced throughassessments and makes

and completion of programsare based on a single or fewassessments. The unithas not examined bias in itsassessments, nor made aneffort to establish fairness,accuracy, and consistency ofits assessment proceduresand unit operations.

and programs. Decisions aboutcandidate performance are basedon multiple assessments atadmission into programs,appropriate transition points, andprogram completion. The unit hastaken effective steps to eliminatebias in assessments and is workingto establish the fairness, accuracy,and consistency of its assessmentprocedures and unit operations.

changes in assessmenttechnology and in professionalstandards. Decisions aboutcandidate performance are based onmultiple assessments made atmultiple points before programcompletion and in practice aftercompletion of programs. Data show

a strong relationship of performanceassessments to candidate successthroughout their programs and laterin classrooms or schools. The unitconducts thorough studies toestablish fairness, accuracy, andconsistency of its assessmentprocedures and unit operations. Italso makes changes in itspractices consistent with theresults of these studies.


5/22


6/22

2c. Use of Data for Program

ImprovementUnacceptable Acceptable TargetThe unit makes limited or nouse of data collected, includingcandidate and graduateperformance information,to evaluate the efficacy ofits courses, programs, andclinical experiences. The unitfails to make changes in itscourses, programs, andclinical experiences whenevaluations indicate that

The unit regularly and systematicallyuses data, including candidateand graduate performanceinformation, to evaluate theefficacy of its courses, programs,and clinical experiences. The unitanalyzes program evaluation andperformance assessment data toinitiate changes in programs andunit operations. Faculty haveaccess to candidate assessment

The unit has fully developedevaluations and continuouslysearches for strongerrelationships in the evaluations,revising both the underlyingdata systems and analytictechniques as necessary. Theunit not only makes changesbased on the data, but alsosystematically studies theeffects of any changes to

strengthen candidatepreparation to meetprofessional, state, andinstitutional standards. Facultydo not have access tocandidate assessment dataand/or data systems.Candidates and faculty are notregularly provided formativefeedback based on the unitsperformance assessments.

.

Candidate assessment data areregularly shared with candidatesand faculty to help them reflect onand improve their performance andprograms.

strengthened without adverseconsequences. Candidates andfaculty review data on theirperformance regularly anddevelop plans for improvementbased on the data.


7/22

What evidence do you collect fromP-12 students that indicatecandidates have a positive effect

on student learning?

Teacher Work Samples

Standardized Achievement Tests

A state system that links student learning tocandidates and/or alumni


8/22

What assessments do youuse? Unit level

State reports Student Teaching Performance Evaluations

TWS

Cooperating Teacher Program Evaluations

rogram eve

Look at required assessments

Program specific Student Teaching Evaluations

Advanced level Internship Evaluations

Advanced candidate projects with rubrics

Follow-up Surveys

Exit Interviews

Alumni Surveys

Employer Surveys


9/22

What assessments do you use?

ProgramInstitution

Unit Community(P-12)


10/22

Key assessments

Institution and unit

Institutional research

NSSE

-

Program

State assessments

Dispositionalassessment

(alumni andemployers)

Tracking programcompleters

FQPD

Faculty evaluations

Field experience andclinical practice

Diversity assessment

Course embedded

assessment Comprehensive exams

or portfolio defense


11/22

The common ground

CandidatePerformance

UnitIndicators

Unit

Performance

Indicators

StudentLearning

Program

Quality


12/22

Alignment and use of

assessments

Core assessments

Delineation of components to be measured

Alignment of assessments

Clustering of unit and program assessments

Continuous improvement


13/22

NUs conceptual framework 4 major

components

1. Mission

2. Dispositions

3. Theoretical dimensions

4. Strategic goals


14/22

Alignment to assessment system

Each component of the conceptual framework

is aligned with the assessment system.

Data are clustered to measure our success and

n assessments.


15/22

Clustering unit and program

assessments (streams of data)

1. Program area data stream

2. Field experience and partnership data stream

3. Diversity data stream

4. Faculty qualifications, performance, anddevelopment data stream

5. Alumni and employer follow-up surveys

6. Advancing research data stream

7. National Survey of Student Engagement (NSSE)


16/22

Conceptual

Framework

Components

Program

Area

Field Experience and

Partnership

Diversity FQPD Alumni &Employer

Follow-Up

Advancing

Research

NSSE

Mission

1.1 Service (Vincentian) X X X

Dispositions

2.1 Responsibility X X

2.2 Relationships X X X

2.3 Critical thinking X X

Theoretical

Dimensions

3.1 Constructivism X X X3.2 Process-product X X X

3.3 Reflective practice X X X

Strategic Goals

4.1 Diversity X X X X

4.2 Faculty X X

4.3 High standards X X X X

4.4 Success grads. X X X

4.5 Partnerships X X


17/22

Linking assessments Timelines

Annual (When, where)

Transparency

What, how

Roles and responsibilities

Who

Meaningful information

Why


18/22

Meaningful information for

Planning, analysis, and advocacy

Measuring progress

Conceptual framework

Unit standards, strategic plan goals, federal andstate standards

Improving and refining assessment system

Assessments

Transition points and quality assurance


19/22

Meaningful information for

Ensuring success of graduates and impact on

student learning Alignment, currency, and rigor of curriculum

Field experiences and clinical placements

Enhancing faculty development

Teaching effectiveness, scholarship, service

Improving structures and operations

Expanding partnerships with P-12 and community


20/22

Culture of continuous improvement

Assessment is purposeful, well-planned (the

assessment system ).

Assessments are aligned to the conceptualframework.

Data from multiple sources is used for decision-making, planning, and continuous improvementat multiple levels (beyond accreditation).

It is the way to do business and it will make adifference.


21/22

Assessment and use of data have led

to (90% of faculty reporting):

96%

90%

95%

100%

81% 81%

70%

75%

80%

85%

Collaboration Structure and

process

Involvement and

leadership

Niagara University, College ofEducation (2009)


22/22

Questions and comments

Using Assessments for Program and Unit Review_Boyd_Colley

Documents

Transcript of Using Assessments for Program and Unit Review_Boyd_Colley