Writing and Using Assessment Plans to Enhance Teaching,

Learning, & Curriculum

November 10, 2006

Today’s Objectives• Understand some reasons why we need to assess student learning

• Identify and discuss each component of an assessment plan

• Brainstorm about some ways to make assessment useful and meaningful

• Brainstorm about appropriate methods to use when assessing student learning

• Introduce ways that the assessment process can impact teaching, learning, curriculum, and other things important to us in the university community

External Pressure to Assess• Federal government pressures to

assess—NCLB Act, Spellings Commission Reports, etc.

• State pressures will not dissipate, given budget constraints

• Regional accreditors (for us, the Middle States Association) require it

• Local constituencies want it—they want to know how well universities and colleges are “adding value” to a student’s education

• The public increasingly expects it

• Even grant funders want it—assessment systems submitted with grant applications, assessment results included with grant reports, etc.

Example of Regional Accreditor’s Review:

Middle States Association Suggestion for Binghamton University

Despite External Pressures, There Are Real Advantages to Program Assessment

• Enables programs to answer external requests for information

• Empowers faculty, not bureaucracies to make decisions about curriculum, instruction, and learning

• Enhances grant application process

• Empowers faculty to reflect on student teaching and learning in a way that is non-threatening

• Enhances faculty’s ability to publish

• Improves student learning

Why An Assessment Plan?• Facilitates periodic, not episodic assessment of student learning

and program outcomes

• Serves as template for conversations about student learning

• Leads to improvement in student learning

• Enables faculty to play central role in managing student learning

• Facilitates information gathering for program review process

• Makes it easier to do something about what assessments have to say

Whose Needs Might We Consider When We Develop Assessment Plans?

University Community

Regulatory Community

Program Community

Speaking Different Languages:Assessment Communities In Practice

•Mission Statement

•General EducationObjectives

•Constituent Expectations and Norms

•Faculty Expectations

•Federal Guidelines

•State Guidelines

•North-Central Association Guidelines

•Public Expectations and Norms

•Dispositional Expectations

•Professional Standards

•Advisory Board•Expectations

•Professional Organization Norms and Expectations

•Employer Expectations

Assessment Plan

Reflect on some of the different audiences that might be interested in or that might require assessment

information about student learning. What are some of those audiences?

University Community:

Example: Faculty

Regulatory Community:

Example: Federal Government

Program (or Discipline) Community:

Example: National Council on Accrediting Teacher Education (program accreditor)

Faculty Involvement Is Crucial

• The focus of assessment is not measurement, but how assessment information is used by faculty

• Only faculty are qualified to make decisions about how to use assessment information for enhancing teaching, learning, curriculum, etc.

• This is an opportunity, not an obstacle

Assessment As A Faculty-Based Process

Faculty

Defined Student Learning Objectives

Meaningful Measurement(Qualitative, Quantitative,

Triangulated)

Focused Reflection/Discussion

Evidence-Based Program Enhancements—

(Curriculum, Learning, Teaching, etc.)

Components of An Assessment Plan

• Student Learning Objectives

• Assessments

• Plan for Faculty Discussion About Assessments and How they Lead to Recommendations to Further Enhance Student Learning Objectives

• Plan to Implement Recommendations in Respect to Curriculum, Instruction, etc.

Identifying Student Learning Objectives

Defining Learning Objectives

• Knowledge—what do we want students to know when they graduate (i.e., content knowledge, etc.)

• Skills—ability to perform specific tasks, think in certain ways, etc.; what should a graduating student be able to do?

• Competencies—ability to perform specific tasks “in real time,” or “authentically” [knowledge + skills competencies]; also, what are some values, attitudes, behaviors we feel are important for graduates to have?

Listing Student Learning Outcomes By Knowledge, Skill, and Competency

1. Think about some of the knowledge, skills, and competencies, you would like a student from your program to graduate with

2. List them

3. Discuss with a partner at your table.

Name of Program: What Are Some Things You Want Them to Know?

What Are Some Things You Want Them To Do?

What Are Some Ways You Would Like Them To Use Their Newly-Acquired Knowledge and Skills?

Assigning Relevant Assessments

Selecting Appropriate and Meaningful Assessments

• At least one should be a “direct assessment” of student learning, meaning that they should involve observations of actual student performance

• Indirect assessments —those that include student opinions about a programs’ ability to deliver on the student learning objectives, etc. can also be very helpful

• Using a combination of these—two or three—might provide meaningful information; “triangulation” is increasingly being required by evaluators

• It is rarely a surprise to find out that departments and programs initially feel that they never directly assess student learning, only to find out after discussion that they have been doing so, only informally

“Not everything that can be counted counts, and not

everything that counts can be counted.”

-Albert Einstein

“Data is the plural of anecdote”

-Ronald Coase

Helpful Tips• The process of selecting appropriate assessments is usually dependent on a number

of factors —experience in process, resource limitations, etc.

• While the preference is to move toward a comprehensive assessment system, reality suggests that the development of an assessment system is very much developmental; it takes time to develop such a system.

• Although the prior point is important to keep in mind, most program and regional accreditation organizations are beginning to expect 3-5 years of periodically collected assessment information

• Therefore, begin incrementally. Start with assessments that create a balance between need to conserve resources and need to maximize the meaning gleaned from the process

• The most important question to ask at this point: “How will this information provide faculty with legitimate information that will affect learning, teaching, and curriculum?”

• A “shameless plug:” Ask the Assistant Provost for Curriculum, Instruction, & Assessment for assistance!

University Community

Regulatory Community

Disciplinary Standards Community

Outcomes Assessments from the Perspective of Different Communities

Examples:Student Satisfaction Surveys

Portfolio Assessments

Graduate Record Exam (GRE) Math Scores

Focus Group Interviews

Unstructured Student Interviews

Examples:

State Licensure Exams(Passage Rates)

General Education Syllabi Reviews

Examples:

Capstone Course Case Study Reviews

Advisory Board Input

GRE Subject Exam Scores

Assessment Plan

Encouraging Faculty Discussion

Defining When Faculty Will Discuss Information

• Faculty (or an assessment committee) should be given information in advance to consider

• Focus of meeting is to discuss what assessments say about student learning objectives and any recommendations that might stem from such discussion; it should not be a “rubber stamping” of findings or solely an opportunity to complain—instead, the focus should be upon recommendations

• It is important to focus on communicating results to faculty, and tracking ways that information is used

Tip on Producing An Effective Meeting:

Try to Put Assessment Results on One Page, If Possible

 

Assessment: Graduate Alumni Surveys Assessment: Graduate Employer Surveys

Assessment: Expert Panel

NBPTS 1: Teachers Are Committed to Students and Their Learning

 

1.1: Teachers recognize individual differences in their students and adjust their practice accordingly

q8: "Using a variety of methods, strategies, and materials to promote development, learning and cooperation" (3.29--satisfactory);

q5: "Using a variety of methods, strategies, and materials to promote development, learning and cooperation" (4.14--good);

Graduate students know about a wide range of research-based pedagogies--expert panel average was 3.7 (satisfactory/well prepared)

1.2: Teachers have an understanding of how students develop and learn

q4: "Using knowledge of child/adolescent development and relationships with students and families to plan instruction based on abilities, interests and needs" (3.14--satisfactory)

q1: "Using knowledge of child/adolescent development and relationships with students and families to plan instruction based on abilities, interests and needs" (4--well prepared)

Graduate students know how to teach based upon what is developmentally appropriate--expert panel average was 3.8 (satisfactory/well prepared)

1.3: Teachers treat students equitably

q6: "Establishing caring, inclusive, stimulating and safe environments that facilitate learning, collaboration, independence, and intellectual risk taking" (3.0--satisfactory)

q3: "Establishing caring, inclusive, stimulating and safe environments that facilitate learning, collaboration, independence, and intellectual risk taking" (3.0--satisfactory)

Graduate students have developed a recognition of professional ethics of their discipline--expert panel average was 4 (well prepared)

1.4: Teachers' mission extends beyond developing the cognitive capacity of their students

q19: "Helping students explore important life issues, and world issues" (2.57--satisfactory)

q16: "Helping students explore important life issues, and world issues" (3.43--satisfactory)

Graduate students appreciate how knowledge is linked to other disciplines--expert panel average was 3.4 (satisfactory).

Implementing Faculty Recommendations, and Affecting

Learning and Teaching

Ideas for Using Assessment • Use information to pursue department or program-level initiatives or projects

on teaching

• Incorporate assessment information into curriculum discussions and recommendations

• Have periodic discussions or reflective discussions on teaching using assessment information

• Use for grant applications

• Use in budget requests

• Use in fundraising letters or alumni functions

• Use to gather further information about student learning from instructors on a periodic basis

Ways to Track Effect of Faculty Discussions/Recommendations

• Encourage inclusion in annual report (section under teaching effectiveness)

• Encourage submissions of faculty narratives —how were faculty discussions and recommendations used in course design, selection of courses, etc.?

• Track how recommendations were enacted —curriculum process, department/program initiatives, course sequencing, equipment requests, etc.

• Not so important to make an academic study out of this, or to make this too huge of a process, but documentation is helpful when writing annual reports on assessment, curriculum, & instruction

Future Directions and Vision

• Will ask for report on four questions at end of each academic year

• Focus is not upon “reporting for reporting’s sake,” but instead upon how we can assist

• Focus will be upon serving faculty—in helping them in accomplishing the last two of the four core questions

• Focus primarily upon impact—how has process impacted learning, teaching, and curriculum processes

Conclusion

• Four foci:

– Objectives

– Assessments

– Faculty Usage

– Impact

All assessment is a perpetual work in progress. -- Linda Suske,

Unknown , May 3, 2005

In assessment, "the perfect is the enemy of the good." Let's keep

striving for the good. -- Tom Angelo, Unknown , Unknown

Questions? Comments?