AssessmentCollege of EngineeringA Key for Accreditation

February 11, 2009

Why do Assessment• What are the students actually learning?

• How can we teach better?

• Are we teaching the right stuff according to our stakeholders?

• Does learning the material lead to successful alumni

Terms• Learning outcome – what the students can

do (or not) by the time that they finish the experience

• Program objective – what the students can do (or not) at the 3-5 year mark after graduation

• Metric – measurement that can be used to assess the attainment of either an outcome or an objective

ABET Criteria• Students - System to evaluate and monitor performance

• Objectives - System to determine objectives covering all constituents

• Outcomes - System to evaluate whether ABET educational Outcomes are met and directions for improvement

• Professional - System to ensure sufficient curriculum content

• Faculty - Competency, advising, service, practice

• Facilities - Classrooms, labs, modern tools, computing infrastructure

• Institutional Support - Funding for faculty, operations, capital

• Program Specific Criteria - Presented by professional societies for each discipline

Criterion 3 - Educational OutcomesStudents should have the ability to:

a. Apply mathematics, science, engineering principles

b. Design and conduct experiments and interpret data

c. Design a system, component, or process to meet desired nee

d. Function on multidisciplinary teams

e. Identify, formulate, and solve engineering problems

f. Understand professional and ethical responsibility

g. Communicate effectively

h. Understand the impact of engineering solutions in a global context

i. Recognize need and to engage in life-long learning

j. Know contemporary issues

k. Use the techniques, skills, and modern engineering tools necessary

for engineering practice

ABET – 4 questions • Do you have a process for determining program

objectives ?• Do you have a process for assessing attainment

of program objectives?• Do you have a process for determining learning

outcomes?• Do you have a process for assessing attainment

of learning outcomes and feeding back results to improve?

What ABET is looking for!!!• Objective process in place that provides for:

– Definition of desired measurable objectives with input from stakeholders

– Collection of data linked to objectives– Analysis of data and evaluation of results– Implementation of changes

• Output process in place that provides for:– Definition of desired measurable outcomes– Collection of data linked to outcomes– Analysis of data and evaluation of results– Implementation of changes

Objective Process• Who are your stakeholders?

• How do you measure 3-5 years after graduation?

• How do you implement a feedback process?– Frequency of change– Process for collecting and evaluating

measurements– Going back to stakeholders

Stakeholders

Example Objectives

Example Metrics

Iterative Improvement Process

Define Outcomes Measure

Outcomes

AnalyzePropose Program

Changes

Example OUTCOMESSIE 265 Engineering Management 1

265.1 Understanding of concepts of the time value of money and interest rates.

• 265.1.1 Ability to define the following different interest rates: discount rate, inflation rate, prime lending rate, federal funds rate, rate of return, MARR

• 265.1.2 Ability to compute the equivalent value of a cash flow series at any point in time including time 0 (present value) and time N (future value)

265.2 Ability to analyze cash flow series using decision criteria of present worth, annual equivalent, and internal rate of return.

• 265.2.1 Ability to compute PW(i) for a cash flow series

• 265.2.2 Ability to compute AE(i) for a cash flow series

• 265.2.3 Ability to compute IRR for a simple investment

• 265.2.4 Knowledge of the correct decision criteria for each of the above computations

• 265.2.5 Knowledge of the assumptions concerning the use of the above criteria

Metrics for Outcomes

• Exam Questions

• Design Report Evaluation

• Oral presentation Evaluation

• National Examination

• External Evaluation Reports

• Placement Evaluation (companies and graduate programs)

• Alumni metrics

Key Ideas• Surveys typically measure attitudes and

not outcomes. A student cannot self-evaluate knowledge in our systems

• Each Metric has a “pass standard” and we are looking to increase the number of students that meet the standard

• Multiple metrics for each outcome and you must set a standard for the entire outcome

Linking Metrics and Outcomes

Outcome Number of

measures

Standard Number Passed

Result

1.1 Formulate Model 10 7 7 passed 1.2 Appropriate Modeling Approach 7 5 7 passed 1.3 Feedback & Sensitivity Analysis 7 5 5 passed 2.1 System Design 3 2 3 passed 2.2 Conflicting Criteria 6 4 5 passed 2.3 Society and Environment 3 2 3 passed 3.1 Teaming 4 3 4 passed 3.2 Communication 5 4 5 passed 4.1 Develop Software 2 2 2 passed 4.2 Modern Tool Usage 4 3 4 passed 5.1 Professionalism 2 2 2 passed 5.2 Lifelong learning 3 2 2 passed 5.3 Ethics 2 2 1 failed