ABET Accreditation Criterion 4: Continuous Improvement Direct Assessment of Learning Outcomes Dr....

ABET AccreditationCriterion 4: Continuous Improvement

Direct Assessment of Learning Outcomes

Dr. Abdel-Rahman Al-QawasmiAssociate Professor

EE Department

11/10/2015 Criterion 4: Improvement Process

Quality Vice-Deanship in the college of EngineeringMajmaah University

ABET?• ABET, incorporated as the Accreditation Board for

Engineering and Technology.• ABET is the recognized U.S. accreditor of college and

university programs in applied science, computing, engineering, and technology


Goal of ABET

• To promote Continuous Quality Improvement in Applied Sciences,

Computing, Engineering, and Technology education through faculty guidance and

initiative


ABET Criteria


Students(1)

Facilities )7( Institutional Support )8(Curriculum )5( Faculty )6(

Criteria for Accrediting Engineering Programs

• Program Educational Objectives • Student Outcomes

• Assessment • Evaluation


Continuous Improvement )4(


Important :Important :Should document your processes for regularly assessing and evaluating the extent to which the student outcomes are being attained

[Exams, student portfolios, senior project presentations]

Assessment: is defined as one or more processes that identify, collect, and prepare the data necessary for evaluation

Evaluation:is defined as one or more processes for interpreting the data acquired though the assessment processes in order to determine how well the student outcomes are being attained

11/10/2015 Criterion 4: Improvement ProcessAssessment for Quality AssuranceAssessment for Quality Assurance©©

LearningLearning OutcomesOutcomes

ConstituentsConstituents

Assessment: Assessment: Collection, Analysis Collection, Analysis

of Evidenceof Evidence

Evaluation:Evaluation:Interpretation Interpretation

of Evidenceof Evidence

Feedback Feedback for for

Continuous Continuous ImprovemeImproveme

ntnt

MeasurablMeasurable e

PerformanPerformance Criteriace CriteriaEducational Educational

Practices/StrategPractices/Strategiesies

MissionMissionEducationaEducational l

ObjectivesObjectives

Assess/Evaluate


ABET Terms Definition

Objectives

Broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve.

Outcomes Statements that describe what students are expected to know and able to do by the time of graduation.

Performance Criteria Specific, measurable statements identifying the performance(s) required to meet the outcome; confirmable through evidence.

AssessmentProcesses that identify, collect, use and prepare data that can be used to evaluate achievement.

EvaluationProcess of reviewing the results of data collection and analysis and making a determination of the value of findings and action to be taken.

Step-by-Step Assessment


Choose your one of the Assigned to Choose your one of the Assigned to your course ABET student your course ABET student

OutcomesOutcomes

Students outcomes are assigned to your course and

Approved

Choose the assessment method Exams, Quiz, Project

Analyze the results of assessment KPIs and Rubrics

Final Result is below the Target for

Performance

Yes

recommendation and action plan

The internal Target for Performance is 75%

Your comments and observations

ABET student Outcomesa-k

• (a) an ability to apply knowledge of mathematics, science, and engineering • (b) an ability to design and conduct experiments, as well as to analyze and interpret

data • (c) an ability to design a system, component, or process to meet desired needs within

realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

• (d) an ability to function on multidisciplinary teams • (e) an ability to identify, formulate, and solve engineering problems• (f) an understanding of professional and ethical responsibility• (g) an ability to communicate effectively • (h) the broad education necessary to understand the impact of engineering solutions in

a global, economic, environmental, and societal context • (i) a recognition of the need for, and an ability to engage in life-long learning • (j) a knowledge of contemporary issues • (k) an ability to use the techniques, skills, and modern engineering tools necessary for

engineering practice.



Data do not assist in driving Data do not assist in driving improvement. To drive any improvement. To drive any

improvement, one needs information - improvement, one needs information - a detailed presentation of the results. a detailed presentation of the results.

SLO, KPIs and Rubrics• ABET learning outcomes, Key performance

Indicators (KPIs) and Rubrics are ready and approved for every program in the college of Engineering.

• Refer to the program coordinator to get all information about your course

• All instructors should be involved in the evaluation process in order to evaluate the program


Information for Driving Improvement


Learning Outcome Assessment

Area1>B Area 2<B Area 3>B Area 5>BArea 4>B

Sub area 1>B Sub area 2<B Sub area 3>B Sub area 4<B

The recommended actions for improvement for the student outcome are related to Sub area 2 and Sub area 4


What is a rubric, anyway?????

• A rubric is a set of categories which define and describe the important components of the work being completed, critiqued, or assessed.

• Each category contains a gradation of levels of completion or competence with a score assigned to each level and a clear description of what performance need to be met to attain the score at each level.

The recommended actions for improvement

• Actions Related to the Course Objectives• Actions Related to Student Outcomes• Actions Related to teaching Strategies• Actions Related to Assessment Strategies• Actions Related to Other aspects


The recommended actions should be written by the end of semester in the course Report


Assessment Methods

• Written surveys and questionnaires

• Exit and other interviews

• Standardized exams• Locally developed

exams• Archival records• Focus groups

• Portfolios• Simulations• Performance

Appraisal• External

examiner• Oral exams• Behavioral

observations


Direct Measures

Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning outcomes


Indirect Measures

Indirect measures of student learning that ascertain the opinion or self-report of the extent or value of learning experiences


Direct Indirect• Exit and other interviews• Standardized exams• Locally developed exams• Portfolios• Simulations• Performance Appraisal• External examiner• Oral exams• Behavioral observations

• Written surveys and questionnaires

• Exit and other interviews

• Archival records• Focus groups

Student Learning Outcomes (code) Assessment Methods

a1 Reports, discussions and presentationsa2 Exams and presentationsb1 Standardized exams, Oral exams, Micro projectsb2 Reports and presentationsb3 Standardized exams, Oral exams, Micro projectsb4 Standardized exams, Oral exams, Micro projectsb5 Behavior observation and reportsc1 Behavior observation and presentationsc2 Discussionsc3 Reports, discussions and presentationsd1 Standardized exams, Oral exams, Micro projectsd2 Reports, discussions and presentationsd3 Exams, quizzes and reportsd4 Standardized exams, Oral exams, Micro projects


LO:An ability to design and conduct experiments, as well as to analyze and interpret data


Example:


Question1: Experiment Lab )measuring current(SLO: An ability to design and conduct experiments, as well as to analyze and interpret dataMaximum grade :15Number of students: 14No Grades No Grades

1 7 8 5

2 8 9 6

3 10 10 7

4 3 11 1

5 11 12 14

6 12 13 12

7 8 14 15

Rubrics

RecommendationsWrite your recommendations based on your observations and the result of your used assessment method

KPI )2(:Experimental plan of data gathering

Unsatisfactory Range)0-9(

)1(No systematic plan of data gathering;

experimental data collection is

disorganized, even random,

and incomplete

DevelopingRange)9-12(

)2(Develops a simplistic experimental plan of data gathering, does not recognize entire scope of study (e.g. not all parameters

affecting the results are investigated)

SatisfactoryRange )12-15(

)3(Formulates an

experimental plan of data gathering to attain

a stated objective (develop correlation,

test a model, ascertain performance of equipment, etc.)

8 4 2

Final Result)1X8 + 4X2 + 2X3(/)3X14( =57.1% )weight(

Or )4+2(/14= 42% )Student percentage(

Results AnalysisStrength points: Strength points: 1- The students average performance is good.2- The plan of the experimental is well defined. Weaknesses points: 1- There is an average problem in applying the procedure.2- There is a problem in Identifying and collecting data in organizing wayRecommended actions: 1- Data collection methods and Statistical analysis should be give more attention2- Including a statistical analysis and data collection method in the course description or in other low level course.


Make it Simple!


Learning Outcome

Assessment Method Question

(a) First Exam Question 1

(e) Second Exam Question 3

(h) Mini-Project Presentation

(h) and (k) Final Exam Question 3 and Question 5

Your Plan for outcomes assessment:

Course Report• The required course Report by ABET at the

end of the semester is simple.• Collect all results from all assessment

methods you used during the semester to fill your report for every learning outcome assigned and approved for your course


ABET LO a b c d e f j h i j k

EE XXX √ √ √ √

(%) 77 67 80 76

EE program SOs


CourseSLO

(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (EE1) (EE2) (EE3) (EE4)EE 101 X X XEE 111 X X X XEE 202 X X X X XEE 205 X X X EE 206 X X X X X EE 207 X X X X EE 208 X X X X XEE 212 X X X X EE 221 X X X X XEE 234 X X X XEE 270 X X X XEE 271 X X X X EE 288 X X X XEE 307 X X X XEE 308 X X X X EE 322 X X X XEE 323 X X X X EE 341 X X X X XEE 360 X X X X XEE 361 X X X

Table 4-1-a: Student outcomes mapping for general courses

Automation of Assessment Process


Thank you for Listening


ABET Accreditation Criterion 4: Continuous Improvement Direct Assessment of Learning Outcomes Dr....

Documents

Transcript of ABET Accreditation Criterion 4: Continuous Improvement Direct Assessment of Learning Outcomes Dr....