The Use of Assessment Information: Root Cause Analysis

Carlo Magno, PhDAsian Psychological Services and Assessment

Answer the following Questions:

What information do I need in this session? (limit to two answers)

What specific school practices do I need to discuss in this session? (Limit to two answers)

What do we need to develop among our students?

Learning and innovation skills

Creativity and InnovationCritical Thinking and Problem SolvingCommunication and Collaboration

Information, media, and technology skills

Information LiteracyMedia LiteracyICT (Information, Communications and Technology) Literacy

Life and career skillsFlexibility and AdaptabilityInitiative and Self-DirectionSocial and Cross-Cultural SkillsProductivity and AccountabilityLeadership and Responsibility

21st century skills

How do you get evidence that your students develop 21st century skills?

21st century

skills

AssessmentInternational Level Assessment

National Level

AssessmentRegional/District

Level

AssessmentClassroom Level

Principles of RCA

The primary aim of root cause analysis is to identify the factors that resulted in the harmful outcomes (consequences) of one or more past events in order to identify what behaviors, actions, inactions, or conditions need to be changed to prevent recurrence of similar harmful outcomes and to identify the lessons to be learned to promote the achievement of better consequences ("Success" is defined as the near-certain prevention of recurrence).

To be effective, root cause analysis must be performed systematically, usually as part of an investigation, with conclusions and root causes that are identified backed up by documented evidence.

Principles of RCA

There may be more than one root cause for an event or a problem, the difficult part is demonstrating the persistence and sustaining the effort required to determine them.

The purpose of identifying all solutions to a problem is to prevent recurrence at lowest cost in the simplest way. If there are alternatives that are equally effective, then the simplest approach is preferred.

Root causes identified depend on the way in which the problem or event is defined. Effective problem statements and event descriptions (as failures, for example) are helpful, or even required.

Principles of RCA

To be effective, the analysis should establish a sequence of events or timeline to understand the relationships between contributory (causal) factors, root cause(s), and the defined problem/ event to prevent in the future.

Root cause analysis can help transform a reactive culture (that reacts to problems) into a forward-looking culture that solves problems before they occur or escalate. More importantly, it reduces the frequency of problems occurring over time within the environment where the root cause analysis process is used.

RCA procedure

RCA-based corrective action1. Define the problem or describe the event

factually. Include the qualitative and quantitative attributes (properties) of the harmful outcomes. This usually includes specifying the natures, the magnitudes, the locations, and the timing of events.

2. Gather data and evidence, classifying it along a timeline of events to the final failure or crisis. For every behavior, condition, action, and inaction specify in the "timeline" what should have been done when it differs from what was done.

Problem EvidenceLow student achievement of HS students in Mathematics for SY 2013-2014

0% of the students are highly proficient fro grades 7 and 8. Majority of the students are progressing towards the standards.

Interpretation: MathGrade 7 Section Paul

Grade 8 Section Paul

What to report Achievement gains per school/locale Achievement gains per subject

area Achievement gains per level Trends: Comparison across school

years Trends: Comparison with other

countries/states/region Red: School Level

Test Information

Proficiency levels in English, mathematics, and Science

Comparison of the class performance with the school, country, and standards.

Proficiency level for each skill

Areas: EnglishEnglish 1 Percent correctAssociating Picture to test Dividing word into syllables Associating text to pictures Dividing words into syllables Noting details Meaning of words Rules of punctuation synonym of a word Cause and effect relationship Summarizing the s try Classifying nouns Plural form of words Contracting pronouns and helping verbs Identifying compound words Arranging word to a sensible sentence Comparative degrees of adjectives Use of apostrophe Substituting pronouns to nouns Riles in capitalization Identifying antonyms of words

Areas: Math

Math 2 Percent correctAlgebra: Sorting Data analysis and Probability: Graphs Data analysis and Probability: Use data Geometry: 2D Geometry: 3D Measurement: Capacity Measurement: Elapsed time Measurement: weight Number operations: Addition Number operations: Fractional part Number operations: Fractions Number operations: Place value Number operations: Rounding off Number operations: Standard form Number operations: Subtraction Number operations: Whole numbers Number oprations: Number fact Number oprations: Odd number Number operations: Estimation Number operations: Money Number operations: Roman numerals

RCA-based corrective action

3. Ask "why" and identify the causes associated with each step in the sequence towards the defined problem or event. "Why" is taken to mean "What were the factors that directly resulted in the effect?"

4. Classify causes into causal factors that relate to an event in the sequence and root causes, that if eliminated, can be agreed to have interrupted that step of the sequence chain.

Problem Evidence Factors

Low student achievement of HS students in Mathematics for SY 2013-2014

0% of the students are highly proficient fro grades 7 and 8. Majority of the students are progressing towards the standards.

Curriculum•Several competencies

Instruction•Fast•Lack time for practice

Causes

Physical causes – Tangible, material items failed in some way (for example, books were destroyed due to flood).

Human causes – People did something wrong, or did not do something that was needed. Human causes typically lead to physical causes (for example, no one was assigned to observe the teacher).

Organizational causes – A system, process, or policy that people use to make decisions or do their work is faulty (for example, no one was given training and updates on teaching the new curriculum).

Factors

Jacobs & Hales-A guide to curriculum mapping Content Assessment Skills/thinking process Standards Objectives Instructional practices Resources

Factors

McKinsey 7S Framework 

Strategy Structure Systems Shared values Skills Style Staff

4Ps of Marketing 

Product Place Price Promotion

RCA-based corrective action

5. Identify all other harmful factors that have equal or better claim to be called "root causes." If there are multiple root causes, which is often the case, reveal those clearly for later optimum selection.

6. Identify corrective action(s) that will with certainty prevent recurrence of each harmful effect. Check that each corrective action would, if pre-implemented before the event, have reduced or prevented specific harmful effects.

RCA-based corrective action

6. Identify solutions that, when effective, and with consensus agreement of the group, prevent recurrence with reasonable certainty, are within the institution's control, meet its goals and objectives and do not cause or introduce other new, unforeseen problems.

7. Implement the recommended root cause correction(s).

8. Ensure effectiveness by observing the implemented recommendation solutions.

Problem Evidence Factors Strategy

Low student achievement of HS students in Mathematics for SY 2013-2014

0% of the students are highly proficient fro grades 7 and 8. Majority of the students are progressing towards the standards.

Curriculum•Several competencies

Instruction•Fast•Lack time for practice

Corrective action•Convert extra-curricular time to mathematics review time

Solution•Decongest the competencies

Direction of solutions

Development of academic programs and special programs

Further improvement of the curriculum Identifies students that needs further

help Reflection on how to teach or deliver

the curriculum better Allocation of resources and priorities Informs what is happening in the

schools

RCA-based corrective action

9. Identify other methodologies for problem solving and problem avoidance that may be useful.

10.Identify and address the other instances of each harmful outcome and harmful factor.

Problem Evidence Factors Strategy AlternativeStrategies

Low student achievement of HS students in Mathematics for SY 2013-2014

0% of the students are highly proficient fro grades 7 and 8. Majority of the students are progressing towards the standards.

Curriculum•Several competencies

Instruction•Fast•Lack time for practice

Corrective action•Convert extra-curricular time to mathematics review time

Solution•Decongest the competencies

•Facilitate curriculum mapping

•Review of DepEd Standards

Input for management planning

Integrate the evidences and solutions in strategic planning or school improvement planning.

Monitor the implementation of solutions

Gather evidence if the problems were reduced.

Workshop

Groupwork Come up with a RCA Tabulate the problem, evidence,

factors, strategy, and alternative strategies.

Use actual test data for the evidences

Put your work in a powerpoint presentation