Statistical Process Control

Todd Pawlicki

tpaw@ucsd.edu

University of California, San Diego

Dept of Radiation Medicine & Applied Sciences

1 of 40

Disclosures

• Founding partner of TreatSafely, LLC – www.treatsafely.org

– i.treatsafely.org

• Founding partner of Oncology Owl, LLC • SPC-based QA software

• Royalties from textbook – Quality and Safety in Radiotherapy

2 of 40

https://bitbucket.org/tohccmedphys/qatrackplus

What is Statistical Process Control?

• A collection of tools for quality improvement

– Histogram

– Check sheet

– Pareto chart

– Scatter diagram

– Control chart

– Cause-and-effect diagram – Stratification, run chart, flowchart, defect concentration diagram

3 of 40

Background on Quality

• Craftsmanship – Expert and apprentice

• European industrial revolution – Subdivided trades into multiple steps

• Departure to Taylor’s system – Scientific management

A History of Managing for Quality: The Evolution, Trends, and Future Directions of Managing for Quality. Ed. J.M Juran, 1995 ASQC Quality Press

4 of 40

Taylor’s System

• Engineering department

• Production department

• Inspection department – Separate the good from the bad

• AT&T; creating a long distance telephone system – Needed reliable, interchangeable parts

– Western Electric; manufacturing arm of AT&T

Juran. Early SQC – A historical supplement. 1997:Quality Progress

5 of 40

Interesting Outcome

• Western Electric engineers adjusted the processes when they found that parts were out of specifications

• Result…quality decreased!

• Now what to do?

Remember Dr. Bissonnette’s funnel and marbles example from Tuesday.

6 of 40

Enter Walter A. Shewhart

• 1917 PhD in physics

• Shewhart’s insight

– Deviation of manufactured parts is resultant from one of two reasons:

• Common (chance) causes

• Assignable (special) causes

• Considered the father of process control

7 of 40

Shewhart’s Definition of Control

“A phenomenon will be said to be controlled when, through the use of past experience, we can predict, at least within limits, how the phenomenon may be expected to vary in the future.”

Page 6

8 of 40

Shewhart’s View of Special Causes

A special (assignable) cause of process variation is one that can be found by experiment without costing more than it is worth to find it.

9 of 40

Take Home Points

• Process control is an empirical tool that can be used for quality improvement

– Control charts have a basis in statistical theory but are not dependent on it

• Statistical process control (SPC) is a way of thinking with tools attached

10 of 40

Stewart’s Characteristics of Original Data

• Numbers representing the numerical values of the measurements

• Text describing the condition under which each measurement was made, including a description of the operation of measurements

• Human element or observer

• Order in which the numbers were taken

11 of 40

The order of data matters

• 𝑥 = 0.0, 𝑠 = 1.0 Chamber Readings - Random Ordered

1.930

1.935

1.940

1.945

1.950

0 5 10 15 20 25 30

Reading number

Ch

am

ber

read

ing

Chamber Readings - Time Ordered

1.930

1.935

1.940

1.945

1.950

0 5 10 15 20 25 30

Reading number

Ch

am

ber

read

ing

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 5 6 7 8 9 10 11 12 13

12 of 40

Process Perspective on Data Analysis

• Sub-optimal ways to understand a process

– Year-to-date

– Compared to last year this time

– What did the last data point tell us?

• Better way to understand a process

– Plot data as a function of time and calculate limit lines around the historical average

13 of 40

Anatomy of a Control Chart

Plot data as a function of time, number, patient, etc.

Point outside the limits – find out why

14 of 40

This is called a Shewhart-type control chart

Entr

op

y

Qu

ality Imp

rovem

ent

Wheeler & Chambers

1992 Figure 1.10

State IV Out-of-control

Outside action limits

State III Out-of-control

Within action limits

State II In-control

Outside action limits

State I In-control

Within action limits

Action: Continue to monitor indefinitely Action: Re-engineer process or widen action limits Action: Analyze process, remove assignable causes Action: Analyze process, re-commission and/or re-engineer process

15 of 40

Continual Quality Improvement

16 of 40

Pause for Discussion

• Questions or comments?

• Is this idea different that what we do now?

• Is it useful or trivial / not useful?

17 of 40

How to determine the control limits?

• Shewhart used the standard error

– Subgroup data as a function of time

– Calculate mean and standard deviation of subgroups as a function of time

• We’ll use individual values

– Point-to-point data should be the same thing

18 of 40

Control Chart Strategy

• Choose a metric, 𝑥

– Stratify your data as best you can

• Construct the control chart limits as 𝜇 ± 3𝜎

• How to estimate the parameters 𝜇 and 𝜎?

19 of 40

Control Chart Strategy

• For a sample 𝑛, 𝑥 is an unbiased estimate of 𝜇

– Therefore, 𝑥 is a reasonable estimate of 𝜇

• For a sample 𝑛, 𝑠 is a biased estimate of 𝜎

– For large 𝑛 • 𝑠 is still a reasonable estimate of 𝜎

– For small 𝑛 • 𝑠 can be a very poor estimate of 𝜎

20 of 40

Remember, we’re using 𝑛 = 1.

Another estimate of 𝝈

• It is “well known” that the sample range (𝑅) can be used as an unbiased estimate of 𝜎

𝑊 =𝑅

𝜎 (relative range dist., mean of 𝑊 is const.)

𝜎 =𝑅

𝑑2 (𝑑2 is a function of sample size, 𝑛)

• The range has been historically used to calculate control chart limits

21 of 40

For a sample size of 1

• The range estimate of 𝜎 also works well as the difference between successive points

– Called the moving range

𝜎 =𝑚𝑅

𝑑2

22 of 40

Actually, It’s 2 Control Charts

Individuals Chart

Moving Range Chart

23 of 40

Strategy to Create Charts

• Choose a quality metric

• Calculate 𝑥 over a period where the process is carefully monitored (chart center line)

– Then, calculate the average moving range, 𝑚𝑅 , over the same points

• Calculate the limits for individual values:

𝜇 ± 𝑡 ∙ 𝜎 = 𝑥 ± 3 ∙ 𝑚𝑅 𝑑2 = 𝑥 ± 2.660 ∙ 𝑚𝑅

24 of 40

𝑥 ± 𝑡 ∙ 𝜎 ; why t = 3?

• Some theoretical argument using

Tchebycheff’s inequality; 𝑃 > 1 −1

𝑡2

– At least 1 −1

𝑡2 of the values (𝑥) will fall with 𝑡

standard deviations of the mean

– When 𝑡 = 3

• 89% of the values will be within ±𝑡 ∙ 𝜎𝑥 limits irrespective of the distribution

25 of 40

Stewart’s final explanation of 𝒕 = 𝟑

• “Experience indicates that 𝑡 = 3 seems to be an acceptable economic value.”

Page 277

26 of 40

Moving Range Chart

• Estimate the limits for moving range chart (𝜇𝑅±3𝜎𝑅)

𝑚𝑅 = 𝑑2𝜎

𝜎𝑅 = 𝑑3𝜎

• Calculate the range charts limits as:

𝑈𝑅𝐿 = 1 + 3𝑑3𝑑2

∙ 𝑚𝑅

27 of 40

𝑑3 is a function of sample size, 𝑛

Individual Values: I Chart

Sample number or Time

Ind

ivid

ual

va

lues

1x x

n

(use d2=1.128 for n = 2)

3 2.6601.128

mRUL x x mR

2.660LL x mR

28

28 of 40

Individual Values: mR Chart

Sample number or Time

Mo

vin

g R

ange

1

2

1

1

N

j j

j

mR x xN

3.268URL mR

(use d3=0.8525 for n = 2)

29

29 of 40

IMRT QA

• Less consistent process.

• Less chance to detect a problem.

• Low quality process.

• More consistent process.

• More chance to detect a problem.

• High quality process.

30 of 40

State IV Chaos

State I Ideal

Inappropriate use of control charts

8th

21st 28th

31 of 40

An Example Using Control Charts

• Charge

– Standardize the time it takes from CT to first treatment

• Goal to reduce the time

Start with a process map Focus on a specific area

Example; from my DB, I know these two steps take the longest

32 of 40

Process: MD Plan Approval Time

• Process description

– CMD completes the plan, pages or emails MD, and then waits until MD approves or rejects the plan

• Process metric

– Duration of time that it takes the MD to approve the plan after CMD contacts MD

• Process requirement

– Plan completed to plan approved ≤ 25 hrs

33 of 40

Pla

n a

pp

rova

l (h

rs)

Pla

n a

pp

rova

l (h

rs)

Interpreting Control Charts

7.2 hrsx

8.0 hrsx

25 hrs

25 hrs

29.4 hrsUCL

23.6 hrsUCL

MD #1

MD #2

Process change. Find out why.

> 25 hrs. Normal operation for this MD.

(𝑛 = 31)

(𝑛 = 26)

1st 10 points

1st 10 points

34 of 40

Process (Quality) Improvement

8.0 hrsX

25 hrs

23.6 hrsUCL

MD #1

12.0 hrsUCL

Pla

n a

pp

rova

l (h

rs)

4.0 hrsX

MD #1

35 of 40

Summary of Control Chart Use

Montgomery figure 5.5

Process Input Output

Measurement System

Detect assignable cause

Identify root cause of the problem

Implement corrective action

Verify and follow up

36 of 40

Leading vs Lagging Metrics

• MD plan approval example

– Lagging indicator

• Time from CT Sim to First Treatment

– Leading indicator

• Time from plan completed to plan approved

• May be different depending on the intent

– Time from CT Sim to First Treatment is a leading indicator to…patient satisfaction, throughput, etc

37 of 40

More About Control Chart Use

• Real-time process analysis tool based on retrospective data – Control charts won’t fix a problem at commissioning…only

highlight the problem

0.0% commissioning error -2.1%

38 of 40

Parting Comments

• Control limits are point estimates

– The limits have an associated uncertainty

• Shewhart-type (ImR) charts are good at detecting large process deviations

– Slow drifts are better detected by other charts, e.g., EWMA, CUSUM

39 of 40

What to do when I get home?

1. Pick a metric

2. Create a control chart

3. Wait for out of control point

4. Identify cause(s)

5. Possibly remove causes

6. Have fun knowing that your improving quality!

40 of 40