C13 LTMS discussion Follow-up to San Antonio O&H meeting

Elisa SantosNovember 29th, 2005

2

Objective

• Reevaluate the decision made during the O&H meeting– The group decided to use targets and standard deviations based

on 29 tests (matrix plus concurrent tests) without taking into account

• that Lab F is different from all the other labs in Delta OC• and that Lab B is different from the other labs in TLC

• The immediate consequence of using inflated standard deviations is that all stands will calibrate

• Also, the control chart bounds generated for the chosen reference oil will be affected by this decision, making it harder to detect Lab shifts

3

Data source

• 29 tests from the ltms file– 23 matrix tests for PC10 oils– 3 tests for PC10 G– 3 concurrent tests

• Critical parameters:– Delta OC– TGC– TLC– R2TCA (not included)

4

Delta OCIncluding Lab F• Transformation: square root• Oil discrimination• Lab F is different from all Labs• Final model: Lab & Oil Type• RMSE = 0.57

Excluding Lab F• Transformation: square root• Oil discrimination• Labs are similar• Final model: Lab & Oil Type• RMSE = 0.59

Del

ta O

C

0

10

20

30

40

50

60

PC10A PC10B PC10C PC10D

PC10E PC10F PC10G

IND

A B D F G

LTMSLAB

Delta OC By IND

5

square root of Delta OCsq

rt D

elta

OC

2

3

4

5

6

7

8

PC10A PC10B PC10C PC10D

PC10E PC10F PC10G

IND

A B D F G

LTMSLAB

sqrt Delta OC By IND

6

Means, Standard Deviations, LSMEANS & Oil discriminationBefore and After removing Lab F

BeforeOIL N Row s Mean(sqrt

Delta OC)Std Dev(sqrt

Delta OC)Least Sq

MeanLevel Least Sq

Mean PC10A 3 6.27 0.82 6.56 PC10F A 7.20 PC10B 8 5.73 0.73 5.94 PC10A A B 6.56 PC10C 2 5.72 1.90 4.87 PC10B A B 5.94 PC10D 3 3.84 1.85 3.94 PC10E B C 5.21 PC10E 7 4.86 1.43 5.21 PC10C B C 4.87 PC10F 3 6.59 0.98 7.20 PC10G C 4.37 PC10G 3 3.81 0.85 4.37 PC10D C 3.94

IND N Row s Mean(sqrt Delta OC)

Std Dev(sqrt Delta OC)

Least Sq Mean

Level Least Sq Mean

PC10A 3 6.27 0.82 6.05 PC10F A 6.67 PC10B 7 5.50 0.32 5.52 PC10A A B 6.05 PC10C 1 4.38 . 4.17 PC10B A B 5.52 PC10D 2 2.78 0.27 3.42 PC10E B C 4.61 PC10E 6 4.39 0.76 4.61 PC10C B C 4.17 PC10F 3 6.59 0.98 6.67 PC10G C 3.88 PC10G 3 3.81 0.85 3.88 PC10D C 3.42

After

RMSE = 0.57

RMSE = 0.59

7

Options for Delta OC• Summary for potential reference oils

– PC10 B: • Original RMSE: 0.57• Original STDEV: 0.73; Eliminating Lab F: 0.32

– PC10 E:• Original RMSE: 0.57• Original STDEV: 1.43; Eliminating Lab F: 0.76

• The other oils have only three or two tests associated to them

• Option1: Use LSMEANS and Unique RMSE for Delta (using data from all labs)

• Option2: Use Means and Standard deviations after removing Lab F from the calculations

• Option 3: Use Means and Standard deviations (using data from all labs)

8

Outlier screened TGC• No Oil discrimination• Lab G different from Lab A

TGC1 TGC2 TGC3 TGC4 TGC5 TGC6 mean 50.741 44.974 44.655 42.216 43.931 45.664Cyl effect 5.378 -0.389 -0.708 -3.148 -1.432 0.301

Cylinder Effect

• Final model: Lab & Oil• RMSE = 6.47

Level Least Sq Mean PC10C A 52.57 PC10F A 52.38 PC10E A 46.19 PC10B A 45.33 PC10A A 43.29 PC10D A 43.28 PC10G A 37.18

Level Least Sq Mean A 40.26 B 48.23 D 44.76 F 44.43 G 51.05

LSMEANS for OilLSMEANS for Lab

9

OTGCIND N Rows Mean(OTGC) Std Dev(OTGC) Least Sq Mean PC10A 3 45.55 6.44 43.29 PC10B 8 45.18 7.42 45.33 PC10C 2 54.57 2.92 52.57 PC10D 3 39.18 5.86 43.28 PC10E 7 45.52 8.02 46.19 PC10F 3 54.08 11.09 52.38 PC10G 3 35.85 2.83 37.18

RMSE = 6.47

Lab G different from Lab A

OT

GC

25

30

35

40

45

50

55

60

65

PC10A PC10B PC10C PC10D

PC10E PC10F PC10G

IND

A B D F G

LTMSLAB

OTGC By IND

10

Options for OTGC

• Option1: Use LSMEANS and Unique RMSE for OTGC (using data from all labs)

• Option 3: Use Means and Standard deviations (using data from all labs)

11

scrnd TLC• PC10 F is different from all oils

except for PC10 C & PC10 G• Lab B different from Lab A and Lab

G. • The severity of Lab B is more clear

with the Tech & Base Oil analysis.

Level Least Sq Mean B A 35.88 F A B 28.28 D A B 27.15 A B 24.42 G B 24.04

• Final model: Lab & Oil Type• RMSE= 4.69

Level Least Sq Mean PC10F A 40.11 PC10C A B 28.77 PC10G A B 27.86 PC10B B 26.02 PC10D B 25.83 PC10E B 24.97 PC10A B 22.11

LSMEANS for Oil

LSMEANS for Lab

12

scrnd TLC

RMSE = 4.69

scrn

d TL

C

10

15

20

25

30

35

40

PC10A PC10B PC10C PC10D

PC10E PC10F PC10G

IND

A B D F G

LTMSLAB

scrnd TLC By IND

13

Means and Standard Deviations before and after removing Lab B

IND N Rows Mean(scrnd TLC) Std Dev(scrnd TLC) PC10A 2 20 1.18 PC10B 7 23.13 5.91 PC10C 2 26.98 0.21 PC10D 3 23.58 2.34 PC10E 6 21.50 5.00 PC10F 3 36.32 2.81 PC10G 2 28.83 1.06

RMSE = 4.5

RMSE = 4.69

Before

After

14

Options for scrnd TLC

• Option1: Use LSMEANS and Unique RMSE for scrnd TLC (using data from all labs)

• Option2: Use Means and Standard deviations after removing Lab B from the calculations

• Option 3: Use Means and Standard deviations (using data from all labs)

15

Impact on calibration• Options considered for the three parameters:

– Shewhart calibration 111 (Option 1 for Delta/ Option 1 for OTGC/ Option 1 for scrnd TLC)

– Shewhart calibration 212 (Option 2 for Delta/ Option 1 for OTGC/ Option 2 for scrnd TLC)

– Shewhart calibration 333 (Option 3 for Delta/ Option 3 for OTGC/ Option 3 for scrnd TLC)

– … there are other combinations, but these are the ones I generated

• Look at EXCEL spreadsheet..\Excel files\Discussion about Targets and precision.xls

– Shewhart calculations by parameter– EWMA for severity adjustments (?)

lambda 0.3K shewhart 2K EWMA 2

16

Summary of Impact on Lab/ Stand calibrationRow Color represents oil type

A Shewhart Cal row different from zero indicates that that particular Yi is falling out of the K bounds for at least one parameter