Presentation Outline
description
Transcript of Presentation Outline
![Page 1: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/1.jpg)
1
Presentation Outline
Introduction Objective Sorting Probability and Loss Matrices The Proposed Model Analysis Of Some Loss Functions Case Study Redundancy based methods Illustrative example Summary
![Page 2: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/2.jpg)
2
Introduction
A. Accuracy and PrecisionB. Types of dataC. Binary Situation
![Page 3: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/3.jpg)
3
A. Accuracy and Precision.
Accuracy The closeness of agreement between the
result of measurement and the true (reference) value of the product being
sorted.Precision
Estimate of both the variation in repeated measurements obtained under the same
conditions (Repeatability) and the variation of repeated measurements obtained under
different conditions (Reproducibility).
![Page 4: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/4.jpg)
4
B. Two types of data characterizing products or
processes
Variables (results of measurement, Interval or Ratio Scales)
Attributes (results of testing, Nominal or Ordinal
Scales ).
![Page 5: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/5.jpg)
5
Four types of data
The four levels were proposed by Stanley Smith Stevens in his 1946 article.Different mathematical operations on variables are possible, depending on the level at which a variable is measured.
![Page 6: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/6.jpg)
6
Categorical Variables
1. Nominal scale:
gender, race, religious affiliation, the number of a bus.
Possible operations:
,
![Page 7: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/7.jpg)
7
Categorical Variables(cont.)2. Ordinal scale :
results of internet page rank, alphabetic order, Mohs hardness scale (10 levels from talc to diamond) customer satisfaction grade , quality sort, customer importance (QFD) vendor’s priority, severity of failure or RPN (FMECA), the power of linkage (QFD)
Possible operations:
,,,
![Page 8: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/8.jpg)
8
Numerical data.
3. Interval scale: temperature in Celsius or Fahrenheit scale , object coordinate, electric potential.
Possible operations:
,,,,,
![Page 9: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/9.jpg)
9
Numerical data (cont.)
4. Ratio scale: most physical quantities, such as mass, or
energy, temperature, when it is measured in kelvins, amount of children in family, age.
Possible operations: /,,,,,,,
![Page 10: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/10.jpg)
10
C. The sorting probability matrix
for the binary situationAccuracy is characterized by:1. Type I Errors (non-defective is
reported as defective) –
alfa risk2. Type II Errors (defective is
reported as non-defective) – beta risk
![Page 11: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/11.jpg)
11
The sorting probability matrix
for the binary situation (cont.)
E. Bashkansky, S. Dror, R. Ravid, P. Grabov
ICPR-18, Salerno August, 2, Session 19 6
Factual
Act
ual + -
+-
1-α α
1- ββ
![Page 12: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/12.jpg)
12
Objective
Developing a new statistical procedure for evaluating the accuracy and effectiveness of measurement systems applicable to Attribute Data based on the Taguchi approach.
![Page 13: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/13.jpg)
13
Sorting Probability Matrix
The sorting matrix is an 'm by m' matrix.
Its components Pi,j are the conditional probabilities that an item will be classified as quality level j, given its quality level is i.
A stochastic matrix:
P̂
miPm
jij
1,11
![Page 14: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/14.jpg)
14
Four Interesting Sorting Matrices
(a) The most exact sorting:
(b) The uniform sorting: (designated as MDS: most disordered sorting):
(c) The “worst case” sorting. For example, if m = 4:
ijijP
mPij
1
0001000110001000
P
![Page 15: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/15.jpg)
15
Four Interesting Sorting Matrices (cont.)
(d) Absence of any sorting .For example, if m = 2:
0101
P
![Page 16: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/16.jpg)
16
Indicator of the classification system inexactness
m
P
m
DG
ij
m
i
m
jij
2
)(
2
ˆ2
1 1
2
![Page 17: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/17.jpg)
17
Loss Matrix Definition
Let Lij - be the loss incurred by classifying
sort i as sort j.
![Page 18: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/18.jpg)
18
The Proposed Model
Expected Loss Definition:
Effectiveness Measure:
ijij
m
i
m
ji LPpEL
1 1
)(1
MDStheforELELEff
![Page 19: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/19.jpg)
19
Analysis Of Some Loss Functions
Equal loss: Quadratic loss: Entropy loss: Linear loss:
ijijL 12)( ijLij
ijij PL2
log
)( ijLij
![Page 20: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/20.jpg)
20
Equal loss
mmmPTrace
EffmPTraceEL 1
)ˆ(11)ˆ(1
If there is no preliminary information about pi
![Page 21: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/21.jpg)
21
Equal loss (Cont.)
If any preliminary information about pi
is available :
mm
Pp
EffPpELii
m
ii
ii
m
ii 1
)1(
1)1( 1
1
![Page 22: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/22.jpg)
22
Quadratic Loss
For ordinal data, the total accuracy of the ratingcould be defined as the expected value of Lij.
ij
m
i
m
ji PpijAccuracy
1 1
2)(
![Page 23: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/23.jpg)
23
Quadratic Loss (Cont.)
If there is no preliminary information about pi:
If there is any preliminary information about pi:
61)(1 2
1 1
22
mijm
ELm
i
m
jmds
6)12)(1()()1()( 2
mmXEmXEELmds
![Page 24: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/24.jpg)
24
Entropy loss
If level i is systematically related to level j (Pij=1), there is no “entropy loss”.
The above loss function leads us directly to Eff = Theil’s uncertainty index.
ijij
m
i
m
ji PPpEL 2
1 1
log
![Page 25: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/25.jpg)
25
Linear Loss
Could be useful for bias evaluation:
)(1 1
ijPpBias ij
m
i
m
ji
This measure characterizes the dominant tendency [over grading, if Bias > 0 or under grading, if Bias < 0 ]
![Page 26: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/26.jpg)
26
Case Study (nectarines sorting)
Type 1- 0.860, Type 2 - 0.098 , Type 3 -0.042
ClassificationType1 Type2 Type3 Total
Actual
Type1 446 91 7 544Type2 12 307 33 352Type3 0 11 93 104Total 458 409 133 1000
![Page 27: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/27.jpg)
27
The Classification Matrix
894.0106.00
094.0872.0034.0
013.0167.0820.0
10493
10411
1040
35233
352307
35212
5447
54491
544446
ijP
![Page 28: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/28.jpg)
28
The Loss matrix
000.742.12
78.0042.5
76.198.00
ijL
![Page 29: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/29.jpg)
29
Effectiveness Evaluations
According to proposed approach: Eff = 80%Compare the above to the measures of effectivenessfor different loss functions :
Equal loss: 74%. Taguchi loss: 87%. Disorder entropy loss: 62% Linear penalty function:79% Traditional kappa measure : 79% .
It can be seen that the effectiveness estimatestrongly depends on the loss metric model.
![Page 30: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/30.jpg)
30
REPEATED SORTING
n
nxxxx
xx
n
...21
![Page 31: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/31.jpg)
31
Independency vs. Correlation
22
2
tmeasuremenproduct
product
![Page 32: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/32.jpg)
32
Case A: Two Independent Repeated Ratings Real improvement may be obtained if, in the
case of disagreement, the final decision is made in favor of the inferior sort (one rater can see a defect, which the other has not detected).
Usually, the loss that results from overrating is greater than the loss due to underrating, thus such a redistribution of probabilities seems to be legitimate. Nevertheless, to verify improvement in sorting effectiveness, we need a new expected loss calculation.
![Page 33: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/33.jpg)
33
Case B: Three Repeated Ratings
We add a third rater only if the first two raters do not agree. For most industrial applications this means that a product is passed through a scrupulous laboratory inspection or, for the purpose of our analysis, through an MRB board. The decision could be considered as an etalon measurement.The probability of correct decisions increases, and the probability of wrong decisions, decreases.
![Page 34: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/34.jpg)
34
Conclusions
To decide whether a double or triple rating procedure is expedient the total expenditures
have to be compared
![Page 35: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/35.jpg)
35
Case C: A Hierarchical Classification System
The classification procedure is built on more than one level.
To characterize such a hierarchical classification system G can be utilized as a “pure” indicator of the classification system’s inexactness.
Usually, the cost of classification (COC) has an inverse relationship to the amount of G. In contrast to the COC, the expected loss usually decreases, as the exactness of the judgment improves.
![Page 36: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/36.jpg)
36
Optimization of HCS
If one decides to pass the hierarchical classification subsystem from the lower level (1) up to the K level, the total expenditure can be optimized by looking for the best level minimizing it .
![Page 37: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/37.jpg)
37
A CASE STUDY AND ILLUSTRATIVE EXAMPLE
The proportions of the sort types were: Type 1- 0.53, Type 2 - 0.27 and Type 3 - 0.20. The same loss matrix was considered. From an R&R study, executed in relation to two independent raters’ results, the joint probability matrices were estimated.
![Page 38: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/38.jpg)
38
Summary table
One rater Two
independent raters
( case A )
Three raters (case B)
Sorting matrix
0.89, 0.08, 0.030.07, 0.85, 0.080.06, 0.14, 0.80
0.84, 0.10, 0.060.04, 0.80, 0.160.01, 0.06, 0.93
0.96, 0.04, 0.000.04, 0.96, 0.000.01, 0.92, 0.07
Exp. loss EL = 0.534 EL' = 0.309 EL'' = 0.132
Inexact. Ind. G = 0.0194 G' = 0.0192 G'' = 0.0013
![Page 39: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/39.jpg)
39
Summary
The proposed procedure for evaluation of product quality classifiers takes into account some a priori knowledge about the incoming product, errors of sorting and losses due to under/over graduation.
![Page 40: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/40.jpg)
40
Summary (Cont.1)
The appropriate choice of the loss function (matrix) provides the opportunity to fit quality sorting process model to the real situation.
The effectiveness of quality classifying can be improved by different redundancy based methods. However, the advantages of redundancy based methods are not unequivocal, as is the case in the usual measurement processes, and corresponding calculations according to the technique being used are required.
![Page 41: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/41.jpg)
41
Summary (Cont.2) The conclusion concerning the selection of
the preferred case depends on the losses due to misclassification, as well as on the incoming quality sort distribution.
Possible applications of the proposed approach are not limited only to quality sorting. The approach can be extended to other QA processes concerned with classification
![Page 42: Presentation Outline](https://reader036.fdocuments.in/reader036/viewer/2022081604/56816331550346895dd3b379/html5/thumbnails/42.jpg)
42
Thank You