Development of a Regular CSITC Round Trial Axel Drieling
description
Transcript of Development of a Regular CSITC Round Trial Axel Drieling
1 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Development of a Regular CSITC Round Trial
Axel Drieling Faserinstitut Bremen e.V. (Bremen Fibre Institute)
- Laboratory of the Bremen Cotton Exchange -
James L. KnowltonUSDA, AMS, Cotton Program
Memphis, TN
Beltwide Cotton ConferencesNew Orleans, LA, January 9-12, 2006
Cotton Quality Measurements Conference
2 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
CSITC Task Force
• ICAC Task Force on Commercial Standardization of Instrument Testing of Cotton, formed in December 2003
• 15 international members: cotton exporters, importers, all segments of the world cotton industry and research
• Based on the consensus that instrument testing is superior to traditional hand classing• Objective: Introduction of a worldwide acceptable, adoptable and reliable
instrument based cotton quality assessment – with defined test rules– based on reliable instrument test results– based on a worldwide cotton testing laboratory evaluation system
• Options for instrument testing– Fixed calibration with Universal Standard Material (HVICCS etc.)– Choice of suitable parameters: at present Mic, Strength (HVICCS), Length (UHML HVICCS),
Uniformity Index, Color Rd and Color +b– Only / All instruments are suitable
• that conform to Universal Standard Material results for the chosen parameters • that do not show systematic deviation • that do not show extended result variability
3 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
CSITC Round Trial
• It is essential to check the reliability of cotton testing laboratories and the test results provided by the laboratories
• Round Trial results will allow the laboratories to demonstrate their capability to meet recommended standards, but certification cannot guarantee the accuracy of individual tests
• Round Trial results will help laboratories to achieve more accurate results
(Participation in the CSITC Round Trial is anonymous and individual results will only be given to the individual participant)
4 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Excursion: Definitions / Wording
• Wording is chosen based on ISO 5725
• Trueness refers to the closeness of agreement between the arithmetic mean of a large number of test results (of one laboratory) and the accepted reference value
• Precision refers to the closeness of agreement between (single) test results / variability of instrument results on tested cottons
• Accuracy includes trueness and precision
5 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Attributes of Different International Round Trials
Attribute USDA HVI Checktest Bremen Round Trial CSITC Round Trial
Number of participants 50 to 80 SITC 90 to 110 SITC ??
Kinds of instruments High volume (SITC) Every kind High volume (SITC)or similar results
Standard material Universal Calib. Std. Various std. materials Universal Calib. Standards
Cottons: Origin and type
USA;Upland
World;broad range of prop.
USA / world;Upland or similar
Frequency 12 times/yeareach 2 samples
4 times/yeareach 1 sample
4 times/yeareach 4 to 5 samples
Number of tests per sample
Asked for 12 testsfor each cotton
Proposed: typical number in the lab for
testing of cotton types
Fixed number: 5 days with each 6 tests on each
cotton = 30 test per cotton
Aim Information for the laboratory
Information for the laboratory
Official laboratory evaluation and detailed analysis for the laboratory
Evaluation of Laboratory average Laboratory average Laboratory average and all single data
Evaluation of Trueness only Trueness only Trueness and precision
6 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Aims of the CSITC Round Trial
a. Evaluation of the test methods / Test result variability• inter-laboratory variability
• in-laboratory variability
• possbible additions: variations between instrument types etc.
b. Evaluation / rating of the participating laboratories- based on the trueness of the results -
c. Detailed analysis of laboratory results to achieve more accurate results- based on trueness and precision -
7 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Configuration of the Round Trial Tests
Cotton 1 Cotton 2 Cotton 3 Cotton 4 Cotton 5
day 1 6 tests 6 tests 6 tests 6 tests 6 tests
day 2 6 tests 6 tests 6 tests 6 tests 6 tests
day 3 6 tests 6 tests 6 tests 6 tests 6 tests
day 4 6 tests 6 tests 6 tests 6 tests 6 tests
day 5 6 tests 6 tests 6 tests 6 tests 6 tests
Sub Total 30 tests 30 tests 30 tests 30 tests 30 tests
Total 150 tests for each Round Trial
Only cotton 1 to cotton 4 are used to evaluate laboratoriesCotton 5 offers the chance to include cottons with different behavior / from different origins
8 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Basis: Reference Results
• Laboratory results are compared to reference results based on the interlaboratory average of the participating laboratories
– In case that the number of participating laboratories is not satisfactory to calculate reliable reference results, the USDA Established Results will be added for consideration
• Possibilities for the consideration of outliers
– No exclusion of outliers
– 90% trimmed mean (exclusion of 5% of the lowest and 5% of the highest results)
– Statistical detection of outliers: (e.g.) Grubbs Method
Grubbs Method was chosen, as only a statistical detection of outliers allows both:
– to calculate a reference result that is not influenced by outliers and
– to calculate inter-laboratory variability results that are not systematically biased
9 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Excursion:Reference Results Consideration of Outliers
Average Inter-Laboratory CVsbased on 30 tests
0
1
2
3
4
5
6
7
Micronaire Strength Length Uniformity Color Rd Color +b
Coe
ffici
ent o
f Var
iatio
n (C
V%
)
All results
OutliersGrubbs90% trimmed mean
10 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Excursion:Reference Results Consideration of Outliers
Average Inter-Laboratory CVs for Micronaire
0
0,5
1
1,5
2
2,5
3
3,5
4
Micronairebased on 30 tests
Micronairebased on
6 tests
Micronairebased on
single tests
Coe
ffici
ent o
f Var
iatio
n (C
V%
)
All results
OutliersGrubbs90% trimmed mean
11 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Micronaire based on 30 tests
02468
101214161820
3,1
3,2
3,3
3,4
3,5
3,6
3,7
3,8
3,9 4
Micronaire, Mic
Num
ber o
f Lab
s Micronaire based on 6 tests on single days
0
20
40
60
80
100
120
3,1
3,2
3,3
3,4
3,5
3,6
3,7
3,8
3,9 4
Micronaire, Mic
Num
ber
of L
abs
x Da
ys
Exemplary inter-laboratory distributions of the results:Micronaire for Cotton 1a) Based on 30 testsb) Based on 6 tests on one day
12 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 3,83 5,17 4,40 3,81Number Of Labs/Instruments 46 46 46 46 46
SD 0,07 0,06 0,06 0,07 0,06CV % 1,76 1,14 1,46 1,73 1,52SD 0,07 0,07 0,10 0,07 0,08CV % 1,93 1,38 2,20 1,95 1,87SD 0,09 0,09 0,11 0,09 0,09CV % 2,25 1,69 2,44 2,30 2,17SD 0,03 0,03 0,04 0,03 0,03CV % 0,78 0,67 0,80 0,82 0,77SD 0,04 0,05 0,04 0,05 0,05CV % 1,10 1,00 0,97 1,24 1,08SD 0,05 0,06 0,06 0,05 0,06CV % 1,34 1,17 1,28 1,37 1,29
Micronaire
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
Strength
Length
Uniformity
Color Rd
Color +b
13 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 32,82 28,22 25,54 32,90Number Of Labs/Instruments 46 46 46 46 46
SD 0,89 1,12 1,31 1,00 1,08CV % 2,72 3,95 5,11 3,03 3,70SD 1,08 1,21 1,45 1,20 1,24CV % 3,30 4,29 5,67 3,66 4,23SD 1,32 1,41 1,75 1,45 1,48CV % 4,03 4,99 6,84 4,42 5,07SD 0,59 0,51 0,51 0,55 0,54CV % 1,79 1,80 2,00 1,67 1,81SD 0,65 0,65 0,62 0,69 0,65CV % 1,99 2,29 2,42 2,10 2,20SD 0,88 0,84 0,78 0,97 0,87CV % 2,68 2,96 3,07 2,95 2,92
Strength
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
14 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 1,207 1,136 0,948 1,177Number Of Labs/Instruments 46 46 46 46 46
SD 0,011 0,011 0,011 0,009 0,011CV % 0,941 1,012 1,162 0,755 0,967SD 0,014 0,013 0,013 0,011 0,013CV % 1,121 1,185 1,425 0,948 1,170SD 0,018 0,018 0,018 0,015 0,017CV % 1,494 1,576 1,910 1,278 1,564SD 0,008 0,007 0,006 0,006 0,007CV % 0,637 0,609 0,627 0,541 0,604SD 0,010 0,012 0,010 0,009 0,010CV % 0,869 1,029 1,105 0,760 0,941SD 0,014 0,014 0,011 0,012 0,013CV % 1,122 1,199 1,212 1,005 1,135
Length
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
15 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 82,42 81,90 78,53 83,65Number Of Labs/Instruments 46 46 46 46 46
SD 0,46 0,44 0,50 0,47 0,47CV % 0,56 0,54 0,64 0,56 0,57SD 0,58 0,65 0,60 0,60 0,61CV % 0,70 0,80 0,77 0,72 0,74SD 0,80 1,19 0,82 0,84 0,91CV % 0,97 1,45 1,04 1,00 1,12SD 0,30 0,35 0,27 0,26 0,29CV % 0,36 0,43 0,34 0,31 0,36SD 0,53 0,59 0,55 0,52 0,55CV % 0,64 0,72 0,70 0,62 0,67SD 0,58 0,69 0,61 0,61 0,62CV % 0,71 0,84 0,78 0,73 0,77
Uniformity
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
16 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 76,31 78,06 74,86 76,08Number Of Labs/Instruments 45 45 45 45 45
SD 1,15 1,34 0,88 1,03 1,10CV % 1,50 1,72 1,17 1,35 1,44SD 1,28 1,42 0,99 1,12 1,20CV % 1,68 1,82 1,32 1,47 1,57SD 1,36 1,47 1,04 1,18 1,26CV % 1,78 1,88 1,39 1,56 1,65SD 0,26 0,32 0,26 0,23 0,27CV % 0,34 0,41 0,35 0,30 0,35SD 0,24 0,26 0,20 0,25 0,24CV % 0,32 0,33 0,27 0,33 0,31SD 0,40 0,41 0,37 0,41 0,40CV % 0,52 0,53 0,49 0,54 0,52
Color Rd
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
17 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
a) Test Result Variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 12,14 11,53 10,86 10,98Number Of Labs/Instruments 45 45 45 45 45
SD 0,48 0,46 0,29 0,37 0,40CV % 3,96 3,98 2,67 3,35 3,49SD 0,54 0,49 0,32 0,40 0,44CV % 4,47 4,23 2,98 3,67 3,84SD 0,57 0,51 0,35 0,44 0,47CV % 4,67 4,45 3,18 3,97 4,07SD 0,13 0,16 0,10 0,15 0,13CV % 1,03 1,40 0,95 1,33 1,18SD 0,13 0,15 0,10 0,13 0,13CV % 1,11 1,31 0,97 1,15 1,13SD 0,19 0,23 0,15 0,19 0,19CV % 1,59 2,00 1,41 1,74 1,68
Color +b
Inter-Laboratory Variation (Grubbs)
Typical In-Laboratory Variation (Median of all labs)
based on 30 tests
based on 6 tests
based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days
18 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
b) Evaluation / Rating of Each Participating Laboratoryfor the example of laboratory 115
Micronaire Strength Length Uniformity Color Rd Color +bReference Values Cotton 1 3,83 32,82 1,207 82,42 76,31 12,14
Cotton 2 5,17 28,22 1,136 81,90 78,06 11,53Cotton 3 4,40 25,54 0,948 78,53 74,86 10,86Cotton 4 3,81 32,89 1,177 83,65 76,08 10,98
Laboratory Average of All Days Cotton 1 3,80 33,62 1,207 82,71 75,37 11,38Cotton 2 5,23 28,50 1,134 81,44 76,05 10,82Cotton 3 4,36 26,11 0,969 76,13 73,62 10,41Cotton 4 3,79 32,72 1,182 83,83 75,29 10,17
Rel. Distance to Reference Cotton 1 -0,03 0,80 0,000 0,29 -0,94 -0,76Cotton 2 0,06 0,28 -0,003 -0,46 -2,00 -0,71Cotton 3 -0,04 0,57 0,021 -2,40 -1,24 -0,45Cotton 4 -0,02 -0,18 0,005 0,18 -0,79 -0,81
Mean Absolute Distance to Reference 0,04 0,46 0,007 0,83 1,24 0,68
Scale Factor (Based on USDA Reproducibility Limits except Rd) 0,10 1,50 0,02 1,00 1,50 0,50Summary Evaluation for Each Property(=Mean Abs. Distance divided by Scale Factor)
0,38 0,31 0,36 0,83 0,83 1,37
Relevance of Property 1,00 1,00 1,00 1,00 1,00 1,00
Summary Evaluation of All Properties(=Average of all properties)
Performance of Laboratory 115
0,68
19 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
b) Evaluation / Rating of Each Participating Laboratoryfor the example of laboratory 115
Evaluation of the Laboratory 115 0,68
Average 0,67Median 0,53Best Lab. 0,32Worst Lab. 3,16
Statistics- All Laboratories -
Evaluation Combined Prop.
Laboratory No. Evaluation Combined Prop.
Evaluation Results- Combined Properties -
0
2
4
6
8
10
12
14
00,
10,
20,
30,
40,
50,
60,
70,
80,
9 11,
11,
21,
31,
41,
51,
61,
71,
81,
9 22,
12,
22,
32,
42,
52,
62,
72,
82,
9 33,
13,
2
Evaluation Results
No.
of L
abor
ator
ies
Same kind of presentation
is done for every single
property
20 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
c) Detailed Laboratory Analysisfor the example of laboratory 115
Laboratory 115 - Color Rd
R2 = 0,4498
-4
-3
-2
-1
0
1
2
70 71 72 73 74 75 76 77 78 79
Reference Value
Devi
atio
n fr
om R
efer
ence
Color Rd - AverageSingle Days
1. Detailed Trueness Evaluation to find systematical deviations / biases
21 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
c) Detailed Laboratory Analysisfor the example of laboratory 115
2. Precision Evaluation to show / compare laboratory test result consistency/variability
Cotton 1 Cotton 2 Cotton 3 Cotton 4 Average RemarkAverage Test Result 12,14 11,53 10,86 10,98 --- ---
between different days with each 6 tests SD 0,13 0,16 0,10 0,15 0,13 ---between single tests on one day SD 0,13 0,15 0,10 0,13 0,13 ---between all tests on different days SD 0,19 0,23 0,15 0,19 0,19 ---
between different days with each 6 tests SD 0,27 0,24 0,09 0,10 0,17higher than typical variation
between single tests on one day SD 0,30 0,27 0,17 0,21 0,24higher than typical variation
between all tests on different days SD 0,41 0,35 0,19 0,26 0,30higher than typical variation
Typical In-Lab. Variation (Median)
In-Lab. Variation (Your Lab.)
Laboratory 115 - Color +b
22 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01
Lab of the Bremen Cotton Exchange
Conclusion
• CSITC Round Trial– Is based on the recommendations of the CSITC Task Force– Was developed in co-operation between the Bremen Fibre Institute and
USDA-AMS – Is headed by the International Cotton Advisory Committee (ICAC)– Will be conducted regularly in co-operation between the Bremen Fibre Institute and
USDA-AMS
• Timetable– 1. Pilot Round Trial was conducted in 2005– 2. Pilot Round Trial was conducted in 2006, with 46 laboratories/instruments– Regular Round Trials are starting now (2007-1)
• Samples are already shipped• Approx. 60 laboratories will participate with approx. 90 instruments
– For registration in future Round Trials, please contact the ICAC