Korea Scheme for ISO 17025 Proficiency Testing in CC ... Lee.pdf · Korea Scheme for ISO 17025...

http://www.security.re.kr

Korea Scheme for ISO 17025 Proficiency Testing

in CC Evaluation FacilitiesHyunjung Lee, Dongho Won

Sungkyunkwan University, Information Security Group, Korea


http://www.security.re.kr2

Contents

§ Requirements for the CC evaluation facilities§ ISO 17025§ ISO 17025: Proficiency testing§ Korea proficiency testing scheme§ Proficiency testing results§ Proposition


Requirement for the CC evaluation facilities

§ General requirement for the CC evaluation facilities:§ Accreditation to the requirements of ISO Standard 17025


Requirement for the CC evaluation facilities

§ Korea Evaluation and Certification Scheme(KECS)§ Criteria for acknowledging an evaluation facility

• a corporation in Korea• 3 or more senior evaluators including a technical manager; and

2 or more evaluators• an independent decision making system; and

an office and devices required for evaluation activities• a corporation’s own quality manual and procedural

documents drawn out according to the ISO/IEC 17025• other particulars considered necessary by the certification body• the requirements regarding acknowledgment of an evaluation

facility stated in Annex B.3 of the CCRA


§ ISO/IEC 17025 contains the criteria necessary for a laboratory to implement in order for it to perform its test work competently

§ The criteria that a laboratory must comply with to be internationally acceptable is ISO/IEC 17025.

§ Testing and calibration laboratories that comply with ISO 17025 will also operate in accordance with ISO 9001.

What is ISO 17025?


The Contents of ISO 17025

§ The ISO 17025 standard itself is comprised of 5 elements: 1. Scope 2. Normative References 3. Terms and Definitions 4. Management Requirements 5. Technical Requirements

§ Elements 4 and 5 contain the actual accreditation requirements.

ISO 17025ISO 17025

Management Requirements

(15 Elements)

Technical Requirements

(10 Elements)


The Contents of ISO 17025

§ Management Requirementsü Organizationü Quality systemü Document controlü Review of requests, tenders and contractsü Subcontracting of tests and calibrationsü Purchasing services and suppliesü Service to clientü Complaintsü Control of nonconforming testing and/or

calibration workü Improvementü Corrective actionü Preventive actionü Control of recordsü Internal auditsü Management reviews

§ Technical Requirementsü Generalü Personnel - Accommodation and

environmental conditionsü Test and calibration methods and method

validationü Equipmentü Measurement traceabilityü Samplingü Handling of test and calibration itemsü Assuring the quality of test and calibration

resultsü Reporting the results

§ Elements 4 and 5 contain the actual accreditation requirements.


International accreditation scheme

ILAC(International Laboratory Accreditation

Cooperation)

APLAC(Asia Pacific Laboratory Accreditation

Cooperation)

EA(European co-operation for

Accreditation)

KOLAS(Korea Laboratory Accreditation Scheme)

Testing/Calibration bodyISO/IEC 17025

KOLAS-G-009

KOLAS-R-001

Inspection bodyISO/IEC 17020

KOLAS-G-010

Proficiency testing administration body

ILAC-G13KOLAS-R-006

Proficiency testing administration

ISO/IEC Guide 43KOLAS-R-003

Proficiency testing material

ISO/IEC Guide 35ISO 13528

···

···


Related documents

Doc. No. Title

Inspection body

ISO/IEC 17020 (KS Q 17020)

General criteria for the operation of various types of bodies performing inspection

KOLAS-G-010 17020 guide

Testing/Calibration

body

ISO/IEC 17025 (KS Q ISO/IEC 17025)

General requirements for the competence of testing and calibration laboratories

KOLAS-G-009 17025 guide

KOLAS-R-002 International laboratory accreditation scheme (Korea)

Proficiency testing

administration

scheme

ILAC-G13 ILAC Guidelines for the Requirements for the Competence of Providers of Proficiency Testing Schemes

KOLAS-R-006 Criteria and procedure for appointment of proficiency testing administration bodies

ISO/IEC Guide 43-1, 43-2 (KS A ISO/IEC Guide 43-1, 43-2) Proficiency testing by inter-laboratory comparisons

KOLAS-R-003 Criteria for proficiency testing administration

ISO/IEC Guide 35 (KS A ISO Guide 35) Reference materials - General and statistical principles for certification

ISO 13528 (KS Q ISO 13528)

Statistical methods for use in proficiency testing by inter-laboratory comparisons


What is proficiency testing?

KS Q ISO/IEC 17025 5.9 Assuring the quality of test and calibration results

“The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where applicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not limited to, the following:”

a) regular use of certified reference materials and/or international quality control using secondary reference materials;

b) participation in interlaboratory comparison or proficiency-testing programs; c) replicate tests or calibrations using the same or different methods;

d) retesting or recalibration of retained items;

e) correlation of results for different characteristics of an item.

NOTE The selected methods should be appropriate for the type and volume of the work undertaken.


KOLAS-R-002 Korea Scheme for ISO 17025

KOLAS-R-002 Clause 26 Proficiency testing

① The head of the accreditation body shall manage a proficiency testing programs based on the criteria for proficiency testing administration in order to assess the competence of accredited testing laboratories and improve the confidence of test reports.

② To demonstrate its competence, an accredited testing laboratory shall take part in a proficiency testing of relevance that is supervised or approved by the head of the accreditation body at least once every three years.

In case a factor that caused a proficiency testing to fail is considered to have significant influence on the test result, the head of the accreditation body can require the technical manager or experimenter in question to complete a relevant training course.

③ Though an accredited testing laboratory has participated in a proficiency testing according to article 2, it shall also take part in the proficiency testing the head of the accreditation body requests. If it doesn’t, the head of the accreditation body can conclude that the laboratory would yield a nonconforming result.



KOLAS-R-002 Clause 26 Proficiency testing

④ To be an accredited testing laboratory, a testing laboratory shall participate in a domestic or international proficiency testing of its interested area approved by the head of the accreditation body and submit the result at the time of application for accreditation. In case that a proficiency testing cannot be conducted, a measurement audit based on the regulation explicitly arranged by the head of the accreditation body can replace it; or, during an on-the-job assessment, an assessor can provide a reference test item for the applying laboratory to test so that he can assess its competence.

⑤ The head of the accreditation body can conduct a proficiency testing as part of follow-up management.

⑥ A testing laboratory that has yielded 2 consecutive nonconforming results may be required to undergo an on-the-job inspection, which the laboratory should pay for.

⑦ A proficiency testing administration body appointed by the head of the accreditation body can make it possible for the participating laboratory to make a test item and operate the test at minimum cost.



KOLAS-R-002 Clause 29 Revocation of accreditation

③ The head of the accreditation body can revoke accreditation in whole or in part in the following cases:

8. If an accredited body didn’t take any corrective action in a timely manner after receiving a notice of nonconforming proficiency testing result.

9. If an accredited body produces 2 consecutive nonconforming results out of a proficiency testing for a same test item.



KOLAS-R-002 Clause 36 Appointment of proficiency testing administration bodies

① The head of the accreditation body can appoint a proficiency testing administration body for performing proficiency testing.

② A body appointed as a proficiency testing administration body is required to submit a documented plan for a proficiency testing of the following year by the end of December every year.

③ A proficiency testing administration body shall report the results of a proficiency testing to the head of the accreditation body.

④ Particulars of the appointment of proficiency testing administration bodies will be explicitly notified by the head of the accreditation body according to the international criteria.

(KOLAS-R-006 Criteria and procedure for appointment of proficiency testing administration bodies)


KOLAS-R-006 proficiency testing administration bodies

KOLAS-R-006 Criteria and procedure for appointment of proficiency testing administration bodies § Technical requirements based on the ILAC Guidelines for the

Requirements for the Competence of Providers of Proficiency Testing Schemes(ILAC G13:2007 ); and criteria and procedure for appointment of proficiency testing administration bodies

§ Management requirements• Organization• Management system• Document control

§ Technical requirements• Proficiency testing plan • Consistency and stability testing• Statistical plan • Performance assessment


KOLAS-R-003 Proficiency testing administration


1) Consideration on appointing a proficiency testing program

• The type of test, measurement, or calibration should match that of a participating

body or testing laboratory.

• The KOLAS shall, with the consent of participating testing laboratories, obtain the

test results of those laboratories along with the facts regarding the testing program,

deciding an assigned value, guidelines for participation, statistical process of data,

and a final report of each proficiency testing.

• Period of operation

• Appropriateness of a program in which an involved accredited laboratory is

interested, including time, location, safety of test material, program distribution, etc.




1) Consideration on appointing a proficiency testing program

• Capability of a participating testing laboratory to use the acceptance criteria

(that is to judge whether the proficiency testing succeeded)

• Cost of participating the program

• Program to ensure confidentiality for the participants

• Time for reporting test results

• Appropriateness of the test material and measurement artifact that are used in

a special assessment of consistency, stability, or (if applicable) traceability to

the international standard




2) Types of proficiency testing programs approved by the KOLAS• Proficiency testing program conducted by the KOLAS or a proficiency testing

administration body appointed by the KOLAS

• Proficiency testing program conducted by the APLAC, EA, or other international

accreditation cooperation schemes

• Proficiency testing program conducted or approved by the ILAC MRA accreditation

bodies

• Inter-laboratory or inter-experimenter comparison that is approved by the head of

the KOLAS

• Measurement audit, in case there is no proficiency testing program




3) Difference between proficiency testing, inter-laboratory comparisons, and inter-experimenter comparisons

§ Proficiency testing: more than 8 testing laboratories involved

Inter-laboratory comparisons: 2 ~ 7 testing laboratories involved

Inter-experimenter comparison: 1 testing laboratory

§ Inter-laboratory or inter-experimenter comparison carried out with prior consent

of the KOLAS on the proficiency testing plan is considered a proficiency testing

conducted according to the “International laboratory accreditation scheme” and

“Korea calibration laboratory appointment scheme.”




4) Corrective actionAn accredited testing laboratory shall report the result of corrective action to the KOLAS in 30 days after receiving a notice of nonconforming result. The result shall include the following:

• potential cause identified;

• corrective actions taken;

• a document, if necessary, that proves the effectiveness of the corrective actions taken; for example, a result of retesting using a reference material;

• validation plan, in case reporting the result of validation of effectiveness in 30 days is not practicable.


Proficiency testing administration process

Planning

Make test item and test consistency

Apply to join

Distribute test item

Check test item and forward a delivery

note

Submit result

Preliminary report

Require corrective action

Final report

Take corrective action

Require corrective action Take corrective

action

End


KOLAS PT 2009-06

§ Inter-laboratory proficiency testing by CC evaluation facilitiesCategory Description

Period of time Apr.5.2010 ~ May.17.2010 (15 days)

Test method 2 evaluators from each facility (including at least 1 senior evaluator)

Evaluation facilities involved

KTL, KOSYAS, KSEL, and TTA

Test item for proficiency testing

Title Neo Watcher@ESM Package V4.0

Scope - ST: ST introduction, Conformance claim, and Security problem definition-Operational user guidance

CC version Common Criteria for Information Technology Security Evaluation Part 1: Introduction and general model, CCMB-2006-09-001, Version 3.1 Revision 1, 2006. 9. Common Criteria for Information Technology Security Evaluation Part 2: Security functional components, CCMB-2007-09-002, Version 3.1 Revision 2, 2007. 9 Common Criteria for Information Technology Security Evaluation Part 3: Security assurance components, CCMB-2007-09-003, Version 3.1 Revision 2, 2007. 9 Common Methodology for Information Technology Security Evaluation, CCMB-2007-09-004, Version 3.1 Revision 2, 2007. 9.


KOLAS PT 2009-06

§ Results

§ Things to improve:§ Increase test items§ Expand test material§ Involve more evaluators

Test item Evaluation Facility 1

Evaluation Facility 2



ASE_INT PASS PASS PASS PASSASE_CCL PASS PASS PASS PASSASE_SPD PASS PASS PASS PASSFinal result All results proved to be suitable


Proficiency tests managed by the APLAC

Calibration

Program Coordinator Starting Date Final Report Distribution

M021 Volume KAN May 2007 Status Uncertain

M022 Gauge Blocks ONAC December 2010 Late 2011

M024 Plain Plug Gauges KOLAS November 2009 Late 2011

M025 Hydraulic Pressure Standards Malaysia September 2009 Early 2011

Program Coordinator Starting Date Final Report Distribution

T061 Electromagnetic Interference VLAC Oct 08 End 2011

T074 Pb and Cd in Rice CNAS September 2010 Mid 2011

T075 Paralytic Shellfish Poison CNAS September 2010 Mid 2011

T076 Iron Ore CNAS August 2010 Early 2011

T077 Elements in Bovine Liver HKAS August 2010 Early 2011

T078 PAH in Sediment HKAS August 2010 Mid 2011

T079 Phthalates in Plastic TAF November 2010 Mid 2011

※ No program related to the CC or SW under progress.

Test


Proposition

§ Organizing a new working group§ Purpose: Inter-laboratory proficiency testing among CCRA

members§ To prepare:

§ Guideline for a proficiency testing: Process and method§ Test material for a proficiency testing


Thank You!E-mail : [email protected]

Korea Scheme for ISO 17025 Proficiency Testing in CC ... Lee.pdf · Korea Scheme for ISO 17025...

Documents

Transcript of Korea Scheme for ISO 17025 Proficiency Testing in CC ... Lee.pdf · Korea Scheme for ISO 17025...