51°EOQ CongressCompetitiveness through excellence

Challenge for EuropePrague 22-23 May 2007

THREATS TO THE CREDIBILITY OF ACCREDITED MANAGEMENT SYSTEM CERTIFICATIONS AND RELATED CORRECTIVE AND IMPROVEMENT

MEASURES

Lorenzo ThioneChairman EA

European co-operation for Accreditation

Purpose of accreditation

• Accreditation shall ensure that accredited certifications of management systems (quality, environment, information security, occupational health and safety, food safety, etc.) are reliable indicators of the capability of the certified organizations to consistently meet the objectives of the relevant standard (output focussed accreditation). For example:- Is an ISO 9001-certified organization giving confidence that it can consistently provide conforming product to its customers?

- Is an ISO 14001-certified organization committed and able to preventing pollution, meeting legislation, and continually improving?- Is an OHSAS 18001-certified organization committed and able to ensure health and safety in the working place?

- and similar.

Criticalities of management system certification

• Management system certification has many criticalities.

It is an indirect form of quality assurance (quality = ability of satisfying the pertinent economical and social needs), having the related advantages (straightforwardness and generality of applicability) and disadvantages (indirect guarantee, significant residual risk of non-quality).

• On the other hand, it has acquired a large diffusion and visibility (accredited ISO 9001 certifications worldwide exceeds nowadays 1.000.000 and ISO 14001 certifications exceed 120.000) and has induced great expectations in the social-economical context.

The threats to the credibility of

accredited certifications

• For many well known cultural and operational reasons, in the present European and international scenario, the risks of deceiving such expectations – causing loss of credibility of the entire third party conformity assessment “chain” (standards, accreditation, certification) and the collapse of the related infrastructure – are very high.

• It is thus necessary to identify the main weaknesses of the chain and define and implement the required corrective and improvement measures, starting from actions which could be implemented within the existing frameworks and practices and coming, if and as far as possible, to more substantial innovations representing a real breakthrough in the approach to accredited MS certification (re-engineering).

THE IMPROVEMENTS

• In principle, all the components of the “chain” shall be considered, namely:- management system standards, that can certainly be improved, although the main point is their substantial and non minimalist application by the user organizations;

- standard and criteria for certification bodies (CBs), to be improved and implemented under the aegis of the respective accreditation bodies (ABs);

- standards and criteria for ABs to be enhanced and enforced under the aegis of regional and international co-operations (e.g. EA, PAC, IAF).

* This presentation mainly deals with the issues concerning accreditation and certification. Moreover, most of it is devoted to the analysis of short/medium term improvements which can be adopted within the current culture and practices and represent, as such, the first line of defence against the incumbent risks. A few considerations, however, are put forward also concerning a possible substantial redesign of the approach to accredited certifications which might include, amongothers, a quite different relation between accreditation bodies and accredited subjects.

SHORT / MEDIUM TERM IMPROVEMENTS

1. Improving effectiveness of accreditation(to be achieved under the aegis of EA, IAF and others)

1.1 Competence of AB assessors

AB assessors shall be able to follow a process oriented

approach and shall be knowledgeable of technologies, products and processes pertaining to the sectors of social economical activities in which the accredited/applicant CBs issue their MS certifications.

Clear requirements must be defined in this respect in ad-

hoc guidance documents issued by the regional and international co-operations among ABs (e.g. EA, IAF).

The possibility should be explored of establishing a mechanism of “central” approval of the ABs lead assessors (at both regional and international levels).

1. Improving effectiveness of accreditation

1.2 Performance based assessment

The assessment should be focused on assessing the performance of the CBs, including recognition of “excellence”. This should not necessarily lead to the establishment of orders of merits among CBs but might imply diversified “treatments”.

1.3 Conduct of the assessment and reporting

Criteria for the optimal conduct of the assessment shall be defined, regarding aspects such as the processes to be assessed, persons to be interviewed, sampling criteria, etc. Also guidance for appropriate preparation of the assessment report should be provided.

1. Improving effectiveness of accreditation

1.4Surveillance mechanisms

Surveillance is, with no doubts, the core issue for improving the effectiveness of accreditation, provided it is carried out duly focusing on the output.Regular surveillance shall be enhanced with systematic review ofthe certification files. Extraordinary surveillance shall also be reinforced, particularly based on feedbacks from the marketplaceand stakeholders in general, regarding both office audits and witness audits. Also remote electronic surveillance and periodic self-evaluations submitted by the CB to the AB should be considered.Some kind of “proficiency test” should be developed to allow both to check the professionalism of the CBs (e.g. by circulation of “anonymous” case studies) and the effectiveness of the certified MSs.Moreover, within the surveillance activities, the possibility should be given to the AB to go to assess directly the compliance of the certified MS with the applicable requirements (concept of “validation” audits).

1. Improving effectiveness of accreditation

1.5 Sanction measures

The ABs must adopt more rigorous policies in terms of sanction measures to be applied where there is evidence accredited certifications not being effective. These may include imposing suspension or withdrawal of the certifications improperly released, as well as sanctions directly against the defective CB (intensification of surveillance in the concerned sectors, reduction or suspension of accreditation, etc..). To be really effective, the adopted sanctions should be transparent and duly publicized.

Financial sanctions could also be considered as effective sanction measures.

1. Improving effectiveness of accreditation

1.6 Issues arising from cross-frontier accreditation

It should be kept in mind that a significant fraction of “poor” MS certifications are related to accreditation of CBs in countries /economies different from that of the AB (due to difficulties of the AB of exerting proper oversight) and to situations of CBs havingtheir main premises in the country in which they are accredited and certifying under this accreditation in other countries (due to difficulties of the CB of exerting control on the local branches and of collecting appropriate feed back from the market).

The rules on “cross frontier” policy – issued by regional and international organizations – should try to limit these negative effects. Note that the best guardian of the credibility of accredited certifications is the local AB. Cooperation among ABs is thus the key issue to solve these problems.

1. Improving effectiveness of accreditation

1.7 Feedback mechanisms

ABs shall have proper mechanisms in place to

search for and obtain feedbacks from the direct and indirect customers of certified organizations.

This can be achieved by various means and some

kind of guidance could be useful in this respect.

The inputs from the feedbacks shall be integrated in

the accreditation policy and used accordingly.

Committees representative of the interested parties

should be properly involved.

1. Improving effectiveness of accreditation

1.8 Harmonization of the behaviour of the ABs

The role of the regional and international MLAs

Improving the effectiveness and harmonization of the operation of the signatory ABs – based on the enhanced rules as above – is a task pertaining to the supra-national organizations managing the Mutual Recognition Agreements (e.g. EA and IAF). In this regard,the following actions are needed:

- exerting more rigor in granting the status of AB member of supra-national organizations;

- improving the efficacy and consistency of the peer evaluations,making the process more continuous, incisive and reproducible;

- better implementing the principle of cooperation among ABs, within the MLA, as a fundamental support to harmonization.

2. Improving effectiveness of certification(to be achieved under the aegis of the respective ABs)

2.1 Competence of CB auditorsThe criteria for auditor qualification shall be deeply reconsidered with respect to the traditional practices, including reconsidering the added value of auditor certification. Real knowledge is required of the technical and managerial issues concerning the business sectors where the audited organization operates.The capability of conducting process oriented audits shall be assured and the existence of the necessary personal attributes shall be objectively demonstrated. The entire mechanism of auditor training shall thus be re-examined introducing – in addition to the obvious requisites on knowledge of MS standards and audit procedures – clear requirements on understanding and mastering of basic concepts such as organisational learning, capability maturity model and process oriented audit. Formal requirements of little substantial value, as the duration of the training courses, years of working, number of audits, etc.., shall not be any longer the (unique) basic reference for demonstration of auditor competence.

2. Improving effectiveness of certification

2.2 Conduct of the assessment / audit times

Criteria for the optimal conduct of the audit shall be defined regarding aspects such as the processes to be assessed, the timespent in each process, the type and number of persons interviewed, the kind of samples picked up, the notes made, etc.. Also guidance to the appropriate preparation of the audit report shall be provided.Concerning the audit times, the rigid numerical approach foreseen by Table of Annex 2 of IAF Guidance to ISO/IEC 62 shall be overcome. The CB shall demonstrate it has in place appropriate criteria to ensure – in all respects and circumstances – that it is able to display the manpower resources (man days) necessary for the appropriate conduct of the audit, based on risk assessment and on other applicable elements. It can be shown that this approach may lead, in some cases, to the allotment of audit times shorter than those derived from the above cited table but, in many other cases, it could require much longer times.

2. Improving effectiveness of certification

2.3 Counteracting “price” competition among CBs

Remedies are included in what specified above concerning the strengthening of accreditation rules to prevent unreliable CBs to obtain and maintain accreditation.

Also an educational action towards the organizations seeking “cheap certifications”, explaining the caveat they may face, could be beneficial.

Another aspect to be considered could be to require CBs subject to accreditation and organizations subject to certification to self declare that they comply, to the best of their knowledge, with the applicable requirements prior to seeking accreditation and certification, respectively. This would then provide a useful contractual basis for placing such parties in front of their responsibilities.

3. Improving accountability towards stakeholders

• The objective of improving the value and credibility of accredited MS certification can hardly be achieved without the awareness and the political and technical support of all the actors involved in the conformity assessment chain. Accreditation bodies shall therefore be fully accountable towards the interested parties (standardization bodies, Regulators, CABs, industry, consumers, etc.), promote their involvement and seek their cooperation.

• This already happens at the national level and should be fostered, as well, at the level of regional and international co-operations among ABs (e.g. EA and IAF).

• The role of the Advisory Boards (gathering the above stakeholders) should be thus confirmed and reinforced – where existing, as in case of EA – or such Boards shall be formed where not yet in place (see the IAF initiative of establishing an End User Advisory Board to be possibly further expanded, giving to the Board also a monitoring role of the functioning of the certification system operating under the umbrella of the IAF MLA).

4. Other actions

Within the improvement strategy being considered, other possible actions include:

* make clear that the guidance documents established at the regional and international level are mandatory – when intended for this purpose and issued as such – and strictly enforce their observance within the applicable MLAs.

* make best use (benchmarking) of the experience of the most advanced sector schemes (automotive, aerospace, telecommunications, etc..) which have tried (may be with only partial success) to be more output focused than the accreditation community has been so far.

* promote and develop the educational role of the supra-national co-operations about accredited certifications.

LONG TERM IMPROVEMENTS

The measurements

In parallel with the implementation of the improvement actions outlined above, we shall try to identify more substantial innovations to the approach to accreditation of MS certifications.

To this purpose, a better knowledge of the “output” is required and this can only be achieved by some kind of “measurements”.

Information on the “quality” of the output (namely on the real performance of the organizations owning accredited MS certifications) may be obtained by:

* collecting information from second party audits of large public and private purchasers and legal audits from regulators.

* collecting feedback from intermediate and final users of products and services.

* collecting information from media.

LONG TERM IMPROVEMENTS

Analysis of data

Based on data collected as above, it will be possible to identify the critical points and to determine the actions required to re-engineer the conformity assessment chain (organization → certification → accreditation).

It is likely that the investigations will confirm the adequacy of most of the actions outlined above but there could be the need for deep re-thinking of some issues.

As an output of the analysis, it will also be possible to definereliable means to “measure” the effective performance of certified MS and to have them implemented in the current practice.

LONG TERMS IMPROVEMENTS

Possible actions

These might include considering:

- a change in the relations between ABs and CBs, moving towards a stricter cooperation and partnership, including joint operation of market surveillance facilities.- a substantial lifting up of the conditions for obtaining and maintaining accreditation, not so much in terms of more numerousand stricter rules but of real and substantial implementation of the existing ones, with consequent reduction on the number of accredited CBs in the social-economical contexts where excessive and quality-detrimental proliferation has occurred.- the need for strongly regulating/limiting “price” competition, by implementing some kind of price control mechanism – based on suitable criteria – particularly when protection of fundamental rights of the society is directly or indirectly involved.

CONCLUSIONS

• A substantial re-design of the accreditation and certification processes – in the desired direction of rendering MS accredited certification a credible guarantee of the capability of the certified organizations to deliver the pertinent quality (economical, social) – cannot be based only on defining and enforcing better rules (this being a condition necessary but not sufficient), but implies a major cultural/political change.

• This, in turn, cannot be achieved without the consciousness and cooperation of all the interested parties, including the operators in charge of developing and disseminating the quality culture.

THANK YOU FOR YOUR ATTENTION