IAF Certification/ Registration Bodies’ Member Satisfaction Program September 19, 2003 Final...

www.burke.comwww.burke.com

Applying KnowledgeTM

Improving DecisionsApplying Knowledge

TM

Improving Decisions

IAF Certification/

Registration Bodies’

Member Satisfaction Program

IAF Certification/

Registration Bodies’

Member Satisfaction Program

September 19, 2003September 19, 2003

Final Report Summary

Applying Knowledge

Improving DecisionsTMApplying Knowledge

Improving DecisionsTM

www.burke.com

Agenda

Research Objectives

Executive Summary

ABs: Interacted and Satisfaction With

Accreditation Bodies’ Performance Analysis

Mutual Recognition (MLA)

Applying Knowledge



www.burke.com

Research Goals

Measure CRBs’ overall satisfaction with Accreditation Bodies.

Identify areas warranting improvement by Accreditation Bodies.

Examine the perceived value of MLA (Mutual Recognition)

Applying Knowledge



www.burke.com

Research Design

Web-based interviewing:

An email invitation is sent to CRBs which contains the URL where the web survey is administered.

Reminder emails are sent to non-respondents.

Respondent names and email addresses are provided by IAF members to Burke.

Survey is conducted in English, French, German, Spanish, Korean, Mandarin and Japanese.

Applying Knowledge



www.burke.com

Executive Summary

This 2003 program is the first comprehensive assessment of CRB perceptions worldwide.

Contact information was provided by over 40 Accreditation Bodies and the survey was conducted by the IAF.

Email invitations were sent by Burke with a message from the IAF asking for their participation.

In addition, two email reminders were sent.

Applying Knowledge



www.burke.com

Executive Summary Key Findings

We sent out 704 invitations. Of these, 99 completed the survey. This yields a response rate of 14%, fairly typical for an online survey.

Those CRBs completing the survey worked with 31 of the 43 Accreditation Bodies listed. On average, CRBs have been accredited by 3.36 ABs.

We asked respondents to name the two primary ABs with whom they work and we then gathered information about these two organizations.

We gave respondents the opportunity to rate other ABs at the end of the survey.

Applying Knowledge



www.burke.com

Executive Summary Key Findings (cont.)

Overall Satisfaction with ABs: Two-thirds of respondents (64.8%) are at least somewhat satisfied

with the Accreditation Bodies with whom they work.

However, only 13.7% were “very” satisfied.

Typically, we would expect 60 - 65% to be very satisfied.

The proportion of CRBs who express dissatisfaction (32.8%) is very high.

Applying Knowledge



www.burke.com


How to Improve AB Satisfaction Levels: We have conducted over 1,500 similar programs.

Historically, organizations would focus on customer complaints to direct improvement efforts.

The problem with this approach is that these issues might not drive customer/ stakeholder satisfaction or purchase.

Organizations then moved (10-15 years ago) to key driver analysis. Identify those issues driving customer satisfaction and focus on those.

The problem with this approach is that an organization might already perform well on several of the key drivers and thus focusing on them is a waste of those resources.

Applying Knowledge



www.burke.com


How to Improve AB Satisfaction Levels: We combine these approaches: we focus on key drivers of satisfaction but only

on those where customers rate your performance low. We call these issues “target” issues.

Improvement on target issues represents an organization’s best chance of improving overall satisfaction. The target issues identified include:

Completes accreditation procedure in a timely manner Responds quickly to requests for extensions of scope Has open, clear communication with me Treats me as a valued customer Provides services that are a good value for the money

Applying Knowledge



www.burke.com


How to Improve AB Satisfaction Levels: Several other issues should be monitored as they are nearly target

issues themselves:

Has a witness process that adds value Allows the CRBs to participate in the governance of the

Accreditation Body Has reasonable office audit fees

Applying Knowledge



www.burke.com


Mutual Recognition (MLA):

58.6% of CRBs believe having one accreditation body recognized worldwide is very valuable and 31.3% feel it is somewhat valuable.

This differs across regions:

North (84.6%) and South America (100%) and Australia (78.6%) believe one accreditation body recognized worldwide is very valuable.

In Asia (29.4%), Europe (53.1%) and South Africa (50%), the value is seen by fewer CRBs.

Somewhat fewer (71.7%) saw the value (very or somewhat valuable) of an IAF accreditation mark recognized worldwide.

Applying Knowledge



www.burke.com

Executive Summary

Key Findings (cont.)

Mutual Recognition (MLA):

While 61% see the MLA as at least somewhat valuable, over a third (35%) do not see the value.

More CRBs (45%) are dissatisfied with the progress of the ABs in achieving mutual recognition using MLA than are satisfied (40%).

Applying Knowledge



www.burke.com

Executive Summary Conclusions

It should be remembered that only 14% responded to our invitation to participate in the survey. They may, or may not, be representative of the entire CRB population.

This survey especially cannot be used to evaluate individual Accreditation Bodies as the base sizes per AB are just too small.

Among those who participated, the levels of dissatisfaction would be a matter of grave concern were this program sponsored by a commercial firm in a competitive market.

Applying Knowledge



www.burke.com

Accreditation Bodies Worked With and Evaluated

Applying Knowledge



www.burke.com

Four or More, 17.1%

Three, 8.1%

Two, 16.2% One, 58.6%

Q.1: Number of Accreditation Bodies By Whom They’ve Been Accredited

Mean = 3.36

Base: All Respondents: 99

Applying Knowledge



www.burke.com

Q.3: Overall Satisfaction With Accreditation Bodies

13.7

51.1

22.1

10.7

2.3

0

10

20

30

40

50

60

Per

cen

t

VerySatisfied

SomewhatSatisfied

SomewhatDissatisfied

VeryDissatisfied

Don't Know

Base: Total Accreditation Bodies Rated: 131

Applying Knowledge



www.burke.com

Performance Attribute Analysis1. What is important to our customers?

3. So what do we do now?

2. How are we doing?

Attribute LeverageAttributes that are found to be

drivers of satisfaction via correlation

Attribute LeverageAttributes that are found to be

drivers of satisfaction via correlation

Attribute PerformanceCRBs' attribute ratings evaluating performance.

Attribute PerformanceCRBs' attribute ratings evaluating performance.

Issue PrioritizationContrasting attribute leverage with attribute

performance to identify improvement opportunities

Issue PrioritizationContrasting attribute leverage with attribute

performance to identify improvement opportunities

Stated ImportanceCustomer importance ratings

indicate stated attribute importance

Stated ImportanceCustomer importance ratings

indicate stated attribute importance

Applying Knowledge



www.burke.com

Attribute Performance

Then rate how satisfied you are with each Accreditation Body’s performance on that attribute.

Not At All Very Important Important

1 2 3 4 5 6 7 8 9

Not At All Very Important Important

1 2 3 4 5 6 7 8 9

Please rate how important you feel each attribute is to you.

Very Very Dissatisfied Satisfied

1 2 3 4 5 6 7 8 9

Very Very Dissatisfied Satisfied

1 2 3 4 5 6 7 8 9

Applying Knowledge



www.burke.com

Q.4: Attribute Leverage- Most Important Performance Issues -

0.565

0.566

0.569

0.583

0.631

0.673

0.678

0.687

0.687

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Is fair in handling complaints from my clients orcompetitors

Has a complaint resolution process that is easy to dealwith

Completes accreditation procedure in a timely manner

Has a process for scope addition that helps my abilityto do business

Has an effective process for scope additions

Offers a fair complaint resolution process

Overall accreditation body's conduct of my assessment

Treats me as a valued customer

Has open, clear communication with me

Attribute Leverage (Impact On Overall Satisfaction)


Applying Knowledge



www.burke.com

Attribute Performance Deficiencies

2. How are we doing?

Attribute Performance CRB attribute ratings evaluating Accreditation Bodies’

performance. Showing proportion of customers getting less than they would like (dissatisfied).

Attribute Performance CRB attribute ratings evaluating Accreditation Bodies’

performance. Showing proportion of customers getting less than they would like (dissatisfied).

Attribute performance ratings show how Accreditation Bodies perform on each attribute in the eyes of its customers.

We provide the mean rating to give an overall assessment of AB performance.

The primary metric, however, is a “Deficiency” score which indicates the percentage of CRBs who give a 1-4 rating (on the 9-point scale) for their Accreditation Body’s performance.

Applying Knowledge



www.burke.com

Q.4: Accreditation Body Performance- Proportion of CRBs Giving a Bottom 4 Box (1 - 4) Rating –

Issues With the Highest Levels of Dissatisfaction

33.6

34.4

40.5

42.0

42.7

44.3

44.3

45.0

45.0

0 5 10 15 20 25 30 35 40 45 50

Witnesses an appropriate number of your audits

Completes accreditation procedure in a timely manner

Provides services that are a good value for the money

Treats me as a valued customer

Has reasonable office audit fees

Has a reasonable accreditation application fee

Has reasonable surveillance audit fees

Has a witness process that adds value

Has reasonable accreditation fees

Proportion of CRBs Who Rate ABs' Performance Low (1-4)


Applying Knowledge



www.burke.com

Performance Issue Classification & Prioritization-The Targeting Matrix -

Work here last;check to see if “over-delivering”or if able to redirect resources

ABs are under-performing

on these impactful attributes

Reduce deficiencies,but lower impact on satisfaction;

less “bang for the buck”

Must maintain:continuous improvement candidates

TARGETISSUES

Less ImportantStrengths

Less CriticalIssues

Important Strengths

AB Performance

More CustomersPerceiving Deficiency

Fewer CustomersPerceiving Deficiency

Weaker

Stronger

Att

rib

ute

Im

pac

t on

Sat

isfa

ctio

n

Focus on

these issues

Applying Knowledge



www.burke.com

Higher

Opportunity Identification

Important Strengths

Important Strengths

Less Important Strengths

Less Important Strengths

Less Critical Issues

Less Critical Issues

Lower Higher

Lower

Att

rib

ute

Im

pa

ct

On

CR

B S

ati

sfa

cti

on

Proportion of CRBs Giving a 1-4 Rating

1

2

3

4

5 Completes accreditationprocedure in a timely manner

6

7

8

9

10 Responds quickly to requests for extensions of scope

11

12

13

14

15

16 Has a witness process that adds value

17

18

19 Has open, clear communication with me

20 Allows the CRBs to participatein the governance of the Accreditation Body

21 Treats me as a valued customer

22 Provides services that area good value for the money

23

24

25

26

27 Has reasonableoffice audit fees

28

29

TargetIssues

TargetIssues


Applying Knowledge



www.burke.com

Mutual Recognition: MLA

Applying Knowledge



www.burke.com

Q.10a: Value of One Accreditation Body That Is Recognized Worldwide

58.6

31.3

6.11.0 3.0

0

10

20

30

40

50

60

Per

cen

t

VeryValuable

SomewhatValuable

Not TooValuable

NotValuable At

all

Don't Know


Applying Knowledge



www.burke.com

Q.10b: Value of an IAF Accreditation Mark That Is Used Worldwide

38.433.3

14.1

5.19.1

05

10152025303540

Per

cen

t

VeryValuable

SomewhatValuable

Not TooValuable

NotValuable At

all

Don't Know


Applying Knowledge



www.burke.com

Q.11: Value of Current IAF MLA (Mutual Recognition Arrangement)

30.3 31.327.3

8.1

3.0

0

510

1520

2530

35

Per

cen

t

VeryValuable

SomewhatValuable

Not TooValuable

NotValuable At

all

Don't Know


Applying Knowledge



www.burke.com

Q.12: Overall Satisfaction With the Progress of the Accreditation Bodies in Achieving

Mutual Recognition Using the MLA

5.1

35.4

29.3

16.214.1

05

10152025303540

Per

cen

t

VerySatisfied

SomewhatSatisfied

SomewhatDissatisfied

VeryDissatisfied

Don't Know


Applying Knowledge



www.burke.com

Action steps to move forward

Applying Knowledge



www.burke.com

• Communicate the results to the GA

• Establish team to address issues

• Communicate the study results to all members

• Communicate appreciation to the CRBs

• Communicate the establishment of the team to the CRBs

• Proactive Communiqué

• Seek input from members

• Identify corrective actions

• Communicate – develop consensus

• Execute

• Resurvey

Action steps moving forward

IAF Certification/ Registration Bodies’ Member Satisfaction Program September 19, 2003 Final...

Documents

Transcript of IAF Certification/ Registration Bodies’ Member Satisfaction Program September 19, 2003 Final...