Workshop on 3D Coordinate Metrology Certification CMS ...

Workshop on 3D Coordinate

Metrology Certification

CMS Certification Committee

July 26, 2011

CMSC 2011 Phoenix, Az.

Sign Up Sheets


• Recap of 2009/2010 activities & workshop

results

• Review of 2009/2010 work

– Refinement of Body of Knowledge

– Creation of Bylaws

– Accreditation Research

– Preliminary Market Study

– Request for Information from other organizations

– Development of skills study at CMSC 2010

Agenda


ContributorsActive Members

Angelo Beraldin National Research Council Canada

Gary Confalone ECM Global

Talion Edwards Boeing

Ulysses Green Retired - Boeing, GKN Aerospace

Randy Gruver Boeing

Pat Hammett University of Michigan

Alan Metzel Northrop Grumman Electronic Systems

Chuck Pfeffer FARO

Dave Sommers Babcock & Wilcox

Observers

Keith Bevan National Physical Laboratories – UK

Georgia Harris NIST, NCSLI


Response Cards

• Do NOT use the “GO” button

• Depress the button that correlates with your answer to a slide that has a question

• The green light will indicate that your response has been sent

• You will have 10 seconds to input your selection…watch the timer!


What is the average “high”

temperature in Phoenix during

July?


90 95 105

Hotte

r th

an a

?

0% 0%0%0%

What is the average “high” temperature in Phoenix during

July?


1. 90

2. 95

3. 105

4. Hotter than a ?

10

What industry best describes

your company?


Gove

rnm

ent/M

ilita

ry

Aer

ospace

Auto

motiv

e

Metr

ology

Har

dwar

e/Soft

w...

Serv

ice

provider

Scie

ntific

/Res

earc

h Educa

tion

Pow

er gene

ratio

n

Ship

buildin

g

0% 0% 0% 0%0%0%0%0%

1. Government/Military

2. Aerospace

3. Automotive

4. Metrology Hardware/Software manufacturer

5. Service provider

6. Scientific/Research Education

7. Power generation

8. Shipbuilding

What industry best describes your company?


10

What is your job title?


Sal

es

Met

rolo

gy Tec

hnic

al

Man

agem

ent

Oth

er

0% 0%0%0%

1. Sales

2. Metrology

Technical

3. Management

4. Other



10

How much experience do you

have in Metrology?


None

1-5

6-1

0

11-

20 2

1+

0% 0% 0%0%0%

How much experience do you have in Metrology?

1. None

2. 1-5

3. 6-10

4. 11-20

5. 21+


10

• Industry Requested Certification

• Potential Growth

Nuclear – Increase in production and reset

Aerospace – Extended Service Life

Automotive – Efficiency requirements

• Contractual Requirements

• Standardized Practices

• Customer Demand of Qualifications

• ISO certification requires standard qualification

Need


2009/2010 Workshop Attendees -

Industry

What industry best describes your company?

0%

10%

20%

30%

40%

50%

Gove

rnm

ent/M

il...

Aero

space

Auto

motiv

eM

etrolo

gy

Ha...

Serv

ice p

rovi

der

Scie

ntif

ic/R

es...

Pow

er genera

...

Ship

build

ing

Perc

en

t

2009

2010


2009/2010 Workshop Attendees –

Job Title

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

Pe

rce

nt

Sales Metrology

Technical

Management Other


2009

2010

Phoenix, Az.CMSC 2011

2009/2010 Workshop Attendees –

Experience

0.00%

10.00%

20.00%

30.00%

40.00%

Pe

rce

nt

None 1-5 6-10 11-20 21+

Years of Experience

How much experience do you have in Metrology?

2009

2010


Have you ever attended a

certification committee

presentation at CMSC?


Have you attended a certification committee

presentation at CMSC

Louisv

ille 2

009

Reno 2

010

I’m

a n

ewbie

0%0%

0%0%

1. Louisville 2009

2. Reno 2010

3. I‟m a newbie


10

Review of 2009 Activities

• Formation of committee

• Research existing certifications

• Survey of CMS membership (2009 conf)

• Development of preliminary Body of

knowledge (BOK)


Original Assumptions

• Need to justify the need

• Equipment operators and data processors

are target audience

• CMS should investigate a partnership in

administering certification

• Training provided by 3rd Party

(manufacturers, service providers,

academia, national institutes)


Committee Bylaws

• Established Charter

• Defined committee membership

• Defined reporting responsibility

• Updated CMS Bylaws to make certification

committee a standing committee


Accreditation Research

• ISO 17024 Personnel Certification

• ANSI accredits to ISO 17024 in US

• Attended ANSI workshop on ISO 17024

• The Council of Engineering & Scientific

Specialty Boards (CESB)


Market Research

• Baseline with CMS membership

• Also contacted UAW International through

CMS automotive connected members

• Contacted each represented company by

phone

• Asked to provide how many 3D

metrologists worked at the facility

• Total number of users from survey: 2407

CMSC 2011 CMSC 2011

Portable 3D Metrology users by sector

64.37%

20.71%

5.36%

3.95%

1.00%

2.54%2.08%

Aerospace Automotives Shipbuilding Misc

Nuclear H. Equipment Measurement

Services


Body of Knowledge

• Started with developing hardware categories

• Listed knowledge and skills a mid level operator

should have to be proficient with each type of

hardware

• Sorted each list for common requirements


Core and hardware specific

Certification should include both general certification and endorsements for specific hardware groups

•Core Specific

Common characteristics to all technologies.

•Hardware Specific

Unique characteristic to each technology.


Hardware Specific

• Laser Tracker

• PCMM

• Photogrammetry

• Area Scanner

• Hand Scanner

• Spherical Scanner

• Post Processor

CMSC 2011 CMSC 2011

What type of equipment do you

usually use?


What type of equipment do you usually use?

1. L

aser T

rack

er

2. P

CMM

3. P

hotogr

amm

etry

4. A

rea Sc

anner

5. H

and Scanner

6. S

pherical S

canner

7. P

ost P

roce

ssor

0% 0% 0% 0%0%0%0%

1. Laser Tracker

2. PCMM

3. Photogrammetry

4. Area Scanner

5. Hand Scanner

6. Spherical Scanner

7. Post Processor


10

2009 Workshop Feedback

• 80% of those in attendance learned OJT

• 62% recommended proficiency testing

• 80% recommended graduated levels

• Accreditation was strongly recommended


• A properly structured certification program would be of definite value.

• Currently equipment must be calibrated but the operator, the greatest

potential source of error, is not required to be certified.

• Certification should be multi-level to delineate degree of capability and

responsibility.

• Certification should indicate mastery of a core body of knowledge with

additional certifications for equipment/software.

• Hardware/software certification should demonstrate appropriate technical

knowledge as well as proficiency.

• There should be certified examiners for each hardware group.

• There may be areas where certification would be application specific.

Conclusions drawn from 2009 Work


Review of recent Activities

• 2010 CMSC workshop

• Negotiate an alliance with selected RFI

respondent

• Market CMS certification by presenting at

complimentary organization‟s conferences

– SME AeroDef 2011

– Quality Conference 2011

– Hexagon 2011


(CMSC 2010) Skills Development

• Committee developed idea to perform

statistical study at CMSC

• Study would tie to identified skill gaps in

general metrology community

• Workshops at CMSC would relate content

to data developed in study


Request for Information

• Initial formal contact with possible partners

• Included background information on CMS and Certification Committees

• Description of work to date and CMS resources.

• Request information about specific criteria related to interest in 3D metrologists cert

• RFI sent to 5 industry organizations with related certification programs


RFI Components

• Introduction to the Coordinate Metrology Society

• CMS Certification Committee

• Conclusions from Certification Committee work

• Coordinate Metrology Society resources

• Statement of Work and information requested


• Over 1600 Members

• Quarterly Newsletters sent to 20,000 through CMS Media

Partner Quality Digest

• Semi Annual Publication “Journal of CMSC”

• Partnerships & media exchange with SME, SPAR, LVMC

• Conference attracts over 45 vendors

• CMS membership includes Training and Service Providers

• CMS Annual Conference is attended by over 550 metrology

professionals

CMS Resources


Information Requested

• Does this certification fit within your organizations profile?

• How is your organization credentialed or accredited to deliver

personal certification?

• What experience does your organization have in developing and

delivering personnel certification?

• Do you currently have the staff and resources to support additional

certifications?

• Does your organization have a mechanism to partner with an

organization such as CMS to provide technical content for the

development of a new certification?

• How do you see CMS playing a role in the development of the

proposed certification?


Solicited Organizations

• American Society for Quality (ASQ)

• American Society for Nondestructive Testing

(ASNT)

• North American Coordinate Metrology Association

(NACMA)

• ASTM International E57 Committee

• American Society of Photogrammetry and Remote

Sensing (ASPRS)


Request for Information

Responses


Rating the RFI responses

(categories)

• Fit with portfolio

• Credentials

• Experience

• Support staff

• CMS role

• Practical element

• Overall structure


CMS selected:

American Society of

Photogrammetry and

Remote Sensing (ASPRS)


RFI respondent summary

• ASPRS (American Society of

Photogrammetry and Remote Sensing)

– Timely response

– Resources to support

– Experience administering certifications

– Willing to partner with CMS

– Applied for CESB accreditation


ASPRS Meeting


Mike Renslow

ASPRS Certification

Chairman


ASPRS Certification

ProgramCoordinate Metrology Society

Annual Conference

Mike RenslowJuly 26, 2011


Topics

• Brief Overview of ASPRS Certification

• Brief History of the Certification

Program

• The ASPRS Certification Process

• Preparation for Certification

• Re-Certification

• Future Programs


ASPRS Committee Organization

• 20 Standing Committees

– Represents Every Aspect of the Spatial Sciences and the „Practice‟ Issues

• „Evaluation For Certification Committee‟

Established in 1975

• History of Certification (~700 active)

– 1502 as Photogrammetrist

– 190 as Mapping Scientist in Remote Sensing

– 241 as Mapping Scientist in GIS/LIS


Background of Certification• Voluntary Program Open to Members and

Non-Members

• Originally Established in 1975 for „Certified Photogrammetrist‟ and Expanded for MS-RS and MS-GIS/LIS in 1991

• Added the Certified Technologist Program in October, 2003 for 3 Specialist‟s Areas

• ASPRS Manages Certification Nationally and Internationally

• Provisional Certification in-place since 2007

CMSC 2011 Phoenix Az.

Certification Program Structure• Peer Review Process (1st Step)

– Evaluation of Professional Experience, Training,

Activities, and 4 References

– Reviewed by Evaluation Sub-Committees

2 Review Committees (Photogrammetry & Certified Mapping

Scientist)

Points Awarded Based on the Application

• Examination Process (2nd Step)

– In-Place Since January 1999

– Customized By Specialty Area

• Knowledge of All 3 Technical Areas is Required

– Photogrammetry, Remote Sensing, GIS/LIS, plus,

Standards, Ethics, and Practice Issues


Certification Designations• Certified Photogrammetrist (ASPRS)

• Certified Mapping Scientist, Remote Sensing

(ASPRS)

• Certified Mapping Scientist, GIS/LIS (ASPRS)

• Certified Photogrammetric Technologist

(ASPRS)

• Certified Remote Sensing Technologist

(ASPRS)

• Certified GIS/LIS Technologist (ASPRS)


Experience RequirementsCertification Type Minimum Total

Experience

Professional

Experience

CP 6 Years 3 Years

MS-RS 6 3

MS-GIS/LIS 6 3

Specialty

Experience

P-T 3 2

RS-T 3 2

GIS/LIS-T 3 2

Plus………CMSC 2011 Phoenix, Az.

General Certification

Requirements• Completed Application with Fee

• 4 References with Knowledge About Your Work and Experience

• Compliance with the ASPRS Code of Ethics

• Peer-Review by Sub-Committee

• Successful Completion of a Written Examination


Examination Requirement• Final Step in Certification

• Designed to Assure a Minimum Level of

Technical and Professional Knowledge

• Covers a Variety of Focus Areas with

Emphasis on the Specific Certification

• Format for Professional Certification Exams

100 Questions, Multiple-Choice, Pass/Fail

May Have – Hand-held Calculator & One Bound Reference

May Not Have – Laptop, Cell Phone, PDA

4 Hour Time Allotment – Proctored Environment


Question Matrix

Focus Area Photogrammetrist Mapping Scientist

GIS/LIS

Mapping Scientist

Remote Sensing

Math/Science 13-15% 14-16% 14-16%

Engineering/Surveying 6-7% 5-6% 5-6%

Physics 9-11% 7-9% 9-10%

Imaging 12-15% 18-20% 25-28%

Photogrammetry 25-27% 10-13% 10-14%

GIS 12-15% 21-23% 12-15%

Ethics/General 15% 15% 15%

Table Extracted From the ASPRS Web

Page


ASPRS Technologist Program• Newer Program Designed for Entry-level and

Technician Positions– Perhaps a Stepping Stone to „Professional‟

– Recognition for Career Technical Positions

• Similar to the Professional Certification, but Emphasizing Technical Tasks and Minimal Project Management Responsibility

• Implemented in October 2003

• Similar Track Requiring Peer Review and An Examination– Application and Processing Fee Required

– Examination of 50 Questions with 2 Hour Time Limit


Preparation For Certification• ASPRS Workshop “Preparing for ASPRS

Certification”– Offered at Spring and Fall ASPRS Conferences

– Can be Offered at Regional ASPRS Meetings

– Webinars every 3 months

• Maximizing the ASPRS Web Site Resource– On-line Fillable Forms, Instructions, References

Forms, Information

• Applicants Advised To Seek Support from Associates– Mentoring is Encouraged


Re-Certification Program• „Active‟ Status Requires Re-Certification

– Every 5 Years for Certified Professional

– Every 3 Years for Certified Technologist

• Submit Completed Application with Fee

• 4 References

• No Written Examination Required

• Sub-Committee Evaluation– Document Experience and Activity During the

Previous 5 Years

– Must Satisfy Minimum Requirements


Re-Certification Criteria

• Up to 20 Points– Active in area being re-certified, or in academic arena involved

directly with the subject matter

• Up to 8 Points– Participation in panels, presented or published papers

• Up to 8 Points– Attended/taken workshops or classes in related subjects

• Up to 4 Points– Attended technical conferences


Certification Status• Active

– Currently Certified or Re-certified

– Fully Certified Status

• Inactive

– Certified, but not Re-certified, and Practicing

– Cannot Use „Certified‟ Title

• Retired

– Certified, Not Re-Certified, and No Longer

Practicing in the Field

– Cannot Use „Certified‟ Title


On-Going Certification Activity• Certification „Marketing‟ Effort

• Maintain the ASPRS Web Site

– New Information Twice a Year

– New Forms and Information

• New Editions of the Exam

– New Question Bank with New Technology

Questions

– Maintain Exams for Each Specialty

– In-Place since Jan. 2005


On-Going Activity• Continuing Marketing / Outreach Program

• Interaction with State Licensure and NCEES Model

Law Implementation

– Partnering with the Colonial States Board for Licensure

of Land Surveyors to Develop a New Exam for Licensing

Photogrammetrists

– Exam Ready for the States in Spring 2007

– NCEES Houses the Exam Items and Provide

Administration

• Update the ASPRS List of Certified Individuals

– On the Web Site since the end of 2005

Accreditation• ASPRS Membership in CESB

– „Council of Engineering and Scientific Specialty Boards‟

– Membership as a Board Member in January 2011

• Full Accreditation for all Certifications

– Application Process underway (March 2011)

• Minor program adjustments to meet CESB Guidelines

– Accreditation in place in late-summer 2011

– The Only Accredited Geospatial Certification

• All previous certifications accredited

– All new certificates will have both the ASPRS and CESB

logos


Provisional Certification• Available to all graduating Students (within 1 year

of graduation)

• Same Requirements- different timeline– Application, Partial Fee, 1 Reference, Pass Peer-review,

and Pass the Exam

– Achieve Experience Requirement• 10 years for Professional Certification / 5 years for Technologist

Certification

– Submit Experience Application, Complete Fee, and 3 references & Pass Peer-review

– Become fully certified


Information

• Mike Renslow

– Chair, Evaluation for Certification Committee

– [email protected]

– 541-335-1251

• ASPRS

– www.asprs.org / certification


mailto:[email protected]

Would you volunteer your time to

support a sustaining effort?


Would you volunteer your time to support a

sustaining effort?

Yes N

o

0%0%

1. Yes

2. No10


New Assumptions

• We should pilot an initial

“Level 1” certification

• The Level 1 certification should be

cognitive, and encompass a core body of

knowledge

• Proficiency testing should follow, as “Level

2” certifications

• Levels system must be developed in

respect for limitations due to work rules


Recommendations

• Basis: Trade Study, Survey, 2009/2010 Workshop

• CMS to support creation of required BOK

• CMS to identify partner organization

• CMS to provide ongoing support of the BOK

• Assist developing the market potential to support

certification

• Give input to 3rd party training providers


WHERE DO WE GO FROM

HERE?


•Formalize relationship

•Provide high level plan for the path forward

•Define resource commitments

•Define responsibilities

•Define Liaison relationships

Memorandum of Understanding


• Reconfigure the CMS Certification

Committee structure to efficiently and

effectively support the Certification

Program and Processes

Certification Committee

Structure



CMS Executive

Committee

Accredited Partner

Governing Body

Memorandum of

Understanding

CMS

Certification

Executive

Committee

Business Plan

Partner

Certification

OrganizationEthics Subcomm.

Certification

Administration

Body of

Knowledge

Question

Database

Proficiency Test

Oversight

Know Ye

By These

Presents

That ….

Training

(3rd Party)

TESTING

Liaison

• Reconfigure the CMS Certification

Committee structure to efficiently and

effectively support the Certification

Program

Certification Committee

Structure


SME for each identified Body of Knowledge to:

• Fully develop the subject BOK

• Develop the requisite Training Outline (TO)

for that BOK

• Determine interrelation of the BOK‟s

• Develop examination materials based on

the BOK and TO

• Assist in validation of examination materials

Recruit Subject Matter Experts


Outline committee meeting rhythm and time

commitment provide details…

Recruit Subject Matter Experts


Would you be interested in

contributing to a Body of

Knowledge?


Would you be interested in contributing to a Body

of Knowledge?

Yes N

o

0%0%

1. Yes

2. No

10


Overview of Workshops

and Study


Open measurement workshop inviting conference delegates

to participate in a measurement study

• Subject: First principle measurement study based on a variety of

„hand tools‟ used in dimensional measurement

• Objectives:To look at the importance of

• core measurement principles

• instilling the right measurement strategy

• observing behaviours when dealing with measurements

• instilling questioning

2010 measurement study


Activity

• The measurement studies were to be undertaken over 2 days in 2

separate areas using various defined first principle tasks.

• The criteria of the tasks was modified to allow for various training &

assessments techniques to be undertaken such as:

– Assessment of prior learning and experience

– Questioning techniques

– Practical task monitoring

– Demonstration


4 measurement tasks with and without instruction covering diameters,

length, depths and gauge block wringing

• Task 1: Without procedures 55 readings ranged from 0,155 to

0,191mm

• Task 2: Without procedures 43 readings ranged from 0.0015 to 1.21

inches

• Task 3: With procedures 39 readings ranged from 0,009 to 0,057mm

• Task 4: Without procedures 28 readings varied by 0.0045 inches

• Key observations and behaviours across a wide range of expertise

Data


Open measurement workshop inviting conference delegates to participate

in a 3D co-ordinate measurement study on ‘How behaviours impact your

Measurement’

Objectives: Over a 3 day period of the conference participants are

• Days 1 & 2: Invited to undertake 3D co-ordinate measurements by a

variety of methods whilst looking at underpinning measurement core

principles and behaviours.

• Day 3: Invited to the measurement study workshop where the findings of

the study will be presented by the study group.

2011 CMSC measurement study

Sponsored By Metrologic


2011 CMSC measurement studyhttp://www.youtube.com/watch?v=o0j6KOGkBMU



http://www.youtube.com/watch?v=o0j6KOGkBMU

Activity:

– Day 1 (Tuesday): 3D measurement tasks with little or no instruction where the

participant will need to determine the requirements to take measurements or

instruct the operator as to the measurements needed. The resultant data will

be stored in an appropriate system

– Day 2 (Wednesday): 3D measurement tasks with instructions or procedures

where the participant will be required to take measurements or instruct the

operator as to the method to collect the measurements needed. The resultant

data will be stored in an appropriate system

– Day 3 (Thursday – PM): Workshop presentation of the observations and data

from the measurement study.

The criteria of the tasks will allow for various training & assessments techniques

to be undertaken such as: Assessment of prior learning and experience,

questioning techniques, practical task monitoring and demonstration

2011 CMSC measurement study


CMSC 2011 CMSC 2011

How many of the other

workshops do you plan to

attend?


How many of the other workshops do you plan to

attend?

0 1 2 3

0% 0%0%0%

101. 0

2. 1

3. 2

4. 3


Feedback

After seeing the results of the survey and

certification research:


Was this workshop valuable to you for

understanding the components of

developing a certification?


Do you agree?

Yes N

o

0%0%

1. Yes

2. No

10

CMSC 2011Phoenix, Az.

Did the workshop give you a

good picture of the Committee‟s

work to date?


Do you agree?

Yes N

o

0%0%

1. Yes

2. No

10


Would you personally pursue a

certification based on the

committees recommendations?


Do you agree?

Yes N

o

0%0%

1. Yes

2. No

10


Thank You!


Workshop on 3D Coordinate Metrology Certification CMS ...

Documents

Transcript of Workshop on 3D Coordinate Metrology Certification CMS ...