How to Develop and Administer a Job Analysis Survey€¦ · How to Develop and Administer a Job...

How to Develop and Administer a

Job Analysis Survey

Presented by Dr. Jim Higgins

Executive Director, BCGi

Biddle Consulting Group Institute for

Workforce Development

• BCGi Memberships (free): ~7000+ members /

15,000 HRCI credits to-date

– Online community

– Monthly webinars on EEO compliance topics

– EEO Insight Journal (e-copy)

• BCGi Platinum Membership (paid)

– Includes validation/compensation analysis books

– EEO Tools including those needed to conduct AI analyses

– EEO Insight Journal (e-copy and hardcopy)

– Access to the BCGi library of webinars, training materials, and much more …

• www.bcginstitute.org

A few housekeeping items:

• Webinar slides have been sent out, if you didn’t receive them in your

email please follow up with us at [email protected]

• HRCI & SHRM credits are no longer offered for free through BCGi,

to get the credits you must register through both the BCGi website

and GoToWebinar. The cost is $99 and you receive both credits, if

you meet their requirements.

• Webinars are still offered for free!

• This webinar, along with all of our previously recorded webinars, is

accessible to BCGi Platinum Members on our website.

mailto:[email protected]

How to Develop and

Administer a Job Analysis

Survey

Dr. Jim Higgins, Ed.D.

Executive Director

BCGi

Definition of Job Analysis

• A detailed statement of work behaviors and other information relevant to the job (Uniform Guidelines on Employee Selection)

• A purposeful, systematic process for collecting information on the important work-related aspects of the job (Gatewood & Field, 2001)

Why Conduct a Job Analysis?

• Meet legal and professional standards for employment

• Select the best qualified candidates

• Use Job Analysis to make employment and management decisions

7

The Important Uses for a Job Analysis

Job Analysis

Testing

Screening

Appeals

Evaluating

Promoting

Recruiting

Legal Background

Title VII, Equal Employment Opportunity, of the Civil Rights Act of 1964 states in part:

– SEC. 703. (a) It shall be an unlawful employment practice for an employer:

o to exercise any employment-related decisions resulting in discrimination due to an individual’s or group’s race, color, religion, sex, or national origin and to use a selection instrument that results in adverse impact unless the employer can demonstrate the job-relatedness of the procedure

Case Law

Griggs vs. Duke Power Company (1971)

• Ruled that selection instruments had to be demonstrably related to job performance (i.e., job-related)

• Found that the absence of the intent to discriminate does not justify use of selection procedures that resulted in discrimination

• Established that companies could not use artificial, arbitrary, and unnecessary obstacles to employment that serve as discriminatory barriers

Case Law

Albemarle Paper Company vs. Moody (1975)

• Ruled that employment tests must be “predictive of—or significantly correlated with—important elements of work behavior which comprise or are relevant to the job or jobs which candidates are being evaluated”

Case Law

Kirkland vs. New York State Department of Correctional Services (1983)

• Established that identifying critical tasks and skills as well as the

competency required to fulfill the various aspects of the position is

an essential part of job analyses

• Ruled that the foundation of a content valid examination is the job

analysis

Uniform Guidelines on Employee Selection

Procedures Requirements (1978)

• Adopted by the Department of Labor, the Equal Employment Opportunity Commission, the Civil Service Commission, and the Department of Justice

• Promotes a uniform set of principles to help employers comply with Federal laws that prohibit discrimination

• Establishing guidelines for the proper validation and use of selection procedures

Standards for Educational and

Psychological Testing (1999)

• Prepared by a joint committee (American Psychological Association, American Educational Research Association and the National Council on Measurement in Education)

• Provides assessment professionals with guidelines for the evaluation, development and use of testing instruments

– “When evidence of validity based on test content is a primary source of validity evidence in support of the use of a test in selection or promotion, a close link between test content and job content should be demonstrated.”

Americans with Disabilities Act (ADA)

• The ADA requires that employers evaluate candidate capabilities

in relation to their ability to perform the essential functions of

the job and prohibits discrimination against individuals with

physical or mental disabilities

Validity

• The extent to which a test measures what it purports to measure

• Content Validity: extent to which a measurement reflects the specific intended domain of content

– Job Analysis is a process for establishing Content Validity!

Job Analysis Process:

1. Determine the scope of the job analysis

2. Determine the procedures

3. Draft preliminary Task and KSAPC statements based on archival research

4. Identify subject matter experts (SMEs)

5. Conduct job audits

6. Edit previously drafted Task and KSAPC statements

Job Analysis Process:

7. Finalize Task and KSAPC statements

8. Organize the Task and KSAPC statements into logical content areas

9. Obtain SME ratings regarding Task and KSAPC statements

10. Link critical Task and KSAPC statements

11. Select employment selection method(s)

12. Create an examination outline

13. Document processes in a job analysis report

TEST

CONTENT

The Job

JOB ANALYSIS

TASKS KSAs Task/KSA

Links

Test

Content/KSA

Links

Job Analysis Process - Overview

Collecting Job Analysis Information

• Subject Matter Experts (our best resource)

• Archival Research

• Job Audits / Interviews

• Job Observations

• Committee-Based Data Collection

Definition of Subject Matter Experts

• Subject Matter Experts (SMEs) are expert-level incumbents and

supervisors, knowledgeable about the duties and requirements

of the job.

• SMEs should be top performers in the job, first-level

supervisors or hiring managers

• SMEs have direct knowledge of the job and know what is

required to succeed in the job

Subject Matter Experts

Gather a representative sample:

– Gender

– Race

– Age

– Length of time at job

– Work location (e.g., field office)

– Different divisions or work units

– Shift work, i.e., morning, swing, evening (if applicable)

Archival Research

• Written information pertaining to the job

– Job descriptions/class specifications

– Rules and regulations

– Training manuals

– Performance appraisals

– Previous job analyses

– Web sites (O*Net, opm.gov, job-analysis.net)

• Collect and review prior to meeting with SMEs

Job Audits / Interviews

• Most widely used method for data collection!

• Gather information from SMEs

– Face-to-face meeting

– Phone interview

• Typically conducted at incumbent work location


• Ask SME to describe the work that they perform in their own words – Have SMEs describe a typical day and then month

– Have SMEs list major functions and then tasks

– Probe, probe, probe!

• Identify duties (key, major responsibilities) and then work down to tasks for each)

• Begin to prepare tentative list of Tasks and KSAPCs


Interview Questions:

• Tell me what you do in a typical day?

• What is important about your work?

• What tasks are performed most frequently?

• What must an incumbent be able to perform at entry to the job?

• Why is the work performed?

• How do you do this work?

• What is your role?

Tips for Conducting an Audit

Brainstorm KSAPC information

• “What knowledge skills, and abilities are needed to perform the critical tasks of this classification?”

• Review tasks with the SME(s), then ask them what they need to know to complete the tasks

• “How do your duties differ from the classification just below yours? Above yours?”

– Technical knowledge

– Tools and equipment

– Reading materials

– Software knowledge

– Interpersonal and communication


• Establish rapport

• Maintain interest (watch non-verbal cues)

• Review your written information for clarity, completeness and understanding

• Ask open ended questions

• Walk through the worksite; look at any equipment used by incumbents


More Tips

– Be courteous

– Explain what you are doing and why

– Maintain comfort level

– If interviewing, use a protocol

– Remember the incumbent is the expert

– Keep the incumbent on track

– Provide feedback

When are you finished with your audit?

• SMEs begin to repeat themselves

• SMEs begin to identify overly detailed tasks and KSAPCs

• SMEs begin to identify insignificant tasks and KSAPCs

• SMEs have no more information to provide

WHEN YOU ARE CONFIDENT THAT YOU HAVE MET YOUR

OBJECTIVE!

Job Observations

• Observe SMEs who are actually performing their job

• Serves as an unobtrusive data collection technique

• Most appropriate for jobs that require:

– Field work

– Manual labor

– Interpersonal communication

Job Observations

Observation Questions:

– What are you doing?

– Why are you doing it?

– What information and training is required?

– What equipment is used, and could you demonstrate it?

– Are there different ways of doing this?

– What is the consequence of error?

Committee-Based Job Analysis

– What if there are not enough Subject Matter Experts (SMEs)

available?

– What if the objective of the job analysis is to establish a new

classification?

Committee-Based Job Analyses

• Committee-based Job Analyses are performed when a lack of staff or

SMEs prevents a typical job analysis from being performed

– A panel of employees from all levels of the organization is

assembled

– Expectations of the job are developed

– The panel develops a list of tasks and KSAPCs, then rates each

statement in terms of position criticality

So which data-gathering method should you

use?

• Its not necessary to use all the data-gathering methods for

every job analysis

• Simply choose the best method for getting the data necessary to

achieve your desired goal

TEST

CONTENT

The Job

JOB ANALYSIS

TASKS KSAs Task/KSA

Links

Test

Content/KSA

Links

Job Analysis Process - Develop Task Statements

Task Statements

• A discrete sentence containing one action verb that concisely

describes a single observable activity

• Address important aspects of the job

• Task statements are drawn from the information you collect in

archival research, interviews, observations, etc.

Task Statements

Five elements

1. Action verb

2. Object of the verb

3. Rationale or expected output of the action

4. Materials, tools, procedures, or equipment used

5. Under what direction or guidelines

Example Task Statement

Task Components

1. Action verb

2. Object of the verb

3. Expected output (purpose) or work product

4. Materials, tools, procedures, or equipment used

5. Under what direction or guidelines

Transcribe board meetings discussions to maintain Record of Minutes

1 2 3

using transcription equipment at the request of the Manager.

4 5

More Task Statement Examples

Enter employee training requests into online personnel tracking

1 2

system using database software in order to keep up-to-date 4 3

employee development records in accordance with departmental

5

training delivery policies.

More Task Statement Examples

Perform routine tractor maintenance

1 2

by checking fluids (e.g. coolant, oil, power steering etc.), examining seals,

4

pistons, and chambers in order to identify safety issues and/or replace

3 necessary equipment in accordance with the XYZ maintenance repair and

5

safety checklist.

• You may find it difficult to include all five elements into

every task statement within a job analysis

• What’s important: Observable behaviors and work

products

Task Statements

Task Statements

• Task statements should be written at the same level of specificity.

Avoid being overly broad or overly specific

– What is insufficient information in a task statement. . .

Poor Answer office correspondence.

Fair Write replies to letters asking for job information using the computer.

Better Compose written responses to inquiries regarding job openings using

word processing software to provide application information within

departmental eligibility guidelines.

Task Statements

…and what is too much information in a task statement?

Poor Prepare travel form #625(a) in triplicate giving airline flight numbers

and hotel contact information to obtain travel advances and distribute

one copy each to supervisor, manager, and file twelve days prior to

departure date using #2 lead pencil at the request of staff.

Better Complete travel forms in a timely manner to obtain travel advances

following departmental guidelines at the request of staff.

Task Statements

Essentially, Tasks should:

– Start with an action verb

– Describe work done by one person only

– Give mention to an identifiable product or outcome

– Be specific enough to derive associated KSAPCs

TEST

CONTENT

The Job

JOB ANALYSIS

TASKS KSAs Task/KSA

Links

Test

Content/KSA

Links

Assessment Development Process - Develop KSAPC Statements

KSAPC Statements

KSAPC: Knowledge, Skills, Abilities, and Personal Characteristics

– Derived from essential job tasks

– Address a single related knowledge, skill, ability, or personal characteristic

– A single task statement may generate several KSAPCs, and each KSAPC statement may apply to more than one task

– Every task should have a minimum of one KSAPC

Note: KSAPC statements must be operationally defined, meaning they provide enough information to be consistently understood by any reader, with minimal ambiguity or confusion.

KSAPC Statements

Three elements

1. The knowledge, skill, ability, or personal characteristic required.

2. Degree of proficiency or level used

3. How/why the KSAPC is used or under what circumstances (context).

Example

Knowledge of multivariate statistics (factor analysis, discriminant

1 2

analysis, logistic regression) to identify population characteristics

3

and make relevant predictions.

KSAPC Statements

Knowledge:

– A body of information applied directly to the performance of a function

– Facts, procedures, etc.

Skill:

– A present observable competence to perform a learned psychomotor act

– Manual manipulation – with precision

Ability:

– Present competence to perform an observable behavior or a behavior that results in an observable product

– Describe capacity to perform activities, typically cognitive

KSAPC Statements

Knowledge of basic spreadsheet software functions (e.g., data entry, arithmetic

1 2

formulas, logic tests) to generate monthly expense reports.

3

Skill to mince various food products such as garlic, onion and ginger using

1 2

standard kitchen cutlery to assist in food preparation.

3

Ability to interpret and explain policies, procedures, and regulations to non-

1

technical individuals (e.g. departmental employees, the public, vendors, other

2

State agencies) to ensure they are well-informed.

3

KSAPC Statements

Personal Characteristics

– Other elements/aspects relevant to the position may

include:

o Unique working condition

o Unusual job locations

o Physical demands

o Personality traits

KSAPC Statements

Example of Personal Characteristics:

– Willingness to work nights, weekends, and holidays

– Conscientiousness: Take responsibility and initiative for

completing work in a timely manner with an appropriate focus

on details

– Honesty: Refrains from making statements or taking actions

that are knowingly false, deceptive, or fraudulent

KSAPC Statements

• KSAPCs should:

– Address only one worker trait or human characteristic

– Be operationally defined

– Be related to one or more task or job dimension

– Be observable

– Be easy to read and understand

TEST

CONTENT

The Job

JOB ANALYSIS

TASKS KSAs Task/KSA

Links

Test

Content/KSA

Links

Assessment Development Process - Develop Job Analysis Survey

Job Analysis Finalization Meeting

• SMEs must review and finalize all task and KSAPC statements

developed through archival research and job audits etc. prior to

survey (i.e. JAS) administration

• Edit task and KSAPC statements, add statements, delete

statements, refine language, clarify, etc.

• This meeting should contain between 10 – 12 SMEs

Job Analysis Finalization Meeting

• As with all parts of the Job Analysis process, use a

demographically representational sample of SMEs:

– Gender


– Race


– Age



Job Analysis Survey (JAS)

Purpose

• SMEs evaluate task and KSAPC statements in terms of their relative importance and other relevant data

• Essential to the job analysis process, as it determines the appropriateness of using specific Tasks and KSAPCs for selection

Job Analysis Survey

• Things to include:

– Letter of explanation

– Contact information (in case of questions).

– Clear return instructions (paper-form and electronic)

– A reasonable deadline (in a prominent place)

– Demographic questions o Job relevant questions

o EEO information (Optional response item)

Job Analysis Survey

– Things to include (continued):

– Demographic Questions

o Job relevant questions

– Current job classification

– Relevant expertise (incumbent, supervisor, trainer..)

– Classification filling out the survey for (if the survey is evaluating

more than one job as in a classification series job analysis)

– Program area/division

– Office location

– Work location (Field, Headquarters…)

o EEO information (Optional response item)

– Race

– Gender

– Age

Job Analysis Survey

The number of SMEs needed to

complete the JAS varies depending on

how many incumbents are currently in

the job.

Number of

Incumbents

Job

Analysis

Sample

Number of

Incumbents

Job

Analysis

Sample

1 1 21 16

2 2 ... …

3 3 25 18

4 4 ... …

5 5 30 21

6 6 ... …

7 6 40 25

8 7 ... …

9 8 50 29

10 9 ... …

11 10 … …

12 10 100 41

13 11 … …

14 12 … …

15 12 150 47

16 13 … …

17 14 … …

18 14 200 51

19 15 … …

20 16 240+ 20% (70 max)

Job Analysis Survey

Tasks and KSAPCs are

separated into two lists

because they are

evaluated using different

ratings scales.

TASKS

Importance

Frequency

KSAs

Importance

Expected at Entry

Relationship to

Performance

Task - Importance

IMPORTANCE: How important is this task to successful

job performance?

(0) Does Not Apply

(1) Moderately Important

(2) Important

(3) Very Important

(4) Critical

Task - Frequency

FREQUENCY: How often is this job task performed?

(0) Does Not Apply This task is not an essential function of the job, or this task is not performed

on the job. (1) Less than Once a Month This task is performed at least one time a year, but less than once a month

(includes quarterly). (2) Monthly This task is typically performed at least one time a month but less than once

in a typical week. (3) Weekly This task is typically performed once to several times each week but less than

once a day. (4) Daily This task is performed one or more times a day.

KSAPC - Importance

IMPORTANCE: How important is this KSAPC to successful

job performance?

(0) Does Not Apply (1) Moderately Important (2) Important (3) Very Important (4) Critical

KSAPC – Expected at Entry

EXPECTED AT ENTRY: When is a person expected to have this

KSAPC? Is it required before being hired or do they learn it on the

job?

(0) Not Needed

This KSAPC is not needed on the first day of the job. It is either learned on the job or through training, or is not an essential component of the job.

(1) Needed Possession of this KSAPC is needed and expected on the first day of the job. Possession of MOST of this KSAPC is needed and expected upon entry to the job.

(2) Essential Possession of this KSAPC at entry to the job is essential. Possession of ALL of this KSAPC is expected upon entry to the job.

KSAPC – Relationship to Job Performance

RELATIONSHIP TO JOB PERFORMANCE: Does possession of

more of this KSAPC beyond minimum requirements lead to better

job performance?

(0) No Observable Relationship This KSAPC is not required to perform the job, or possession of more of this KSAPC (beyond the minimal level required) does not result in better job performance.

(1) Observable Relationship

Possession of more of this KSAPC (beyond the minimal level

required) does result in better job performance.

Survey Administration

As usual, strive for a representative sample:

– Gender


– Race


– Age



Linking Task and KSAPC Statements

The Linking process is used to identify which KSAPCs are

required to perform which tasks

– Linking evidence should be provided in the final Job

Analysis report

– If a KSAPC does not link back to a specific task it is

dropped from the final job analysis


KSAPC

Ability to ascend a

twelve-foot ladder to

perform rooftop

maintenance.

Task

Repair shingles, gutters, and drainage

systems of government owned

facilities in compliance with

applicable building codes by

replacing missing and damaged

components in a timely manner.

KSAPC

Expert Knowledge of

rooftop drainage systems

to complete building

maintenance projects. KSAPC

Skill to operate electric

powered drills to repair

damaged or broken

building components.


SMEs are presented with the edited list of Tasks and KSAPCs. They are

instructed, for each KSAPC, to read through every task and identify those

which require or utilize the specific KSAPC. With this process, one KSAPC

may be linked to several tasks.

Example:

# KSAPC Statement Linking Task # Classification

Specification #

Scientific Knowledge and Research

1

General knowledge and understanding of scientific research

principles and investigatory methods and statistical analysis

as applied to integrated waste management to achieve

program goals.

6,7,9,32,33 1,2

2

Ability to extract relevant facts and information from written

documents in order to perform effective project planning, solve

problems, or summarize information.

18,19,20,21,22,28 1,2,4

3 Knowledge of chemical, biological, physical, and

environmental sciences as applied to waste management. 9,20,21,24,34 5,6

4 Ability to identify the information needed to answer research

questions. 6,7,9,10,13,28,34 1,2,3

Data Interpretation and Computation

5 Ability to interpret graphical data to extract or identify the key

points or issues that are presented.

6,7,9,10,13,20,21,22,2

3 7

6 Ability to interpret quantitative or statistical data to extract key

information and make valid inferences. 6,9,10,13,28 8

7 Ability to perform basic computations using addition,

subtraction, multiplication and division to summarize data. 6,9,10,13,20,21,22,23 8,10

Linking


Review linkage data for agreement

– Agreement is generally found if two or more SMEs endorse the linkage

– If a group setting is used, the SMEs establish consensus on which KSAPCs link to which tasks

– KSAPCs that do not link are dropped from the final job analysis Note: Linked KSAPCs become the basis of the examination plan and may be used to select testing methods.

Task/KSA Linking Requirements

Technically…

– You really only need to link those KSAPCs for which you

are testing.

– Therefore, it is not a requirement that you link every

KSAPC.

o …especially if linking will be arduous, or interfere with

examination timelines.

Online Survey Tools to use with Job Analyses!

Making your life just a little bit easier.

74

Different Online Survey Options

• Survey Monkey

• Zoomerang

• EZquestionnaire

• KeySurvey

Online Surveys

1. High levels of Customization allowed

o Color

o Logos

o Size and Placement

2. Collects demographic and job information from SMEs

Online Surveys

Benefits of Online Surveys

3. Collects SME ratings of Tasks and KSAPCs

Many different scale option and designs available

Forced choice (No more missing data )

Online Surveys


4. Can disseminate the JAS to hundreds of SMEs

simultaneously

o Can set up a link in an email

o Customized message can be created and sent to a pre-

established list of SMEs

– Can have Survey Monkey resend the JAS to only

those who have not finished the survey


5. Reduce mailing costs

– Faster responses

– Higher response rate


6. Reduces Data Entry

o Can cut and paste Tasks and KSAPCs straight from

a Word or Excel document into the JAS

o Avoid wrist fatigue and injuries!


7. Improves Quality of Data

o Reduces human error

o Easier to fill out than a Scantron


8. Analyzes Data

– Summarizes data automatically

– Responses can be downloaded into a variety of

formats

o including an Excel spreadsheet where results can

be copied and pasted right into a job analysis

report

Summary

• Job analysis surveys are a critical part of many

HR responsibilities—particularly in selection

• There is no one single best way but you should

strive for objective quantitative data

• Information from job analysis can come from

many sources such as duty statements, class

specifications, previous job analyses, O*NET,

job audits and observations

Summary

• Rely heavily on SMEs when developing or

finalizing task and KSA statements

• Job analyses can be administered in groups, via

paper-and-pencil surveys or online

• Online surveys have many advantages

Jim Higgins, Executive Director

BCGi

[email protected]

Upcoming Webinars

• June 21 – Ask a Consultant

• July 12 – The Role of Talent Acquisition in

Affirmative Action Planning

• July 19 – VETS 4214

• July 24 – How to Create Statistically Valid

Samples Using Microsoft Excel

Visit bcginstitute.org to register!

How to Develop and Administer a Job Analysis Survey€¦ · How to Develop and Administer a Job...

Documents

Transcript of How to Develop and Administer a Job Analysis Survey€¦ · How to Develop and Administer a Job...