To assign the right person to the right job …

… who masters the required skills…

… in order that one’s work output meets the organization’s needs for operations

and sustainable growth

Isn’t it critical ?

Copyright 2010. All rights reserved.

Claude TrépanierProfessional in measurement, assesssment and development of skills

Extended version of a scientific communication delivered at the 2008 Annual conference of the Canadian Educational Researcher’s Association (http://www.csse.ca/CERA/home.htm)

Content

1. Practical problem addressed

2. Study of research problem

3. Selected a skills typologies

4. Action-research undertook

5. Applications of skills typologies

6. Further scientific researches required

7. Conclusion

8. References

1. Practical problem addressed

Drawbacks : The skills developed don’t yield optimal correlation with skills expected in the job, they don’t yield optimal performance (effectiveness = results) and competence (efficiency = use of resources).

Research object : Skills set of a position

Problem observed: Teachers and workplace trainers cannot easily reuse job skills set, as defined by employers, to design courses aimed at developing skills in a valid and effective fashion.

Cost of the problem

Source : www.worldbank.org/ieg/training,

Cost = billions of dollars $ wasted each year worldwide

“The World Bank finances about US$720 million in training every year, through both its lending projects and its in-house World Bank Institute (WBI).

The evaluation found that while most of the training reviewed resulted in demonstrable participant learning, this learning frequently did not lead to real change in participants' workplace performance.

Poor training outcomes most often resulted from training content that wasn't relevant to the needs and goals of the target institutions, or from the trainees' lack of incentives or resources to apply learning in their workplaces.

These findings highlight how important it is for training to be embedded in broader capacity-building programs that identify and address organizational and institutional capacity constraints alongside human ones.”

2. Study of research problem

Hence enabling us as educational researchers, to become better at:

So … : teachers and workplace trainers can easily reuse job skills

set, as defined by employers, to design courses aimed at developing skills in a valid and effective fashion.

Research problem : How to define the skills set of a position in order to measure,

assess and develop them in a valid and reliable fashion ?

1. Identifiyng skills to develop;2. Teaching skills;3. Measuring ans assessing skills.

2. Study of research problem

1. Problem observed within two Delphi sessions on ICT skills set definition www.ictc-ctic.ca.

2. Studied occupational skills Profile Models (frameworks).

4. Draw specifications for a universal skills typology deemed universal (workplace and classroom).

Methodology

3. Studied various skills typologies.

2.1 Problem observed within Delphi sessions

Occupational levels

The expert discuss until a consensus is reached

A facilitator asked experts of a workplace position (Intellectual Property Manager), to validate proposed set of accountabilities and skills, to precise them and to propose additional ones.

Accountabilities

Skills

What happened

2.1 Problem observed within Delphi sessions

Scope of essential

accountabilities and skills for a job

Observed accountabilities and

skills through practical experience alone

Accountabilities and skills which cannot be observed by practical

experience alone

Intuitive observer

Job

Job experts recalled their experience of the job activities and competencies only through their memory, hence inducing biases : interpretations, unbalanced weightings, emotive recalls, etc..

About the validity of the sessions

How valid were the accountabilities and skills hence defined ?

It is fine for experts to recall past experiences and memories of a position to define accountabilities and skills.

1. How can they be confident they have covered all aspects?

2. How do they know they have not been too focused on certain aspects and omitted some?

3. Can they ensure their wording of competencies and skills can be reused by teachers in order to design courses developing and assessing exactly the competencies sought?

There are hence serious limitations stemming from the absence of an underlying conceptual framework in teaching and learning

But :

2.2 Studied Occupational Skills Profile Models

(frameworks)Logical representation of factors contributing to understanding, developing and managing occupational skills.

Core competencies

Ancillary competencies

Secondary competencies

ICTC

CSTD Training Competency Architecture1. Analyzing performance/training needs2. Designing training3. Facilitating/instructing4. Evaluating training5. Coaching the application of training

DACUM (Developing a Curriculum)

Tasks, knowledge, behaviour, skills, tools and equipment

Competency A Prepare mealsSub-competency A1 Buy foodSub-competency A2 Wash foodSub-competency A3 Cut food

2.2 Studied Occupational Skills

Profile Models (frameworks)

6. Non-systemic, loosely structured frameworks.

3. None based on any theories (educational, organizational, psychological).

4. Equivocal relation between activity, competency and skill : « a skill is the capacity to undertake an activity ». Circular definitions.

5. Wording of skills using a language’s syntax, no framework used. How one can be sure a skill is properly worded ?

Findings :1. Mainly facts, information and tasks-based.

7. None enable alignement of workplace and human capital resources.

2. Most are groupings of competencies in homogeneous categories, sometimes in a sequence of visible tasks.

2.3 Studied various skills typologies

•Affective•Cognitive

•Conceptual•Moral

•Perceptual•Psychomotor

•Social

1. Based on groups of skills applications (areas in which they are used)

Typology of skills typologies

2. Skills based on a framework of learning styles and experiential learning theory (domain specific)

• Knowledge• Attitude• Know-how

• Knowledge• Declarative Knowledge• Process knowledge

• Very loosely defined. Not grounded much in theories• No clear statement guidelines. Proned to interpretations.• Proned to mainly state visible behaviors (tasks)

• Cognitivism-based• What about other people’s dimensions ?

Affectivity, social, imagination, physical skills ?

• Not integrated within a framework preventing inclusion of common skills traits

• Grounded in theory• Observable and measurable behaviors• Cover most of a person’s skills areas

Educational researchers beware !

If skills typologies lack :

1. Grounds in psychological and educational theories;

3. Universal, objective, observable and measurable criteria to define specific skills …

2. An underlying framework/model depicting the functioning of a person;

Is … similar to looking through capped binoculars

Drawbacks of typologies of skills studied

… hence yielding researchers and educators to …

Characteristics sought from an ideal skills framework

2. Skills cover all of a person’s set of observable and measurable behaviors (including internal dimensions affecting demonstration of behaviors)

5. Prescribes a clear, unequivocal structure to state skills;

1. Skills are observable and measurable behaviors;

3. Grounded on an integration of theories pertaining to personal functioning and development;

4. Skills of the same type are grouped within functions;

7. Universal uses : personal development, interpersonal relationships, teaching learning and coaching within a classroom as well in workplaces with persons of all age ranges, allowing to align personal and professional development.

6. Prescribes a clear, unequivocal structure to link skills with competencies, teaching models and measurement and assessment instruments;

3. Selected a skills typology

• Selected the Synergistic Person’s typology of skills (Racine, 2003)

• Based on how a person functions (behave) and learn

Paradigm and theories underneath a proposed skills typologies

• Positivist paradigm putting a person’s functioning model at the core of the teaching and learning system (observable and measurable behaviors)

Cognitivism A person’s observable behaviors are linked to skills which we may measure.

Constructivism One learns and develops skills through tasks or roles learned socially. One fuses and integrates, through weighting them according to their moment’s importance, many skills achieved beyond a performance threshold: hence building knowledge and skills.

Functionalism (Claude Bernard)

Every organ has a function (a task). A person regularly using skills of a same type, crafts his/her personality : « We become what we do ».

The synergistic person model©

Resources• Material• Human• Financial•

Information

Inputs

Results of functions in action

OutputsFunctions-based process transforming

inputs in outputs

PhysiquePhysical

Imaginative

Affective

Social

Cognitive

Each function integrates a typology of skills

Skill : Definition, structure and formula

Definition: Any learned human process deemed with a certain value acting on an object and generating a product (effect).

Formula

Skill = Object (resource)

Process (activities processing resources in outputs)

Product (output is an observable and

measurable behavior or deliverable

generated by the act)

+ +

Structure A skill is composed of three elements:

1. An object2. A process

3. A product

Typologies of skills of the Synergistic person

Functions

Synergy levels

5

4

3

2

1

Imaginative (Gestalt)

To meditate (to give a meaning)

To solve problems

To create artistically

To produce images

To visualize images

Affective (Krathwohl)

To characterize

self

To organize self around a

value

To value an option

To respond

To accept

Racine, S. (2003)

Physical (Harrow)

To speak in public

To express non verbal language

To live a healthy life

To act with dexterity and

adroitness

To perceive (5 senses)

Cognitive (D’hainaut)

To mobilize cognitive resources

To explore

To apply

To conceptualize

To reproduce knowledge

Social (Saha)

To negotiate collegially

To cooperate

To help

To listen

To act with courtesy

Skills : Examples

I4 To solve (process) a problem (object) to generate a solution (product)

A4 To organize (process) self around a value (object) in order to live by those values (product)

C3 To apply (process) rules (object) in order to generate a deliverable expected by our role/position (product)

S3 To help (process) customers satisfy their needs (object) in order they take decisions in their best interest (product)

P5 To convey a message (object) through speaking in public (process) in order to inform, entertain, convince or mobilize (product)

A thoroughly validated typology of skills

Validity based on :1. How well a respondent recognizes one’s PFT (over 97%)2. Content validity :

• Skills’experts through consensus within a Delphi session• Measure of how well items are linked to a function

3. Construct validity : Through structural equation modeling, confirmatory factor analysis

4. Diverging validity : Divergence between negative and positive items (forced choice items)

5. Predictive validity : Regression indice6. Internal coherence = Internal fidelity : Homogeneity of test with sub-test7. Theoretical validity : link between items and taxons

Validated with over 5,000 persons who completed a personal functioning test (PFT) on 7 occasions.(Racine, 1999, 2000, 2002, 2003, 2005, 2006, 2007)

Competence : Definition, structure and formula

Definition

Socially acceptable human activity;

integrating various skills of various functions (imaginative, affective, physical, cognitive and social);

realised beyond a measurable performance threshold.

Competence structure

integration of skills of various functions in an action:

Skill of the I function = object –> process –> product

Skill of the A function = object –> process –> product

Skill of the P function = object –> process –> product

Skill of the C function = object –> process –> product

Skill of the S function = object –> process –> product

Skill #1Skill #2 Skill #

3

Skill #4 Skill #5

Skill #6

Competence formula

Comp. = (Iskill * weightI)+ (Askill * weightA) +

(Pskill * weightP) + (Iskill * weightC) + (Iskill * weightS)

Comp. = Σ weighted skills

Examples:

I To know thyself (A2) in order to give a meaning to one’s life (I5) and a direction

A To organize self (A4) and to explore (C4) with others means of living in harmony (S4)

C To write a technical report according to industry’s guidelines (C3), which help target readers make sound decisions based on the findings (S3)

S To help people (S3) find what they are looking for, through applying bureaucratic rules (C3) and showing them a hand-made picture (I3)

P To express verbally (P5) one’s emotions (A2) in order to willingly cooperate with others in solving relationship issues (S4)

A definition of learning

Learning is :

1. A change in a person’s imaginative, affective, physical, cognitive and social functions;

2. as aimed by the objectives (operational formulation of skills and competencies based on typologies);

3. amplified by the teaching (application of the appropriate teaching models) and;

4. noticed through using appropriate measurement and assessment instruments.

4. Action-research undertook

Methodology followed

4. Used the skills position profiles as exit profiles in the design of three university courses.

3. Analysed results, gathered experts for a Delphi session until consensus was reached.

5. Delivered the courses.

1. Identified experts for each position.

2. Devised questionnaires to draw competency profiles for the positions.

6. Measured and assessed the developed skills.

Goal: To design three competency-based university courses according to the competency profiles of the sought positions : R&D Manager, IP manager, Events organizer.

5. Delphi sessions for three workplace positions

Position Course

R&D Manager Technology management

Intellectual property manager

Intellectual property protection

Events organizer Sponsorship and special events

Synergistic Workplace System Model

Every job transforms inputs into outputs through a process using 5 functions :

Outputs

1. Accomplishment of job’s mission meeting organizational requirements

2. Impacts in community

3. Environnemental impacts

Feedback loops

Inputs

• Resources• Material• Human• Financial• Information

To assess :to process informationCognitive

To communicateSocial

To execute tasksPhysical

To choose Affective

To conceiveImaginative

Corresponding job functions

Person’s functions

Process

Defined a position competency profile

1. Sent a survey questionnaire to experts of the positions.

2. Synthesized results.

3. Set-up a Delphi session to reach a consensus.

4. Used a job-person matching test to weigh I, A, P, C and S functions deemed most desirable in the position.

Author: Serge RacineAvailable for free on : www.synergimax.com

Job-Person Matching Test©

Auteur:

- Serge Racine. Ph.D

Desired Position Functioning Profile

Once skills set required are known for a job, skills of a person (candidate to the job or worker actually filling the job), have to be

assessed. Then, one has to assess gaps between …

Gaps suggest some learning is required !

… and those of an individual… skills deemed desirable in the job…

Result of the Job-Person Matching Test (JPMT) (consensus of experts)

Results to the skills test

Colored cells suggest the level of skills required in each function, (based on the typology of skills)

Identification of skills requiring developmentTo meditate (to give a meaning)

To speak in public

To solve problems

To organize self around a value

To express non verbal

language

To explore

To create artistically

To value an option

To live a healthy life

To apply To help

To produce images

To respond To act with dexterity and

adroitness

To conceptualize

To listen

To visualize images

To accept To perceive (5 senses)

To reproduce knowledge

To act with

courtesy

Results of the JPMT : Skills sought in the

job=>

Results to a skills test ; Skills mastered by an

individual=>

To meditate (to give a meaning)

To speak in public

To solve problems

To organize self around a value

To express non verbal

language

To explore

To create artistically

To value an option

To live a healthy life

To applyTo help

To produce images

To respond To act with dexterity and

adroitness

To conceptualize

To listen

To visualize images

To accept To perceive (5 senses)

To reproduce knowledge

To act with

courtesy

Skills requiring development=>

Upon having identified, within the skills’typologies, the generic skills within the I, A, P, C and S functions which are deemed desirable for a particular role or job, one has to tailor them to a particular role or job.

Tailoring of skills selected from typologies, to a role or a job

Application for a trainer’s role :

Function of a person Skills

Imaginative To initiate : To set the teaching’s perspective, competencies and skills

Affective To manage : to organize the teaching/learning framework and to commit

Physical To teach : To deliver teaching and learning activities

Cognitive To assess : To measure and assess learning as well as job and business impacts

Social To communicate : To help participants and to acknowledge successes

Functioning types deemed most desirable for the three positions surveyed

R&D Manager Intellectual property manager Events organizer

1

Social skills

Cognitive skills

Physical skills

Affective skills

Imaginative skills

Skills typologies

2

Models of teaching

3

Measurement and assessment instruments

Used the skills position profiles as exit profiles in the design of three university

courses

Allows to grade

change in skills as a

scorecard in Excel

Kirkpatrick’s level #2

Delivered the courses, measured and assessed the skills

5. Applications of skills typologies

1. Personal Functioning Test2. Job-Person Matching Test3. Course design (choice of objectives, teaching models

and measurement and assessment instruments)

4. Course satisfaction assessment5. Competence and skills measurement and

assessment 6. Assessment of educational research7. Diagnostic of an organization8. Achitecturing information/content/template tools

within organization’s databases

Framework for any task aimed at measuring, assessing and developing any human organizational element:

Th

e u

se o

f th

e sa

me

fram

ewo

rk i

nsu

res

con

gru

ent

lin

ks

• Production of a formative and sommative evaluation• Prescription of skills enhancement activities

Organization’s diagnostic

Measurement and assessment ok key performance indicators :

Functions : Direction, Management, Production, Assessment and Sales

Management dashboard, Prescription of enhancement activities

Competencies and skills profiles : Imaginative, Affective, Physical, Cognitive and Social

skills sought in a role or a job

Job situational assessment

Competencies development program :

Program, Course outlines, Pre-test, Educational material, Measurement and assessment instruments, post-test

6. Further scientific researches required

1. Further validate skills by function with skills experts (Delphi)

2. Further define statement of skills by type (I, A, P, C and S) and provide application examples for various roles and jobs

3. Validate corresponding teaching models and measurement and assessment instruments

4. Validate with lots of positions (hundred subjects per position)

7. ConclusionSkills typologies are an invaluable

science-based tool, an enabling technology, used at the heart of humans and organization’s functioning.

1) Know more precisely how we behave;

2) Significantly learn better through more adapted teaching, thus becoming empowered;

They allow us to:

3) Assign the right person in the right job in order to deliver the expected performance, in alignment with the organizational‘s objectives.

8. References• BANDLER, R. GRINDER, J. (1982), Les secrets de la communication,

Actualisation edh.• D'HAINAUT, L. (1988), Des fins aux objectifs de l'éducation, Labor Nathan, Coll

2000.• GORDON, W. J. (1987) The New Art of the Possible: The Basic Course in

Synectics, Cambridge: Porpoise Books.• HARROW, A. (1972), a Taxonomy of the Psychomotor Domain. McKay.• KOHLBERG, L. (1969), Stage and sequence: The cognitive-developmental

approach to socialization.• KRATHWOHL, D.R. & al. (1964), Taxonomy of Educational Objectives. Handbook

II: Affective Domain, McKay.• RACINE, S., (2003), La personne synergique : théorie et applications, Ste-Foy :

Septembre.• RACINE, S., (1996), A dynamic and holistic model for formulating competence,

choosing teaching strategies and evaluating synergic effects. In H.M. Kandarakis (Ed.) New Horizons in learning assessment: Conference Proceeding of the 21st annual meeting of the International Association for Educational Assessment. Montréal, Que: Université de Montréal).

• RACINE, S., (2007) Étude de validation du test de fonctionnement personnel, www.synergimax-international.com.

• TORRANCE, E. P. (1966), Torrance Tests of Creative Thinking. Normal Annual, Research Edition, Personnal Press.

About Claude TrépanierProfessional services consultant in Human Capital Management, educational researcher, educational advisor, university lecturer in management and free-lance corporate trainer.

Undergoing theoric and field researches along the organizational competencies development chain, from needs assessment to the assessment of competencies developed.

Develops practical in-class and workplace practices and tools for the measurement, assessment and development of skills and competencies.

Has delivered since 1998, more than 1 000 hours of courses with syllabus.

Has developed and delivered since 2003 more than 15 competency-based courses, developed and used numerous performance assessment tools (skills assessment, teamwork’s diagnostic, project’s diagnostic, organization’s diagnostic).

Electronic portfolio : http://eduportfolio.com/22421Claude.trepanier@videotron.ca Ph. : 450-674-5904 (Canada)