supp.apa.orgsupp.apa.org/.../apl0000081/z2j011163111so1.docx · Web viewIt seems that the...

APL-2015-1267 Supplemental Material 1

Supplemental Materials

Solving the Supreme Problem: 100 Years of Selection and Recruitment at the

Journal of Applied Psychology

by R. Ployhart et al., 2017, Journal of Applied Psychology

http://dx.doi.org/10.1037/apl0000081

Online Appendix A: Summary of Historical and Scientific Developments that Influenced Recruitment

and Personnel Selection Research and Practice

A variety of economic, business, social, legal, and political forces have shaped the nature

of selection and recruitment research. Throughout the first decades of the 20th Century, Western

society experienced rapid industrialization and movement of masses of people from rural areas to

urban locations as well from agrarian occupations to non-farm employment, particularly factory

and clerical work. This shift in types of employment generated a need to optimally select and

place people in increasingly specialized jobs. At the same time, there was a need for workers to

be more interchangeable and able to fill multiple roles (so long as they had the necessary “mental

equipment”). As work was becoming increasingly more specialized, early JAP research focused

on identifying KSAOs (“mental equipment”) required for different occupations and evaluating

methods for assessing the KSAOs individuals possessed.

Historical Context

In both World Wars, a combination of patriotism and a national draft produced large

numbers of candidates for the military. World War I and the development of the Army General

Classification Test (Alpha and Beta) encouraged post war interest in the application of individual

differences research. Likewise, World War II stimulated research in many different areas of

applied psychology, and for a decade beginning in the mid 1940’s, several military applications

(e.g., new tests, performance evaluations) in the selection area were published annually.


Interestingly, the Great Depression did not seem to generate much selection research. The

likely causes were the absence of significant hiring for a number of years and the lack of

discretionary funds for research, although it might be argued that a large number of applicants

for any job did exist, and selection research would have benefited the employer. Roosevelt’s

response to the Great Depression produced a number of hiring programs for public works

projects, but most hiring was based on the individual employer’s need. However, one of the

programs did produce the Dictionary of Occupational Titles, the precursor of O*NET.

The common, implicit employment contract following World War II was based largely

on commitment between an individual and a single firm. Because employees tended to stay with

an employer throughout their careers, initial selection was the focus through the 1950s and

1960s. In addition, a large number of applicants who had developed skills through military

service and education obtained through veterans’ benefits were available. These two factors

probably combined to lessen the attention on recruitment relative to selection. Most research

published in JAP at this time followed one of two major themes. The first was exploration and

refinement of different types of predictor methods for individual difference constructs. The

second was an increasing understanding of different statistical methods (e.g., regression, factor

analysis). These changes reflected an evolving understanding of validity but also an implicit

preference for empirical evidence and thus criterion-related validity.

The main practical challenge that originated during the 1960s came from the Civil Rights

Act in 1964 and Executive Order 11246 in 1965, both of which prohibited discrimination against

members of protected classes (11246 also set requirements for affirmative action related to the

hiring of members of protected classes). These federal mandates changed practice and stimulated

a great deal of research over the next 20 years. Between the mid-1960s through the early 1990s,


federal guidelines for employment tests (i.e., Uniform Guidelines for Employee Selection

Procedures) were published and introduced the “4/5ths rule” for determining adverse impact in

1978. In addition, new legislation related to employment was enacted (e.g., Age Discrimination

Act, Americans with Disabilities Act), and various court cases (e.g., Griggs v. Duke Power

Company 1971; United States v. Georgia Power, 1973; Connecticut v. Teal, 1982) interpreted

these anti-discrimination laws and guidelines. As a result, there was much uncertainty about how

to design and implement legally-compliant selection practices throughout this period. Increased

regulatory attention to disparate impact introduced new challenges for employers and

researchers. In particular, many of the tests used for hiring purposes assessed cognitive ability.

Although scores on these tests were shown to be valid predictors of job performance, they

resulted in significant adverse impact against some protected classes (particularly African-

American and Hispanic applicants), placing employers in a quandary between using valid

cognitive ability test scores and avoiding adverse impact. This challenge was even faced by the

U.S. federal government and a number of states that had been using the General Aptitude Test

Battery (GATB),a cognitively-loaded assessment, as a screening tool. For a period of time, the

government balanced the validity-diversity dilemma by using within-group norming, in which

the highest scoring candidates within each racial group would be selected at some representative

percentage (Hartigan & Wigdor, 1989). However, the Civil Rights Act was amended in 1991,

making within-group norming illegal, and thus stimulating the search for effective alternatives

without large mean group differences.

Notably, the U.S. federal government’s use of the GATB for hiring in multiple jobs and

locations had another effect—it provided the data that led to studies of validity generalization

and ultimately to improved meta-analytic techniques. Although there were prior attempts to


summarize validity coefficients across jobs (Ghiselli & Barthol, 1953, was among the earliest),

research on meta-analysis extended these prior attempts in important ways by identifying a

number of statistical artifacts, sample characteristics, and design issues that could affect the

magnitude of validity coefficients. And because there were now a sufficient number of primary

validity studies involving workers in different occupations, it was possible to make relative

comparisons of the validity of different predictors across jobs through meta-analysis (Hunter &

Hunter, 1984; Schmidt & Hunter, 1998).

Within the context of the 1991 Civil Rights Act and emerging meta-analysis research,

personality testing was ripe for resurgence (Barrick & Mount, 1991). Research on other

predictors that were also “lost” during earlier decades (e.g., situational judgment tests) was

similarly reinvigorated (e.g., Motowidlo, Dunnette, & Carter, 1990). An additional stimulus to

this research was the U.S. Army Project A, a large scale selection and classification project that

included alternatives to cognitive ability and broadened the performance domain. The

examination of validity, diversity, adverse impact, and predictors that were non-cognitive (e.g.,

personality) defined selection research in the 1990s.

One noteworthy change that reflected shifting business dynamics was a growing

emphasis on recruitment. By the 1970s, the U.S. was transitioning from an industrial economy to

a knowledge and information economy. The nature of work increasingly required advanced or

highly specialized skills and education, which in turn heightened the need to find the right person

to do this work. At the same time, large numbers of the Baby Boom generation began entering

the work force, the assumptions of the implicit contract between employer and employee started

to erode, and the loyalty to one employer began to diminish, resulting in greater employee

mobility. Adding to these societal trends was a shift in employee demographics. The workforce


started to become more diverse in terms of race, ethnicity, culture, and gender with new entrants

into the workforce. As these trends accelerated through the 1990s and into the 2000s, a “War for

Talent” began to be waged globally.1 It is no coincidence that research on selection from the

applicant’s perspective (e.g., applicant reactions) started to occur at this time, and recruitment

research expanded.

Technology further revolutionized the nature of selection and recruitment. The advent of

the Internet in the early 1990s allowed both organizations and applicants to learn more about

each other more easily than any other time in history. By 2000, many organizations used

complex job boards that provided information about the firm and the available jobs and

presented testimonials from employees designed to attract individuals. Internet testing, first

proctored and then unproctored, captured the attention of practitioners. Social media (becoming

widespread in the mid-2000s) changed the way applicants were sourced and became a potential

source of information about candidates. Research is starting to systematically examine many of

these topics.

The Great Recession of 2008-2010 left a splintered labor market in its wake: Many

occupations that require advanced education or training in certain fields still face severe labor

shortages and require extensive recruiting efforts, while other occupational groups have an

abundance of applicants in many geographic areas. At the same time, selection work is

increasingly outsourced to specialized vendors with the expertise and infrastructure to build and

administer assessments in paper, online, and mobile versions. For a while, the number of these

vendors grew rapidly; however, more recently, significant consolidations into a smaller number

of large-scale vendors have occurred. Recruiting functions have followed a similar path

1 Published in 1987, Workforce 2000 predicted dire shortages of skilled labor and greater competition for employees. In 1997, Steven Hankin of McKinsey & Company coined the term “the war for talent” to indicate the intense competition for talent. This concept was later expanded by McKinsey in a book titled “The War for Talent.”


beginning with in-house recruiters, followed by outsourced recruiters and evolving to

recruitment agencies that specialize in particular occupations.

In the early days of selection research and JAP, research and practice appear to have been

closely aligned. Research was conducted and published by those employed by business and

industry as well as by academic institutions. By the 1950s, the number of practitioners publishing

research began to diminish (Meltzer, 1960). While the issues facing practitioners in the area of

selection largely mirror the research interests of academics, the same may not always be said in

the area of recruiting. The recruiting methods used by practitioners are often ahead of the

research. Frequently, the research focuses on parts of the recruiting process without integrating

them. The result is a fragmented perspective of the recruiting process. In addition, much of the

research on recruiting practices used by organizations is not performed by applied psychologists;

instead, the data on the effectiveness of recruiting sources are often studied by experts in

statistics or data mining who are not grounded in psychological theory.

Despite the tensions between the interests of research and practice, the practical nature of

the Supreme Problem of recruitment and selection has remained surprisingly unchanged. The

Supreme Problem faced in 1917 remains with us today. Although fewer organizations steer

applicants to their “fittest place,” most are still faced with the challenge of recruiting and

selecting each applicant to determine the degree of fit of his or her capabilities with the

requirements of the job.

Scientific Context

The broader historical context certainly influenced the topics both practitioners and

researchers addressed in the first 100 years of JAP publications, but there were also trends in

psychological theory and research that had an impact on JAP authors. An early focus on


empiricism (James, 1911) was certainly consistent with the fact that the first several decades of

publications in JAP consisted mostly of demonstrations of predictor-criterion relationships with

little or no attention to underlying theory. This was evident in Hall’s (1917) statement that with

the advent of selection research, psychology was now on a par with other “scientific” disciplines.

The emphasis was on data collected by observation or experiment, empirical evidence rather than

theoretical argument or belief and the reporting of experimentation in ways that would allow for

replication. Early work was also influenced by the work on individual differences originating

with Galton’s (1869) work and the work of Pearson (1904), who is credited with developing the

correlation coefficient as a method of quantifying relationships between variables (i.e., the

predictor and the criterion in selection research). The influence of individual differences on

organizational psychology and JAP publications is documented in a separate “centennial” paper

by Sackett et al. (in press) so we will not elaborate on that research here.

In the 1950s and 1960s, selection research was influenced by behaviorism (Watson,

1925) in psychology. It seems that the introduction of behaviorally anchored rating scales (Smith

& Kendall, 1963) and the development and use of more “behavioral” tests such as assessment

centers, in-baskets, behavioral interviews and others must have been influenced by the notion

that observable behavior should be the central focus of psychology. This was articulated in the

signs versus samples paper by Wernimont and Campbell (1968). Even the underlying premise of

the use of biodata measures (i.e., the best predictor of future behavior is past behavior) is

consistent with the Zeitgeist of that period.

The emphasis on behavior was frequently criticized by Schmidt and colleagues (e.g.,

Schmidt, Hunter, & Pearlman, 1981), and they and others espoused a more cognitive approach to

the measurement of individual differences. Such an approach was consistent with the “cognitive


revolution” that began in psychology in the late 1960’s. The Schmidt and Hunter work

established the importance of cognitive ability as a predictor of performance across the many

contexts in which people work. The centrality of cognitive ability is evident in the various

process models of performance. Schmidt and Hunter rejected the situational specificity of

predictor tests that seemed to have influenced most of the early work on selection.

More recently, the resurrection of personality testing is directly attributable to the work of

Digman (1990) and personality psychologists who proposed the Big Five as the overarching

framework within which to study the personality determinants of work behavior. The Barrick

and Mount (1991) review of personality correlates of job performance and most subsequent

studies of personality-performance relationships were based on this taxonomy even though some

researchers point to the inadequacies of this framework (Hough & Oswald, 2008).

Personnel selection has also been heavily influenced by quantitative data-analytic

approaches developed by researchers in other areas of psychology and other disciplines. No one

would argue with the notion that Pearson’s (1904) introduction of the correlation coefficient and

the related development of regression analyses (Fisher, 1922) were central to the progress of our

discipline. Over the years, increasingly sophisticated statistical methods have been developed

and used in research in recruiting and selection. For example, the use of item response theory and

computer adaptive testing originated in ideas probably first brought to our attention by Lord

(1980) and used in educational contexts. The exploration of differential item and test functioning

and the use of computer adaptive testing based on item response theory is now commonplace in

most moderate to large testing programs. Similarly, exploratory factor analysis that was common

in the 1960s gave way to confirmatory factor analysis and structural equation modeling

(Joreskog, 1969), which have become frequently used tools for those examining the factor


structure of their scores and testing hypothesized models of the theoretical process models of

predictor-criterion relationships (e.g., Barrick, Stewart, Neubert, & Mount, 1998). At the current

time, hierarchical linear modeling (HLM), whose origins trace to sociology (e.g., Firebaugh,

1978) and education (Bryk & Raudenbush, 1992), is the most commonly used tool by those

interested in testing levels of analysis issues in personnel selection and other areas of

organizational psychology. Personnel selection is based on a predictive hypothesis, and as such,

we occasionally (although less often than we should) test longitudinal models. Hence methods

used in developmental psychology (Collins & Horn, 1991) are of direct applicability and are

increasingly being used in organizational psychology and selection.

As is the case for societal context, there are multiple factors in the scientific context that

have affected the growth of our field, and selection and recruitment research is partly a function

of scientific theory and development in other areas of psychology and in other scientific

disciplines.

References to Appendix A

Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M.K. (1998). Relating member ability

and personality to work-team processes and team effectiveness. Journal of Applied

Psychology, 83, 377-391.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models. Thousand Oaks, CA:

Sage.

Collins, L. M., & Horn, J. L. (Eds.) (1991). Best methods for the analysis of change.

Washington, D. C. American Psychological Association.

Firebaugh, G. (1978). A rule for inferring individual-level relationships from aggregate data.

American Sociological Review, 43, 557-572.


Fisher, R. A. (1922). The goodness of fit or regression formulae and the distribution of

regression coefficients. Journal of the Royal Statistical Society, 85, 597-612.

Galton, F. (1869). Hereditary genius. London: MacMillan.

Hartigan, J. A., & Wigdor, A. K. eds. (1989). Fairness in employment testing: Validity

generalization, minority issues, and the General Aptitude Test Battery. National

Academy Press, Washington, D.C.

Hough, L. M., & Oswald, F. L. (2008). Personality testing and industrial-organizational

psychology: Reflections, progress, and prospects. Industrial and Organizational

Psychology: Perspectives on Science and Practice, 1, 272-290.

James, W. (1911). The meaning of truth.

Joreskog, K. (1969). A general approach to confirmatory maximum likelihood factor analysis.

Psychometrika, 34, 183-202.

Lord, F. M. (1980). Applications of item response theory to practical testing problems.

Hillsdale, NJ: Lawrence Erlbaum Associates.

Meltzer, H. (1960). Industrial psychology in Psychological Abstracts: 1927-1959. Journal of

Applied Psychology, 44, 111-114.

Michaels, E., Handfield-Jones, H., & Axelrod, B. (2001). The war for talent. Harvard Business

Press.

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure:

The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.

Pearson, K. G. (1904). On the theory of contingency and its relation to association and normal

correlation. London: Dulau & Co.

Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1981). Task differences as moderators of aptitude


test validity in selection: A red herring. Journal of Applied Psychology, 66, 166-185.

Smith, P., & Kendall, L. M. (1963). Retranslation of expectations. Journal of Applied


Watson, J. B. (1925). Behaviorism. New York: W. W. Norton & Company.

Wernimont, P., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied



Online Appendix B: A Timeline of Noteworthy Papers Published in JAP Over 100 Years2

Year/Significant Event3 Article/Year Comment1914: World War I starts in Europe. Scientific activity in European countries declines dramatically; remains constant or increases in the U.S.1917: U.S. enters World War I which creates the need to select and classify large numbers of soldiers.

Nearly all articles on selection; usually describing a new test and providing simple descriptive data on some group(s) of examinees

1920: Thorndike, E. L. (1920). A constant error in psychological ratings. Pp. 25-29.

First mention of ratings, which have become a major outcome variable in selection research

1922: Kitson, H. D. A shift of emphasis needed in personnel research. Pp.141-148.

Call for change in emphasis from a focus on test performance to a consideration of the occupations and jobs in which a person works

1925: Brandenburg, G. C. Analyzing personality, Part 1.Pp. 139-155. First mention of personality (there were three other papers on personality in the September issue)

1928: Popencoe, P. Eugenic sterilization in California. XII. Social and economic status of the feebleminded. Pp. 304-316.

Paper describing some form of intelligence test used to make decisions on who would be sterilized and examining the SES status of those sterilized

1929: U.S. stock market crashes and the Great Depression begins resulting in high unemployment and large numbers of applicants for few jobs.

1930: Johns, W. B., & Worcester, D. A. The value of the photograph in the selection of teachers. Pp. 54-62.

Description of the use and validity of unusual selection procedure

1931: H. A. Edgerton. Review of Methods of Correlation Analysis. Pp.103- Mentions of multiple correlation and

2 By noteworthy, we mean those papers that were either highly influential or one of the first published on a particular topic.3 Note that only those events that had a significant influence on the practice of recruitment and selection are reported.


104. regression. Previous articles used only bivariate or simple tables and charts to display validity

1933: Hartmann, G. W. The interview as a research and teaching device. Pp. 205-211.

First mention of the use of the interview as a selection device

1933: Kitson, H. D. A further study of the turnover of psychologists. Pp. 542-549.

First use of turnover as a criterion

1934: Remmers, H. H. Reliability and halo effect of high school and college students’ judgments of their teachers. Pp. 619-630.

First mention of halo error in ratings

1934: Kelley, T. L. Note upon Dickey’s formula for the effect of range upon the reliability coefficient. Pp. 855-857.

First mention of range restriction affecting the magnitude of correlations

1935: Schultz, R. S., & McFarland, R. A. Industrial psychology in the Soviet Union. Pp. 265-308.

Interesting description of IO applications in the Soviet Union, reflecting the first interest in work outside of the USA. Interests of Soviet psychologists at the time seem similar to JAP authors.

1936: Hartkemeier, H. P., & Brown, L. M. Multiple correlation and the multiple factor method. Pp. 396-415.

The first use of multiple correlation. The formulas and discussion are familiar, but there seems to be no recognition of the effects of collinearity. Reliance on significance of regression coefficients is complete

1937: Messenger, H. R. An illustration of new techniques in test construction. Pp.311-319.

Test construction using correlations with an external criterion

1937: Pintner, R., & Forlano, G. A comparison of methods of item selection for a personality test. Pp.643-652.

Comparison of nine different methods of selecting test items for inclusion in a test, all of which were some form of item-total test correlations

1938: Scott, C. M. Background and personal data as factors in the prediction of scholastic success in college. Pp.42-49.

First mention of the use of what we now call biodata

1939: World War II starts in Europe. Many scientists, including notable psychologists, move to the U.S.

1939: Taylor, H. C., & Russell, J. T. The relationship of validity coefficients in the practical effectiveness of tests in selection: Discussion and tables. Pp565-578.

First exposition of the utility of tests along with the presentation of the Taylor-Russell tables for the computation of expected performance


when using a test in selection1941: U.S. enters World War II which creates the need to select and classify massive numbers of enlisted personnel. Shortages of labor in private firms leads to more women in the workplace.

1941: Achilles, P. S. Commemorative address on the twentieth anniversary of the Psychological Corporation and to honor its founder, James McKeen Cattell. Pp. 609-618.

Description of the activities of the first consulting firm devoted to test construction, research and applications

1941: Moore, B. V. Analysis of results of tests administered to men in engineering defense training courses. Pp. 619-635.

One of the first papers to use multiple regression to evaluate a battery of tests. R=.75 with 24 tests and small N; no recognition of the shrinkage problem

1942: McGehee, W., & Moffie, D. J. Psychological tests in the selection of enrollees in engineering, science, management, defense training courses. Pp. 584-586.

Description of the use of tests to select soldiers in the war effort. For the next decade, there are several articles on military-related selection

1942: Ghiselli, E. E. A comparison of the Minnesota Vocational Test for clerical workers with the general clerical battery of the United States Employment Service. Pp. 75-80.

First use of an incremental validity argument to support the use of a test

1943: Kerr, W.A. Where they like to work: Work place preference of 228 electrical workers in terms of music. Pp. 438-442.

First article to question whether the work environment could influence job choice

1945: World War II ends. Large numbers of servicemen return home, requiring organizations to screen and process large numbers of applicants (pushes lessons learned from military selection into the private sector).

1947: Kerr, W. A. Labor turnover and its correlates. Pp. 36-371. Turnover used as a criterion. After this point, we see more interest in turnover

1948: Brown, C. W., & Ghiselli, E. E. Accident proneness among street car motormen and motor coach operators. Pp. 20-23.

First mention of safety or accidents as an outcome variable of interest

1949: Karn, H. W. Performance on the File-Remmers test, How Supervise? Before and after a course in psychology. Pp. 534-539.

Discussion of the How Supervise test, which was similar to modern situational judgment tests


1950: Hay, E. H. Cross-validation of clerical aptitude tests. Pp. 153-158. First mention of cross-validation of validity evidence though what is described sounds more like a replication than a cross-validation

1951: Ghiselli, E. E. New ideas in Industrial Psychology. Pp. 229-235. Mention of morale, field theory, job satisfaction, industrial peace, leadership, critical incident collection and personnel classification procedures as new ideas

1951: Bass, B. M., & Norton, F.M. Group size and leaderless group discussion. Pp. 397-400.

First discussion of simulations as a selection technique and the influence of a group variable

1952: Wesman, A. G. Faking personality test scores in a simulated employment situation. Pp. 112-113.

First mention of the impact of faking on personality test scores

1952: Gaier, E. L. The criterion problem in the prediction of medical school success. Pp. 316-322.

Discussion of the criterion problem, which had been mentioned previously, but is the primary focus in this paper

1952: Keenan, V., & Kerr, W. A. Unfair employment practices as viewed by private employment counselors. Pp. 361-364.

First mention of fairness of employment practices

1952: Darley, J. G., Gross, N., & Martin, W. C. Studies of group behavior: Factors associated with the productivity of groups. Pp.396-406.

First paper in which group outcomes are the focus of interest

1953: Fleishman, E. A. The description of supervisory behavior. Pp. 1-6. Description of the precursor of the Leadership Behavior Description Questionnaire

1953: Ghiselli, E. E., & Barthol, R. P. The validity of personality inventories in the selection of employees. Pp. 18-20.

Summary of personality test validities that is similar to modern meta-analyses, but with no corrections for artifacts

1956: Weitz. Job expectancy and survival. Pp. 245-247. First paper to introduce the “realistic job concept”; a precursor to the realistic job preview

1958: Bendig, A.W., & Stillman, E.L. Dimensions of job incentives among college students. Pp. 367-371.

First paper to use college graduates to study how job and work characteristics influence applicant attractiveness

1960: Buel, W. D. The validity of behavioral rating scale items for the assessment of individual creativity. Pp. 407-412.

Use of behaviorally anchored rating scales

1961: Bolster, B. I., & Springbett, B. M. The reaction of interviewers to Examination of the manner in which


favorable and unfavorable information. Pp. 97-103. interviewers make judgments not just the validity of those judgments

1961: Palmer, G. J., & McCormick, E. J. A factor analysis of job activities. Pp. 289-294.

Description of the precursor of the Position Analysis Questionnaire, a structured job analysis instrument

1961: West, W. M., Porter, L. W., & Ghiselli, E. E. Relationship between individual proficiency and team performance and efficiency. Pp. 435-440.

Presentation of what may be the first cross-level analysis in which individual characteristics are related to group outcomes

1963: Ghiselli, E. E. Moderating effects of differential reliability and validity. Pp. 81-86.

First mention and exploration of moderator effects

1963: Smith, P. C., & Kendall, L. M. Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Pp.149-155.

Description of the construction of rating scales that became known as behaviorally anchored rating scales (BARS)

1963: Dunnette, M. D. A note on the criterion. Pp. 251-254. Not the first paper to mention the importance of the criterion in selection research but this paper was a highly influential statement of the importance of considering multiple facets of employee performance

1963: Dunnette, M. D. A modified model for test validation and selection research. Pp. 317-323.

Paper that argues that the relationships between predictors and criteria are moderated and mediated by other variables in complex ways

1964: Civil Rights Act passes and requires equal employment opportunities, creates EEOC responsible for enforcement, focuses on subgroup differences, disparate treatment, and disparate impact.

1964: Locke, E. A., Smith, P. C., Kendall, L. M., Hulin, C. L., & Miller, A. M. Convergent and discriminant validity for areas and methods of rating job satisfaction. Pp. 313-319.

Introduction of the multi-trait multi-method matrix. Although not focused on personnel selection per se, this logic has been used to analyze rating outcomes and predictor and criterion data in personnel selection

1965: Executive Order 11246 is issued and initiates affirmative action, creates Office of Federal Contract Compliance Programs (OFCCP) responsible for enforcement, focuses on subgroup differences, disparate treatment, and disparate impact.

1966: Darlington, R. B., & Stauffer, G. F. Use and evaluation of discrete test information in decision making. Pp. 125-129.

Description of the nature of the personnel classification problem and


the importance of considering base rates and the utility of various outcomes

1967: Fleishmann, E. A. Development of a behavior taxonomy for describing human tasks: A correlational-experimental approach. Pp. 1-10.

Description of a taxonomy that is still being used in various applications

1967: Richards, J. M. Can computers write college admissions tests? Pp. 211-215.

Description and evaluation of an item generation procedure that may be a precursor to today’s attempts to clone test items

1968: Darley, J. G. A journal is born. Pp. 1-9. Review of the first 50 years of the publication of JAP that suggests that many classic problems in test validation research were evident prior to 1922

1968: Bray, D. W., & Campbell, R. J. Selection of salesmen by means of an assessment center. Pp. 36-41.

First description of an assessment center

1968: Wernimont, P., & Campbell, J. P. Signs, samples, and criteria. Pp. 372-376.

Paper emphasizing the use of samples of behavior as predictors of subsequent job behavior

1960s - 1970s: About this time, mainframe computers became commonly available at larger universities and major corporations and substantially increased the data-analytic capacity of researches.

1972: Boehm, V. Negro-white differences in validity of employment and training selection procedures. Pp. 33-39.

The first of many papers dealing with subgroup differences in test validity and job performance

1973: Wanous, J.P. Effects of a realistic job preview on job acceptance, job attitudes, and job survival. Pp. 327-332.

Introduction to realistic job previews

1973: Schmidt, F. L., Berner, J. G., & Hunter, J. E. Racial differences in validity of employment tests: Reality or illusion? Pp. 5-9.

Review of existing literature suggesting that findings of differential validity are a function of sample size and use of tests of single group validity

1976: Schmitt, N., & Coyle, B. W. Applicant decisions in the employment interview. Pp. 184-192.

Paper focusing on decisions made by job applicant as opposed to employer decisions


1976: Rosenbaum, R. W. Predictability of employee theft using weighted application blanks. Pp. 94-98.

Perhaps the first paper to focus on what is now called counterproductive work behavior as a criterion of interest

1976: Chase, L. J., & Chase, R. B. A statistical power analysis of applied psychological research. Pp. 234-237.

Description of the problem of a lack of power in the statistical tests employed by psychologists, including those doing selection research

1976: Schmidt, F. L., Hunter, J. E., & Urry, V. W. Statistical power in criterion-related validation studies. Pp. 473-485.

Paper that displayed the effect of lack of power on conclusions derived from validation studies

1977: Hunter, J. E., Schmidt, F. L., & Rauschenberger, J. M. Fairness of psychological tests: Implications of four definitions for selection utility and minority hiring. Pp. 245-260.

Paper that examined the fairness of selection tests for subgroups under varying definitions of fairness

1977: Schmidt, F. L., & Hunter, J. E. Development of a general solution to the problem of validity generalization. Pp. 529-540.

Paper that demonstrated that the validity of cognitive tests likely generalized across situations. Their method of analysis became the precursor of one form of meta-analysis

1978: The Uniform Guidelines for Employee Selection Procedures are published and introduce the 4/5ths rule.

1979: Schmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow, T. W. Impact of valid selection procedures on work force productivity. Pp. 609-626.

Description of a new way to estimate the standard deviation of worker output enabling the authors to use existing utility formula to estimate worker productivity

1980: Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. The situational interview. Pp. 422-427.

Summary of the development of the situational interview, which generated research on the nature of interview questions and their impact on the utility of the interview as a selection device

1981: The personal computer is launched enabling increasingly sophisticated approaches to assessment, testing, and data analytics, as well as creating the

1981: Dalton, D. R., Krackhardt, D. M., & Porter, L. M. Functional turnover: An empirical assessment. Pp. 716-721.

Recognition that not all turnover is bad and/or voluntary; increased our understanding of the precursors of turnover and its nature


opportunity to use digital simulations and computerized testing.

1982: Sackett, P. R., & Dreher, G. F. Constructs and assessment center dimensions: Some troubling findings. Pp. 401-410.

Emphasis on the fact that the validity of ratings derived from a single exercise exceeded those that were supposed to reference a single construct which led to questions about their construct validity and stimulated a long series of studies

1982: Arnold, J. D., Rauschenberger, J. M., Soubel, W. G., & Guion, R. M. Validation and utility of a strength test for selecting steelworkers. Pp. 588-604.

Paper providing a model of the use of construct validity arguments to support the use of a test

1983: Rynes, S. L., & Miller, H. E. Recruiter and job influences on candidates for employment. Pp. 147-154.

First study of recruiting in JAP that made explicit reference to recruitment as a signaling mechanism

1983: Smith, C. A., Organ, D. W., & Near, J. P. Organizational citizen behavior: Its nature and antecedents. Pp. 653-663.

Introduction of OCBs as an additional outcome of interest in selection research

1984: Hough, L. M. Development and evaluation of the “accomplishment record” method of selection and promoting professionals. Pp. 135-146.

Introduction of a new way to measure employee potential

1984: Mossholder, K. W., & Arvey, R. M. Synthetic validity: A conceptual and comparative review. Pp. 322-333.

An updated presentation of the use of synthetic validation to support test use

1985: Kraiger, K., & Ford, J. K. A meta-analysis of ratee race effects in performance ratings. Pp. 56-65.

Presentation of evidence of race effects in the criterion as opposed to the predictor in selection research

1985: Boudreau, J. W., & Rynes, S. L. Role of recruitment in staffing utility analysis. Pp. 354-366.

Presentation of the argument that recruiting allows (or does not) for improved selection ratios and quality of the applicant pool thus increasing the utility of selection efforts

1986: Sussman, M., & Robertson, D. U. The validity of validity: An analysis of validation designs. Pp. 461-468.

Analysis of various validation designs against Campbell and Fiske criteria for experimental research in psychology

1988: Sackett, P. R., Zedeck, S., & Fogli, L. Relations between measures of typical and maximum job performance. Pp. 482-486.

Introduction of an important distinction in criterion definition and measurement

1989: Vance, R. J. , Coovert, M. D., MacCallum, R. C., & Hedge, J. W. Explicit investigation of the meaning


Construct models of task performance. Pp. 447-455. of criterion constructs1989: Binning, J. F., & Barrett, G. V. A conceptual analysis of the inferential and evidential bases. Pp. 478-494.

Influential analysis of the inferences we make in doing selection research and the support that various validation efforts provide for those inferences

1989: Ryan, A. M., & Sackett, P. R. Exploratory study of individual assessment practices: interrater reliability and judgments of assessor effectiveness. Pp. 568-579.

Systematic study of individual assessment practices and validation efforts

1990: Mathieu, J. E., & Kohler, S. S. A cross-level examination of group absence influences on individual absence. Pp. 217-220.

First mention of cross-level effects

1990: Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. An alternative selection procedure: The low fidelity simulation. Pp. 640-647.

Modern introduction of situational judgment test items

1990: George, J. M., & Bettenhausen, K. Understanding prosocial behavior, sales performance, and turnover: A group-level analysis in a service context. Pp. 698-709.

Another early demonstration of group as opposed to individual correlates of performance

1991: Civil Rights Act is amended and prohibits within-group norming. Increased enforcement leaves employers searching for ways of using cognitive ability tests without creating adverse impact and helps stimulate research on personality and non-cognitive predictors.

The World Wide Web becomes accessible to the general public and leads to organizations using the Internet for recruitment and selection.

1992: Hofmann, D. A., Jacobs, R., & Gerras, S. J. Mapping individual performance over time. Pp. 185-195.

Examination of the correlates of individual performance patterns over time

1992: Ackerman, P. L. Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Pp. 598-614.

Examination of individual differences in ability determinants of performance before and after skill acquisition

1992: Austin, J. T., & Villanova, P. The criterion problem: 1917-1992. Pp. 836-874.

Examination of the history of how IO psychologists have treated performance

1992: Harrell, T. W. Some history of the Army General Classification Test. Pp. 875-878.

Description of the development and history of one of the earliest attempts to use individual differences to select


military personnel1993: Williams, C. R., Labig, C. E., & Stone, T. H. Recruitment sources and posthire outcomes for job applicants and new hires: A test of two hypotheses. Pp. 163-172.

Examination of the sources of applicants and the implications for after hire performance

1993: James, L. R., Demaree, R. G., & Wolf, G. rwg: An assessment of within-group interrater agreement. Pp. 306-309.

Development of an index of agreement between raters as opposed to reliability

1993: Ostroff, C. Comparing correlations based on individual-level and aggregated data. Pp. 569-582.

One of the first large scale studies comparing analyses done at different levels of analysis

1993: Schmit, M. J., & Ryan, A. M. The Big Five in personnel selection: Factor structure in applicant and nonapplicant populations. Pp. 966-974.

Examination of possible differences in the motivation of employees and non-employees as a determinant of personality test factor structure

1994: Motowidlo, S. J., & Van Scotter, J. R. Evidence that task performance should be distinguished from contextual performance. Pp.475-480.

Evidence of a distinction between performance components that were earlier theorized

1994: Van de Vijver, F. J. R., & Harsveldt, M. The incomplete equivalence of the paper-and-pencil and computerized versions of the General Aptitude Test Battery. Pp. 852-859.

One of the first comparisons of computerized and paper-and-pencil versions of tests to appear in JAP

1996: Steiner, D. D., & Gilliland, S. W. Fairness reactions to personnel selection techniques in France and the United States. Pp. 134-141.

Paper considering fairness reactions to tests cross-culturally

1997: Hattrup, K. , Rock, J., & Scalia, C. The effects of varying conceptualizations of job performance on adverse impact, minority hiring and predicted performance. Pp. 656-664.

Examination of the impact of using different conceptualizations of performance as the outcome in regression analyses of performance on predictors

1997: LePine, J. A., Hollenbeck, J. R., Ilgen, D. R., & Hedlund, J. Effects of individual differences on the performance of hierarchical decision-making teams: Much more than g. Journal of Applied Psychology. Pp. 803-811.

One of the first studies to demonstrate the effects of individual KSAOs on individual performance are different from the effects of team KSAO composition on team performance.

1997: Schmitt, N., Rogers, W., Chan, D. Sheppard, L., & Jennings, D. Adverse impact and predictive efficiency of various predictor combinations. Pp. 719-730.

Examination of the impact of adding predictors that display varying levels of adverse impact to a battery of other tests

1999: Zickar, M. J., & Robie, C. Modeling faking good on personality items: An item-level analysis. Pp. 551-563.

Presentation of a process to detect item faking using item response theory analyses


1999: Ghorpade, J., Hattrup, K., & Lackwitz, J. R. The use of personality measures in cross-cultural research: A test of three personality scales across two countries. Pp. 670-679.

Use of test of measurement equivalence to examine responses to personality measures by members of different cultures

1999: DeCorte, W. Weighing job performance predictors to both maximize the quality of the selected workforce and control the level of adverse impact. Pp. 695-702.

One of the first studies to use large scale simulations to examine the impact of various ways to combine predictor data on sometimes conflicting outcomes

2000: Bennett, R. J., & Robinson, S. L. Development of a measure of workplace deviance. Pp. 349-360.

Paper that reflects interest in a broad-based definition of another employee performance outcome

2000: Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. Adaptability in the workplace: Development of a taxonomy of adaptive performance. Pp. 612-624.

Paper focusing on the definition and development of a new aspect of performance

2002: Barrick, M. R., Stewart, G. L., & Piotrowski, M. Personality and job performance: Test of the mediating effects of motivation among sales representatives. Pp. 43-51.

Example of interest in mediators and moderators of personality-performance relationships

2002: Collins, C. J., & Stevens, C. K. The relationship between early recruitment-related activities and the application decisions of new labor-market entrants: A brand equity approach to recruitment. Pp. 1121-1133.

Paper on recruitment activity and application decisions; one of first papers to examine brand image influences in recruitment

2002: Dineen, B. R., Ash, S. R., & Noe, R. A. A web of applicant attraction: Person–organization fit in the context of web-based recruitment. Pp. 723-734.

First paper to explicitly consider internet/website influences on recruitment

2003: Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., deFruyt, F. et al. A meta-analytic study of general mental ability validity for different occupations in the European community. Pp. 1068-1081.

Extension of validity of general mental ability in the European context

2004: Facebook is launched. This leads to firms using social media for recruiting and selection (regardless of their effectiveness or legality).

2004: Stark, S., Chernyshenko, O., & Drasgow, F. Examining the effects of differential item (functioning and differential) test functioning on selection decisions: When are statistically significant effects practically important? Pp. 497-508.

Paper looking at the practical significance of item and test analyses results that indicate subgroup differences

2005: Chapman, D. S., Uggerslev, K. L., Carroll, S. A., Piasentin, K. A., & Jones, D. A. Applicant attraction to organizations and job choice: A meta-analytic review of the correlates of recruiting outcomes. Pp. 928-944.

First meta-analysis examining influences on organizational attractiveness and job choice

2006: Schmitt, N., & Oswald, F. L. The impact of corrections for faking on the validity of noncognitive measures in selection settings. Pp. 613-621.

Demonstration that corrections for faking on personality measures are


futile if one is interested in increases in validity

2006: Edwards, B. D., Day, E. A., Arthur, W. Jr., & Bell S. T. Relationships among team ability composition, team mental models, and team performance. Pp. 727-736.

Examination of relationships among aggregated ability and team performance as modified by perceptions of the task

2006: Arthur, W. Jr., Bell, S T., Villado, A. J., & Doverspike, D. The use of person-organization fit in employment decision making: An assessment of its criterion-related validity. Pp. 786-801.

Investigation of the validity of employee organization fit indices

2006: Harrison, D. A., Kravitz, D. A., Mayer, D. M. Leslie, L.M., & Lev-Arey, D. Understanding attitudes toward affirmative action programs in employment: Summary and meta-analysis of 35 years of research. Pp. 1013-1036.

Exploration of reactions to personnel selection procedures and legal prescriptions of that use

2007: The Apple iPhone is introduced. This leads to firms adapting recruiting and selection strategies that include mobile devices.

2007: Hausknecht, J. P., Halpert, J. A., DiPaolo, N. T., Moriarty, G., & Meghan, O. Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Pp. 373-385.

Review of evidence on two practically important methods of using tests in applied contexts

2007: Hogan, J., Barrett, P., & Hogan, R. Personality measurement, faking, and employment selection. Pp.1270-1285.

Review of the literature on the evidence and magnitude of faking on personality measures and the impact on validity

2008: Arthur, W. Jr., & Villado, A. J. The importance of distinguishing between constructs and methods in personnel selection and research. Pp. 435-442.

Paper that points to the importance of considering the validity of constructs as opposed to methods when comparing predictors

2008: Subramony, M., Krause, N. Norton, J., & Burns, G. N. The relationship between human resource investments and organizational performance: A firm-level examination of equilibrium. Pp. 778-788.

An attempt to measure the impact of HR activities including selection on organizational outcomes

2008: Nguyen, H., & Ryan, A. M. Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence. Pp. 1314-1334.

Examination of evidence regarding one explanation for racial and gender differences on tests

2009: Weller, I., Holtom, B. C., Matiaske, W., & Mellewigt, T. Level and time effects of recruitment sources on early turnover. Pp. 1146-1162.

Use of hazard analyses to show that personal recruitment sources were less likely to result in early turnovers

2010: Aguinis, H., Culpepper, S. A., & Pierce, C. A. Revival of test bias research in preemployment research. Pp. 648-680.

Challenge of the accepted belief that the performance of minority groups is under-predicted using tests

2011: Van Iddekinge, C., Putka, D. J., & Campbell, J. P. Reconsidering vocational interests for personnel selection: The validity of an interest-based

Review of the literature on the use of interest tests as predictors in selection


selection test in relation to job knowledge, job performance, and continuance intentions. Pp. 13-33.

research

2011: Le, H., Oh, I., Robbins, S. B., Ilies, R., Holland, E. et al. Too much of a good thing: Curvilinear relationships between personality traits and job performance. Pp. 113-133.

Examination of evidence for curvilinear relationships between personality test scores and performance; linearity is usually assumed

2013: Jansen, A., Melchres, K. G., Lievens, F. Kleinmann, M., Brandli, M. et al. Situation assessment as an ignored factor in the behavioral consistency paradigm underlying the validity of personnel selection procedures. Pp. 326-341.

Assessment of situations that are assumed to be a factor when we speak of behavior consistency

2013: Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Pp. 1060-1072.

Examination of evidence related to the efficacy of two major ways of combining data to make selection or admissions decisions

2013: McCarthy, J. M., Van Iddekinge, C., Lievens, F. Kung, M., Sinar, E. F.,& Campion, M.A. Do candidate reactions relate to job performance or affect criterion-related validity? A multistudy investigation of relations among reactions, selection test scores, and job performance. Pp. 701-719.

Study of candidate reactions to more objective correlates.

2014: Youngsang, K., & Ployhart, R. E. The effects of staffing and training on firm productivity and profit growth before, during, and after the Great Recession. Pp. 361-389.

Examination of the impact of selection and training on firm outcomes under differing economic circumstances

in press: McFarland, L. A., & Ployhart, R.E. Social media: A contextual framework to guide research and practice.

Conceptual paper that identifies key differences between social media methods and traditional recruitment/selection methods


Online Appendix C:

Speculations about the Future of Recruitment and Selection

Changes in practice have often shaped the nature of research on recruitment and selection for the last 100 years. The speculations

presented below describe future changes in practice that could shape the direction and nature of scholarly research on recruitment and

selection. While some of these changes may seem far off in the future or unrealistic, there are already examples of most of these today.

The intention of these speculations is to prompt researchers and practitioners to think more broadly about the future of our science and

practice and consider how our field will address recruiting and selection in the future.

Practice Trend Implications for Recruitment and SelectionPsychologicalPrediction using the “base” structure of individual differences (e.g., cognitive ability, FFM of personality) wanes. Prediction begins to focus on understanding how “composites” of base KSAOs can be useful for different business needs.

Research focuses less on homogenous constructs (e.g., conscientiousness) and concentrates more on creating composites of KSAOs relevant for specific purposes (e.g., emotional intelligence, employability, flexibility, adaptability, potential).

Research places less emphasis on “universal” relationships across contexts (e.g., cognitive ability and job performance) and greater emphasis on more contextually-embedded relationships (e.g., prediction of service performance).

Validity generalization and meta-analysis will still be important sources of evidence for validity but will also be used to identify the strongest composite of predictors within a given occupation or industry.

Assessment becomes increasingly driven by technology: Assessments, including those based on gaming principles,

lead to higher-fidelity and become more engaging. Technology allows researchers to measure KSAOs in very

different ways, many of which are highly realistic and/or


engaging. Some of these approaches challenge traditional notions of psychometrics (e.g., independence of items; independence of test takers) and generate a number of questions:

o How will game participants be determined?o How will interactive “games” be scored? o How will the reliability, validity, and so on, of these

scores be estimated? o How will construct, content, and criterion-related

validity be established? o Are these new approaches to testing “better” in terms of

validity, reactions, cost, ROI, or minimization of subgroup differences?

o Do games help or hurt the recruitment of the “best” candidates?

Employers blend selection and recruitment, evaluating candidates at all stages of recruitment and actively working to attract candidates throughout the selection process. For example, company-sponsored games draw people in; those who score well are encouraged to pursue application and progress to the next stage of the hiring process.

Recruitment and selection research becomes more integrated. Employers systematically assess their selection processes

against criteria related to applicant reactions, including the applicant’s attraction to the job and company.

Employers look for opportunities to assess job relevant KSAOs from data that are collected at each step of the recruiting process.

The recruitment, assessment, and selection of passive job candidates (or those who have not technically “applied” for a job) becomes a major new stream of research.

Because of affordability and availability, businesses embrace the use of Big Data for selection. Biodata in the form of digital footprints left behind by those using digital media (including the Internet) and interacting with the Internet of Things are analyzed to identify patterns of behavior that predict job performance and other job-

Research needs to provide guidance on how to use digital footprints in a manner that is ethical, predictive, legal, and socially responsible. Just as some biodata items are not legal because of their potential for discrimination, the same issues will occur with Big Data gleaned from digital media. Researchers will address questions such as the following:


related outcomes. o What are the reliability, validity, cost, ROI, subgroup differences, and user acceptability of Big Data?

o How will job relevancy be established?o What other issues related to the use of Big Data for

employee selection exist?o Are such approaches more predictive and cost-effective?o What is the impact of using digital biodata on recruiting

efforts?

Assessment ceases to be a discreet, defined event and becomes part of everyday life; there is no going to “take a test”; rather one’s day-to-day actions are the test. Consequently, the lines between recruitment and selection contexts, versus other contexts (e.g., non-work), become blurred.

A number of important questions about the use of data collected for purposes other than employment must be addressed. Examples include:

o Are data collected from sources related to personal use (e.g., online purchase behavior) a reasonable basis for recruitment and selection?

o Are data collected in different contexts (e.g., at home or on vacation) as appropriate, job-related, or valid, as those collected in selection contexts?

o Are data collected for one purpose and used in a selection context as appropriate, job-related, or valid as those data collected for the purposes of selection?

o Professional standards provide guidelines for the responsible administration of tests. How should these be amended in a “Big Data World”?

o How will potential applicants and incumbents respond to their data being used for recruitment and selection?

o Does the use of data extracted from social media sources help attract candidates to an organization?

o What attempts to fake these measures will occur? Will individuals try to construct an “ideal” applicant online profile?


Biological assessments become more available and more common. Brain imaging will be developed to the point at which biological markers for personality, ability, and related KSAOs can be evaluated quickly and objectively.

A number of questions regarding the validity of such assessments, applicant reactions to them, and their comparative strengths and weaknesses:

o Are approaches based on biological assessment valid? o Are they legally defensible and job-related? o How might such practices violate privacy laws (e.g.,

Americans with Disabilities Act, Genetic Information Privacy Act)?

o How will applicants respond to biological data being used for recruitment and selection?

o What impact would the use of biological assessments have on recruiting (e.g., applicant attraction)?

Social media empowers everyone to comment on products, organizations, and other people. Increasingly, applicants and incumbents will maintain online resumes and rely on a “reputation scorecard,” which is based on others’ providing endorsements and “reviews” that are easily accessible online and linked to a person’s profile.

Before there is widespread acceptance of information gleaned from online sources for recruiting and selection purposes, a number of research questions must be answered:

o What are the most appropriate ways to leverage reputation-based data for recruitment and selection?

o To what degree are these reputation-based data subject to deliberate attempts to fake them?

o How should the accuracy and veracity of data from such sources be determined?

o Do new reputation-based approaches produce scores with reasonable levels of reliability, validity, subgroup differences, and cost effectiveness?

o What practices might be used to make better use of reputation-based data?

o Is the use of reputation-based data for selection purposes lawful, even when such data are collected without the explicit purpose of selection or without the candidate’s knowledge or approval?

o How can the use of reputation-based data for selection


purposes enhance recruiting efforts?

EconomicIf talent is the key determinant of the business’s competitive advantage, comparable or even more important than financial or physical resources, then talent-based data are of primary importance.

Research will examine how individual differences comprise a collective resource that contributes to firm performance and competitive advantage.

Greater attention is given to predicting firm or business unit performance.

Because each firm faces a fairly unique competitive situation, it is unclear whether there will be “universal” aggregate KSAO predictors of firm performance like individual predictors that predict individual performance (e.g., cognitive ability and job performance).

Multilevel validation becomes common. Demonstrating the business impact of recruitment and selection becomes standard practice.

The majority of work becomes even more based on knowledge, information, and collaboration. Machines or computer systems do most of the routine work. This requires workers to know how to understand, collect, manipulate, and make sense of information and data. Perhaps more importantly, workers must have the skills to deal with the anomalies in work that do not follow the patterns that are programmed into software. Jobs will still exist, and tasks will still need to be performed, but successful performance will require collaboration among workers and greater flexibility and adaptability in individual workers. In summary, jobs will place greater requirements for cognitive and interpersonal KSAOs.

Changing work roles and responsibilities may require different job analysis methods. For example, are existing job analysis methods truly capable of capturing highly dynamic and interactive environments and tasks that are typical in team settings?

As work becomes more collaborative, the nature of work becomes more highly influenced by other coworkers. Selection may become less focused on “broad” hiring (e.g., identifying the KSAOs anyone needs to perform the job), and become more focused on “configural” hiring (e.g., identifying the KSAOs one needs to work in a particular group).

Recruitment methods and selection practices become specifically tailored for staffing different types of teams.

Configural approaches to recruitment and selection, whereby hiring decisions are based on the characteristics of existing


team members, become common.

Organizations will continue to engage in hyper-competition. Lasting competitive advantage is untenable, and organizations will need to strive for short-term competitive advantages. In hyper-competitive environments, firm adaptability may actually be enhanced by employees who are higher on broad KSAOs like overall cognitive ability and the components of the FFM of personality—in contrast to predictions raised earlier in this table. Broad competencies, rather than specific KSAOs, may become the focus in selection and recruitment.

Quickly-changing business strategies require rapid redeployment of human capital resources.

o What KSAOs or competencies are critical in such contexts?

Research needs to focus on which KSAOs or competencies best enable firm adaptation and flexibility. The prediction of and adaptation to change as outcomes will become an important selection topic.

o Which approach is more useful to a firm long term -- evaluating KSAOs that are specific to a job and context, or measuring those that are related to the ability to learn new skills and adapt to different environments?

Recruitment and selection for the ability and willingness to continuously learn become more common.

Few firms will be unaffected by global competition and economic conditions. Different regions will become hubs for different types of talent (e.g., knowledge in developed countries, labor in developing countries), requiring organizations to source and hire talent globally. Some firms will also need to determine which employees can competently and willingly fill expatriate roles. Firms will increasingly need to cooperate with competitors, competing in some situations but partnering in other situations.

Research on the sourcing, attraction, and selection of talent globally is needed.

o What is the validity of recruiting and selection practices across cultures and countries? What are their acceptability, predictiveness, and ROI?

o What variables predict success in international assignments?

o Which KSAOs are most related to being able to simultaneously cooperate and compete?

o How do firms use recruitment to poach competitor talent?

Disruptive technologies will radically transform industries. Competition will come not just from within an industry, but from outside the industry. For example, vendors with expertise in supply chain management or operations may start to offer selection services; vendors with expertise in internet

Research will increasingly be dependent on academic-practitioner collaborations to balance rigor against relevance, speed, and cost. Current practice must inform recruitment and selection research so that it focuses on topics that currently challenge practitioners.


analytics may start to offer recruitment services. The historical dominance of I-O psychologists in the field of personnel selection becomes challenged by those with expertise in gaming, simulation design, or Big Data analytics.

o If practitioners feel the science of recruitment and selection (operationalized via journal publications) is not relevant for their work, what happens to the field of I-O psychology and the scientist-practitioner model?

o How can scientists and practitioners be effectively encouraged to collaborate?

Research must study the many alternative approaches to recruitment and selection that are being introduced by those without training in psychology. Regardless of whether these approaches are seen as “relevant” or “scientific,” scientific scrutiny will provide comparative data to separate the worthwhile approaches from the fads and help practitioners make wise decisions.

o How will I/O psychologists stay current with new approaches to recruiting and selection that are introduced by people outside the field?

Smaller, agile firms will use technology to compete against corporate behemoths in certain niches. Although these new firms may have many fewer people, they will need to ensure that the people whom they hire are capable of high performance.

Researchers will need to focus on how current methods for test development, validation, equivalence, etc. can be modified for use in small incumbency populations.

Researchers and practitioners will need to find ways to encourage cooperative studies that allow organizations to pool data.

SocialSeveral demographic forces, including aging populations in developed countries, rapidly growing populations in developing countries, and increasing diversity everywhere, contribute to shifting cultural and national values, which in turn influence political systems. Racial and ethnic groups that were historically in the minority will become the majority in many areas, and in turn, espouse different attitudes about recruiting and selection practices and may ultimately strive to revise existing business and legal regulations. An increasingly

Research on how different generations, races, cultures, and so on view recruitment and selection practices is critical. Such work is vital for understanding how these groups and society as a whole are likely to respond to different types of employment testing and recruitment practices.

o How do different generations, races, cultures, and so on, view different types of employment testing and recruitment practices?

o How do different generations, races, cultures, and so on,


diverse workforce may lead to significant shifts in employment litigation trends. There may be less support for using assessments that have large subgroup differences in selection, particularly from those who have been historically disadvantaged. Standards for the acceptability of recruiting and selection programs may change. For example, concepts such as reliability, validity, and job-relatedness may not be recognized as having the same degree of importance as in the past.

view existing employment laws and regulations? o How do various types of experiences influence these

perceptions? Research on recruitment and selection practices that create high

validity, show no subgroup differences, are cost-effective, and are acceptable to users will remain one of the most important goals for organizations and pressing challenges for I-O psychologists.

Modern businesses operate in dynamic environments that change rapidly due to many factors, including global competition, rapidly developing technology, changing attitudes about acceptable recruiting and selection practices as well as the desirability of various employers, political unrest that interrupts the supply chain or limits the availability of qualified labor, etc. To be successful, businesses must adapt quickly to the changing environment. Consequently, they expect support functions, including recruiting and selection, to quickly adapt to changing business needs.

Research will need to identify effective methods that enable practitioners to adapt recruiting and selection processes quickly to the needs of the businesses they serve:

o How can recruitment and selection be done more efficiently in rapidly changing, global contexts?

o How can firms maintain sufficient contact with “passive” candidates so that there is a sufficient applicant pool in times of rapid expansion?

o What selection practices are most adaptable to changing organizational strategies and goals?

o What selection practices are most predictive and efficient for the prediction of team performance?

o How can I-O psychologists stay abreast of constantly evolving technology and answer questions about its effectiveness relative to recruiting and selection processes?

The opportunities provided by Big Data approaches to recruiting and selection lead to a collision with government regulations and legal guidelines. Issues relating to privacy and job-relatedness will generate new legislation. Discrimination claims will increase, and what is considered “job-relevant”

To prepare for a future in which the use of Big Data is a commonly accepted practice, research must address questions such as:

o What are the perceptions of different stakeholders regarding Big Data?


will be hotly debated. Complaints from candidates about their inability to develop the KSAOs measured through Big Data in a reasonable period of time will surge.

o How can the job-relatedness of Big Data modeling be established?

o How can the job relevancy of elements of Big Data be established? What is the appropriate methodology?

o If views of job-relatedness change, what are the ramifications for validity (and in particular, content validity)?

o How can KSAOs measured through Big Data be developed?

Affirmative action will be gone. It will no longer be needed because businesses must create a diverse workforce or they will not survive. Instead of affirmative action and unbiased selection being compliance issues, organizations will view a diverse workforce as part of their strategy to address competition and differentiate themselves.

To create recruitment and selection programs that result in a diverse workforce that meets the needs of the business, research must focus on questions such as:

o What variables operationalize a diverse workforce?o How can firms find and attract a diverse and qualified

workforce most effectively and efficiently? o How can the selection of a diverse workforce occur in a

legally compliant manner?

supp.apa.orgsupp.apa.org/.../apl0000081/z2j011163111so1.docx · Web viewIt seems that the...

Documents

Transcript of supp.apa.orgsupp.apa.org/.../apl0000081/z2j011163111so1.docx · Web viewIt seems that the...