SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS...
Transcript of SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS...
SABER – SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS
6WORKFORCE DEVELOPMENT
What Matters in Workforce
Development: An Analytical
Framework for the Pilot Phase
Jee‐Peng Tan
Kiong Hock Lee
Alexandria Valerio
Robert McGough
© 2012 The International Bank for Reconstruction and Development / The World Bank
1818 H Street NW
Washington DC 20433
Telephone: 202‐473‐1000
Internet: www.worldbank.org
1 2 3 4 15 14 13 12
This work is a product of the staff of The World Bank with external contributions. The findings,
interpretations, and conclusions expressed in this work do not necessarily reflect the views of
The World Bank, its Board of Executive Directors, or the governments they represent.
The World Bank does not guarantee the accuracy of the data included in this work. The
boundaries, colors, denominations, and other information shown on any map in this work do
not imply any judgment on the part of The World Bank concerning the legal status of any
territory or the endorsement or acceptance of such boundaries.
Rights and Permissions
The material in this work is subject to copyright. Because The World Bank encourages
dissemination of its knowledge, this work may be reproduced, in whole or in part, for
noncommercial purposes as long as full attribution to this work is given.
Any queries on rights and licenses, including subsidiary rights, should be addressed to the
Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax:
202‐522‐2422; e‐mail: [email protected].
Cover design: Patricia Hord.Graphik Design, Alexandria, VA
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase iii
Contents
About the Series ............................................................................................................... v
About the Authors ........................................................................................................ vii
Acknowledgements ........................................................................................................ ix
Executive Summary ........................................................................................................ xi
Introduction ...................................................................................................................... 1
Purpose, Audience, and Relation to Other Skills‐Related Work ............................... 2
A Note on Terminology .................................................................................................. 3
Conceptual Framework ................................................................................................... 6
Linking Polices to System Performance ...................................................................... 12
Data for Benchmarking WfD Policies and Institutions............................................. 15
Piloting the SABER‐WfD Methodology and Expected Outputs ............................. 38
Conclusion ...................................................................................................................... 40
Appendix 1: Functional Dimensions, Policy Goals, and Actions in SABER‐
WfD............................................................................................................................ 41
Appendix 2: Benchmarking Rubrics ........................................................................... 42
References ....................................................................................................................... 51
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase v
About the Series
Building strong education and training systems that promote learning and foster
skills acquisition is fundamental to development and economic growth. Over the
past few years, as developing countries have succeeded in building more
classrooms, getting millions more children into school, and expanding
educational opportunities at higher levels, the education community has begun
to embrace the vision of ensuring that graduates not only exit the system with
employable skills but are also equipped to learn new skills as job opportunities
and labor market conditions evolve over course of their working lives. The
responsiveness and relevance of the education and training system depend not
only on the amount of resources invested, but also on the influence of policies
and institutions on how resources are allocated and how well they are used to
achieve results on the ground.
In May 2011, the World Bank Group launched Education Sector Strategy
2020: Learning for All, which outlines an agenda for achieving “Learning for All”
in the developing world over the next decade. To support implementation of the
strategy, the World Bank commenced a multi‐year program to support countries
in systematically examining and strengthening the performance of their
education systems. This evidence‐based initiative, called SABER (Systems
Approach for Better Education Results), is building a toolkit of diagnostics for
examining education and training systems and their component policy domains
against global standards, best practices, and in comparison with the policies and
practices of countries around the world. By leveraging this global knowledge,
SABER fills a gap in the availability of data and evidence on what matters to
achieve better education and training results.
SABER‐Workforce Development, one of the domains examined within the
SABER program, focuses on documenting and assessing workforce development
policies around the world. Its goal is to shed light on how WfD systems might
bring into closer alignment skills supply and demand, by enabling individuals to
acquire job‐relevant skills while also giving employers access to workers with the
required skills to fill staffing gaps and improve business productivity. To help
explore the state of knowledge in the area, the SABER‐Workforce Development
team invited leading researchers, analysts, and practitioners from developing
and industrialized countries to come together to discuss workforce development
issues. The papers and case studies on workforce development in the pilot phase
are the result of those conversations and the underlying research. Prior to
publication, all of the papers benefited from a rigorous review process, which
included comments from World Bank staff, academics, development
practitioners, and national workforce development experts.
vi Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
All SABER‐Workforce Development papers in this series that are developed
in the pilot phase were made possible by support from the government of the
United Kingdom under its Partnership for Education Development (PFED) with
the World Bank, and from the government of the Republic of Korea under the
Korean Trust Fund.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase vii
About the Authors
Jee Peng Tan is currently Education Advisor in the Human Development
Network of the World Bank where she leads the Network’s program on
workforce development. Between 2005 and 2009 she was Education Advisor in
the Africa Region’s Human Development Department where she led the
Department’s analytical and policy work on education in Africa. A staff member
of the World Bank since 1981, Ms. Tan has worked as an economist in several
Regions of the institution, including South Asia, East Asia, and the Middle East
and North Africa. She has written widely on education‐related policy matters
and taught courses on policy analysis. She pioneered the use of Quality
Enhancement Reviews to improve the quality of World Bank operations in
education; and created the EdStats Website as a knowledge‐sharing platform.
She was also a core member of the team for the 1993 World Development Report on
Investing in Health. Ms. Tan has a BSc in economics from the London School of
Economics, and a PhD in economics from Princeton University.
Kiong Hock Lee is currently a consultant in the Human Development Network
of the World Bank where he works as a member of the SABER‐Workforce
Development team. Mr. Lee worked briefly in the banking sector, then as an
economics lecturer and later professor in the Faculty of Economics and
Administration, University of Malaya, Malaysia. He has written on education
and labor‐related issues and has taught courses in the economics of education,
labor, and personnel economics. He is a graduate of the School of Economics,
University of New South Wales, has a PhD in economics from the London School
of Economics, and an MA in religion from the Reformed Theological Seminary.
Robert McGough is a specialist in technical and vocational education and
training (TVET) with more than 30 years’ experience in the field. He has taught
at the secondary and post‐secondary levels and has held leadership positions in
government and corporate training initiatives in the United States, including the
Dunwoody Industrial Institute in Minnesota, Texas Instruments Corporation,
Westinghouse Corporation, and Minnesota Diversified Industries. He was on the
staff of the World Bank between 1983 and 2003 where he was involved in
training‐related operations around the world. Mr. McGough has published two
books on the organization and administration of TVET, more than 60 refereed
articles and numerous World Bank publications. He has an EdM in industrial
education from the University of Missouri and a PhD in vocational‐technical
education from the University of Missouri.
viii Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Alexandria Valerio is currently a Senior Economist in the Education Department
of the World Bank’s Human Development Network where her work focuses on
workforce development policies, skills measurement, and entrepreneurship
education and training. Prior to her current position, Ms. Valerio was responsible
for leading the World Bank’s policy dialogue and lending programs in education
and training in Argentina, Brazil, Chile, Nicaragua, Paraguay, Panama, Angola,
and Mozambique. She has extensive experience in the areas of education policy,
planning, and finance; workforce development; technical and vocational
education and training; and early childhood development. Her publications
include country and cross‐country studies on skills development, assessments of
the social impact of school fee removal schemes, cost and finance of early
childhood development programs and expansion of secondary education, and
school‐based HIV/AIDS prevention programs. Ms. Valerio holds an MA in
public administration from Syracuse University and a PhD in comparative
education and economics of education from Columbia University.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase ix
Acknowledgements
This paper has benefited from the support of the SABER‐Workforce
Development team, whose past and current members include Richard Arden,
Rita D. Costa, Angela Demas, Ryan Flynn, Manorama Gotur, Lan Joo, Lorelei
Lacdao, Joy Yoo‐Jeung Nam, Marie Madeliene Ndaw, Trinh Nguyen, Sungmin
Park, Brent Parton, John Sloan, Bernardo Vasconcellos, and Cecilia Zanetta. The
authors wish to thank numerous colleagues at the World Bank and members of
two advisory panels for helpful suggestions on the ideas discussed in the paper.
They are grateful for financial support from the government of the United
Kingdom through the Partnership for Education Development (PFED) between
the World Bank and the UK Department for International Development (DfID);
and from the government of the Republic of Korea through the Korean Trust Fund.
None of the above individuals or parties is responsible for the views expressed in
the paper; the authors alone are. Readers are welcome to send comments on the
paper to the authors: Jee‐Peng Tan ([email protected]), Kiong Hock Lee
([email protected]), Alexandria Valerio ([email protected]), and Robert
McGough ([email protected]).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase xi
Executive Summary
Equipping the workforce with job‐relevant skills is a continuing challenge
around the world. In developing and emerging economies, the pool of educated
labor has been rising over time as a result of past investments in education and
training. At the same time, there is growing evidence of persistent mismatches in
skills demand and supply, with many school leavers and graduates unable to
find jobs commensurate with their education and training, while employers
complain of skills shortages and mismatches. Employers also bemoan the
scarcity of “soft skills” (such as problem‐solving, communication, and
effectiveness in team work), skills that are considered essential for business
productivity in today’s global economy. More broadly, skills constraints are
making it difficult for companies to innovate and move into more lucrative areas
of economic activity. The inability to integrate into and ascend the value chain in
global production networks keeps countries in a low‐skills trap and impedes
their efforts to accelerate economic growth and poverty reduction. The
persistence of these problems suggests that workforce development in most low‐
and middle‐income countries is not functioning as well as might be desired.
As a follow‐up to its Education Sector Strategy 2020, which was launched in
May 2011, the World Bank started the Systems Approach for Better Education
Results (SABER) initiative, whose aim is to create diagnostic tools for assessing
the policy and institutional factors that influence the performance of education
and training systems. Workforce development (WfD) is one of several domains
included in the SABER initiative. The focus is on how well the education and
training system is equipping individuals with skills to meet the demand for skills
in the labor market. The tool is expected to help identify promising actions to
achieve better results in light of global good practices in enhancing the impact of
workforce development on economic progress. It places explicit focus on three
functional dimensions of WfD policies and institutions: (a) strategic framework;
(b) system oversight; and (c) service delivery.
In the current phase of the project, the SABER‐WfD tool was applied to a
group of five pilot countries—Chile, the Republic of Korea, Ireland, Singapore,
and Uganda. Because the main purpose is to test the methodology, these
countries were chosen somewhat opportunistically to span a reasonably broad
range of experience. The inclusion of Ireland, Korea, and Singapore allowed the
tool to be tested for its ability to capture key policy and institutional changes in
settings where skill development was an explicit part of the countries’ strategies
for rapid economic growth. The inclusion of Chile and Uganda took advantage
of the availability of significant knowledge available to the research team in the
context recent World Bank operations and analytical work in these countries.
xii Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Insights from the pilot phase are being used in two ways: to sharpen the
SABER‐WfD tool, and to create the beginnings of a new knowledge asset on
promising policies to advance workforce development in the World Bank’s
partner countries.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 1
What Matters for Workforce
Development: An Analytical
Framework for the Pilot Phase
Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Introduction
Many developing and emerging economies have invested in education and
training and have made good progress in expanding coverage at all levels. They
are also paying more attention to learning outcomes, to the broader array of
skills beyond academic achievement, and to technical and vocational education
and training.1 Yet, a growing pool of better‐educated labor has not always
produced the expected results. Indeed, many countries find themselves in a
paradox where large numbers of graduates are unable to find jobs commensurate
with their education and training while employers complain of skills shortages
and mismatches. Viewed from a longer‐term perspective, an added
disappointment is that these countries, especially the poorest ones, continue to
find it difficult to diversify their economies and acquire the business and
technological capabilities required to produce and sell more sophisticated goods
and services. Skills constraints are part of the problem, suggesting that the
system for workforce development in most countries is not functioning as well as
might be desired.2
This paper proposes a new diagnostic tool to enhance the World Bank’s
assistance to its partner countries with regard to the challenges of workforce
development.3 The broader concept is the World Bank’s Skills toward
Employability and Productivity (STEP)4 framework, which sets forth a holistic
model encompassing five components for human development to support
economic and social progress: (a) starting right in early childhood; (b) laying a
strong foundation in basic and secondary education; (c) building and upgrading
job‐relevant skills; (d) fostering innovation and entrepreneurship; and (e)
matching skills demand and supply. Workforce development focuses on—but is
1 One reflection of the increasing focus on learning outcomes is the prevalence of international projects to
measure learning outcomes. Examples include Trends in International Mathematics and Science Study
(TIMSS), Program for International Student Assessment (PISA), and Programme for the International
Assessment of Adult Competencies (PIAAC). 2 A recent review of the problems associated with workforce development through technical and
vocational education and training can be found in Asian Development Bank (2009). 3 The World Bank’s partner countries refer to low‐ and middle‐income countries as well as fragile states. 4 For more details on the STEP framework, see World Bank 2010a.
2 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
not strictly limited to—the third of these five components. The project is being
implemented under a new World Bank initiative known as Systems Approach
for Better Education Results (SABER),5 which was launched following the May
2011 launch of the institution’s Education Strategy 2020.6 SABER‐Workforce
Development (SABER‐WfD) shares the overall initiative’s goal of creating a tool
for systematic documentation and assessment of a country’s policies and
institutions in education and training in light of global good practice.
The rest of the paper is organized into six sections. The next section below
elaborates on the purpose and intended audience of the project; and situates it in
relation to other skills‐related work at the World Bank. The section that follows
clarifies two terms used throughout this paper: workforce development (WfD)
and benchmarking. The next four sections present the paper’s substantive core:
the conceptual framework, key propositions on the link between policies and
outcomes in workforce development, the analytical strategy for data collection,
and the design of the data collection instrument. The penultimate section
explains how the tool is being tested in the pilot cohort of countries—Chile,
Ireland, Korea, Singapore, and Uganda— and the outcomes to date. The last
section concludes the paper.
Purpose, Audience, and Relation to Other Skills‐
Related Work
The World Bank engages as a member of the development community in
assisting partner countries to advance social and economic progress. Its
involvement takes many forms, which for simplicity may be categorized under
three broad spheres of activity: knowledge generation, policy dialogue, and
operational support. SABER‐WfD targets the piece pertaining to policy dialogue.
Its purpose is to create a systematic approach for documenting and assessing
countries’ policies and institutions relating to WfD. The tool is thus intended to
shed light on an important part of the “black box” that separates tangible inputs
from outcomes. Its application in multiple countries can culminate in a
systematic comparative database to foster cross‐country learning and support
policy dialogue on promising approaches for WfD. In the current experimental
phase of the SABER‐WfD initiative, the main purpose of this paper is to explain
the tool’s design and implementation in the five pilot countries. Lessons from
this phase are expected to inform efforts to refine the tool, clarify the protocols
for data collection and analysis, explore formats for reporting the results, and
examine how best to use the results in policy dialogue.
5 See World Bank (2012) and the resources at http://go.worldbank.org/NK2EK7MKV0 (accessed on
November 9, 2011). 6 See World Bank (2011) and http://go.worldbank.org/DTQZ9EKJW0 (accessed on November 9, 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 3
The SABER‐WfD tool complements other skills‐related work at the World
Bank. Two areas of work are worth highlighting. One pertains to the
measurement of skills gaps and mismatches. Recent studies have pointed to the
inadequacy of common indicators such as educational attainment and related
details on fields of study, type of training, and so forth (see, for example,
Hanushek and Wößmann 2006; Sondergaard and Murthi 2012)7 These indicators
also neglect the “soft” skills valued by employers (see, for example, di Gropello
et al. 2011a and 2011b) and rewarded by market forces (see, for example,
Heckman et al. 2006). To narrow the measurement gap, the World Bank is
currently implementing a multi‐country project known as the Skills for
Employment and Productivity (STEP) Skills Measurement Study.8
The second broad area of related work pertains to the impact of policies and
investments in skills development. Promoting the welfare of individuals through
skills development is the focus of many initiatives by governments, international
donors, and private entities. But definitive knowledge about what does and does
not work in different settings is limited (see, for example, Betcherman et al. 2004
and 2007). To fill this knowledge gap, the World Bank has been expanding its
portfolio of impact evaluation with the support of partners in order to narrow
the evidence gap.9 On a more limited and selective basis, the World Bank also
fosters South‐South exchange, including through study visits that enable policy
makers in partner countries to gain first‐hand exposure to and engage in
dialogue and exchange with their peers in other countries.
A Note on Terminology
Workforce development and benchmarking are two terms that are used
frequently in this paper. Because they mean different things to different people,
it is useful to clarify at the outset how we use them in the content of the SABER‐
WfD project.
Workforce development has attracted growing interest in recent years from a
wide range of researchers and policy makers, many of whom are concerned
about the implications for skills development associated with five major factors:
globalization, technology, the role of knowledge and innovation in the “new
economy,” political change, and demographic shifts (Jacobs and Hawley 2008).
Although the term raises issues at the level of individuals, organizations, and
societies, we use it in this paper to refer to the challenges faced by a national,
regional, provincial, or sector‐based system in responding to two expectations:
7 While useful, these indicators are insufficient for policy development, given their generally weak link to
individuals’ labor market outcomes and to countries’ rate of economic growth. 8 Results from the study are expected in 2013. 9 For more information on ongoing evaluations of skills‐related initiatives, see www.worldbank.org/sief
(accessed on November 9, 2011).
4 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Enabling individuals to acquire the knowledge, practical skills, and
attitudes for gainful employment (including self‐employment) or
improved work performance in a particular trade or occupation; and
Providing employers with an effective means to communicate and meet
their demand for skills.10
With regard to the first expectation, it is clear that job‐relevant skills are
acquired through diverse pathways: some are established in early childhood
education (for example, language skills) and reinforced through subsequent
general schooling at the primary and secondary levels (for example, literacy and
numeracy); others rely on pre‐employment vocational training at the secondary
and post‐secondary levels; and still others through apprenticeships, on‐the‐job
training via formal and nonformal arrangements, and informal learning. An
evaluation of workforce development thus requires a comprehensive assessment
of the entire education and training system and its effectiveness with regard to
the skills of both the future pipeline of workers and that of incumbent workers.
Such a comprehensive assessment is beyond the scope of SABER‐WfD. Instead
we narrow our focus to those parts of the system with an explicit mission of
enabling individuals to acquire job‐relevant skills, either through initial
preparation or continuing education and training. The tool thus puts little focus
on early childhood education, primary education, and general secondary
schooling, even though these are all clearly essential in laying a strong
foundation for skills acquisition. To avoid overlap with the SABER tools
addressing higher education, SABER‐WfD focuses on the segment of the
workforce filling mid‐level positions (for example, skilled craftsmen, technicians,
production supervisors, and so forth). Formal training in the skills associated
with such positions typically begins at the secondary or post‐secondary non‐
university levels and can progress up to the tertiary level in some fields.11
The second expectation of workforce development puts the spotlight on
employers as users of skills. As such, their involvement in WfD is essential to
ensure a responsive system for skills supply. In a well‐functioning WfD system,
employers are able to meet their demand for skills and compete successfully in
product and services markets while individuals are able to acquire the
knowledge and skills to prosper in the labor market. Such a system can help
boost economic productivity and growth; creating and maintaining its
10 This dual expectation is shared by the U.S. National Governors Association, which notes that to
develop the skilled and knowledgeable workforce required for economic competitiveness in the global
economy, “state workforce and education systems are designed to provide the skilled workers employers
need to thrive and the education and training individuals need to prosper in todayʹs labor market.” See
http://www.nga.org/cms/home/nga‐center‐for‐best‐practices/center‐issues/page‐ehsw‐issues/page‐workforce‐
development.html (accessed on August 12, 2012). 11 University‐level training typically supplies workers with advanced technical and research skills that
are essential for advancing the development of knowledge‐intensive economic sectors. It is the focus of a
separate domain in the SABER project and is therefore treated more fully there.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 5
performance is thus a key concern of policy makers pursuing skills‐intensive
strategies for economic development.
Benchmarking is another term used in this paper. The concept gained
prominence in 1979 in light of what Xerox Corporation did to stem the loss of its
photocopier business to Japanese competitors (Camp 1989). The company
undertook a systematic assessment of its business practices, in the process
identifying ways to improve manufacturing design, boost production efficiency,
and lower costs, all of which enabled it to regain its competitive edge. The
concept and practice of benchmarking have since become widespread as a well‐
regarded and versatile tool for evaluating organizational and institutional
performance (Dattakumar and Jagadeesh 2003).
Often used as a continuous process for enhancing outcomes, benchmarking
now finds wide application in the public and private sectors throughout the
developed world.12 An organization that embarks on benchmarking assesses
various aspects of its processes in relation to those of leading organizations,
usually from a peer group defined for the purposes at hand. The exercise poses
such questions as the following for self‐examination: (a) how do we compare to
others in the same business; (b) how good do we want to be; (c) who is doing it
the best; (d) how do they do it; (e) how can we learn from and adapt what they
do to suit our circumstances and needs; and (f) how can we excel and become a
standard of performance for others to aspire to?13 The process allows an
organization to develop plans for adapting specific best practices to improve its
own performance.
For our purpose, national or subnational systems of WfD rather than
companies are the relevant units of analysis.14 Conceptually, the foregoing
questions relating to the benchmarking of companies remain relevant when
considering systems of WfD. Many countries are interested in knowing how well
their policies and institutions compare to those in nations with more effective
systems for workforce development. To systematize the comparison, we develop
performance metrics for key aspects of WfD policies and institutions and define
the corresponding stages of development based, whenever possible, on the
experiences of nations with successful WfD systems. Where data are scarce, the
benchmarks are based on a review of the relevant literature and on the logic of
practical progression as a system improves. Finally, it is important to emphasize
that the purpose of benchmarking is not to rank countries but to create a
12 See, for example, http://www.well.com/user/benchmar/germany/Germany.html (accessed on
November 9, 2011). 13 Adapted from Alstete (1995). 14 See O’Lawrence (2007) for an application of the benchmarking approach to a system for skills
development in the state of California in the United States. Although the approach may be used to
evaluate specific programs for skills development (for example, adult worker retraining and re‐
employment schemes, formal industry‐based apprenticeship schemes, or disadvantaged youth
employment/training schemes), the SABER‐WfD project will focus on national systems or subnational
systems.
6 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
common frame of reference for policy discussion and to facilitate self‐diagnosis
as a basis for improvement.
Conceptual Framework
Key elements of the conceptual framework for the SABER‐WfD project appear in
figure 1.15 The framework recognizes that achieving coherence in WfD policies
requires a simultaneous consideration of the demand for skills as well as the
supply. A well‐functioning WfD system with a high degree of coherence
between skills demand and supply achieves faster economic growth by: (a)
improving trainees’ employability; (b) supplying the workers needed by firms to
improve productivity, product quality, and competitiveness; and (c) enabling
firms and entrepreneurs to expand their technological capability and move up
the value chain where profit margins are often more attractive. Where this match
is poor, slower growth and the associated problems of joblessness and
underemployment, brain drain and technological stagnation or laggardness are
an ever‐present prospect.
Figure 1: Conceptual Framework for Workforce Development
Source: Authors’ construction.
15 The proposed benchmarking approach complements the World Bank’s MILES framework, which is
used by some to evaluate employment issues through five lenses: Macroeconomic conditions, Investment
climate and infrastructure; Labor market regulation and institutions; Education and skills development;
and Social protection, where the capitalized letters are used to form the acronym MILES (see Banerji et al.
2008).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 7
In developed countries, the economic crisis of 2008 and its aftermath have
increased the urgency of addressing WfD issues (see, for example, OECD 2010a).
In less‐developed countries, skills to increase the employability of youth and to
help accelerate overall growth also feature regularly in both national and
international dialogue (see, for example, World Bank 2010b, 2010c; World Bank
2012). Below we elaborate on the main components of the framework relating to
skills demand and supply and to the linkages between these two parts of a WfD
system.
The demand for skills. The government’s economic policies set the overall
context for skills demand in multiple ways. Wage policies and related legislation,
for example, influence the flexibility of the labor market, the hiring practices of
firms, and the aggregate demand for labor and its distribution between formal
and informal sectors.16 Tax policies affect firm and worker behavior, including
their incentives to train. The spending policies of governments often have a
significant impact on skills demand. Large infrastructure investments, for
example, create a demand for various construction, engineering, and specialized
skills. In some countries, governments also make strategic investments to nurture
or grow selected sectors and these may call for new skills beyond what is
available in the current labor pool (see, for example, UKCES 2009). In countries
where the government takes a proactive approach to economic management, its
policies may also affect firms’ business strategies and the incentives for
innovation and adoption of new technologies. These forces in turn shape the
demand for skills that may be required for technology absorption and
advancement.
Forward‐looking policies must obviously take into account global trends
and practices and evaluate their implications for the local economy.
Demographic changes, globalization and technological advances, urbanization,
and climate change are some of the key influences on the dynamics of skills
demand. The world’s population is aging as a result of declining fertility and
rising life expectancy.17 In developed economies, the trend is already translating
into a growing demand for health and personal care services and for related
skilled workers, often imported from abroad, required to provide these
16 In Mexico, for example, Levy (2008) notes that the government requires firms to provide health
insurance, pensions, and other social benefits directly to salaried workers, whereas non‐salaried workers
get these services through government pension and insurance programs. The difference in policy
effectively puts a tax on salaried workers but offers a subsidy to their peers in the informal arrangements.
Levy estimates that salaried benefits in Mexico are 30–35 percent less than their actual value to workers;
and that the gap is sufficiently large to keep workers in low productivity work in the informal sector,
thus slowing overall economic growth. 17 In developed economies, 33 percent of the population in 2050 is projected to be at least 60 years old,
compared with just 22 percent in 2010. In developing economies, the corresponding percentages are 20
percent and 9 percent. Africa has the youngest population, but even there, the share of the population at
least 60 years of age is projected to rise from 5 percent in 2010 to 11 percent by 2050 (UN 2009).
8 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
services.18 In less‐developed economies, the share of younger workers—many of
them new entrants to the labor force and often in the midst of starting their
families—is increasing rapidly. They represent both a pool of potential
employees and a market for consumer goods and services and the associated
demand for skilled workers to produce and sell these products. In most
countries, however, the potential has yet to translate into tangible results in the
form of jobs and sustainable income growth.
Globalization and technology, particularly in information and
communication, are an even more obvious influence on the demand for skills.
Firms everywhere face increasing pressures to innovate and diversify into newer
and more lucrative markets, as profit margins in traditional industries are
squeezed by intense global competition and the shortening of product shelf life.
Among the more dynamic firms that compete in export markets, skills
bottlenecks are a chronic complaint, even in countries with highly educated
workforces.19 Rapid urbanization, a trend that some countries actively promote
in order to foster dynamic clusters of economic activity, is also ratcheting up the
demand for talent and skilled workers, including service workers to build and
maintain lively and livable urban spaces. In some settings, the lure of overseas
jobs has fueled the demand for skills, encouraging potential migrants to obtain
training to improve their prospects abroad. Climate change is yet another global
trend with important implications for skills demand. As countries scale up
mitigation or coping responses, they will require workers skilled in a wide range
of “green” technologies.
These global trends combine with conditions in a particular sector or a
locality to shape the demand for specific skills. In the United States, for example,
a 2009 survey of manufacturing businesses found emerging skills shortages in
three sectors (aerospace and defense, life sciences and medical devices, and
energy and resources) and highlighted bottlenecks for skilled production
workers and for scientists and engineers (DMIO 2009). In Europe, WfD polices
are being informed by detailed analyses by the European Center for the
Development of Vocational Training for the 27 countries of the European
Union.20 In India, the boom in information technology (IT) and IT‐enabled
services is increasing the demand for technicians—estimated at 500,000 a year—
as well as for English language skills (Sudan et al. 2010). One careful study
18 At the same time, older workers in developed economies are finding it necessary to postpone their
retirement and seek new skills in an effort to remain employable and productive. As a result the concept
of lifelong learning has gained growing relevance in public policy debates. 19 See, for example, data from the World Bank’s Enterprise Surveys posted at
http://enterprisesurveys.org. 20 According to CEDEFOP (2010), projections for 2020 suggest a net loss of some 2.5 million jobs in the
primary sector (mainly agriculture) and 2.0 million in traditional manufacturing, balanced against gains
of some 7.0 million jobs in business and other services, 3.4 million in distribution, transport, hotels and
catering, and 1.0 million in education, health, and other nonmarketed services. Demand is expected to
rise for highly‐ or medium‐skilled workers, while it will shrink for workers in clerical services, crafts, and
related trades and agriculture.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 9
shows that each new call center raises enrollments in English‐language schools in
the center’s vicinity by nearly six percent (Oster and Millett 2010).
The demand for quality skills. A large number of surveys confirm that
employers value not just a worker’s ability to perform specific technical tasks,
but also his or her possession of “soft” skills or behaviors (Ferrier et al. 2003a;
UKCES 2009). In the United States, manufacturers have consistently ranked
highly skilled and flexible workers as one of two top drivers of performance (the
other is new product innovation).21 In the Philippines and Indonesia, employer
surveys conducted in 2008 revealed a high demand for three “soft” skills for
managers and production staff alike: problem‐solving, leadership, and
communication skills. In Vietnam, the top behavior prioritized by employers in a
2009 survey was punctuality. In the same survey, employers also valued
workers’ practical and technical knowledge more highly than theoretical
knowledge. A recent survey of Indian employers highlighted deficiencies in the
“soft” skills of the graduates from engineering colleges (Blom and Saeki 2011).
The importance placed on “soft” skills is not surprising given the nature of work
in today’s economy.
An increasingly integrated global economy with intense competition, a
growing body of scientific and technological knowledge, and widening access to
knowledge via digital media are reinforcing the demand for skills that go beyond
simple book learning and/or the competence to execute only specific tasks. The
ability to solve problems, to learn new skills, and to adapt to new conditions, all
of which require a solid foundation in literacy and numeracy, are especially
valuable in contexts of rapidly evolving business conditions (Wang 2012).22 These
profound changes are transforming today’s jobs, increasing the premium on
workers that can handle nonroutine tasks as a normal part of their jobs (Levy and
Murnane 2005). In settings where most school leavers must find their livelihood
in the informal sector, having the flexibility, entrepreneurship, and initiative to
find work must also be added to the list of core skills that young people will
require for a successful transition from school to work (for example, Adams
2007).
The supply of skills. The technical and vocational education and training
(TVET) system is often perceived as the main source of workforce skills
development.23 This is understandable as public TVET institutions often host
21 See DMIO 2009. 22 Because of the importance of these cognitive skills, the World Bank recently embarked on the STEP
Skills Measurement project, which involves administering tests to adults ages 15–64 as part of the effort
to document the levels and distribution of such skills in the workforce and to relate the possession of
these skills to individuals’ earnings and employability (among other outcomes). 23 It is useful to note that the acronym TVET represent the first letter of four words with specific
meanings: “technical” and “vocational” typically refer, respectively, to technician‐level and craftsmen‐
level programs; “education” refers to exposure to the world of work and preparation for further studies
in technical and vocational fields; and “training” refers to preparation for entry into, or upgrading in,
specific occupations or clusters of occupations.
10 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
major government‐financed programs for formal skills training. However, in
many countries, other sources of training exist, including private TVET
institutions and employer‐based training programs. In middle‐ to high‐income
nations, and in particular in large industrialized nations such as the United
States, Germany, and Japan, the main suppliers of middle‐ to higher‐level skills
training are the employers themselves. Because firm‐based training activities and
programs are fragmented across the sector, often unpublicized outside the firms,
and typically paid for by the firms themselves, their national contribution to WfD
is not always well‐understood or recognized.
A wide range of programs in fact offers opportunities for skills training in
most countries. They include formal, nonformal, and informal programs, each of
which may require different levels of prior educational attainment and may
operate under different institutional homes, typically though not exclusively, the
ministries of education and of labor. While both formal and nonformal TVET
refer to structured programs, they often fall under the mandate of different
parent ministries. Formal programs usually take place in the education system and
end with formal certification or accreditation of qualifications; nonformal programs
often occur outside the education system (hosted by such ministries as labor,
social welfare, industry, and so forth), in or outside the workplace, and may or
may not lead to an accredited qualification. Where granted, the qualification may
not relate to the certification system hosted by the education ministry. Informal
TVET comprises unorganized and unstructured learning that is seldom the
responsibility of any single agency; it often occurs in the workplace, for example,
when a junior worker learns the ropes simply by working alongside or watching
more experienced colleagues.24 Despite their informality, these arrangements can
be a significant source of learning for tacit skills; their prevalence often depends
on firms’ workplace practices and organizational culture.
To add to the complexity, the system for skills provision may also include
such programs as: (a) formal, industry‐based apprenticeships, usually for
training in advanced skills; (b) training partnerships involving organized labor
and firms, often with formal curricula taught by training providers; (c) technical
training for and within the military; and (d) special needs training (that is, skills
training for select populations, such as prisoners, persons with disabilities, and
the disadvantaged).
In all TVET systems, government policies influence to varying degrees the
way the system is organized, governed, and financed and how skills attainment
is certified and recognized. They therefore affect the efficiency of service
provision, the effectiveness of the system in meeting training objectives, the
quality and relevance of skills that it produces, and the accessibility of training
opportunities, particularly among disadvantaged population groups. They also
24 Nonformal TVET should not be confused with TVET programs that are intended to equip trainees for
jobs in the informal sector of the economy.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 11
shape the incentives for both training providers and learners and influence the
aggregate supply of skills and its diversity by level and specialization.
The impact of policies on the role of employers in WfD planning and
implementation—at both the strategic and operational levels—warrants close
attention. In well‐performing systems, policies are in place to improve the
responsiveness of training providers by encouraging them to view employers,
both public and private, as critical partners whose input and feedback help
ensure the continued relevance of training services as labor market conditions
evolve. Such systems are demand‐led. It is important to clarify that being
demand‐led does not always mean that the system is driven by employers alone.
For countries trying to move into higher value‐added production, politicians’
policies regarding the economy’s strategic direction also influence the demand
for skills. A demand‐led system would be sensitive to such demand as well. By
contrast, in systems where the relation between training providers and
employers is weak, the risk of mismatches in both the quantity and quality skills
is ever present. In such supply‐driven systems, employers that face critical skills
gaps may resort to addressing the problem themselves, as in India in such sectors
as information technology and communication and pharmaceuticals (as
documented, for example, in Tan and Nam 2012).
Alignment of skills supply and demand. A key challenge in all WfD systems
is to ensure a good match between skills demand and supply. Where the match
is good, significant benefits can accrue in the form of a more productive
workforce, higher rates of employment and labor utilization, progress in the fight
against poverty, and, for the economy as a whole, tangible movement up the
value chain of economic activity (see, for example, Ferrier et al. 2003b). The
alternative scenario is that of a weak system delivering a poor match between
skills demand and supply. In such systems, the risks include high
unemployment and underemployment, often coexisting with chronic skills gaps
felt by employers, emigration of skilled workers, and an economy showing few
signs of diversifying and upgrading of its technological capabilities.
In reality, skills misalignment is a common problem in most countries.
Numerous surveys of employers consistently reveal concerns about skills
constraints on business performance and about the lack of certain types of skills
among employees (for example, de Gropello et al. 2011a and 2011b). These
problems stem to a large extent from what economists describe as market and
government failures. Below we elaborate on an analytical strategy to benchmark
the policies that governments might use to overcome such flaws and enhance
outcomes in workforce development.
12 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Linking Polices to System Performance
We shall focus on actions by government to promote a closer connection between
skills demand and supply. This emphasis does not mean that the government is
the most important actor among those with a role in WfD. Rather, it is intended
to facilitate dialogue on how the government might be more effective, whether
acting directly or through the intermediation of other actors, in advancing its
goals for workforce development. Below we begin by elaborating on three broad
areas where weaknesses in policies and related institutions and processes might
arise; we then state three hypotheses to guide our strategy for data collection and
analysis.
Impediments to Better Outcomes
Economic theory about market and government failures identifies several
conceptual obstacles to a well‐functioning WfD system. In their comprehensive
review, Almeida et al. (2012), for example, discuss failures stemming from: (a)
imperfections in capital markets which prevent individuals and firms from
borrowing for training investments; (b) limited information, uncertainty, and
myopia that distort the actions of individuals and firms with regard to training
investments; (c) coordination failures arising from weak linkages among key
stakeholders (for example, employers, and training providers); and (d) labor
market imperfections (for example, firms demanding suboptimal levels of
training for their workers for fear of losing their trained workers to competitors).
Theoretical considerations alone are insufficient, however, for designing
policies to improve the system’s performance. Also needed is a deeper
understanding of the specific policies that matter and the circumstances under
which they matter. The evidence on this score is, unfortunately, sparse at present,
particularly as it pertains to the experience of developing and emerging
economies. We can nonetheless begin to contribute to the dialogue by creating
and testing a new methodology that documents existing policies and institutions
systematically and benchmarks them against practices in systems that have made
significant progress over time. This approach is characteristic of the overall
SABER initiative. Its application in multiple countries creates a comparative
database that can be analyzed to narrow gaps in our knowledge about effective
policies and institutions.
Our data collection effort focuses on three areas where we believe market
and government flaws are particularly relevant as impediments to better
outcomes in workforce development: governance, finance, and information. We
briefly elaborate on them below to set the stage for discussing the design of our
data collection instrument.
Governance. Workforce development is a complex policy arena. It involves
multiple stakeholders at different levels of decision‐making whose goals, roles,
and responsibilities may overlap or be in conflict (for example, ADB 2009 and
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 13
2010). At least four sets of stakeholders can be discerned: (a) public officials
representing different ministries and levels of authority; (b) employers in their
role (individually and through their trade associations) as a source of jobs,
market intelligence, expertise, and advice to inform the training curricula, and
possibly also of gifts and advocacy to support training programs; (c) training
providers (again individually and corporately) who offer pre‐employment, on‐
the‐job and targeted training and career development services, networking
among trainees, and a pool of potential recruits for employers; and (d)
individuals who have dual roles, as trainees and as incumbent or future workers
(who may be represented through unions), whether working for others or in self‐
employment. In this complex environment, appropriate governance
arrangements can help clarify roles, responsibilities, and accountabilities among
the stakeholders and enhance the quality of key relationships among them.
Finance. All WfD systems depend on adequate and well‐deployed
resources, financial and non‐financial, to function effectively. Funding
arrangements—how money is mobilized, allocated, and channeled—shaped
incentives and therefore the decisions made by individuals, training providers,
as well as employers and firms in their capacity as users of skills. Key design
choices include: (a) the aggregate level and pattern of distribution in budget
allocations; (b) the criteria for deciding on funding allocations, and on recipients’
access to and continuation of funding; and (c) diversity in the sources mobilized
to support investments in WfD. Funding arrangements can exert an influence
through channels on both the demand side and supply side and thus have the
potential to help narrow gaps in skills supply and demand. In‐kind resources,
such as employers’ contribution of machinery or staff time for training purposes,
are often difficult to document but their presence is an important indicator of
engagement with a key group of stakeholders.
Information. Reliable and timely information provide signals that condition
the behaviors of key actors on both the demand and supply sides in the market
for skills. Useful information include the following: the skills gaps faced by
employers; the menu and quality of training options and their costs; the
employment and earnings of the graduates of training programs; and the
trustworthiness of the formal certification of skills attained. The availability of
such information can help influence the decisions of both individuals and
training providers and align them to the demand for skills in the labor market.
However, if an important economic goal of the country is to grow and upgrade
into new and emerging areas of economic activity that require new technological,
managerial, and operational capabilities, then passive information flows alone
may not suffice to minimize skills gaps and mismatches. In such settings
proactive coordination to bring the behaviors of the relevant parties into closer
alignment with the areas of emerging skills demand may be appropriate.
14 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Hypotheses on Improving WfD Outcomes
Improving the performance of the WfD system is akin to moving up the rungs of
a ladder. For example, four rungs on the ladder corresponding to poor, modest,
good, or great outcomes may be characterized as follows:
Poor outcomes:
– Few employers value the WfD system; and
– Interest in hiring graduates from the system is low.
Modest outcomes:
– System produces some useful outputs;
– Trainees and employers are generally dissatisfied; and
– Access to training is limited and quality of skills imparted is poor.
Good outcomes:
– System trains large numbers;
– Most trainees and employers are satisfied with the system;
– Some scope exists for continuous learning and skills upgrading; and
– System lags behind in skills for innovation and technology upgrading.
Great outcomes:
– System makes a difference and enjoys employers’ and trainees’
confidence;
– Employers participate actively to provide feedback and support;
– Graduates secure gainful employment;
– System encourages continuous learning and skills upgrading; and
– System adapts quickly to new economic conditions and opportunities.
The challenge for policy makers is to situate their WfD system in the
continuum of outcomes and to determine a context‐appropriate strategy for
progressing to the next rung of performance. Their task is arduous, given our
current incomplete knowledge about effective policies. An important objective of
the SABER‐WfD initiative is therefore to help narrow this knowledge gap by
accumulating and analyzing the evidence in this regard. To guide our data
collection effort and analytical strategy we proceed by proposing three
hypotheses on the links between polices and institutions in WfD and the
system’s performance.
Hypothesis 1. We contend that well‐functioning WfD systems require
mutually reinforcing policies that simultaneously address weaknesses in
governance, finance, and information. Policy gaps in any of these areas yield
subpar outcomes across countries in a consistent and similar way. Accordingly,
examining WfD policies in countries that have improved the performance of
their WfD systems will provide insight into priorities for reform and suggest
strategies for plugging the most damaging gaps at a given level of system
development.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 15
Hypothesis 2: We suggest that good policies on their own are insufficient to
improve WfD outcomes. Effective institutions for implementation and efficient
feedback mechanisms are also critical in a dynamic reform agenda that delivers
results. These features combine to create a virtuous cycle of informed policy
design, follow‐up action, and self‐correcting adjustments to minimize skills gaps
and mismatches in a timely fashion. Suboptimal outcomes stem, on the other
hand, from any or all of the following: weak or incomplete policy design, lack of
attention to implementation, or disjointed feedback mechanisms.
Hypothesis 3. We assert that a “learn‐to‐do and do‐to‐learn” approach is
essential in the search for better outcomes in workforce development. It
minimizes what some writers have referred to as the risk of “borrowing” policies
that may have worked in one context and applying them in another with little
adaptation or consideration of local conditions (for example, Allais 2010). The
approach reinforces the notion that, in a field of still evolving knowledge,
successful reforms will require intentional learning by national policy makers
through systematic efforts to build domestic capacity for policy design and
implementation.
The three hypotheses influence our approach to data collection and analysis.
The first explains the collection of comprehensive yet specific data on policies
and institutions addressing market or government failures in the three areas
identified earlier: governance, finance, and information. The second motivates
the collection of evidence on policy implementation, not just the existence of
policies on paper. The third hypothesis leads us to document the use of feedback
mechanisms and evidence of intentional learning in the process of policy reform
and implementation. Through carefully crafted questions and answer options,
the SABER‐WfD instrument generates data for scoring a country’s WfD policies
and institutions against predefined rubrics, as explained in a later section and in
Appendix 2. Suffice it here to say that the rubrics describe a progression in scores
corresponding to increasing evidence of institutionalized processes, of policy
implementation informed by robust analysis, and of continuous learning through
robust feedback mechanisms.
Data for Benchmarking WfD Policies and Institutions
The data are generated through a survey‐like data collection instrument (DCI).
Because the respondent is not a person but a country (or subnational entities), an
immediate question concerns the perspective from which the questions would be
answered. Keeping in mind that the SABER‐WfD initiative is intended mainly to
support policy dialogue among interlocutors in the human development sector,
the most appropriate perspective is that of the line ministries responsible for
WfD. These would typically be the ministries of education and of labor, although
other special agencies may be relevant. Below we elaborate on the overall
16 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
structure of the dataset to be generated, elaborate on its content, and summarize
its key features.
Overall Structure of the Dataset
The dataset has a cascading structure: it starts with three broad functional
dimensions of policies and institutional processes; each dimension is associated
with a set of three policy goals; in turn each policy goal is associated with a set of
three actions. The structure is intentionally comprehensive and sufficiently
detailed to endow the contents of the dataset with specific and tangible meaning.
Three functional dimensions of policy choices. The three dimensions are: (a)
the strategic framework, which sets the direction and authorizing environment for
WfD; (b) the oversight arrangements, which guide the functioning of the system;
and (c) the service delivery system, through which outcomes on the ground are
achieved (figure 2).25 From the perspective of the line ministries, strategy is about
sensing, influencing, and responding to the external environment for WfD;
oversight is about governing the activities of all stakeholders with a direct
interest in WfD activities (that is, setting “rules of the game”); and delivery is
about managing the activities of those responsible for training provision.
Figure 2: Functional Dimensions of WfD Policies
Source: Authors’ construction.
25 The functional dimensions of policies in the figure 2 bears some resemblance to the threefold challenge
that the UKCES (2009) believes the United Kingdom faces in its attempt to create a better, more
responsive, demand‐led, and effective WfD system. The three challenges are: (a) “a ‘policy gap’, where a
stronger emphasis on the ‘demand’ side is required to bring about better balance to the policy agenda”;
(b) “a ‘measurement gap’, where developing a more appropriate suite of ‘success measures’ for the
system would help better align policy with the 2020 Ambition, and policy with delivery”; and (c) “a
‘policy to practice gap’, where delivery and arrangements on the ground do not always fulfill the
ambition of the policy promise.”
Strategy:Aligning to national
economic goals
Oversight:Governing to achieve
desired goals
Delivery:Managing for tangible
results
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 17
The three functional dimensions are interconnected, as the figure suggests.
The quality of the strategic framework depends on the role that WfD is expected
and enabled to play in fostering the economic success of individuals, businesses,
and the country as a whole. The resulting authorizing environment sets the stage
for oversight institutions and mechanisms that govern the efforts of the parties
involved—policy makers, employers, training providers, trainees—so as to foster
alignment with desired WfD goals. In turn, the oversight arrangements define
the operational context for training provision by public as well as private
providers. The performance of these service providers in equipping trainees with
skills for employment and productivity provide the feedback that closes the
policy‐making loop.
In essence, the tool is about characterizing a holistic system for WfD in terms
of the workings of two markets: “the [labor] market in which employers and
individuals trade work for wages [or earnings]; and the training market in which
individuals obtain training [and therefore acquire skills] from training
providers” (Karmel 2011).
Drilling down to the policy goals and actions. Each functional dimension is
given a more specific meaning by associating them with three policy goals
(figure 3). In total, nine policy goals are identified that are likely to be relevant
across a wide range of developing and emerging economies. Each policy goal is
disaggregated into three actions to clarify is practical meaning. The grand total of
27 actions in the entire dataset appears in Appendix 1.
Figure 3: Policy Goals under Each Dimension
Source: Authors’ construction.
Note: The figure represents the policy goals through the use of single‐word labels, with each row corresponding
to the concepts of governance, finance, and information respectively. The next section of the paper explains
more fully what lies behind these labels.
Strategy
Direction
Priorities
Coordination
Oversight
Pathways
Resources
Standards
Delivery
Content
Incentives
Outcomes
18 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
The foregoing structure provides a snapshot of the type of policy data
collected by the DCI. While the information may be organized differently and
while the policy goals and action may overlap because of linkages among them,
the approach satisfies the desire for a well‐structured and parsimonious
instrument that suits our purpose in the SABER‐WfD initiative. Below we
develop the ideas in greater detail and elaborate on their links to the questions
that are included in our data collection instrument.
Dimension 1: Strategic Framework
WfD is not an end in itself but an input toward broader goals—of boosting
employability and productivity; of relieving skills constraints on business
development and growth; and of advancing overall economic growth and social
well‐being. Its multidimensional nature poses complex challenges and tradeoffs.
For example, when policy makers refer to skills for growth, they are typically
concerned about skills that can help the economy diversify, innovate, and
upgrade its technological capability. However, because these skills pertain, by
and large, to formal sector jobs they may be irrelevant to the majority of workers
in developing countries who earn their living in the informal sector.
A second source of complexity is that all jobs call for a combination of skills.
Bloom’s (1976) classic taxonomy makes a distinction among cognitive, affective,
and psychomotor skills.26 Development of these skills typically involves different
periods of gestation; the sequencing and timing of investments also depends on
the specific skills required in different jobs. Some skills (for example, basic
literacy and numeracy) entail sustained effort from relatively young ages (for
example, through early childhood education and general basic education); others
call for specific programs that meet employers’ more immediate needs. On a
fundamental level, WfD strategies entail trade‐offs between short‐ and longer‐
term objectives, often because resources are limited and because of differences in
the pattern of costs and benefits across investments in skills development.
A third reason for the complexity of WfD is that outcomes depend on the
actions of multiple stakeholders, including individuals, employers, training
providers, and government officials (often from several ministries or agencies).
Asymmetric access to credit, imperfect or incomplete information, misalignment
26 Cognitive skills refer to a person’s grasp of academic disciplines such as English, mathematics, as well
as various pure and applied sciences, history, and so forth. Affective refer to his or her perceptions about
work, concept of self and others, and attitude toward timeliness, accuracy, quality, and performance.
Psychomotor skills refer to possession of skills required to perform the tasks or duties involved in an
occupation, job, or business (for example, operating a lathe, preparing architectural plans, installing
equipment, and so forth). The Dictionary of Occupational Titles (DOT) in the United States makes a
distinction among jobs according to the degree of involvement with Data, People, and Things (see
http://www.oalj.dol.gov/PUBLIC/DOT/REFERENCES/DOTAPPB.HTM (accessed on October 10, 2011).
The O*NET’s Content Model, which replaced the DOT in 1998, continues to emphasize the multifaceted
skills requirements of jobs (for more information, see http://www.onetcenter.org/content.html (accessed
on October 10, 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 19
of incentives and weak cooperation among these actors are some of the obstacles
that can lead to suboptimal outcomes in WfD.27
These complex objectives and challenges of WfD lead us to emphasize the
following policy goals under the functional dimension relating to strategy: (a)
alignment of the direction for WfD to the country’s goals for economic growth
and poverty reduction, (b) prioritization of a demand‐led approach to WfD, and
(c) facilitation of critical coordination among the actors at the leadership level of
decision‐making. Bringing together these elements requires visionary leadership
based on an understanding of and conviction about the role and nature of WfD
in national development. It necessitates sound judgment on how to solve short‐
run challenges without losing sight of longer‐run ambitions for WfD. And it calls
for clarity in removing practical roadblocks to cross‐sector cooperation among
diverse stakeholders where such cooperation is vital for achieving critical
breakthroughs.
Policy Goal 1: Clarifying the direction for WfD. Advocacy for WfD by
national leaders provides the public support to legitimize it as a priority for
economic development and to ensure that it receives due attention in policy
dialogue and plans for implementation. To be credible, advocacy needs to be
based on realistic and accurate assessment of the priorities for WfD in light of the
country’s economic prospects. To achieve impact, the assessment must be
followed by policies and practices that promote and sustain coherence between
the demand for and the supply of skills. Thus, to flesh out the goal of bringing
clarity to the strategic direction for WfD we identify the following policy actions:
Advocate for WfD as a priority for economic development;
Evaluate economic prospects and its implications for skills; and
Develop strategic polices to align skills demand and supply.
In reviewing the pattern of economic growth over the past few decades, the
Commission for Growth and Development (2008) has emphasized the
importance of “leaders who are committed to achieving growth and who can
take advantage of opportunities from the global economy.”28 In the area of
workforce development to support growth, East Asia’s fastest growing
economies over the past 50 years—the Republic of Korea; Taiwan, China; and
Singapore—provide the clearest examples of the role of leaders in shaping the
skills agenda. These nations have built a strong foundation in basic skills through
general education for all and now consistently appear among the top‐performing
nations in international tests of student learning (OECD 2010b; Mourshed et al.
2010). At the same time, they have also developed training systems that
27 See Almeida et al. (forthcoming 2012) for an elaboration of various market and government failures
that can lea to suboptimal outcomes in workforce development 28 See
http://www.growthcommission.org/index.php?Itemid=169&id=96&option=com_content&task=view
(accessed on November 9, 2011).
20 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
effectively equip workers with job‐relevant skills to expand their own
technological capabilities (see, for example, Ashton et al. 2002; Green et al. 1999;
Kuruvilla et al. 2002). The increasing sophistication of the exports from these
countries is one sign of success. In Korea, for example, electronic, electrical, and
other high‐technology products made up nearly 40 percent of the country’s
exports in 2006, up from less than 5 percent in 1980 (Yusuf and Nabeshima 2010).
Policy Goal 2: Prioritizing a demand‐led approach to WfD. A key principle
is to recognize the role and influence of employers, firms, and individuals in
determining the demand for skills.29 Because these stakeholders are the ultimate
end‐users of skills, a demand‐led approach to WfD at all levels of decision
making is essential to minimize potential mismatches between skills supply and
demand. For the same reason, translating the ambition of skills‐led economic
development will typically require policies to intensify employers’ and firms’
demand for more and better skills to help raise productivity. Because skill sets
take time to develop, policies often produce results only after a lag. The
discrepancy in timeframe suggests that monitoring the pipeline of skills against
emerging and changing trends in demand and taking steps to avert serious skills
imbalances and mismatches are highly critical in setting strategic priorities in
WfD. Taking the foregoing considerations into account, we associate the
following policy actions with the goal of prioritizing a demand‐led approach to
WfD:
Ensure active engagement of employers and industry in developing WfD
policies;
Strengthen firmsʹ demand for skills to improve productivity; and
Address critical challenges in the future supply of skills.
Country conditions influence the specific ways in which governments set
strategic priorities for WfD. To illustrate, in 1996 Costa Rica prioritized the
introduction of new courses in technical training and new curricula as part of its
eventually successful bid. Led by the country’s president, the country succeeded
in persuading Intel, a global computer chip manufacturer, to set up the firm’s
newest plant in the country (Spar 1998; World Bank/MIGA 2006).30 In subsequent
years, the country continued to align its education and training programs to meet
the needs of employers in the information technology industry.31 The IT industry
29 The role of individuals matters not only as potential employees but also as own‐account workers. The
latter are especially important in settings where most job opportunities are in self‐employment or in
family businesses. 30 The Technological Institute of Costa Rica (ITCR) introduced a one‐year certificate program (for high
school graduates), and a one‐year associate degree and English language training programs. The
technical courses focused on new fields such as semiconductor manufacturing and microelectronics, and
later also included materials science (Spar 1998; World Bank/MIGA 2006). 31 These demand‐responsive efforts included, for example, the following: (a) new technical programs and
enhanced curricula at the country’s leading educational institutions— ITCR, the University of Costa Rica
(UCR), and the National Training Institute (INA), especially during 1999–2003; (b) an English
reinforcement program at ITCR; (c) links between UCR’s School of Physics and technical and vocational
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 21
is today a major part the economy, accounting for nearly 19 percent of exports in
2009, compared with just 4 percent in 1997.32 Mexico’s Integral Quality and
Modernization (CIMO) program exemplifies an effort by the government to
foster skills upgrading among firms.33 In Korea, the introduction in 2010 of a
high‐profile government initiative involving “meister” high schools exemplifies
an effort to address emerging shortages of technicians for the country’s priority
sectors (for example, electricity, media, logistics, shipbuilding, IT, and
automotive).34
Policy Goal 3: Strengthening critical coordination to realize WfD
objectives. A key challenge in WfD is that it is by nature complex in terms of the
activities and their timing, involving an intricate web of stakeholders with
diverse interests, roles, and responsibilities. Ensuring that the combined effort of
these partners is consistent with the country’s key priorities for WfD is therefore
an important goal of strategic coordination. Such coordination typically requires
leadership at a sufficiently high level to overcome barriers to cross‐sector or
cross‐ministerial cooperation that are often impossible to resolve among peers
with similar levels of decision‐making authority. Fundamentally, the core issues
pertain to the “interactions among institutions, processes and traditions that
determine how power is exercised, how decisions are taken on issues of public
and often private concern, and how citizens and other stakeholders have their
say” (Abrams et al. 2003). An arrangement that clarifies roles and responsibilities
and fosters communication among the relevant WfD stakeholders sets the stage
for coherent and well‐coordinated actions by these players (see, for example,
ADB 2009). Thus, for the goal of strengthening critical coordination, we identify
the following actions:
Ensure coherence of key strategic WfD priorities;
Institutionalize the structure of WfD roles and responsibilities; and
Facilitate communication and interaction among all WfD stakeholders.
Strategic coordination is particularly relevant for major initiatives in WfD
that are breaking new ground, that involve new or nontraditional partners, and
that possibly also introduces new operational procedures. Such coordination at
schools for electronics; and (d) outreach programs in technology and science for teachers and students in
primary, middle, and high schools. 32 Data from World Development Indicators (see http://data.worldbank.org/data‐catalog/world‐
development‐indicators/wdi‐2011, accessed November 27, 2011). 33 For more information about CIMO, see Tan et al. 2004. 34 These shortages stem from the preference of young Koreans and their families for studies at the
universities. To increase the attraction of a non‐university route to the labor market, the “meister” high
schools follow new curricula that explicitly cultivate “21st century skills.” A personal visit in May 2011 by
two of this paper’s authors to Sudo Electric High School in Seoul, one of the first “meister” schools that
the country’s president inaugurated in March 2010, found the following practices: (a) using a student‐
centered pedagogy; (b) including training on personality development; (c) offering students
opportunities for study abroad; (d) using English for teaching whenever the teachers are able to do so;
and (e) ensuring strong linkages with specific employers.
22 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
both the strategic and operational levels can help avoid duplication—or worse,
misalignment or conflict—of effort and ensure that the most critical initiatives
receive the necessary moral and material support for experimentation,
consolidation, and maturation. Ireland’s experience with the establishment of
new regional technical colleges (RTCs) beginning in the late 1960s is an example
of coordinated effort to supply mid‐level technicians for the country’s emerging
industries (O’Hare 2008).35 The Indian government’s decision to create several
new bodies in a three‐tiered governance arrangement for WfD—the National
Council for Skills Development, chaired by the Prime Minister; the National Skill
Development Coordination Board, chaired by the deputy chairman of the
Planning Commission; and the National Skill Development Corporation, headed
by an eminent private sector industrialist—exemplifies a more recent effort at
strategic coordination at the highest level of government.36
Dimension 2: System Oversight
The second functional dimension of WfD policy choices is primarily concerned
with the design of systemic “rules of the game” to help realize the ambition of
growth‐promoting WfD envisioned in the strategic framework. To make our task
manageable, we focus on components of the WfD system that relate to skills
development for mid‐level rather than highly specialized professionals and
managers. Skills acquisition at this level occurs mainly through pre‐employment
training at the secondary and post‐secondary levels and on‐the‐job training,
whether in formal, nonformal, or informal arrangements, including
apprenticeships. The main objective of oversight is to facilitate efficient and
effective skills acquisition by individuals to improve their employability and
productivity so that employers are able to meet their demand for skilled workers
in a timely manner. The expected result is the minimization of mismatches in
skills supply and demand, thereby enhancing the contribution of WfD to
economic growth and social progress. In countries with large informal sectors
where most people are self‐employed, the oversight function also entails
ensuring that individuals are able to access training services that are effective in
enhancing their earning power.
If the markets for skills functioned perfectly there would be a limited role for
oversight. Individuals would have access at no cost to accurate and timely
information about the types of skills in demand; they would know where and
how to get trained; they could easily mobilize the funds needed to pay for the
35 O’Hare (2008) elaborates that the Ireland Development Authority (renamed since 1994 as the Industrial
Development Agency (Ireland)), in its effort to convince foreign businessmen to invest in the country,
used a portfolio of incentives to encourage a greater output of skilled technicians throughout Ireland,
while simultaneously involving the leaders and staff of the training institutions to participate actively in
its promotional efforts (for example, by attending functions where foreign businessmen were gathered,
and travelling to events in potential investor countries). 36 For more details of the arrangements in India, see
http://labour.nic.in/policy/NationalSkillDevelopmentPolicyMar09.pdf (accessed July 19, 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 23
training; and once trained, they would be able to send accurate and credible
signals to potential employers regarding the skills they possess. Likewise,
employers would have reliable and timely information about the skills of
prospective and incumbent employees; they would be able to make accurate
choices among possible training options for their workers; and they would face
no constraints in organizing or purchasing training. Training providers, on their
part, would be fully aware of the skills that are in demand and could tailor their
programs accordingly; and they would have the information to create efficient
pathways for skills acquisition for their clients, for example, through articulation
agreements with providers that offer complementary services.
This scenario is unrealistic and highly unlikely in practice, however.
Information is costly to generate and to acquire, and it may not be complete,
relevant, accurate, or timely (see, for example, Woods and O’Leary 2007).
Individual firms often cannot spare the time to codify their skills requirements,
particularly if they are small or operate in a fast‐changing business climate. Other
distortions may arise from individuals succumbing to the temptation to overstate
their knowledge and skills in order to land a desirable job or a promotion.
Likewise, training providers may be motivated to mask the quality of their
program offerings in the interest of attracting and retaining their clientele.
Inadequate information aside, financial constraints may also prevent those who
can benefit from training from making the investment, thus leading to
suboptimal investments in WfD.
The foregoing considerations motivate us to specific the following three
policy goals for the oversight dimension of our analytical construct: (a)
diversifying the pathways for skills acquisition so as to encourage lifelong
learning as a basis for adapting to evolving labor market conditions and keeping
up with technology in the workplace; (b) ensuring efficiency and equity in
funding WfD investments, thus fostering the system’s long‐term financial
sustainability; and (c) assuring relevant and reliable standards for quality in
service provision and skills acquisition, thus creating more efficient signals in the
market for skills.
Policy Goal 4: Diversifying the pathways for skills acquisition. The
underlying aim here is to keep the skills of the workforce up to date and adapted
to changing economic conditions and opportunities. Some of the main reasons
that make lifelong learning an imperative today include the emergence of new
occupations and skills, and the rapid transformation of existing ones, the
explosion in knowledge and technology, the shift to an information society, the
introduction of new ways to organize the workplace in an increasingly
integrated economy, and an aging society (see, for example, Kiley and Cannon
2000; Wang 2012).
However, lifelong learning, even for the motivated, may be haphazard or
even impractical if few pathways for skills acquisition exist beyond the initial
level of education and areas of training that a young person started out with. The
24 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
absence of a qualifications framework and of routine procedures for the
recognition of prior learning presents an added impediment. Individuals without
the formal credentials for admission into their chosen training programs are
assumed to have no relevant skills and must thus start their training at a more
basic level than necessary. The result is to prolong training time and increase
costs for some prospective trainees, which may in turn discourage them from
embarking on plans to learn new skills or build on existing ones. Finally, given
that labor market conditions are always evolving, it is important that all available
options for gaining more and better skills, particularly those funded publicly,
remain responsive and sensitive to emerging trends and opportunities for
employment. Thus, the ability to adjust or terminate programs that no longer fit
the bill or to add new ones is a hallmark of effective oversight. Taking the
foregoing considerations into account, we associate with the goal of diversifying
the pathways for skills acquisition the following actions by the government:
Foster articulation across levels of instruction and types of programs;
Promote lifelong learning with recognition of prior learning; and
Set policies and procedures for the renewal of publicly funded programs.
In a well‐articulated system for WfD, students can transfer from course to
course, progress to higher levels of training or gain access to programs in other
fields (MacKenzie and Plovere 2009). Good examples of articulation
arrangements can be found in the United States, where many community
colleges adopt such practices as: (a) give students credit for work experience in
industry; (b) collaborate with high schools to offer preparatory courses so that
students may fulfill the prerequisites for full‐time enrollment in a community
college; and (c) enter into agreements with universities and other colleges, most
often within their respective states, to provide students with the option to
continue on to college‐ or university‐level programs once they have proven
themselves in a community college program.37 In some countries, a national
qualifications framework provides the scaffolding for defining pathways for
lifelong learning and skills acquisition. As Hanf and Hippach‐Schneider (2005)
explain, the framework essentially “takes all of a countryʹs formally recognized
qualifications and arranges them in a clearly defined structure…. [Qualifications
are] understood as sets of certified or documented skills, with no regard given to
the respective learning path.”
Interest in national qualifications frameworks has grown in recent years and
more than 100 countries worldwide are now reportedly at various stages of
building their own.38 A review by Allais (2010) of the experiences in 16 countries
37 For example, the community colleges in the states of Virginia and Maryland have such agreements
with their respective state colleges and universities. 38 Part of the interest stems from employers in countries that employ foreign workers. For these
employers, the qualifications framework in the sending countries is a tool for gauging the quality of
potential recruits from these countries. Another source of interest stems from the growing demand for
intercollegiate credit transfers and articulations across programs and institutions.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 25
cautions against overly ambitious expectations of such frameworks, however.
Success requires significant resources and patient work by skilled personnel over
many years—conditions that are often absent in developing and emerging
economies. In addition to fostering articulation and the mapping of
qualifications, countries have also found it necessary to exercise oversight of
program offerings as well, particularly in publicly funded institutions, to
minimize what ADB (2009) notes is a ubiquitous tendency to repeat courses year
after year without regard to labor market conditions. Insisting on industry
involvement in decisions about program offerings is an increasingly popular
method for providing oversight of the content of publicly funded training
programs.
Policy Goal 5: Ensuring efficiency and equity in funding for WfD. The
underlying design challenge here is to influence WfD catalytically through the
government’s role in funding WfD, ensuring efficient and effective use of the
available funds, and in fostering partnerships that can multiply the resources,
pecuniary and otherwise, that are available to encourage and support investment
in WfD by individuals and employers. Governments fund WfD in part to
counteract possible suboptimal levels of investment arising from such problems
as credit constraints (for example, those faced by students from poor families),
myopia on the part of prospective trainees and firms, incomplete information, or
other imperfections in the market for training. Because funding for WfD is
required on a continuing basis while public budgets are often limited and
subjected to intense competition from other competing projects, a clear strategy
for sustaining support for WfD, one that benefits from continuous assessment
and adjustment, can help ensure that the funding is stable, sustainable, and
sufficient to achieve the most important goals at reasonable cost. Such a strategy
would recognize that besides public budget allocations, resources for WfD may
also come from fees paid by trainees or their sponsors, and contributions by
private entities, nongovernmental organizations, local communities, and/or
external donors. Some of the contribution may materialize in the form of in‐kind
donations that constitute essential inputs for effective training programs,
including, for example, equipment and the services of experienced skilled
workers. In light of the foregoing considerations, we choose the following actions
by the relevant government authorities to flesh out the goal of ensuring
efficiency and equity in funding WfD investments:
Articulate a strategy for funding WfD;
Allocate public funds for WfD to achieve results with efficiency; and
Foster partnership between WfD authority(s) and stakeholders.
In practice, funding is a ubiquitous concern in WfD systems (see, for
example, Wolf and Erdle 2009). In poorly funded systems, providers are hard
pressed to satisfy the expectations of employers who in turn distance themselves
from the providers; the result is to reinforce providers’ tendency toward supply
26 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
driven program offerings, which further undermines their ability to respond to
employers’ needs. These conditions reduce the potential for productive
partnerships in WfD. Many systems in developing and emerging economies fall
victim to this vicious circle because governments simply do not have enough
resources to prioritize TVET when other parts of the education system, such as
primary and secondary schooling, are also poorly funded (see, for example,
Atchoarena 2002; Dunbar 2011). Support for TVET is moreover often weakened
by the fact that most students come from poorer families that typically lack the
political clout to influence budget allocations, as well as by the general
perception of TVET as a “second‐class” route to the labor market with doubtful
impact in equipping students with job‐relevant skills.
Some countries have nonetheless successfully tackled some of these
problems. In Switzerland, for example, TVET tracks at the secondary level offer
excellent programs and pathways to tertiary level courses, which attract a
sufficiently meaningful share of the country’s top students to overcome public
bias against such programs (OECD 2010a; FOPET 2012). In Singapore, sizable
investments in a high‐quality TVET system over the years, coupled with
sustained attention to the employability of graduates, have lowered social
resistance to the TVET programs that the less academically inclined students
enter after 10 years of general schooling (Law 2008).
With regard to funding for the training of incumbent workers, a common
arrangement involves collecting a tax or levy from firms, often in relation to the
size of the payroll, and using the proceeds to reimburse firms that train
(Johanson 2009). Some observers cite Malaysia and Korea as examples of
successful schemes (for example, Lee 2009; Tan 2001). As for allocating funding
to achieve results, having transparent and explicit allocation criteria and
ensuring that the criteria themselves are reviewed and adjusted in light of
experience, are a sine qua non of good oversight of WfD resources. Accordingly,
some recent World Bank operations that support TVET, for example in Pakistan
and India, seek to foster these practices.39 Regarding partnerships, the
relationship between Costa Rica’s training institutions and Intel mentioned
earlier exemplifies arrangements that have been sustained over time because
they generate benefits for all parties: the firm, the participating training
institutions, and the trainees.
Policy Goal 6: Assuring relevant and reliable standards for quality in WfD.
The underlying issue is trust in the services offered by training providers and in
the skills acquired by individuals. Providers are keen to assure potential clients
of the quality of their programs and to market their services, particularly when
39 For example, the Sindh Skills Development Project and the India Vocational Training Improvement
Project; more information can be found on these projects at, respectively, the following sites:
http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&me
nuPK=228424&Projectid=P118177; and
http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&me
nuPK=228424&Projectid=P099047 (accessed on October 13, 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 27
these are offered on a fee‐for‐service basis. Obtaining accreditation from a
recognized authority is an efficient way to do so. The process involves an
authorized accrediting body certifying, typically on a term‐limited and
renewable basis, that the training provider has met externally established
standards for excellence or quality in service delivery (Vlăsceanu et al. 2007).
Standards for skills acquisition are a related but separate aspect of quality
assurance. Often codified in the form of a credential or qualification, they
provide benchmarks for testing and certifying the knowledge and competencies
of the graduates of training programs. Established standards help training
providers clarify expectations about learning outcomes and competencies; they
benefit trainees by providing an easy way to advertise their skills to potential
employers; and, if reliable, the standards also assist employers in expediting the
early stages of recruitment. In light of the foregoing ideas, we associate with the
goal of assuring standards the following actions:
Specify accreditation standards for training providers;
Strengthen skills testing and certification; and
Assure the credibility of accreditation and of skills certification.
Countries face a variety of challenges with regard to quality assurance and
standards in WfD. In recent years, the rapid expansion of enrollments at all levels
of education and training in many developing and emerging economies has
overburdened their public institutions and stimulated an explosive growth of
nonstate providers (see, for example, King and Palmer 2010). The diversity in
options for WfD investments is a positive development, but in the absence of
adequate standards or regulation it could also mean a proliferation of
substandard programs (see, for example, Atchoarena 2002; Dunbar 2011).
Specifying minimum standards is one way to weed out the worst operators
while allowing others to reach for a higher bar, if desired—for example, by
obtaining accreditation with internationally recognized bodies such as the
International Standards Organization (ISO).40
Experience with standards for skills certification is highly diverse as well.
Ganzglass et al. (2011) stress the importance of several factors, including
stakeholder involvement in setting standards, benchmarking of the standards to
employers’ requirements, and reliable processes for validating the standards and
for assessing learning. The system adopted by the National Institute for
Metalworking Skills in the United States is an example of the application of these
principles.41 In less developed nations, where the informal sector dominates the
economy, the testing and certifying of skills present special challenges, in part
because many skills are acquired through traditional apprenticeships and
40 Singh and Sareen (2006) note the rising popularity of ISO 9000 certification among Indian educational
institutions. 41 National Institute for Metalworking Skills. Undated. “Skills Standards,” online: https://www.nims‐
skills.org/web/nims/7 (accessed June 17, 2011).
28 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
standards are often tacit and probably uneven across practitioners (Atchoarena
2003; de Largentaye 2009; Janjua and Naveed 2009). Nonetheless, in the Indian
government’s nationwide Modular Employable Scheme, launched in 2010, a key
feature is to codify the standards for skills acquisition and certification for a wide
spectrum of jobs, many of them in the informal sector.42
Dimension 3: Service Delivery
This functional dimension of policy choices shifts our focus from the strategic
and systemic to the operational. It addresses implementation challenges in WfD
that stand between a country’s ambition for WfD to support economic
development and what materializes in reality. Training providers, both
government and nonstate, are the main channels through which the country’s
strategic and systemic policies are translated into operational applications. Thus,
we disaggregate this policy dimension into the following three policy goals: (a)
fostering relevant curricula content in training programs; (b) designing
incentives to promote excellence in training provision; and (c) enhancing
accountability for results in WfD, notably through appropriate monitoring and
evaluation arrangements.
Policy Goal 7: Fostering relevant content in training programs. Because a
key objective in WfD is to equip graduates with job‐relevant skills, the content of
training programs warrants close attention. In order to offer market‐relevant
programs, training providers require access to reliable sources of information
about current and emerging skills demand. Employers and industry associations
are one source of such information. However, in the absence of systematic
arrangements it may be difficult and probably too costly for individual training
providers to gather the desired information. Another source of information,
particularly in relation to future skills needs, is research institutions. Because the
mission of such institutions places them at the forefront of developing, adapting,
and introducing products and services that are new to the country or region,
their activities can offer useful insights into the future demand for skills. Besides
the linkages at the group level, each training provider can also benefit from close
ties with the specific employers or industry that they serve. Such ties are an
important source of inputs and guidance on ways that an individual provider
can improve the market relevance of its program offerings.
Important as the curriculum is, it is insufficient by itself to ensure effective
training provision; needed too are skilled instructors and administrators as well
as appropriate facilities, equipment, and materials. Each provider must be able to
attract the number, types, and quality of staff required to deliver the expected
services within its budget constraints. Because good TVET teachers are highly
sought after by industry for their technical skills, training providers are often
hard pressed to satisfy employers’ demand for skills unless they are able to find
42 See http://dget.nic.in/mes/sectors.htm (accessed on July 30. 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 29
creative solutions to their staffing constraints. Training providers must also have
access to the facilities, equipment, and instructional materials required for
implementing their training programs. These items in the WfD system are the
next most costly after teachers; their costs per instructional hour per trainee are
often several times as high as those in academic programs at comparable levels
of instruction. In modern WfD systems, the challenge of providing trainees with
experience in handling actual production equipment is sometimes met through
the use of competency‐based modular curricula that include on‐the‐job training
as part of the program. While attractive, this option is often difficult to organize
in low‐income settings.
Taking into account the foregoing considerations, we choose the following
policy actions to define the goal of fostering relevant content in training
programs:
Strengthen linkages among training institutions, industry and research
institutions;
Integrate industry inputs into the design of training programs; and
Enhance competence of WfD administrators and instructors.
The relationship between the Korea University of Technology and Education
(KUT) and Samsung exemplifies all three facets of the delivery of highly relevant
training content in a high‐tech industry.43 KUT hosts the Employee Vocational
Education Program (EVEP), which offers short courses to Samsung’s own
workers as well as those from the firm’s subcontractors. KUT provides the
premises, and the teaching and administrative staff. Samsung, for its part, leads
in defining the course curricula, provides the equipment, and lends in‐house
experts to co‐teach the courses. The collaboration benefits all parties. Samsung
gains from having subcontractors that can meet its product specification and
quality standards; its subcontractors gain from being able to retain or expand
their business with the company. Most importantly, the university gains from the
opportunity to involve its faculty in organizing and teaching the courses. This
involvement enhances the competence of its administrators and instructors in the
design and delivery of industry‐relevant training programs.
Linkages of this nature are not confined to the industrial sector or to new
and emerging industries. In China, the Beijing Agricultural School (BAS), for
instance, has established linkages with “industry” in the form of farmers and
village communities to disseminate the fruits of agro‐technology research in the
agricultural sector and upgrade the technological competencies of farmers—the
incumbent workers—through formal and nonformal training programs
(UNESCO 2002). The collaboration has resulted in a diversification of
agricultural and animal husbandry products, higher yields, and higher incomes
for the farmers and villages. BAS’s faculty benefit from their hands‐on
interactions with the farmers and villagers, for instance, through problem‐
43 This information is based on a personal visit to KUT in May 2011 by two of the authors of this paper.
30 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
solving and developing new managerial practices to meet the needs of their
trainees. The school itself has benefitted through the broadening of its
curriculum, which has become more responsive to the needs of the rural
population. BAS’s infrastructure has been strengthened through the
establishment of experimental facilities in the form of test fields, animal
management technique testing grounds, and rural mechanics services.
Policy Goal 8: Incentivizing excellence in training provision. Training
providers play a key role in equipping individuals with skills for employment
and productivity. The incentives they face are therefore an essential link in the
results chain in workforce development. Where providers must compete for a
share of the training market they are more likely to customize program offerings
to meet individuals’ and firms’ demand for training. This responsiveness in
training provision is critical, given the nature of employers’ demand for skills in
today’s highly competitive economic environment. Globalization and
technological advances, in particular, have produced conditions for substantial
churning in the labor market as new jobs replace existing ones and create a
continuous demand for fresh skills. Because this demand is impossible to predict
with a high degree of precision, the responsiveness of the WfD system depends
not on top‐down planning but on having a diversity of providers that are
appropriately motivated to meet the demand for skills (see, for example, Booth
and Snower 1996).
Financial viability is a powerful incentive that conditions the behavior of
many nonstate training providers. Providers that depend largely on the fees paid
by individuals or firms (typically on behalf of their employees) have strong
incentives to explore the specific skills required by employers and to tailor their
training programs accordingly. By working closely with employers and
developing a reputation for responsive and effective training, these providers
can improve the attractiveness of their services, thereby assuring the commercial
viability of their business. A downside of the profit motive, however, is that it
may drive providers to cut corners as a cost‐saving measure or hide serious flaws
in their operations that are difficult for a poorly informed public to detect. In
some countries, weak regulation of standards in a context of rapid expansion has
led to a proliferation of private sector providers, many of them offering
substandard services (see, for example, Atchoarena 2002).
Relying on state providers is no panacea, however, for the problems of poor
quality services. In many developing and emerging economies, constraints in
public funding typically limit the scope of publicly provided training programs.
Moreover, state provision suffers from a catalogue of common deficiencies that
include excessive centralization of management, limited scope for initiative and
flexibility at the level of individual institutions, outdated curricula, inadequate
infrastructure, poorly trained staff, and tenuous or nonexistent links with
employers (see, for example, Dunbar 2011; ADB 2009). The combination of weak
incentives and low capacity imply that providers will, at best, be slow in
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 31
addressing employers’ demand for skills and, at worst, be completely
unresponsive.
In light of the foregoing considerations, we propose the following actions for
the policy goal of providing incentives to foster excellence in training provision:
Promote diversity in training provision;
Incentivize private providers to meet WfD standards; and
Motivate public training institutions to respond to the demand for skills.
An example of action to diversify training provision includes Kenya’s large‐
scale voucher program for workers in the informal sector (also known as the “Jua
Kali” sector). The intervention increased competition and stimulated the
provision of training tailored to the unique needs of informal sector workers
(Johanson and Adams 2004). More recently, in 2008, another experiment with
vouchers in Kenya offers suggestive evidence that institutions that enrolled
voucher winners were more likely to expand their course offerings compared to
institutions that did not enroll any voucher recipients (Hicks et al. 2011). In
Australia, the government of the state of Victoria introduced a user‐driven
funding model that it argues would “drive greater competition (thereby
achieving increased accountability, accessibility and quality) and respond more
effectively to changing labour market needs” (Skills Victoria 2011).
To reinforce the benefits of competition in training provision, it is important
to ensure that private providers satisfy some minimum standards for quality.
The Singapore Accreditation Council (2008), for example, encourages providers
to seek accreditation in order to assure potential clients of the quality of the
services being offered. Linking access to public funding to accreditation status is
one way to minimize the proliferation of low‐quality programs while promoting
diversity in the training market. For providers in the public sector, the more
important challenge is to improve their responsiveness to employers’ demand
for skills. A common approach is to stipulate the inclusion of industry or
employer representatives on the boards of training institutions. In the United
States, for example, Lindburg (2009) notes that most of California’s community
colleges have institutionalized advisory boards to which they turn for guidance
and inputs to ensure that course offerings are relevant to industry needs. Among
developing countries, the practice is spreading, although evidence on its impact
is still unclear; Mozambique and Pakistan are two examples where boards are
being set up under ongoing World Bank–financed operations.
Policy Goal 9: Enhancing accountability for results in WfD. Systematic
monitoring and evaluation of skills development policies and programs and their
impact on outcomes are a hallmark of strong WfD systems. This function gives
policy makers and other interested parties valuable information about the
current status of WfD and its alignment with the country’s economic
development strategy. It also enables overseers of the system to keep track of the
activities and progress of training providers. Appropriately designed, it can also
32 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
help to separate promising options from dead ends for improving WfD, thereby
enriching the evidence that informs policy dialogue and reform. Monitoring and
evaluation are thus essential tools for ensuring that the country’s current WfD
policies and institutions are in fact delivering the expected outcomes.
Three aspects of monitoring and evaluation warrant special attention. The
first is the pervasiveness of the culture and practice of collecting, sharing, and
analyzing data to discern trends in skills demand and supply, and emerging
challenges in gaps and mismatches. In developing and emerging economies, data
on WfD and related issues are often limited in quality, scope, timeliness, and
accessibility. Educational attainment and self‐reported literacy are frequently the
only measures of the supply of “skills” that are available to inform policy
analysis and dialogue. For a few countries, scores on international cognitive tests
provide additional insights. These data are useful but insufficient in light of
recent research and feedback from employers about skills mismatches and the
importance of cognitive skills (as opposed to educational attainment alone) as
well as noncognitive skills for workplace success in the twenty‐first century (for
example, di Gropello et al. 2011a and 2011b; Sondergaard and Murthi 2012).
A second aspect of monitoring and evaluation that is relevant from the
perspective of the overseers of the WfD system is the reporting required of
training providers. Trainees’ employment, earnings, and retention by their
employers are the key measures of the success of training programs. Once this
principle is understood, the focus of monitoring and evaluation can be more
clearly defined and targeted. Although training programs may have several
monitoring requirements, the evaluator will be most interested in how cost‐
effective they are in equipping trainees to satisfy the needs of the employers. By
requiring providers to report on their operations and outcomes, the overseers of
the WfD system would be able to create a centralized database for tracking
activities in the system as a whole. This can also provide the comparative data
for individual providers to evaluate their own performance.
Finally, monitoring and evaluation also play a critical role in helping to
identify promising innovations in WfD. This function is essential for keeping up
with changes in skills demand and new ideas in training methodology. While the
innovations may relate to any aspect of the system (for example, finance,
governance, and administration), their common goal is to boost the system’s
performance, either at the systemic or institutional levels. Adapting from the
model in Mulgan and Albury (2003), the process of innovation typically passes
through four phases: (a) generating possibilities through the stimulation and
support of new ideas; (b) incubating and prototyping to develop and test the
most promising of the new ideas; (c) replicating and scaling up the best (that is,
most cost‐effective and most relevant to the local context) among the tested ideas
in a timely and sustainable manner; and (d) distilling and codifying lessons to
institutionalize knowledge and foster a culture of continuous learning and
improvement.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 33
The foregoing considerations suggest that the goal of enhancing
accountability may be advanced through actions that:
Strengthen WfD monitoring and evaluation;
Specify reporting requirements by training institutions; and
Increase the focus on outcomes, efficiency, and innovation in service
delivery.
Efforts to strengthen the monitoring and evaluation function in WfD systems
are being made in many countries. A recent example is the collection of new,
internationally comparable data on workforce skills by the member countries of
the Organization for Economic Co‐Operation and Development (OECD) under
the Programme for the International Assessment of Adult Competencies
(PIAAC), which aims to provide a new source of information about the quality of
skills in the adult population and the link between skills and employment and
productivity (Schleicher 2008). A similar effort by the World Bank known as the
Skills toward Employment and Productivity (STEP) Measurement Study is
underway to support the collection of similar data by national teams in eight
low‐ and middle‐income countries. Korea is an example of a country where
policy makers have access to a compilation of comprehensive and up‐to‐date
data from surveys and other sources to inform policy design (see, for example,
KRIVET 2007). In terms of reporting requirements, WfD systems such as
Singapore’s follow the good practice of requiring public sector training providers
to issue annual reports as an arrangement for accountability.44 With regard to
strengthening the focus on outcomes, efficiency, and innovation, a welcome
trend is the growing practice of rigorous impact evaluation in the field of skills
development, including in the setting of developing and emerging economies.
Recent examples include evaluations of youth training programs in several Latin
American countries (for example, Ibarraran and Shady 2009) and of programs
targeted at adolescent girls in countries around the world.45
Summary of the Functional Dimensions and Related Policy Goals
In summary, the SABER‐WfD dataset has an analytical structure that recognizes
three broad functional dimensions of policy choices and institutional processes
for workforce development: strategy, oversight, and service delivery. For each
functional dimension, it highlights policy goals and related actions that address
market and government failures in governance, finance, and information. Figure
4 recapitulates the policy goals explained above while Appendix 1 provides
additional details.
44 For examples, see the annual reports of the Singapore Workforce Development Authority at
http://www.wda.gov.sg/content/wdawebsite/L209‐001About‐Us/L209D‐AnnualReports.html (accessed
August 12, 2012). and of one of Nanyang Polytechnic, at http://www.nyp.edu.sg/about‐nyp/media‐
centre/publications/annual‐report (accessed August 12, 2012). 45 For more details on the Adolescent Girls Initiative at the World Bank see http://go.worldbank.org/
5PYHEZS360 (accessed on November 9, 2011).
34 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Figure 4: Summary of the Functional Dimensions and Policy Goals in the SABER‐WfD Framework
Dimension Policy Goal Action
9 Actions
9 Actions
9 Actions
Source: Authors’ construction.
Note: See Appendix 1 for details on the policy actions.
Design of the Data Collection Instrument
For each policy action the SABER‐WfD data collection instrument (DCI) poses a
set of questions to which answers are supplied from the perspective of the line
ministries responsible for various aspects of WfD. In general, the answers are
designed to document the existence of policies and their scope, as well as
evidence on policy implementation and feedback loops to inform policy
adjustments and improvements.
The instrument is similar to other surveys in some ways but also differs from
them in important aspects. Like other surveys, the SABER‐WfD DCI has an
intentionally parsimonious design in order to manage the costs and timeliness of
data collection. Because the data are meant for benchmarking, the DCI
emphasizes the collection of diagnostic rather than comprehensive data about
WfD policies and institutions. Further, the fact that the respondent is not a
person but a country and its system of WfD implies that the data must be
gathered indirectly, typically from documents and credible informants. Most of
the data are, moreover, qualitative rather than quantitative in nature, which
presents challenges of precision and objectivity in data collection. We discuss
below the approach taken to address these multiple concerns.
Data for benchmarking. Following the practice in the World Bank’s overall
SABER initiative, WfD policies are benchmarked to four stages of development:
latent, emerging, established, and advanced. The options for answering
questions in the data collection instrument are thus designed to help locate
policies in relation to these four stages. Given the qualitative nature of the data
• 1. Clarifying the direction for WfD
• 2. Prioritizing a demand‐led approach
• 3. Strengthening critical coordination Strategy
• 4. Diversifying pathways for skills acquisition
• 5. Ensuring efficiency and equity in funding
• 6. Assuring relevant and reliable standardsOversight
• 7. Fostering relevance in training programs
• 8. Incentivizing excellence in training provision
• 9. Enhancing accountability for results Delivery
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 35
being collected, we begin with a generic rubric (see figure 5) and adapt the scale
to the data collected by the instrument for each of the 27 actions identified in our
framework.
Figure 5: Generic Rubrics for Benchmarking WfD Policies
Source: Authors’ construction.
To illustrate, consider the functional dimension relating to strategy. One of
the three policy goals focuses on clarifying the direction for workforce
development. This goal is itself disaggregated into three actions, one of which
relates to advocacy for workforce development in line with national goals for
economic growth and poverty reduction. The DCI poses two questions on this
action: (a) the type of leadership that advocates for WfD; and (b) the
prioritization of WfD in the country’s agenda for economic development. The
answers are used to benchmark the policy action according to the following
rubric:
Latent: WfD is not prioritized; it does not enjoy the advocacy of visible
champions;
Emerging: Political and other leaders recognize the importance of WfD
for economic development; steps are being taken to develop relevant
policies for skills development and to build institutional capacity for
policy implementation;
Established: Political and other key leaders in industry and the administration provide sustained support for WfD; the institutional
arrangements have been established and WfD priorities are being
implemented; and
Latent•Limited engagement
Emerging
•Some instances of good practice
Established
•Systemic good practice
Advanced
•Systemic good practice meeting global standards
36 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Advanced: WfD is fully integrated into national policies and strategies,
reflecting a holistic approach to WfD; 46 the institutional arrangements for
implementation are fully functional and capable of continuous self‐
evaluation and improvement.
The rubrics for the other 26 policy actions in our tool appear in Appendix 2.
Where the questionnaire relating to a particular action contains several
questions, the answers are aggregated by putting more weight on praxis than on
concepts or plans that exist on paper.47 By assigning a value of 1 for a latent level
of policy development, 2 for an emerging level, 3 for an established level, and 4
for an advanced level, we can create scores for all 27 policy actions for which the
SABER‐WfD data collection instrument gathers data. Taking advantage of the
nested structure of our analytical approach, we compute the scores for the nine
policy goals from the scores for the corresponding policy actions; the calculation
simply involves averaging the scores for the three policy actions associated with
each policy goal. Likewise, the scores for the functional dimensions are a simple
average of the scores for the relevant policy goals.
Balance between diagnostic and comprehensive data. The idiosyncrasy and
complexity of WfD systems present challenges for the design of the DCI. Because
the data are intended for benchmarking WfD systems across countries, all the
questions in the instrument offer the data collector a fixed menu of answers from
which to choose as well as the option to provide additional information that may
be taken into account, as appropriate, in scoring the choices made from the fixed
menu. This feature helps to avoid the loss of potentially important country‐
specific information while minimizing the difficulty of processing data from a
fully open‐ended questionnaire design.
A more difficult challenge relates to the complexity of WfD systems. This
complexity comes to the fore in the context of questions in the DCI relating to the
operational aspects of WfD system. WfD systems typically involve multiple
authorities, often with complementary as well as overlapping responsibilities.
The training programs that are available may target clients at different levels of
instruction, and training providers are likely to serve a diversity of employers
from different sectors of the economy. In large countries, the WfD system may be
highly differentiated across regions. In light of this complexity, the temptation is
to collect comprehensive data on all aspects of the system. However, such a
strategy requires substantial time, effort, and resources, and may in fact exceed
the data required for our needs.
46 A holistic approach is one that ensures systemic coherence by addressing multiple dimensions of skills
development, including: (a) aligning skills training to employers’ needs and national investment
priorities; (b) building strong foundational skills through early childhood development and general
education; and (c) meeting the needs of vulnerable populations (for example, informal sector, traditional
agriculture, and so forth). It also recognizes that interventions often require different time frames to
achieve their impact. 47 Details on our weighting procedure are available from the authors upon request.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 37
As a practical strategy, we proceed on the assumption that the WfD system
may be benchmarked based on the functioning of its most salient or dominant
components. It puts all systems in the best possible light, thus removing some of
the ambiguity inherent in the benchmarking exercise. Unless a single WfD
authority oversees both pre‐employment and in‐service training, the DCI
requests information about the ministries most likely to be involved in oversight
and service delivery functions; in most cases, the ministry of education and the
ministry of labor are the key WfD authorities. To take into account country
specificities, the DCI includes an option for the data collector to include other
ministries should these play an important role in the system. The DCI also
requests information separately for two levels of instruction: secondary and post‐
secondary (non‐university). For a few questions in the DCI, meaningful answers
are possible only in relation to particular economic sectors (for example,
regarding the linkages between training provision and industry) or occupations
(for example, the organization of skills testing). In such cases, the DCI requests
information for the two most dominant or fastest‐growing sectors or
occupations. For large countries, data collection through the DCI is more feasible
and meaningful at the subnational level rather than for the country as a whole.
Data on the de jure and the de facto situation. In line with its intended
purpose, the DCI contains many questions about WFD policies, processes, and
institutions. However, the responses to the questions may reflect the de jure
situation rather than the reality on the ground, thus compromising the validity of
the benchmarking results. For example, many low‐income countries have well‐
written plans to make WfD a key asset for economic development and some even
report plans for sophisticated funding systems for WfD (for example, vouchers
for individuals to pay for skills development). Yet, for a variety of reasons,
including weak capacity and lack of resources, the plans are often not
implemented. In such situations, it would be inappropriate to benchmark the
system based entirely on the intended policies rather than on actual practice. The
difficulty, however, is that a full‐fledged data collection effort to verify
implementation of the stated plans and policies would cost too much and take
too long.
To strike a balance, we adopt the strategy of adding a few more questions
under each topic of inquiry in the DCI to find out if the most important plans,
policies, or programs among those listed, typically no more than the top two,
have been evaluated for impact, and if the recommendations from the
evaluations have led to follow‐up actions. The information clearly does not fully
capture the de facto situation and it is certainly no substitute for a more
comprehensive documentation of the status of policy implementation. The
approach nonetheless provides sufficient information to make it possible to
situate the policy action being benchmarked in relation to the four stages of
development used in our benchmarking exercise.
38 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Piloting the SABER‐WfD Methodology and Expected
Outputs
The SABER‐WfD benchmarking project is best viewed as an exploratory effort by
the World Bank and its development partners to systematize and share cross‐
country knowledge about effective policies and practices. Thus, a spirit of
caution and openness to learning accompanies the implementation of the data
collection instrument described above. Our hope is that this learning by doing
approach would provide a basis for refining both the survey instrument and the
process of data collection. As more data are accumulated, the project should help
deepen our knowledge about the links between policies and approaches and their
impact on outcomes in WfD. This knowledge would in turn inform dialogue about
reform priorities to advance WfD in low‐ and middle‐income countries.
An implementation plan with four phases. The four chronologically
sequenced phases are: (a) creation of a data collection instrument; (b) testing of
the data collection instrument in a first cohort of countries; (c) data analysis and
production of benchmarking results for use in preparing a SABER‐WfD
barometer report for each country; and (d) distillation of lessons to improve the
analytical strategy, the data collection instrument, and the protocol for data
collection and analysis. These resources would facilitate efficient application of
the SABER‐WfD approach in other countries.
As of this writing, the first phase has been completed following
consultations with key experts from around the world.48 The second and third
phases have been completed or are nearing completion in the following
countries: Chile, Ireland, Korea, Singapore, and Uganda. These countries were
selected opportunistically to test the methodology and the working hypotheses.
Ireland, Korea, and Singapore exemplify countries that have made skills
development a critical part of the development strategies and have achieved
undisputed progress over several decades. The inclusion of Chile and Uganda in
the pilot cohort took advantage of the availability of significant knowledge
available to the research team in the context recent World Bank operations and
analytical work. They also allow us to test the relevance of the methodology in
the context of low‐ and middle‐income countries.
As of this writing, a process of review is underway to collect feedback in
order to improve and update the data collection instrument, to enhance the
48 The formal consultations include: (a) a July 2010 meeting with World Bank staff involved in skills‐
related operations and analytical work; (b) an October 2010 consultation with a 16‐person external
advisory group comprising of seasoned TVET leaders from around the world; and (c) a January 2011
workshop with World Bank retirees with significant experience and expertise in the field of skills
development and policy dialogue in developing countries. The interlocutors represent the perspectives
of academics, practitioners, and development specialists and bring together knowledge and expertise
covering a wide range of country perspectives. Feedback from the group has helped to refine all
dimensions of the data collection instrument.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 39
presentation of the findings and to reflect on effective ways to apply the findings
in policy dialogue.49 In response to requests from various quarters, data
collection has started in a number of countries in East Asia (for example,
Malaysia and Vietnam), the Middle East (for example, Turkey), and Europe and
Central Asia (for example, Bulgaria and Romania).
Key processes. Work on the first cohort of countries has proceeded smoothly.
The experience is already informing the approach for countries interested in a
SABER‐WfD benchmarking study.50 Once the decision has been taken to prepare
such a study, the first step is to decide on the assignment of responsibility for
data collection and report writing. Typically, a national team comprising a
seasoned principal investigator and other researchers would perform these tasks.
As the data are gathered, the team may organize work‐in‐progress discussions
with the relevant government counterparts as well as World Bank staff and
others for the purpose of enhancing the completeness and reliability of the data.
Following completion of data collection the data will be processed based on
standard scoring rules in order to generate benchmarking scores that place the
policy actions, policy goals and functional dimensions on the four‐point rubric
developed for the SABER‐WfD study. Upon validation of the benchmarking
results through consultation with national counterparts and other interested and
knowledgeable parties, the national team would prepare a SABER‐WfD Country
Report to summarize the findings. The report would serve as a resource for
ongoing dialogue among the national team and its development partners
regarding WfD policy priorities and reform.
As the SABER‐WfD benchmarking study expands to include more countries,
a systematic cross‐country database on WfD policies and institutions is expected
to emerge. This resource could provide insights for testing and refining the three
hypotheses presented earlier in the paper. At the same time, the accumulation of
experience would shed light on the relevance and usefulness of the
benchmarking approach for supporting policy dialogue on WfD challenges. The
experience is expected to generate ideas for continuous improvements in the
technical resources associated with the SABER‐WfD initiative, including the
conceptual framework, the data collection instrument and protocols, and the
approach for data scoring and report writing.
Expected outputs. Consistent with the experimental nature of the SABER‐
WfD project, the expected outputs are best seen as vehicles for learning. Specific
outputs from this phase of the work includes: (a) a first version of the SABER‐
WfD data collection instrument, which will be updated based on the experience
in applying it in the pilot cohort; (b) rubrics for benchmarking policies for WfD
(see Appendix 2); (c) codified procedures and templates to guide data collection
and processing; (d) SABER‐WfD Country Reports for the first cohort of countries;
49 See, for example, http://blogs.worldbank.org/education/voices‐from‐former‐african‐ministers‐of‐
education‐on‐challenges‐of‐workforce‐development. 50 This interest would typically be conveyed via the World Bank’s task team leader for the country.
40 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
and (e) the beginnings of a network of WfD practitioners within the World Bank
and in partner organizations and countries that are equipped to implement the
benchmarking tool and process.
Conclusion
In the quest for economic growth, many countries see workforce development,
often in the form of technical and vocational education and training, as an
essential investment. However, TVET has continued to pose daunting challenges
because it is expensive and has often failed the test of meeting employers’ skills
requirements. Although the mismatch between what the system supplies and
what employers demand has many causes, some of them beyond the control of
training programs, policy makers are keen to learn from the experiences of
countries that have made progress in equipping trainees with job‐relevant and
productivity‐enhancing skills. Some of them have turned to the World Bank,
among others, for assistance in mobilizing this knowledge to inform efforts to
improve the effectiveness of their workforce development strategies. This paper
is a response to this growing interest among the Bank’s partner countries. The
work is admittedly experimental at this stage. It has involved the articulation of a
conceptual framework and an analytical strategy as well as the creation of a new
data collection instrument and its application in a first cohort of countries. It is
hoped that the accumulation of experience from this phase marks the beginning
of a process that would in time add a new tool to the toolkit that policy makers
and their development partners can use in the search for better policies and
practices to improve outcomes in workforce development.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 41
Appendix 1: Functional Dimensions, Policy Goals,
and Actions in SABER‐WfD
Dimension Policy goal Action
Strategic Framework
Articulating a Strategic Direction
Advocate for WfD as priority for economic development
Evaluate economic prospects for skills
Develop policies to align skills demand and supply
Prioritizing a Demand-Led Approach
Promote a demand-driven approach to WfD
Strengthen firms' demand for skills to improve productivity
Address critical challenges in the future supply of skills
Strengthening Critical Coordination
Ensure coherence of key strategic WfD priorities
Institutionalize the structure of WfD roles and responsibilities
Facilitate communication and interaction among all WfD stakeholders
System Oversight
Diversifying Pathways for Skills Acquisition
Foster articulation across levels and programs
Promote lifelong learning
Set policies and procedures to renew programs
Ensuring Efficiency and Equity in Funding
Articulate funding strategy
Allocate funds to achieve efficient results
Foster partnerships
Assuring Relevant and Reliable Standards
Specify accreditation standards
Strengthen skills testing and certification
Assure credibility of accreditation and of skills testing
Service Delivery
Fostering Relevance in Training Programs
Link training providers, industry and research institutions
Integrate industry inputs into the design of training programs
Enhance competence of WfD administrators and instructors
Incentivizing Excellence in Training Provision
Promote diversity in training provision
Incentivize private providers to meet WfD standards
Motivate public training institutions to respond to the demand for skills
Enhancing Accountability for Results
Strengthen the WfD monitoring and evaluation system
Specify reporting requirements by training institutions
Increase focus on outcomes, efficiency, and innovation in service delivery
Source: Authors’ construction.
42 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Appendix 2: Benchmarking Rubrics
Dimension 1: Strategic Framework
Goal Action Level of Development
Latent Emerging Established Advanced
Art
icul
atin
g a
Str
ateg
ic D
irec
tion
for
Wor
kfor
ce D
eve
lopm
ent
Advocate for WfD as a priority for economic development
WfD is not prioritized in national economic development.
Political and other leaders recognize the importance of WfD for economic development; economic development plans have identified a few WfD priorities.
Political and other key leaders in industry provide sustained support for WfD; economic development plans assess and specify several WfD priorities that are being implemented.
WfD is fully integrated into national policies and strategies, reflecting a holistic approach51 to WfD; economic development plans formally assess and specify a wide range of WfD priorities that are supported by implementation plans and budgets; these are subject to continuous evaluation and improvements.
Evaluate economic prospects and its implications for skills
Policies are being developed but are not based on formal analyses of skills demand.
A few policies have been developed on the basis of occasional assessments to address imbalances between skills demand and supply; these policies and interventions are subject to in-house reviews.
A range of policies based on occasional and routine assessments by government and independent WfD stakeholders have been implemented to address skills imbalances; these are subject to routine in-house reviews and independent external evaluations.
Policies are formulated on the basis of well-informed analyses, including assessments by independent organizations, and they are routinely reviewed and updated with inputs from relevant stakeholders to ensure program offerings fit the economic climate and demands for new skills.
Develop polices to align skills demand and supply
The concept of a demand-driven approach52 to WfD has yet to emerge.
A demand-driven WfD strategy is beginning to take shape but policy reforms are often impeded by various difficulties.
A demand-driven WfD strategy informed by appropriate analyses is accompanied by some policy reforms that have been implemented.
A well-informed demand-driven WfD strategy with continuous evaluation and improvements has been internalized as a standard practice.
51 A holistic approach is one that addresses multiple dimensions of skills development, including: (a)
aligning skills training to employers’ needs and national goals for productivity, growth, and poverty
reduction; (b) governing the system to achieved the desired national goals, and (c) ensuring tangible
results on the ground. 52 In a demand‐driven strategy, the demand for skills drives the supply of training services.
Arrangements to achieve this relationship between skills supply and demand include: the involvement
of employers in shaping training policies and provision, financing tied to employment outcomes, and so
forth.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 43
Dimension 1: Strategic Framework
Goal Action Level of Development
Latent Emerging Established Advanced
Prio
ritiz
ing
a D
eman
d-le
d A
ppro
ach
Promote a demand-driven approach
There is limited or no attempt to incorporate business and industry inputs in establishing and implementing WfD priorities.
Business and industry play an advisory role in establishing and implementing WfD priorities based on occasional studies and assessments.
A demand-driven approach to WfD is in place with business and industry providing inputs for setting WfD priorities based on routine assessments provided by government agencies, employers, trade associations, and labor unions.
A demand-driven approach to WfD has been fully established with business and industry playing both advisory and executive roles supported by routine assessments from government agencies, other key WfD stakeholders, and independent organizations.
Strengthen firms' demand for skills to improve productivity
Few incentives and services exist to support skills development for technology upgrading by firms.
Incentives and services are in place to provide selective support for technology-related skills upgrading; incentive programs are subject to occasional reviews but often without adequate follow-up of recommendations.
Incentives and services enable firms to expand the skills sets of workers to facilitate technology adaptation and adoption for greater productivity; these measures are supported by routine reviews followed by implementation of some review recommendations.
Incentives and services enabling firms to address skills constraints impeding their ability to upgrade technologies and productivity are well established; these are routinely reviewed and adjusted for impact; all key review recommendations are implemented.
Address critical challenges in the future supply of skills
There is limited or no formal assessment of the future supply of skills.
Future supply of skills is assessed on an occasional basis; recommendations from assessments are implemented with some delay, often without adequate funding and assignment of responsibility for implementation.
Assessments of future skills supply are routinely conducted for key sectors at the regional and national levels; recommendations are implemented with little delay; responsibilities for implementation of recommendations are made explicit but without explicit attention to monitorable goals.
Future skills supply is routinely assessed for multiple industries and sectors at the national and international levels; recommendations are implemented promptly; responsibilities for implementation are clearly spelled out and attention is given to the realization of monitorable goals.
44 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Dimension 1: Strategic Framework
Goal Action Level of Development
Latent Emerging Established Advanced
Str
engt
heni
ng C
ritic
al C
oord
inat
ion
Ensure coherence of key strategic WfD priorities
There is no mechanism in place to ensure coherence of key strategic WfD priorities among WfD leaders.
Coherence of key strategic WfD priorities at the leadership level is achieved through informal processes that yield limited WfD outcomes.
Coherence of key strategic WfD priorities at the apex leadership level is achieved through formal and informal mechanisms that yield positive WfD outcomes.
Formal mechanisms overseeing coordination and implementation of WfD strategies are in place and they support strengthening structures of WfD policy development, budget allocations, and assessments of future skills demand and supply.
Institutionalize the structure of WfD roles and responsibilities
Roles and responsibilities for WfD are not formally defined, leaving the WfD authority without a clear mandate.
Roles and responsibilities of WfD stakeholders are poorly defined, leaving the WfD authority with a limited mandate and limited resources to discharge its responsibilities effectively.
Roles and responsibilities are well-defined and supported by legislation and resources that enable the WfD authority and key stakeholders to discharge their respective functions effectively.
Clear WfD roles and responsibilities have been institutionalized through legislation and the WfD authority has the mandate to formulate and request resources that are needed for the relevant authorities to discharge their responsibilities in a transparent and effective manner.
Facilitate communication and interaction among all WfD stakeholders
No formal process exists for engaging all stakeholders.
Informal structures exist that facilitate communication and interaction among key stakeholders.
Formal structures exist in key economic sectors that support extensive communication and interaction among the relevant stakeholders.
Formal structures fostering extensive interactions among WfD stakeholders that culminate in consensuses on WfD priorities and policies are in place in most sectors.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 45
Functional Dimension 2: System Oversight
Goal Action Level of Development
Latent Emerging Established Advanced
Div
ersi
fyin
g P
ath
way
s fo
r S
kills
Acq
uisi
tion
Foster articulation across levels and programs
No functioning articulation arrangements.
Ad hoc articulation arrangements exist within secondary schools and post-secondary institutions; only ad hoc incentives are in place to foster articulation across levels of instruction.
Ad hoc articulation arrangements exist across institutions at the secondary and post-secondary levels; a program of incentives is in place to foster articulation arrangements.
Standardized articulation arrangements exist across secondary and post-secondary programs as well as between TVET and higher education; a system of incentives is in place to foster articulation across programs and levels of education and training.
Promote lifelong learning
No arrangements or public resources are in place to support lifelong learning, recognition of prior learning, and disadvantaged groups.
Ad hoc private resources and arrangements support lifelong learning and recognition of prior learning; publicly funded training programs exist with for disadvantaged groups subject to some restrictions.
School- and community-based resources and arrangements support lifelong learning and recognition of prior learning; publicly funded training programs with minimal restrictions are available for most disadvantaged groups.
Integrated regional or national system with one-stop online resources and standardized arrangements support lifelong learning and recognition of prior learning; publicly funded training programs provide open access to all disadvantaged groups.
Set policies and procedures to renew programs
There are no set policies to manage program offerings; training providers may introduce, adjust, or close publicly funded programs at will.
Introduction, adjustment, and closure of publicly funded programs are made through ad hoc, nonstandardized processes; applications for these changes must be done personally and are vetted by ad hoc committees.
Introduction, adjustment, and closure of publicly-funded programs are based on a few explicit and standardized requirements; applications can be made online and they are vetted by formal committees with some representation from other WfD stakeholders.
Management of publicly funded training programs are made on the basis of comprehensive and explicit requirements that include labor market analyses; applications can be made online and they are vetted by formal committees with representation from other WfD stakeholders and they operate with a commitment to act in a timely manner.
46 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Dimension 2: System Oversight
Goal Action Level of Development
Latent Emerging Established Advanced
Ens
urin
g E
ffici
ency
an
d E
quity
in F
undi
ng
Articulate funding strategy
Ad hoc funding of WfD by multiple stakeholders; no evaluation of funding allocation and strategy.
Systematic funding of WfD is determined by government agencies with annual budget appropriations and line-item allocations; only occasional evaluations of funding allocation and strategy.
Systematic funding of WfD is determined by government agencies with advice from key stakeholders; annual budget appropriations are supported by detailed spending plans; there are routine evaluations of funding allocation and strategy.
Systematic funding of WfD is determined through consensus building among government agencies and key stakeholders; annual budget appropriations are supported by detailed spending plans to foster improved performance; routine evaluations of funding allocation and strategy are accompanied by appropriate reforms as needed.
Allocate funds to achieve efficient results
No formal process for allocating public funds for WfD.
A formal process without explicit criteria is in place; there are no reviews of allocation criteria.
A formal process for allocating public funds based on explicit criteria exists; there are periodic reviews of the criteria but recommended changes face relatively long implemented lags.
Allocation of WfD funds is based on explicit criteria aligned with WfD priorities, including efficiency in resource utilization; there are frequent reviews of the criteria and recommendations are implemented in a timely manner.
Foster partnerships
Limited or no partnership between WfD authority and stakeholders in business and industry; key stakeholders provide few, if any, resources toward meeting WfD priorities.
Limited partnership with business and industry is in place; partners have access to some public resources; key stakeholders contribute a small range of resources toward WfD priorities.
Extensive partnership between WfD authority and key stakeholders in business and industry; partners have access to some public resources; key stakeholders contribute a broad range of resources for WfD.
An institutionalized partnership network with open membership for all WfD stakeholders is in place; partners have access to wide range of public resources; key stakeholders contribute an extensive range of resources to meet WfD priorities.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 47
Dimension 2: System Oversight
Goal Action Level of Development
Latent Emerging Established Advanced
Ass
urin
g R
elev
ant a
nd R
elia
ble
Sta
ndar
ds
Specify accreditation standards
No accreditation standards have been established; training providers are free to offer any program.
Some accreditation standards have been established; standards are infrequently reviewed; accreditation applies to public institutions only.
An accreditation agency has been established with standards developed jointly with relevant stakeholders; standards are reviewed internally on a regular or as needed basis; accreditation applies to public institutions and nonstate providers receiving public funding; renewal applies to the latter only.
An accreditation agency with standards reflecting international best practices is in place; accreditation standards are reviewed frequently both internally and by independent parties; accreditation and renewal of accreditation is compulsory for all public institutions and nonstate training providers, regardless of their sources of funding.
Strengthen skills testing and certification
Competency-based testing has yet to be introduced; testing is largely based on theoretical knowledge and administered by training providers themselves.
Competency-based testing applies to critical occupations in key sectors; testing may focus on a mix of theory and practice and is administered and certified by independent third parties.
A standardized competency-based testing system is in place and applies to most occupations; testing may focus on a mix of theory and practice and is administered and certified by independent third parties.
A standardized competency-based testing system has been established for most occupations; IT-based testing focuses on theory and practice and is administered and certified by independent third parties.
Assure credibility of accreditation and of skills certification
There is limited attention to accreditation standards.
Accreditation standards are established through ad hoc arrangements; some support is provided to encourage nonstate providers to seek accreditation; credibility of skills testing is ensured through explicit standardized testing protocols.
Accreditation standards established with inputs from WfD stakeholders apply to all institutions and providers receiving public funding; credibility of skills testing is ensured through explicit standardized testing protocols and accreditation of testing centers.
A license to operate is issued only to institutions and providers meeting accreditation standards determined by key WfD stakeholders; credibility of skills testing is ensured through standardized testing protocols, accreditation of testing centers, and random audits.
48 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Dimension 3: Service Delivery
Goal Action Level of Development
Latent Emerging Established Advanced
Fos
terin
g R
ele
vanc
e in
Tra
inin
g P
rogr
ams
Link training industry and research institutions
Weak or no links between training institutions and industry and research institutions.
Informal links exist between some training institutions and industry and research institutions to improve training relevance and quality.
Formal links exist between some training institutions and industry and research institutions, leading to significant collaboration in several activities.
Formal links exist between most training institutions and industry and research institutions, leading to significant collaboration in a wide range of activities such as the provision of industry internships and training, and the introduction and redesign of training programs.
Design training with industry inputs
Industry has limited or no role in identifying, prioritizing and designing publicly funded programs.
Industry has an advisory role in identifying, prioritizing, and designing publicly funded programs in some training institutions, usually through informal contacts.
Industry has both an advisory and a decision-making role in identifying, prioritizing, and designing publicly funded programs in some training institutions.
Industry has a widespread advisory and decision-making role in identifying, prioritizing, and designing publicly funded programs in most training institutions.
Improve competence of WfD administrators and instructors
Few or no measures are in place to enhance the competence of WfD administrators and instructors.
Recruitment of administrators and instructors is based on minimum academic qualification(s), with provisions for some in-service training and performance-based recruitment and retention measures based on occasional evaluations.
Recruitment of administrators and instructors is based on minimum academic qualification(s), with provisions for in-service training and performance-based recruitment and retention measures that are based on routine evaluations.
Recruitment of administrators and instructors occurs through a competitive process based on both academic qualification(s) and industry experience, with a wide range of in-service training programs and performance-based recruitment and retention measures based on routine evaluations.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 49
Goal Action Level of Development
Latent Emerging Established Advanced
Ince
ntiv
izin
g E
xcel
lenc
e in
Tra
inin
g P
rovi
sio
n
Promote diversity in training provision
Training occurs through state provision only, with no incentives to promote nonstate provision of training.
Training policies allow some private providers to operate; training is provided mainly by nonprofit providers with few incentives in place to foster nonstate provision of training.
Training policies facilitate participation of nonstate training providers; training is offered mainly by NGOs, with a system of incentives that are evaluated routinely are in place to foster nonstate provision.
A highly diverse mix of nonstate training providers offer training within a comprehensive system with a wide range of incentives is in place to foster nonstate provision; incentives are subject to evaluations and the recommendations are implemented.
Incentivize private providers to meet WfD standards
No incentives are in place to encourage nonstate providers to meet WfD standards.
At least one incentive that is subject to occasional evaluation is in place to encourage nonstate providers to meet WfD standards, but no review system exists to ensure continued adherence to WfD standards.
A system of financial and nonfinancial incentives that are subject to occasional and routine evaluations is in place to encourage nonstate providers to meet WfD standards; periodic audits are conducted to ensure continued adherence to WfD standards.
A comprehensive system of incentives that are subject to both occasional and routine evaluations and adjustments is in place to encourage nonstate providers to comply with WfD standards; periodic audits with penalties for noncompliance are conducted and enforced to ensure continued adherence to WfD standards.
Motivate public training institutions to respond to demand for skills
No mechanism or process is in place to ensure training institutions are demand-driven.
Training institutions are expected to meet target repetition and graduation rates but few incentives are in place ensure they are demand-driven.
Training institutions are expected to meet a wider range of WfD outcomes; some incentives and penalties that are subject to both occasional and routine evaluations are in place to ensure these institutions respond to the demand for skills.
Training institutions are expected to meet a wide range of WfD outcomes; a robust system of incentives and penalties that is subject to both occasional and routine evaluations and adjustments is in place to ensure that the training institutions are driven by employers’ demands for skills.
50 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Goal Action Level of Development
Latent Emerging Established Advanced
Enh
anc
ing
Acc
ount
abi
lity
for
Res
ults
Strengthen WfD monitoring and evaluation
Limited attention is placed on the monitoring and evaluation of skills demand and supply; an overview of WfD data is available through informal channels only.
Occasional monitoring and evaluation of skills demand and supply is in place; an overview of WfD data is available only in government agencies.
Routine monitoring and evaluation of skills demand and supply is in place; an overview of WfD data is available in published reports and Websites.
Skills demand and supply are monitored and evaluated through routine surveys and specially commissioned studies; WfD data are available from a consolidated Website.
Specify reporting requirements by training institutions
No specific data collection and reporting are required; training providers maintain their own databases.
Public institutions and nonstate training providers are required to collect and maintain administrative and graduation statistics; data reporting is voluntary for nonstate providers but they may be notified of noncompliance.
Public institutions and nonstate training providers are required to collect, maintain and submit a comprehensive list of data through an integrated management information system to the WfD authority; timely submission is fostered through incentives for compliance and penalties for noncompliance.
Both public institutions and nonstate training providers are required to collect, maintain, and submit a comprehensive list of data, including client feedback, to the WfD authority using an integrated management information system; incentives, penalties, and data quality audits are performed to ensure timely reporting of reliable data.
Increase focus on outcomes, efficiency and innovation
No system of evaluation and monitoring is in place to ensure efficiency in delivery of training services.
Occasional evaluation and monitoring of limited aspects of training services is in place with results used to provide feedback to the training institutions; information on labor market outcomes of graduates is publicly available for some institutions only.
Routine evaluation and monitoring of several key aspects of training services is in place with results used to provide feedback to training institutions, to prioritize funding allocations, and identify good practices in service delivery; information on labor market outcomes of graduates is publicly available for all institutions.
Institutionalized routine evaluation and monitoring of all key aspects of the delivery of training services with results used to provide feedback to institutions, to prioritize funding allocations, indentify good practices, and to identify options for system-level improvements; online dissemination of labor market outcomes of graduates is available to all users.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 51
References
Abrams, Peter, Grazia Borrini‐Feyerabend, Julia Gardner, and Pippa Heylings.
2003. “Evaluating Governance: A Handbook to Accompany a Participatory
Process for a Protected Area.” Unpublished manuscript. Gatineau, Quebec:
Parks Canada and TILCEPA. http://cmsdata.iucn.org/downloads/
evaluating_governance_handbook.doc (accessed on October 27, 2011).
Adams Arvil V. 2007. “The Role of Youth Skills Development in the Transition to
Work: A Global Review.” HDNCY 5, Human Development Network
Children and Youth Department, World Bank, Washington, DC.
http://siteresources.worldbank.org/INTCY/Resources/395766‐
1187899515414/RoleofYouthSkills.pdf (accessed on October 27, 2011).
Allais, Stephanie. 2010. The Implementation and Impact of National Qualifications
Frameworks: Report of a Study in 16 countries. Geneva: Skills and
Employability Department, International Labour Office (ILO).
http://www.ilo.org/wcmsp5/groups/public/‐‐‐ed_emp/‐‐‐ifp_skills/
documents/meetingdocument/wcms_126589.pdf (accessed on October 27,
2011).
Almeida, Rita, Jere Behrman, and David Robalino. 2012. “The Economic
Rationale for Skills Development Policies.” In The Right Skills for the Job?
Rethinking Training Policies for Workers, ed. Rita Almeida, Jere Behrman, and
David Robalino. Washington, DC: Human Development Perspectives,
World Bank. http://elibrary.worldbank.org/content/book/9780821387146
(accessed on August 13, 2012).
Alstete, Jeffrey W. 1995. “Benchmarking in Higher Education.” ASHE‐ERIC:
Higher Education Report 5. The George Washington University Graduate
School of Education and Human Development, Washington DC.
Ashton, David, Francis Green, Johnny Sung, and Donna James. 2002. “The
Evolution of Education and Training Strategies in Singapore, Taiwan and S.
Korea: A Development Model of Skill Formation.” Journal of Education and
Work 15(1): 5–30.
———. 2009. Good Practice in Technical and Vocational Education and Training.
Manila: Asian Development Bank. http://www.adb.org/
Documents/Guidelines/Good‐Practice‐Education‐Training/Good‐Practice‐
Education‐Training.pdf (accessed on October 27, 2011).
Asian Development Bank (ADB). 2010. Education and Skills: Strategies for
Accelerated Development in Asia and the Pacific. Mandaluyong City,
Philippines: Asian Development Bank.
52 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Atchoarena, David. 2002. “Private Training.” In World Bank Study on: Vocational
Skills Development in Sub‐Saharan Africa—a Working Group Review, ed.
Martha Caddell and Keith Holmes. Edinburgh: Working Group for
International Cooperation in Skills Development. http://www.norrag.org/
medias/documents/pdf/publications/papers/Paper7.pdf (accessed on
October 27, 2011).
Banerji, Arup, Robert Holzmann, Pierella Paci, Carmen Pages, Stefano Scarpetta,
and Milan Vodopivec. 2008. “MILES to Go: A Quest for an Operational
Labor Market Paradigm for Developing Countries.” Unpublished
manuscript. Social Protection and Labor Sector, World Bank, Washington,
DC. http://siteresources.worldbank.org/INTLM/Resources/390041‐
1212776476091/5078455‐1267646113835/MILESThequestoperational
LMparadigm_Jan212008.pdf (accessed on October 27, 2011).
Betcherman, Gordon, Karina Olivas, and Amit Dar. 2004. “Impacts of Active
Labor Market Programs: New Evidence from Evaluations with Particular
Attention to Developing and Transition Countries.” Social Protection
Discussion Paper Series 0402, World Bank, Washington, DC.
http://info.worldbank.org/etools/docs/library/251019/day6DiscussionPaper
Series0402April6Se1.pdf (accessed on October 27, 2011).
Betcherman, Gordon, Martin Godfrey, Olga Susana Puerto, Friederike Rother,
and Antoneta Stavreska. 2007. “Global Inventory of Interventions to
Support Young Workers: Synthesis Report.” Social Protection Discussion
Paper 0715, World Bank, Washington, DC. http://siteresources.
worldbank.org/EXTECAREGTOPYOUTH/Resources/YEI_Synthesis_Paper
_Final_July_13_2007.pdf (accessed on October 27, 2011).
Blom, Andreas, and Hiroshi Saeki. 2011. “Employability and Skill Set of Newly
Graduated Engineers in India.” World Bank Policy Research Working
Paper No. 5640, World Bank, Washington, DC.
Bloom, Benjamin.S. 1976. Human Characteristics and School Learning. New York:
McGraw‐Hill.
Booth, Alison L., and Dennis J. Snower. 1996. “Introduction: Does the Free
Market Produce Enough Skills?” In Acquiring Skills: Market Failure, Their
Symptoms and Policy Responses. Alison L. Booth and Dennis J. Snower, eds.
Cambridge: Press Syndicate of the University of Cambridge.
Camp, Robert C. 1989. Benchmarking: The Search for Industry Best Practices that Lead
to Superior Performance. Milwaukee, Wisconsin: ASQC Quality Press.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 53
Center for the Development of Vocational Training (CEDEFOP). 2010. Skills
Supply and Demand in Europe. Medium Term Forecast Up To 2020.
Luxembourg: Publications Office of the European Union.
http://www.cedefop.europa.eu/download‐manager.aspx?id=15535&lang=
en&type=publication (accessed on October 27, 2011).
Commission for Growth and Development. 2008. The Growth Report: Strategies for
Sustained Growth and Inclusive Development. Washington, DC: World Bank.
http://cgd.s3.amazonaws.com/GrowthReportComplete.pdf (accessed on
October 27, 2011).
Dattakumar, R., and R. Jagadeesh. 2003. “A Review of Literature on
Benchmarking.” Benchmarking: An International Journal 10(3): 176–209.
de Largentaye, Armand Rioust. 2009. “Vocational Training and the Informal
Economy.” In Promoting Pro‐Poor Growth Employment. Paris: OECD.
http://www.oecd.org/dataoecd/63/11/43514554.pdf (accessed on October 27,
2011).
Deloitte, The Manufacturing Institute, and Oracle (DMIO). 2009. “People and
Profitability: A Time for Change: A 2009 People Management Practices
Survey of the Manufacturing Industry.” http://www.deloitte.com/
assets/Dcom‐UnitedStates/Local%20Assets/Documents/us_pip_people
managementreport_100509.pdf (accessed on October 27, 2011).
di Gropello, Emanuela, with Aurelien Kruse and Prateek Tandon. 2011a. Skills for
the Labor Market in Indonesia: Trends in Demand, Gaps, and Supply.
Washington, DC: World Bank. http://ddp‐ext.worldbank.org/EdStats/
IDNpub11.pdf (accessed on October 27, 2011).
di Gropello, Emanuela, with Hong Tan and Prateek Tandon. 2011b. Skills for the
Labor Market in the Philippines. Washington, DC: World Bank.
Dunbar, Muriel. 2011. “Engaging the Private Sector in Skills Development.”
Guidance Notes: A DFID Practice Paper. London: Department for
International Development, Government of the United Kingdom.
Federal Office for Professional Education and Technology (FOPET). 2012.
Vocational and Professional Education and Training in Switzerland: Facts and
Figures. Basel: Federal Office for Professional Education and Technology.
Ferrier, Fran, Clifford Trood, and Karen Whittingham. 2003a. Going Boldly into the
Future: A VET Journey into the National Innovation System, Adelaide: National
Centre for Vocational Education Research (NCVER), Australian National
Training Authority. http://www.ncver.edu.au/research/proj/nr9036_
vol1.pdf (accessed on October 27, 2011).
54 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Ferrier, Fran, Clifford Trood, and Karen Whittingham. 2003b. Going Boldly into
the Future: A Series of Case Studies of Co‐operative Research Centres and their
Relationships with the VET Sector. Adelaide: National Centre for Vocational
Education Research (NCVER), Australian National Training Authority.
http://www.ncver.edu.au/research/proj/nr9036cs.pdf (accessed on October
27, 2011).
Ganzglass, Evelyn, Keith Bird, and Heath Prince. 2011. Giving Credit Where Credit
is Due: Creating a Competency‐Based Qualifications Framework for Postsecondary
Education and Training. Washington, DC: Center for Postsecondary and
Economic Success. http://www.clasp.org/admin/site/publications/files/
Giving‐Credit.pdf (accessed on October 27, 2011).
Green, Francis, David Ashton, Donna James, and Johnny Sung. 1999. “The Role
of the State in Skill Formation: Evidence from the Republic of Korea,
Singapore, and Taiwan.” Oxford Review of Economic Policy 15(1): 82–96.
Hanf, Georg, and Ute Hippach‐Schneider. 2005. “What Purpose do National
Qualifications Frameworks Serve? A look at Other Countries.” Vocational
Training in Research and Practice (Bundesinstitut für Berufsbildung) Special
Edition H 20155 (Promoting Innovations in Vocational Education and
Training: An Exchange of German Experiences): 9–14. http://www.bibb.de/
veroeffentlichungen/en/publication/download/id/1692 (accessed on
October 27, 2011).
Hanushek. Eric A., and Ludger Wößmann. 2006. “Does Educational Tracking
Affect Performance and Inequality? Differences‐in‐Differences Evidence
Across Countries.” Economic Journal 116(510): C63–C76.
Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of
Cognitive and Noncognitive Abilities on Labor Market Outcomes and
Social Behavior.” NBER Working Paper 12006, National Bureau of
Economic Research, Cambridge, MA.
Hicks, Joan Hamory, Michael Kremer, Isaac Mbiti, and Edward Miguel. 2011.
“Vocational Education Voucher Delivery and Labor Market Returns: A
Randomized Evaluation among Kenyan Youth.” Report for Spanish Impact
Evaluation Fund (SIEF) Phase II, World Bank, Washington, DC.
Ibarraran, Pablo, and David Rosas Shady. 2009. “Evaluating the Impact of Job
Training Programs in Latin America: Evidence from IDB Funded
Operations.” Journal of Development Effectiveness 2(1): 195–216.
Jacobs, Ronald L., and Joshua Hawley. 2008. “Emergence of Workforce
Development: Definition, Conceptual Boundaries, and Implications.” In
International Handbook of Education for the Changing World of Work (Vol. 6), ed.
R. MacLean and D. Wilson. Dordrecht, Netherlands: Springer Science +
Business Media B.V.
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 55
Janjua, Shehryar, and Arif Naveed. 2009. “Skills Acquisition and the Significance
of Informal Training System in Pakistan—Some Policy Implications.”
RECOUP Policy Brief No. 7, Research Consortium on Educational
Outcomes and Poverty, Cambridge. http://www.dfid.gov.uk/r4d/PDF/
Outputs/ImpOutcomes_RPC/pb7.pdf (accessed on October 29, 2011).
Johanson, Richard. 2009. “A Review of National Training Funds.” Social
Protection Discussion Paper 0922, World Bank, Washington, DC.
http://siteresources.worldbank.org/SOCIALPROTECTION/Resources/SP‐
Discussion‐papers/Labor‐Market‐DP/0922.pdf (accessed on October 29,
2011).
Johanson, Richard, and Arvil Van Adams. 2004. Skills Development in Sub‐Saharan
Africa. World Bank Regional and Sectoral Studies. Washington, DC: World
Bank.
Karmel, Tom. 2011. The Challenge of Measurement: Statistics for Planning Human
Resource Development. Adelaide: National Centre for Vocational Education
Research. http://www.ncver.edu.au/research/core/2418.pdf (accessed on
October 29, 2011).
Kiley, Margaret, and Robert Cannon. 2000. Leap into Lifelong Learning. Adelaide:
Centre for Learning and Professional Development, University of Adelaide.
King, Kenneth, and Robert Palmer. 2010. “Planning for Technical and Vocational
Skills Development.” Fundamentals of Educational Planning No. 94.
UNESCO: International Institute for Educational Planning.
Korea Research Institute for Vocational Education and Training (KRIVET). 2007.
Human Resources Development Indicators in Korea, 2007. Seoul: KRIVET.
Kuruvilla, S., C. L. Erickson, and A. Hwang. 2002. “An Assessment of the
Singapore Development System: Does It Constitute a Viable Model for
Other Developing Countries?” World Development 30(8): 1461–1476.
Law, Song Seng. 2008. “Vocational Technical Education and Economic
Development: The Singapore Experience.” In Toward a Better Future.
Education and Training for Economic Development in Singapore since 1965, eds.
Lee Sing Kong, Goh Chor Boon, Birger Fredriksen, and Tan Jee Peng.
Washington, DC: World Bank and National Institute of Education,
Singapore.
Lee, Young‐Hyun. 2009. “Vocational Education and Training in the Process of
Industrialization.” Understanding Korean Educational Policy, Vol. 5.
Educational Development Institute, Seoul. Korean. http://eng.kedi.re.kr/
khome/eng/education/educationSeries.do# (accessed on October 30, 2011).
Levy, Santiago. 2008. Good Intentions, Bad Outcomes: Social Policy, Informality, and
Economic Growth in Mexico. Washington, DC: Brookings Institution Press.
56 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Levy, Frank, and Richard J. Murnane. 2005. The New Division of Labor: How
Computers Are Creating the Next Job Market. Princeton, NJ: Princeton
University Press.
Lindburg, Lori. 2009. Replicating Success: Innovative Collaborations between the
Biotech Industry and Education. San Francisco: Bay Bio Institute, City College
of San Francisco. http://www.bio‐link.org/home/sites/files/replicating
success_finalscreen.pdf (accessed on October 30, 2011).
MacKenzie, Jeanne, and Rose‐Anne Polvere. 2009. “TVET Glossary: Some Key
Terms.” In International Handbook of Education for the Changing World of
Work, ed. R. Maclean and D. Wilson. Dordrecht, Netherlands: Springer
Science + Business Media B.V.
Mourshed, Mona, Chinezi Chijioke, and Michael Barber. 2010. How the World’s
Most Improved School Systems Keep Getting Better. McKinsey and Company.
http://mckinseyonsociety.com/downloads/reports/Education/How‐the‐
Worlds‐Most‐Improved‐School‐Systems‐Keep‐Getting‐Better_Download‐
version_Final.pdf (accessed on October 30, 2011).
Mulgan, Geoff, and David Albury. 2003. Innovation in the Public Sector. London:
Strategy Unit, Cabinet Office.
O’Hare, Daniel. 2008. “Education in Ireland: Evolution of Economic and
Education Policies since the Early 1990s.” In An African Exploration of the
East Asian Education Experience, ed. Birger Fredriksen and Tan Jee Peng.
Washington, DC: World Bank. http://www.adeanet.org/adeaPortal/
adea/Biennale%202008/Documentation/Papers%20for%20presentation/01.%
20Session%201/Final%20PDF%20documents/Session%201%20Doc%204%20
World%20Bank%20ENG.pdf (accessed on October 30, 2011).
OʹLawrence, Henry. 2007. “A Case Study Report of a Comprehensive
Benchmarking Review of the Status of Vocational Education Students in
California: Building a New Relationship between Community Colleges
and California State University System.” Journal of Career and Technical
Education 23(1): 85–96. http://scholar.lib.vt.edu/ejournals/JCTE/v23n1/pdf/
olawrence.pdf (accessed on October 30, 2011).
Organization for Economic Cooperation and Development (OECD). 2010a.
Learning for Jobs: Synthesis Report of the OECD Reviews of Vocational Education
and Training. Paris: OECD http://www.mskills.com.au/DownloadManager/
downloads/Learning%20for%20jobs%20report.pdf (accessed on October 30,
2011).
———. 2010b. PISA 2009 Results: What Students Know and Can Do: Student
Performance in Reading, Mathematics and Science. Vol. 1. Paris: OECD.
http://www.oecd.org/dataoecd/10/61/48852548.pdf (accessed on October 30,
2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 57
Oster, Emily, and M. Bryce Millett. 2010. “Do Call Centers Promote School
Enrollment? Evidence from India.” NBER Working Paper 15922, National
Bureau of Economic Research, Washington, DC.
Schleicher, Andreas. 2008. “PIAAC: A New Strategy for Assessing Adult
Competencies.” International Review of Education 54(5‐6): 627‐650.
Singapore Accreditation Council. 2008. “Frequently Asked Questions: What Are
the Benefits of Being Accredited?” http://www.sac‐accreditation.gov.sg/
accreditation.asp#qn1 (accessed on June 17, 2011).
Singh, Chandandeep, and Kuldeep Sareen. 2006. “Effectiveness of ISO 9000
Standards in Indian Educational Institutions: A Survey.” International
Journal of Services Technology and Management 7(4): 403–415.
Skills Victoria. 2011. “Victorian Government Response to Skills Australia
Consultation: Creating a Future Direction for Australian Vocational
Education and Training.” Mimeo. Skills Victoria, Department of Education
and Early Childhood Development, Melbourne. http://www.eduweb.
vic.gov.au/edulibrary/public/commrel/about/skills/victoriangovernmentres
ponsetoskillsaustraliaconsultation.doc (accessed on October 30, 2011).
Sondergaard, Lars, and Mamta Murthi. 2012. Skills, Not Just Diplomas: Managing
Education for Results in Eastern Europe and Central Asia. Directions in
Development Series. Washington, DC: World Bank.
Spar, Debora. 1998. “Attracting High Technology Investment: Intel’s Costa Rican
Plant.” Foreign Investment Advisory Service. Occasional Paper 11,
International Finance Corporation and World Bank.
Sudan, Randeep, Seth Ayers, Philippe Dongier, Arturo Muente‐Kunigami, and
Christine Zhen‐Wei Qiang. 2010. The Global Opportunity in IT‐Based Services.
Assessing and Enhancing Country Competitiveness. Washington, DC: World
Bank. http://www.infodev.org/en/Document.875.html (accessed on October
30, 2011).
Tan, Hong. 2001. “Malaysia’s Human Resource Development Fund: An
Evaluation of Its Effects on Training and Productivity.” Mimeo.
Washington, DC: World Bank. http://info.worldbank.org/etools/docs/
library/206512/Malaysia%20Human%20Resources%20Development%20Fu
nd%20English.pdf (accessed on October 30, 2011).
Tan, Hong, Gladys Lopez‐Acevedo, M. Sanchez, and M. Tinajero. 2004.
Evaluating SME Programs in Mexico Using Panel Firm Data. Washington, DC:
World Bank Institute, World Bank.
58 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough
Tan, Jee‐Peng and Joy Yoo‐Jeung Nam. 2012. “Pre‐Employment Technical and
Vocational Education and Training: Fostering Relevance, Effectiveness, and
Efficiency.“ In The Right Skills for the Job? Rethinking Training Policies for
Workers, ed. Rita Almeida, Jere Behrman, and David Robalino. Washington,
DC: Human Development Perspectives, World Bank
http://elibrary.worldbank.org/content/book/9780821387146 (accessed on
August 13, 2012).
United Kingdom Commission for Employment and Skills (UKCES). 2009.
Ambition 2020: World Class Skills and Jobs for the UK. 2009 Report.
London: UKCES. http://www.ukces.org.uk/assets/bispartners/ukces/docs/
publications/ambition‐2020‐the‐2009‐report.pdf (accessed on November 1,
2011).
United Nations Educational, Scientific and Cultural Organization (UNESCO).
2002. “Best Practices” in Technical and Vocational Education: China, the People’s
Democratic Republic of Korea and Mongolia. Beijing: UNESCO.
http://unesdoc.unesco.org/images/0012/001257/125705eo.pdf (accessed on
November 1, 2011).
United Nations Population Division of the Department of Economic and Social
Affairs (UN). 2009. World Population Prospects: the 2008 Revision. New York:
United Nations.
Vlăsceanu, Lazăr, Laura Grünberg, and Dan Pârlea. 2007. Quality Assurance and
Accreditation: A Glossary of Basic Terms and Definitions. Revised and updated
edition. Bucharest: UNESCO‐CEPES. http://www.cepes.ro/publications/
pdf/Glossary_2nd.pdf (accessed on November 1, 2011).
Wang, Yidan. 2012. Education in a Changing World: Flexibility, Skills and
Employability. Washington, DC: Human Development Network, World
Bank.
Wolf, Artjom, and Andrea Erdle. 2009. Key Aspects of Economics of Technical and
Vocational Education and Training (TEVET): Lessons Learned and Gaps to be Filled.
Eschborn: Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ).
http://www2.gtz.de/dokumente/bib‐2009/gtz2009‐0390en‐economics‐tvet.pdf
(accessed on November 1, 2011).
Woods, James F., and Christopher J. O’Leary. 2007. “Principles of Labor Market
Information.” Employment Research Newsletter 14(2). W.E. Upjohn Institute
for Employment Research, Kalamazoo, MI.
World Bank. 2010a. Stepping Up Skills for More Jobs and Higher Productivity.
Washington, DC: Human Development Network, World Bank.
http://siteresources.worldbank.org/EXTHDOFFICE/Resources/5485726‐
1281723119684/Stepping_up_skills.pdf (accessed on November 1, 2011).
What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 59
———. 2010b. “Sustaining Educational and Economic Momentum in Africa.”
Africa Human Development Series, World Bank Working Paper Series 195,
World Bank, Washington, DC. http://siteresources.worldbank.org/
EDUCATION/Resources/278200‐1099079877269/Sustaining_edu_econ_
momentum_Africa.pdf (accessed on November 1, 2011).
——. 2011. Learning for All. Investing in People’s Knowledge and Skills to Promote
Development. World Bank Group Education Strategy 2020. Washington, DC:
World Bank. http://siteresources.worldbank.org/EDUCATION/Resources/
ESSU/Education_Strategy_4_12_2011.pdf (accessed on November 9, 2011).
——. 2012. “SABER Overview Paper: The What, Why, and How of the Systems
Approach for Better Education Results.” Review draft. Education
Department, Human Development Network, World Bank, Washington,
DC.
World Bank/Multilateral Investment Guarantee Agency (MIGA). 2006. The Impact
of Intel in Costa Rica. Nine Years After the Decision to Invest. Investing in
Development Series. Washington, DC: MIGA. http://siteresources.
worldbank.org/EXTEXPCOMNET/Resources/2463593‐1213887855468/44_
The_impact_of_Intel_in_Costa_Rica.pdf (accessed on November 1, 2011).
Yusuf, Shahid, and Kaoru Nabeshima. 2010. Changing the Industrial Geography in
Asia: The Impact of China and India. Washington, DC: World Bank.