Learning Org Paper

8/12/2019 Learning Org Paper

1/33

The Learning Organization: Performance

Technology and the Implications for

Organizational Effectiveness

William C. RedeenDecember 1, 1997

Research Paper- PUBP 802

(The interactive web-based version of this document may be found at:

http://208.208.159.222/PUBP802/)


2/33

Table of Contents

Abstract

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

What is Performance Technology? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Research Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Organization of the Paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Trends in Productivity and Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

The Learning Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Measurement: The Missing Link in Performance Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Performance versus Learning Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Trends in Performance Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Performance Technology Measures in The Studies Reviewed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

The Relationship of Performance Technology Performance Measurement to Organizational StrategicProcesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Performance Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Functional Process Improvement & Business Process Re-engineering (BPR) . . . . . . . . . . . . . 10

Activity-based Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Are the Emerging Performance Technology Development Methodologies Compatible? . . . . . . . . . . . 12Performance Centered Design (PCD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Performance Support Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

An Model of Performance Technology & Organizational Performance . . . . . . . . . . . . . . . . . . . . . . . . 16

The Research Literature: How Effective is Performance Technology? . . . . . . . . . . . . . . . . . . . . . . . . . 18

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20


3/33

Abstract

This paper reviews the current research literature to identify the relationship between performance

technology and organizational effectiveness. Previous research has attempted to quantify the effect of the

training technology on training effectiveness. Unfortunately, the relationship of these outcomes to

organizational effectiveness is typically unclear. With the rapid evolution and convergence of training and

performance technologies, there is a renewed potential for establishing a direct relationship between

performance technology and organizational effectiveness. In addition, the recent emergence of several key

planning methodologies in different disciplinary fields appear to offer an opportunity to address this historic

measurement gap between organizational and training effectiveness. This paper proposes a model for

integrated training and organizational performance measurement, and summarizes the results of the limited

existing empirical studies on performance technology and organizational effectiveness.


4/33

1Barry Raybould. The Role of Technology in Improving Performance. EPSS.COM, November

1997.

1

Introduction

This paper reviews the current research literature on the relationship between performance technology and

organizational effectiveness. Much of the historic research has focused on the training component of

performance technology, and has addressed training effectiveness without establishing a direct relationship

to organizational effectiveness. As a result, the training function has been viewed with scepticism in terms

of its contribution to organizational effectiveness.

The recent rapid evolution and convergence of several key training and performance technologies, has

created a renewed potential for establishing a direct relationship between performance technology and

organizational effectiveness. Specifically, performance technology can now offer on-demand training and

performance support at the employees person or work place. For the first time, central collection and

measurement of performance is possible using distributed client-server architectures based on open

standards, such as the Internet or Intranets. While there is a general consensus that the introduction of this

technology is increasing organizational effectiveness, there have been only limited attempts to synthesize

the existing research to develop performance measurement and quantitative guidelines to help evaluate the

effect of technology on common measures of organizational effectiveness.

What is Performance Technology?

According to Martin Wikoff, performance technology emphasizes the systematic analysis of factors that

influence behavior and performance, with the objective of measurably improving individual and

organizational performance. Performance technology encompasses all tools which enhance human

performance, including traditional training, computer-based training and electronic performance support

systems. Recently and for purposes of this paper, performance technology refers to the new information

technology continuum. Figure 1 displays the performance technology continuum. According to Raybould,

performance technology can be designed with the primary objective of transferring knowledge or of

producing performance.1This paper focuses on the new paradigm of performance-centered design, which

emphasizes the design of a training or performance support system to meet specific measurable

performance outcomes, both at the individual and organizational level. Learning may occur, but it is not theprimary objective. The primary objective is enhanced organizational performance.


5/33

2

The Performance Technology Continuum in 1997

ClassroomTraining

and VTT

ElectronicPerformance

SupportSystems

Computer-Based

Training

Web-basedTraining

Training Performance Support

OnlineReferences

Performance Technology

PaperJob

Aids

Expertsand

Colleagues

Limited

Implementation

Figure 1. The Status of Performance Technology in 1997

Performance-centered systems are technologies that focus on performance as the primary objective. For

example, a software application may be designed to allow immediate performance of a job at a high level

by integrating the knowledge, information and tools a job performer needs into the software application or

a global interface to the organizational knowledge base. Technologies in this category include:

Performance Support Tools:: global or specialized software applications to help people perform job

tasks such as give an employee review, write a legal document or develop a negotiating strategy

Performance-Centered Information Systems: a global system supporting an organizational activity

or function, such as a customer call center application.

Performance-Centered Intranets- Intranets designed around a functional, topical or task-oriented

structure

These technologies

integrate a range of

performance support tools

t h r o u g h t h e

human/computer interface,

creating a partnership

between job performer and

machine. These can

includeon-screen help and

graphics, dialog boxes,

pushbuttons, task bars,

wizards, cue cards,

checklists and agent

metaphors to access a

common organizational

knowledge base.

Performance technology structures can also be designed with a learning focus in which case the primary

objective is that of transferring knowledge to long-term memory. These technologies include:

Computer-based training (CBT)


6/33

3

Web-based training (WBT): CBT delivered via an Intranet, and

Simulation technologies, such as software simulators, business practice simulators or equipment

simulators.

Research Issues

For performance technology to realize its full potential, the relationship to organizational effectiveness

must be clearly defined. This paper addresses the following specific research questions:

What tools and methodologies exist to support the measurement of the effectiveness of

performance technologies and the relationship to organizational performance?

Are the emerging methodologies for performance technology design (Performance-centered

Design) and strategic planning (Performance Management) compatible?

How can these tools be used congruently to measure and optimize organizational performance?

How effective are the new performance technologies? Is there any quantitative data on the

relationship of performance technology and organizational effectiveness?

Organization of the Paper

This paper is organized into six sections. First, we address recent trends in productivity and the nature of

work, establishing the basis for the importance of performance technology. Second, we describe the concept

of the Learning Organization, and the related paradigm shift in the concept of training and performance

technology. Third, we establish the importance of performance measurement as the mechanism for defining

the effectiveness of performance technology. Fourth, we describe emerging performance technology design

methodologies and their relationship to organizational performance measurement. Fifth, we document the

limited empirical research conducted on performance technology and describe some of the limitations.

Finally, we draw conclusions about the potential of performance technologies for enhanced organizational

effectiveness and identify areas for further research.

Trends in Productivity and Work

Productivity growth is recognized as central to maintaining the current U.S. economic leadership.Fundamental changes in the nature of work and the capabilities of emerging technologies have created a

unique opportunity for enhancing individual and organizational productivity.

With the emergence of the information economy, knowledge work is the area that offers the greatest


7/33

2Drucker, Peter F., Management(Harper & Row, 1974).

3Roach, Stephen, "Services Under Siege-The Restructuring Imperative," Harvard Business

Review(September-October 1991), pp 82-83.

4The Gartner Group, Annual Survey of Information Technology Costs, 1996.

4

Figure 2. Information Technology Costs, 1996

opportunity to increase productivity within the U.S. workforce.2 Knowledge workers make decisions that

significantly impact organizational resources and are themselves a significant and costly resource.

Knowledge workers compose 43 percent

of the white-collar sector, which in turn

comprises 67 percent of the service

sector.3 Most of these personnel use

information systems as a component of

their work. Traditionally, knowledge work

demands substantial training to establish

and maintain competency.

With the recent radical reductions in

hardware and software cost, personnel

costs and training have become the

largest cost component of Information

Systems (IS) implementation and life

cycle support. 4 Figure 1 presents IS costs by major category of expenditure, according to the Gartner

Group. Despite this recent data, the production work environment continues to dominate productivity

enhancement efforts in spite of evidence that the returns on further refinements do not equal those possible

in the knowledge worker environment.

Thus, because of the convergence of these trends, the enhancement of knowledge worker performance

has become the paramount focus for increasing productivity in the new information economy.

The Learning Organization

The concept of a learning organization, as described by Peter Senge, calls for an organization that fosters

group and individual learning through changes in organizational culture and the implementation of systems.


8/33

5David Garvin, Building a Learning Organization, Business Credit, 96(1): 19-28. January 1994.

6

American Society of Training & Development (ASTD), Training & Development Magazine,September 1996.

7Barry Raybould, EPSS: Unlocking Its Potential in Your Organization, Technical & SkillsTraining February/ March 1996

8Diane Gayeski, From Training Department to Learning Organization. Performance &

Improvement, Volume 35 #7, August 1996.

5

The need for learning organizations is driven by the increasing complexity, dynamism and competitiveness

of the global economic environment. Excelling in todays dynamic organizational environment requires more

understanding, knowledge, preparation, and agreement than one person's expertise and experience

provides. According to David Garvin of Harvard University, "Continuous improvement requires a

commitment to learning." 5 Specifically, creation of a learning organization requires the following actions:

Create continuous learning opportunity

Promote inquiry and dialogue

Encourage collaboration and team learning

Establish systems to capture and share learning

Empower people towards a collective vision

Connect the organization to its environment

The concept of a learning organization provides the foundation for a paradigm shift in the concept of

training. In North America in 1995, more than $55.3 billion was spent on formal training and development

of employees6. It is estimated that on average, only 10 to 20 percent of training transfers to the job

leading to increased employee performance, and thus, organizational performance7. Learning organizations

are not about training.8In fact, many managers believe training has failed to prepare employees for the

future and harness organizational knowledge. Managers are demanding that training be held accountable,

with specific measurable contributions to organizational performance.

The new paradigm of performance technology emphasizes the support of job performance, with the

establishment of specific measures. The new approach establishes linkage to organizational performance

measures during the performance technology design process through the use of performance-centered


9/33

9Barry Raybould. The Role of Technology in Improving Performance. EPSS.COM, November

1997.

6

design. The current definition of electronic performance support by Barry Raybould represents this new

paradigm:

An electronic infrastructure that captures, stores, and organizes corporate knowledge

assets throughout an organization, enabling individuals to achieve required levels of

performance in the fastest possible time and with a minimum of support from the people.

Performance is achieved by designing the computer/human interface using the principals

of Performance-Centered Design (PCD), which focuses on the audiences as performers of

work, rather than as users of a system. (Raybould, 1996)

Measurement: The Missing Link in Performance Technology

A key element of performance technology is performance measurement. Indeed, improvement is often

impossible or ineffective in the absence of measurement. Measurement provides the benchmark against

which continuous process improvement may occur.

Until recently, performance technology focused on traditional training. The measurement of training

effectiveness had been conducted in relative isolation from organizational performance measurement. With

the rapid evolution and convergence of training and performance technologies, there is a renewed potential

for establishing a direct relationship between performance technology and organizational effectiveness.

Specifically, the shift away from traditional training to electronic performance support (EPS) is forcing theintegration of the measurement effectiveness of performance technology and organizations.

Performance versus Learning Focus

The key to this shift is the nature of performance technology. According to Raybould, performance

technology can be designed with the primary objective of transferring knowledge or of producing

performance.9 In the performance support literature it has been shown that it is possible to design an

intervention that can generate performance without pre-requisite learning. Learning often occurs as a

by-product but it is not the primary objective.


10/33

10Public Law: 103-62, An Act to provide for the establishment of strategic planning and

performance measurement in the Federal Government, and for other purposes.

11Banerji, Ashok. Electronic Performance Support Systems. Proceedings of International

Conference on Computers in Education (ICCE 95), 1995.

7

Trends in Performance Measurement

The renewed emphasis on performance measurement in the private and public arena, including the

passage of the Government Performance & Results Act of 199410, has sought to establish a public strategic

planning process driven by a balanced and comprehensive set of performance measures. This new

planning process is being referred to as Performance Management (PM).

At the same time, there has been an historic shift in the traditional paradigm of training design. Until

recently, a linear process referred to as Instructional Systems Design (ISD) created training systems focused

on the achievement of learning objectives. These learning objectives were typically not directly related to

the principal organizational performance measures. The new paradigm, referred to as Performance-centered

Design (PCD), argues that all training and performance support be driven by specific measurable

performance outcomes, thus providing continuous and seamless integration of all human performance

development activities in the strategic planning process. Recently, there has been an additional shift which

says that performance-centered design should be renamed customer-centered design, where all support

should directly impact the cost, quality, effectiveness of a product or service delivered to the ultimate

customer.

Performance Technology Measures in The Studies Reviewed

Our literature survey indicated a greater focus on establishing measures with a direct relationship to

organizational performance. We reviewed three articles which studied the effectiveness of performance

technology.

According to Banerji, measures of the effectiveness of performance technology can include:11

error rate

time to complete a task

the quality of outcome

cost of task execution


11/33

12Krien, Theodore and Maholm, Timothy. CBT has the Edge in a Comparative Study.Performance & Instruction. August 1990. p.23.

13Mackay, Betty. EPSS technology in Banking. Interactive '94 (proceedings). 1996.

14 Gloria Gery, Attributes and Behaviors of Performance-Centered Systems, Performance

Improvement Quarterly, Vol 8 No.1, 1995.

8

According to Krein and Maholm, they used Mean test Scores and Time to Complete. They also referred

to Travel and Per Diem costs. These measures reflect the learning centered paradigm, and have little direct

relationship to organizational performance.12

According to Gery, measures may include reduced initial training time, task accuracy, time on task, time

to mastery and the ability to achieve day one performance (acceptable performance in the first day of work).

Mackay used error rate , task time per task , initial training time and on-the-job training time. 13

Most of these measures addressed individual performance, although most could be extrapolated to an

organizational performance measure. A limitation of most of these studies was that they did not attempt to

address the other benefits of performance technology, including greater cross-utilization of personnel, ability

to quickly reassign or replace personnel, and the benefits of a corporate knowledge base which captures the

expertise of the workforce.

The Relationship of Performance Technology Performance Measurement to Organizational Strategic

Processes

Over the last five years, several new management planning and process improvement methodologies

have emerged. These include:

Performance Management

Business Process Re-engineering

Activity-based Management

In the arena of training and performance support, a new paradigms of Performance-centered Design14


12/33

15 Barry Raybould, Performance Support Engineering: An Emerging Methodology for EnablingOrganizational Learning, Performance Improvement Quarterly, Vol 8 No.1, 1995: 1.

16Public Law: 103-62, An Act to provide for the establishment of strategic planning and

performance measurement in the Federal Government, and for other purposes.

17Executive Guide: Effectively Implementing the Government Performance and Results Act.,

06/01/96, GAO/GGD-96-118)

9

andPerform ance Suppo rt Engineering15have challenged the traditional linear instructional systems design

model.

All of these focus on measuring and enhancing some aspect of organizational performance. Although

these concepts have developed within different communities, they all possess common elements of

measurement. These methodologies are briefly described below.

Performance Management

Performance measurement became central to effective public strategic planning. With the passage of

the Government Performance and Results Act of 199316GAO published an executive guide on implementing

the Government Performance and Results Act (GPRA), which describes the focus of the law. Specifically,

the act established the following requirements: (1) GPRA forces federal agencies to focus on their missions

and goals, how to achieve them, and how to improve their structural organizations and business processes;

(2) agencies must define their missions and desired outcomes, use strategic planning, involve stakeholders,

assess their environments, and align their activities, core processes, and resources to support

mission-related outcomes; (3) agencies need to measure their performance to ensure that they are meeting

their goals and making informed decisions; (4) performance measures need to be based on program-related

characteristics and performance data must be sufficiently complete, accurate, and consistent; (5) agencies

must use performance data to improve organizational processes, identify performance gaps, and set

improvement goals; and (6) GPRA success depends on strong leadership practices that devolve decision-

making authority with accountability, create incentives, build expertise, and integrate management

reforms17.

Performance Management is a broader concept which has evolved from GPRA implementation.

Performance Management incorporates the establishment of performance measures under GPRA, but also


13/33

18 Michael Hammer. Reengineering Work: Dont Automate, Obliterate. Harvard Business

Review. July/August 1990. pp.104-105.

19Thomas, B., Baron, J. and Schmidt, W.. Evaluating a Performance Support Environment for

Knowledge Workers . USACERL Technical Report 95/32 , September 1995

10

addresses the need for continuous process improvement, which uses data from good performance

measurement.

To be effective, the performance-centered design process must start with organizational performance

objectives produced from performance planning. Performance technologists may also want to graphically

map the relationship between performance technology measures of effectiveness and organization

measures.

Functional Process Improvement & Business Process Re-engineering (BPR)

According to Michael Hammer, BPR is:

The fundamental rethinking and radical redesign of business processes to achieve dramatic

improvements in critical, contemporary measures of performance, such as cost, quality,

service and speed.18

The BPR concept specifically incorporates performance measurement, but also identifies the tools to

conduct process redesign. The concept of performance -centered design represents a new set of tools that

can help focus training resources on achieving organizational objectives.Performance technology is now

just another technology option for process improvement, in the context of BPR. A missing element

has been evaluation techniques for dec ision makers.

Our literature review identified a paper which addressed this issue. Thomas, Baron and Schmidt identified

five evaluation and measurement techniques to assess the feasibility and effectiveness of performance

technology, in the context of BPR: 19These included Work Profile Analysis, Direct to Indirect Ratio, Time

Saved Times Salary (TSTS), Activity Based Costing (ABC) and Quality Assessment.


14/33

20 Paula Spinner. Using Activity-Based Management to Improve Operations. Department of

Defense, 1996, p. 27.

11

This research offered a different perspective on measurement, focusing on predictive methodologies for

decision makers.

Activity-based Management

Activity-based Management (ABM) is business management in which process owners have the

responsibility and authority to control and improve operations, and that uses Activity-based Costing (ABC).

ABC is a set of management information and accounting methods used to identify, describe, assign costs

to, and otherwise report on the operations in an organization. Performance measurement data is of little use

without accurate representation of costs on an activity basis, which is the level at which process

improvement occurs.

A well-designed ABM/ABC systems directly supports appropriate operational and financial performance

measurements, provide cost information for business decisions such as make vs. buy, outsourcing, product

pricing, and capital investment justification, and provides timely and actionable information for management.

Specifically, ABC produces the following data related to performance measurement 20:

Activity Cost - Value added, Cost of Control, Cost of Quality

Performance Measurement - Unit Cost, Quality

Business Process Analysis - Process Flow, Cost Drivers, Cycle Time, Process Control

Business Process Reengineering - Selection, Process Scope, Process Modeling

Benchmark Activities - Organization, Best in Class, Competition

Planning and Budgeting - Workforce, Target Costing, Core Competencies


15/33

12

ABC Model

GeneralLedger Costs

Resources

Activities

Outputs

ActivityAnalysis

Input

Input

Input

Input

Output

Activity Costs Process Costs Product Costs Unit Costs

Trace

Trace

Trace

Figure 3. Activity Based Cost Process Model

Until recently, there have been limited measures of activity costs. With Activity-based management, there

is a methodology for measuring the cost of specific activities. By combining performance-centered design

with activity-based cost information, training and performance support development cost-effectiveness can

potentially be measured on a task or activity basis. This provides an important link between the

measurement of performance technology and organizational effectiveness.

Are the Emerging Performance Technology Development Methodologies Compatible?

This paper evaluated the two emerging training development methodologies as described in the research

literature, including:

Performance-centered Design

Performance Support Engineering

Each of these is critically evaluated relative to its potential compatibility with Performance Management,and thus its ability to support direct measurement of performance technology effectiveness.

Performance Centered Design (PCD)

Performance-centered design is a development methodology which defines performance outcomes


16/33

21

Barry Raybould. Performance Support Engineering: An Emerging Methodology for EnablingOrganizational Learning, Performance Improvement Quarterly, Vol 8 No.1, 1995: p.2.

22 Craig Marion. What is Performance-Centered Design?.WWW.EPSS.COM. p.2.

23 Gloria Gery, Performance Support: Performance Centered Design. Gery Associates, 1995.

13

(measures) early in the training and information systems development process. By defining these

performance outcomes, the complete design process focuses on achieving these specified outcomes, and

not some intermediate training effectiveness outcome.

Thus, a level of job or task performance is specified, and the training and performance support system

is customized to achieve that result. PCD is an umbrella concept that can result in a solution of job aids,

electronic performance support tools, computer-based training, video teletraining or traditional classroom

training, depending on the nature of the task to be supported.

An underlying element of the philosophy of PCD is that learning is most effective when performed in the

context of actual work. Many organizations report that 85-90% of a person's job knowledge is learned on

the job, and only 10-15% is learned in formal training events. PCD recognizes this reality21.

The PCD concept is focused on the integration of training and performance support with information

systems and applications. PCD infuses information systems with knowledge, structures tasks, and enables

performers to achieve the required level of performance as quickly as possible with minimum support from

other people22. Software that is designed around performance is intuitive to its users and enables them to

perform their normal work with obvious gains in speed and efficiency without ever attending training classes

or looking things up in books. It reflects their own conceptualization of their work and incorporates their

language, idioms, metaphors, and understanding of how to perform tasks.

According to Gery, the focus of PCD is on the efficient performance of individual tasks23. Thus, PCD

establishes performance measures at a lower level than those measures established under the concept of

performance management. To be compatible, the measures established must be coordinated. Figure 1

displays the relative focus of PCD and performance management.


17/33

24 C. Winslow & W. Bramer. Future Work: Putting Knowledge to Work in the Knowledge

Economy. NY: The Free Press, 1994.

14

MISSION

OUTCOMES

OUTPUTS

INPUTS

PERFORMANCEMANAGEMENT

PERFORMANCEMEASUREMENT

UNDER GPRA

PERFORMANCEMEASUREMENT

UNDER PCD

ORGANIZATIONOUTPUT

ACTIVITYOUTPUT

INDIVIDUALOUTPUT

TASK

OUTPUT

Figure 4. Comparison of Performance MeasurementGranularity

PCD by its very nature results in greater congruence with organizational performance measures because

the task analysis targets a level of task performance. The task performance is based on a desired outcome

for the organization as a whole.

Winslow and Bramer of Andersen Consulting explain it like this24:

Traditional transaction-based systems are designed with an emphasis on process and

data modeling. In a way, they are designed inside-out. The user interface modeling is

derived from the process and data structures. For example, the layout of screens is often

a reflection of a record structure,

and the menus of the system

reflect the functional structure.

Performance systems

are more interactive,

and more oriented

toward actual work

circumstances. Thus,

these systems need to

be designed from the

outside-in. The character

of process and data

modeling in itself does

not change so much, but

the (outside) user

interface model ing

drives the (internal)

process modeling anddata modeling.


18/33

25 Gloria Gery. Performance Support: Performance Centered Design. Gery Associates, 1995.

26 Barry Raybould. Performance Support Engineering: An Emerging Methodology for Enabling

Organizational Learning, Performance Improvement Quarterly, Vol 8 No.1, 1995: p.2.

15

According to Gery, When designers have the point of the view of the performer situated

in a real work context, success is inevitable. If the point of view does not closely match the

situation, usability and performance problems are inevitable."25

According to Gery, PCD is about human performance, not system performance. In fact, the philosophy of

PCD is about targeting the attention of software designers away from systems, and on job performance.

Performance Support Engineering

A second major emerging training & performance development methodology is performance support

engineering. This concept was introduced by Dr. Barry Raybould. In a paper entitled Performance Support

Engineering: An Emerging Methodology for Enabling Organizational Learning, Raybould presents a

conceptual model called the Organizational Performance/Learning Cycle26. This model describes the

dynamics of the organizational learning process and provides a framework for thinking about the

technologies and methodologies that enable that process. Raybould argues that previous definitions of

EPSS were too limited and restricted in their scope and expands on these earlier definitions to take into

account this new model of organizational learning. The new definition clearly distinguishes EPSS from

traditional systems development

According to Raybould, this new view of EPSS offers a clear opportunity for organizations to recognize

the strategic importance of managing their knowledge assets. Most of the current methodologies in use by

different functional groups are limited in their capability to enable this model and the expanded definition of

EPSS. The Performance Support Engineering methodology is intended to overcome these limitations.

Rayboulds central thesis focuses on broader strategic aspects of performance design than PCD. He

recognizes that there are synergies to be realized from creating a learning organization, and that knowledge

is a strategic asset. For example, the new concept of dynamic performance support, where the knowledge

and task input are captured fall within the broader concept of performance support engineering.


19/33

16

Raybould proposes a broader definition of performance support systems based on a systems approach.

An Electronic Performance Support System (EPSS) is the electronic infrastructure that

captures, stores and distributes individual and corporate knowledge assets throughout an

organization, to enable individuals to achieve required levels of performance in the fastest

possible time and with a minimum of support from other people.

Rayboulds revised image of performance support envisions a dynamic performance support environment

where the performance support system not only provides access to knowledge, but captures and

disseminates new knowledge.

The implications of performance support engineering for performance measurement are significant. First,

there are no incompatibilities between performance centered design and performance support engineering.

Both may be applicable. PCD addresses the task level of performance support. Performance support

engineering, on the other hand, addresses a new set of organizational performance measures which address

an organizations learning capability. For example, these measures might address how quickly information

is captured and disseminated.

An Integrated Model of Performance Technology & Organizational Performance Planning and

Management

Based on our research, Figure 5 displays a proposed integration model for performance technology and

organization performance planning and management. From our literature survey, it was clear that while both

address performance, the perspective is at completely different level. Performance Technology addresses

individual and group performance, which performance management and strategic planning address

organizational performance. The two are compatible, but it requires a better understanding on the part of

both to ensure goal congruence.


20/33

17

CONTINUOUS PROCESSIMPROVEMENT

MISSION

OUTCOMES

INPUTS

PerformanceManagement

ORGANIZATIONOUTPUT

ACTIVITYOUTPUT

INDIVIDUALOUTPUT

TASK

OUTPUT

2. Change

- How can we

3. Measurement

- How are we

doing?

1. Identification

- What are our

opportunities?

affect change?

Performance-centered DesignStrategic

PlanningActivityBased

Management

Figure 5. An Integrated Model of Performance Technology & Organizational Performance

Management and Planning


21/33

18

10

20

30

5

15

25

TimeinMinutes

E1 N1 N2 N3 N4 N0

Control Groups

E1 Expert

N0 Novice

Test Groups

N1 Paper docs

N2 Expert users help

N3 Text-based EPSS

N4 Multimedia EPSS

Performance Time With Various Supports

Groups(c) Dr Ashok Banerji Sing apore Polytechnic ICCE 95

Figure 6. Performance Time for Various Forms ofSupport

Figure 5. Memory Rentention Rates

The Research Literature: How Effective is Performance Technology?

We reviewed three research articles to identify data on the effectiveness of performance technology. The

consensus of the research is that performance technology can offer specific measurable benefits to

organization effectiveness.

In one of the only control studies on performance supports systems, Banerji compared the performance

of different groups in performing a defined sets of information systems tasks. He provided various support

tools (on-line help, expert user) and developed two electronic performance support systems variants. His

conclusions were that EPSS technology :

Leads to 25%-40% savings in cost

Decreases 20%-50% training time

Decreases 30%-100% training

delivery time

Decreases 20%-40% document

reading time

Leads to 33% reduction in paper

documentation

Increases memory and retention by

16%-24%

Increases productivity by 25%

Figure 6 displays the results of the control groups performance for various measures of effectiveness.

In a control study by Krein and Maholm, computer-based training demonstrated higher test scores and

reduced training time. Figure 7 displays the results.

Average Time to Complete Mean Score on Final Test

Classroom Group (n=61) 3 hours, 52 minutes 13.4

CBT Group (n=29) 2 hours, 48 minutes 15.6

Figure 7. Effectiveness of CBT vs. Classroom Training


22/33

27 Mackay, Betty. EPSS Technology in Banking. Interactive '94 (proceedings). 1994.

19

In a study by Mackay, she testing an EPSS-style front end for a legacy system in bank operations, tested

on groups of new and experienced employees. The results included: reduced errors (73% for existing

employees, 87% for new), reduced time per task (33% for existing employees, 77% for new), reduced

training and on-the-job training time (75% reduction in on-the-job training time).27

Conclusions

The emergence of performance planning and performance-centered design methodologies offers an

opportunity to increase the accountability of training functions within organizations. The following conclusions

are suggested, based on the review conducted and the research questions posed::

There are no inherent incompatibilities between performance management and the emerging

training and performance support development methodologies.

The concept of performance-centered design is narrow in scope, addressing performance measures

for the performance of specific tasks.

Performance-centered design and performance management functions should coordinate design

activities to ensure the compatibility of measures selected.

Performance-centered design and performance support engineering are not mutually exclusive. Both

can proceed at the same time.

New measures may be required to address the effectiveness of learning organizations.

Performance-centered design can ensure goal congruence between training system design and

organizational performance measures.

The performance system developed may incorporate data collection tools to support the

performance measurement process.

To achieve this paradigm shift, the strategic planners, trainers and systems developers must

coordinate the establishment of performance measures.

There are limited empirical studies of the effect of training and performance support technologies

on organizational effectiveness.

The studies that have been conducted indicate substantial increases in productivity and reductionsin cost.


23/33

20

References

*"The Learning Organization." Chief Executive. #101 (March 1995): 57-64.

*Chalofsky, Neal E. "A New Paradigm for Learning in Organizations." Human Resource Development

Quarterly. 7 #3 (Fall 1996): 287-293.

*DiBella, Anthony J. "Developing Learning Organizations: A Matter of Perspective." Academy of

Management Journal. Best Papers Proceedings (1995): 287-290.

*Garratt, Bob. "An Old Idea That Has Come of Age." People Management. 1 #19 (September 21, 1995):

25-28.

*Gephart, Martha A. and others. "Learning Organizations Come Alive." Training & Development. 50 #12

(December 1996): 34-36+.

*Kaplan, Robert S. and David P. Norton. "Strategic Planning and the Balanced Scorecard." Strategy &

Leadership. 24 #5 (September/October 1996): 18-24.

*Kuchinke, K. Peter. "Managing Learning for Performance." Human Resource Development Quarterly. 6 #3

(Fall 1995): 307-316.

*Rheem, Helen. "The Learning Organization." Harvard Business Review. 73 #2 (March/April 1995): 10.

*Schein, Edgar H. "Three Cultures of Management: The Key to Organizational Learning." Sloan Management

Review. 38 #1 (Fall 1996): 9-20.

*Senge, Peter. "Learning to Alter Mental Models." Executive Excellence. 11 #3 (March 1994): 16-17.

*Senge, Peter. "The Learning Organization Made Plain." [interview] Training & Development. 45 #10

(October 1991): 37-44.

*Senge, Peter. "Mental Models." Planning Review. 20 #2 (March/April 1992): 4-10+.


24/33

21

*Senge, Peter. "The Message of the Quality Movement." Executive Excellence. 12 #7 (July 1995): 5-6.

*Shaw, Diana V. and others. "Learning from Mistakes." Quality Progress. 28 #6 (June 1995): 45-48.

American Society of Training & Development (ASTD), Training & Development Magazine, September 1996.

Anthony, G. Michael, "IE's Measure Work, Write Standards for White Collar Workers at Financial Institution,"

Issues in White Collar Productivity (Industrial Engineering and Management Press, Institute of Industrial

Engineers, 1984), pp 84-87.

Argyris, Chris. "Teaching Smart People How to Learn." Harvard Business Review. 69 #3 (May-June 1991):

99-109.

Argyris, Chris. Overcoming Organizational Defenses: Boston: Allyn & Bacon, 1990.

Argyris, Chris. Organizational Learning II: Theory, Method, And Practice. Reading, MA: Addison-Wesley,

1996.

Argyris, Chris. On Organizational Learning. Cambridge, MA: Blackwell, 1993.

Argyris, Chris. (1994), Good Communication that Blocks Learning. Harvard Business Review, July-August.

(HBR Reprint No. 94401)

Argyris, Chris. (1991), Teaching Smart People How to Learn. Harvard Business Review, May-June. (HBR

Reprint No. 91301)

Banerji, A.K.,(1995). Designing Electronic Performance Support Systems, PhD Thesis, School of Computing,University of Teesside, Cleveland, UK.

Banerji, Ashok. Electronic Performance Support Systems. Proceedings of International Conference on

Computers in Education (ICCE 95), 1995.


25/33

22

Barker, P.G. (1993). Exploring Hypermedia, Kogan Page, London.

Barker, P.G. and Banerji, A.K. (1993). Designing Electronic Performance Support Systems, 140-142,

Proceedings of the 10th International Conference on Technology and Education, Boston, USA.

Barker, P.G. and Banerji, A.K. (1995). Designing Electronic Performance Support Systems, Educational and

Training Technology International, Special Edition on Electronic Performance Support Systems, 32(1).

Barker, P.G. and Banerji, A.K. (1994). Evolving Principles of Performance Support, Association for Learning

Technology Conference'94, Enabling Active Learning, University of Hull, 19-21 September.

Bernard, Paul, "Structured Project Methodology Provides Support for Informed Business Decisions," IE

(March 1986), pp 52-57.

Beruvides, M.G., and D.J. Sumanth, "Knowledge Work: A Conceptual Analysis and Structure," Productivity

Management Frontiers-I(Elsevier Science Publishers B.V., 1987), pp 127-138.

Bohl, Don Lee, ed. The Learning Organization in Action. New York: American Management Association,

1994.

Brassard, Michael, The Memory Jogger-A Pocket Guide of Tools for Continuous Improvement(Goal/ QPC,

1988).

Brown, J.S. and Puguid, P. (1991), Organizational Learning and Communities of Practice. Organization

Science, 2(1), 40-57.

Drucker, Peter F. (1994), The Theory of the Business. Harvard Business Review, September-October.

Bussell, Linda. A Performance Technology Case Study: The Medisoft International/Omnicom Alliance.

Performance & Instruction, Volume 34, #5. June 1995.


26/33

23

Chase, Nancy. Training effectiveness measures and scoring schemes: A comparison. Personnel

Psychology. v44 n2, Summer 1991, p. 353.

Chavalier, Roger. Improving Efficiency & Effectiveness of Training: A Six Year Case Study of Systemic

Change. Performance & Instruction. .May/June 1996.

Chawla, Sarita and John Renesch, eds. Learning Organizations: Developing Cultures for Tomorrow's

Workplace. Portland, OR: Productivity Press, 1995.

.

Blueprint for Quality : How Training Can Turn Strategy into Real Improvement. San Francisco: Jossey-Bass,

1993.

Cleary, B.A., "Company Cares About Customers' Calls," Quality Progress, vol 26, No. 11 (November 1993),

pp 69-73.

Cox, Thomas, "The Myth of the Commodity Database or How To Pick the Best Technology for You," Oracle

Integrator(January/February 1993), pp 19-21.

Davidove, Eric. Evaluating the Return on Investment in Training. Performance & Instruction. January 1993.

De Geus, Arie. (1988), Planning as Learning. Harvard Business Review, 66(2), March-April, 70-74.

Dennison, D. R., Corporate Culture and Organizational Effectiveness, (New York: NY: John Wiley, 1990).

Drucker, Peter F., Management(Harper & Row, 1974).

Duchastel, Philippe C.; Lang, Joyce. Raise your training ROI. Quality. v36n9. Sep 1997. p. 28-41.

Dumaine, Brian (1994) Mr. Learning Organization. Fortune, October 17, 147-157.

Executive Guide: Effectively Implementing the Government Performance and Results Act., 06/01/96,

GAO/GGD-96-118)


27/33

24

Gery, G. (1991). Electronic Performance Support Systems: How and Why to Remake the Workplace through

the Strategic Application of Technology. Boston, MA: Weingarten Publications.

Gery, G. Performance Support: Performance Centered Design. Gery Associates, 1995

Gery, Gloria, Attributes and Behaviors of Performance-Centered Systems, Performance Improvement

Quarterly, Vol 8 No.1, 1995.

Gery, G. J. "Attributes and Behaviors of Performance-Centered Systems," Performance Improvement

Quarterly, 8 (1), pp. 47-93, 1995.

Gery, G.J. (1991). Electronic Performance Support Systems - How and Why to Remake the Workplace

Through the Strategic Application of Technology, Weiengarten Publications, Boston, USA.

Gore, A., From Red Tape to Results. Creating a Government That Works Better and Costs Less: Report

of the National Performance Review, (Washington, DC: GPO, 1993)

Gore, A., From Red Tape to Results. Creating a Government That Works Better and Costs Less: Report

of the National Performance Review, (Washington, DC: GPO, 1993)

Hammer, Michael and Champy, James, Reengineering the Corporation, 1993.

Hawkins, P. (1991), The Spiritual Dimension of the Learning Organization. Management Education and

Development, 22(3), 172-187.

Helton, B. Ray, "Quality and the Bottom Line, Part 3: The Inner View," The Quarterly Observer(April 1992),

p 3.

Helton, B. Ray, "Achieving White-Collar Whitewater Performance by Organizational Alignment," National

Productivity Review(Spring 1991), pp 227-244.

Helton, B. Ray, "Quality and the Bottom Line, Part 2: The Company's Side," The Quarterly Observer(March


28/33

25

1992), p 5.

Helton, B. Ray, "More of the Right Stuff," The Quarterly Observer(September 1993), p 5.

Liggettt, Trevino, and Lavelle, "Activity-Based Cost Management Systems in Advanced Manufacturing

Environments," in Parsaei et al. (eds.), Economic and Financial Justification of Advanced Manufacturing

Technologies(Elsevier, Amsterdam, 1992).

Hybert, Peter and Perry, Dreama. Beyond the Buzzwords- TQM, HPT, & Shareholder Value. Performance

& Instruction. Volume 34, #2. February 1995.

Johnson, H. Thomas, "It's Time to Stop Overselling Activity-Based Concepts," Management Accounting

(September 1992), pp 26-35.

Kelly, Robert, and Janet Kaplan, "How Bell Labs Creates Star Performers," Harvard Business Review

(July-August 1993), pp 128-139.

Kim, D. H. (1993), The Link between Individual and Organizational Learning. Sloan Management Review,

Fall, 37-50. (Reprint 3513)

Kirkpatrick, D. L. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler

Publishers, 1994.

Krein, Theodore & Maholm, Timothy. CBT has the Edge in a Comparative Study. Performance & Instruction.

August 1990.

Laffey, J., Machiraju, N. Rao, Chandhok, R. (1991). Integrated Support and Learning Systems for

Augmenting Knowledge Workers. Proceedings of World Congress on Expert Systems, Orlando: Pergamon

Press.

Laffey, J., Machiraju, N. Rao, Chandhok, R. (1991). Organizational Memory as a Support for Learning and

Performance: Prototypes and Issues. Proceedings of the International Conference on the Learning Sciences,

1991.


29/33

26

Leeuw, Frans L.Can Governments Learn?: Comparative Perspectives on Evaluation & Organizational

Learning. New Brunswick: Transaction Publishers, 1994.

Mader, R., (1992). Managing and Organising an Electronic Performance System Project, 10th Annual

Computer-Based Training Conference and exposition, March 2-5, Orlando, USA.

Marquardt, Michael J. Building the Learning Organization. New York: McGraw-Hill, 1996.

McDermott, Richard. "Designing and Improving Knowledge Work." Journal for Quality and Participation. 18

#2 (March 1995): 72-77.

Noer, David M. Breaking Free: A Prescription for Personal and Organizational Change. San Francisco:

Jossey-Bass, 1997.

McDonald, J. Performance support systems for learning. Journal of Educational Technology Systems v. 24

no1 ('95-'96).

McGraw, K.L., (1994). Performance Support Systems: Integrating AI, Hypermedia and CBT to Enhance User

Performance, Journal of Artificial Intelligence in Education, 5(1), 3-26.

National Research Council, Information Technology in the Service Society (National Academy Press,

Washington, DC, 1994).

Nay, D. C. The Effectiveness of a Multimedia Training System on Organizational Performance. International

conference -- 1995 May : Atlanta; GA, Information Resources Management Association: Managing

information and communications in a changing global environment. pp. 342-353.

Nevis, E. C., DiBella, A. J. and Gould, J. M. (1995), Understanding Organizations as Learning Systems.

Sloan Management Review, Winter 1995, 73-85.

Nonaka, I. (1991), The Knowledge-Creating Company. Harvard Business Review, 69(6),96-104.


30/33

27

Norman, D. (1993). Things That Make Us Smart - Defending human attributes in the age of the machine.

Reading, MA: Addison-Wesley Publishing Co.

Ostroff, Cheri.. Performance measurement and decision support systems. Management Accounting:

Magazine for Chartered Management Accountants. v71 n10, Nov 1993, p. 10.

Parker, Marilyn M., Robert J. Benson, and H.E. Trainor, Information Economics Linking Business

Performance to Information Technology(Prentice-Hall, Englewood Cliffs, NJ, 1988).

Partovi, "An Analytic Hierarchy Approach to Activity Based Costing," International Journal of Production

Economics, vol 22 (1991), pp 151-161.

Partovi, "An Analytic Hierarchy Approach to Activity Based Costing," International Journal of Production

Economics, vol 22 (1991), pp 151-161.

Peterson, I., Probing a Computer Productivity Paradox," Science News (1 Jan 1994), p 7.

Rittenhouse, Robert G. "Productivity and the Microcomputer," Management of Technology III (Institute of

Industrial Engineers, 1992).

Phillips, J. J. Handbook of Training and Evaluation and Measurement Methods (2nd ed.). Houston, TX: Gulf

Publishing Company, 1991.

Public Law: 103-62, An Act to provide for the establishment of strategic planning and performance

measurement in the Federal Government, and for other purposes.

Puterbaugh, G. "CBT and Performance Support," CBT Directions, pp. 18-25, June 1990.

Ray, James and Sword, Stanley. Reengineering and Human Performance. Performance & Instruction.

August 1993.

Raybould, B., (1990). Solving Human Performance Problems with Computers - A Case Study: Building an

Electronic Performance Support System, Performance and Instruction, 29, 4-14.


31/33

28

Raybould, B. Info-Line: Making EPSS Work for Your Organization. Alexandria, VA: American Society for

Training and Development, January 1995.

Raybould, Barry. (1994) Performance Support Engineering: An Emerging Development Methodology for

Enabling Organizational Learning. Performance Improvement Quarterly 8(1) pp. 7-22.

The Effectiveness of Computer-based Training. Institute for Defense Analysis, 1991.

Raybould, B. (1990). Solving Human Performance Problems with Computers. Performance & Instruction,

29(11), 4-14.

Raybould, Barry. Solving Human Performance Problems with Computers- A Case Study: Building an

Electronic Performance Support System. Performance Improvement Quarterly. November/December 1996.

Roach, Stephen, "Services Under Siege-The Restructuring Imperative," Harvard Business Review

(September-October 1991), pp 82-83.

Robinson & Robinson, Training Magazine, March, 1996.

Sassone, Peter G., "Cost Benefit Methodology for Office Systems," ACM Transactions on Office Information

Systems, vol 5, No. 3 (July 1987).

Sassone, Peter G.. "A Survey of Cost-Benefit Methodologies for Information Systems," Project Appraisal,

vol 3, No. 2 (June 1988), pp 73-84.

Sassone, Peter G., "Office Productivity: The Impacts of Staffing, Intellectual Specialization and Technology"

(The Georgia Institute of Technology-School of Economics, September 1991), pp 1-35.

Sassone, Peter G., "Survey Finds Low Office Productivity Linked to Staffing Imbalances," National

Productivity Review (Spring 1992), pp 147-158.

Sassone, Peter G., "Three Approaches for Estimating the Value of Office Work," Office Technology and


32/33

29

People, vol 6, No. 1 (1992).

Senge, Peter. The Fifth Discipline: the Art and Practice of the Learning Organization. New York: Doubleday,

1990.

*Senge, Peter. "Leading Learning Organizations." Training & Development. 50 #12 (December 1996): 36-37.

Senge, P. (1990). The Fifth Discipline - The Art and Practice of the Learning Organization. New York, NY:

Doubleday Currency.

Senge, Peter. (1992), Building Learning Organizations., Journal for Quality and Participation, March.

Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction.

Reading, MA: Addison-Wesley Publishing Company, 1987.

Silberman, M., and Auerbach, C. Active Training: A Handbook of Techniques, Designs, Case Examples and

Tips. New York, NY: Lexington Books, 1990.

Stata, R. (1990), Organizational Learning: The Key to Managment Innovation. Sloan Managment Review,

30(3), 63-74.

Stone, Deborah and Villachica, Steven. Performance Support for Knowledge Workers: Practical Strategies

based on Research and Practice. Performance Improvement Quarterly, Volume 36, #3, March, 1997.

Tate, William. Developing Corporate Competence: A Performance Agenda for Managing Organizations.

Brookfield, Vt: Gower 1995.

Thomas, B., Baron, J. and Schmidt, W., USACERL Technical Report 95/32 , Evaluating a PerformanceSupport Environment for Knowledge Workers September 1995.

Thomas, B.E., and J.P. Baron, Evaluating Knowledge Worker Productivity: Literature Review, Interim Report

(IR) FF-94/27 (U.S. Army Construction Engineering Research Laboratories [USACERL}, June 1994).


33/33

30

Today's Need for Viable Training Measures of Effectiveness. CONFERENCE: 13th -- 1991 Dec Orlando;

FL, Interservice/industry training systems conference, 1991 , pp. 401-405

Weston, D. M. (1994). Organizational Learning in Practice. (Publication D94-1817) Menlo Park, CA: SRI

International.

Weston, D. M. (1994), Organizational Learning as Strategy. (Publication R827) Menlo Park, CA: SRI

International.

Wikoff, Martin. The Quality Movement Meets Performance Technology. Performance & Instruction. Volume

3, #8. September 1994.

Winslow, C. & Bramer, W. (1994). Future Work: Putting Knowledge to Work in the Knowledge Economy.

NY: The Free Press.

Wood, Del. A Framework for Re-engineering Traditional Training: Interactive Training and Performance

Support. Performance Improvement Quarterly, Volume 35, #8, September 1996.

Learning Org Paper

Documents

Transcript of Learning Org Paper