Trends in End-User Training: A Research Agenda Fred Niederman

Management Information Systems Area University of Baltimore

Baltimore, Maryland 21201

fniederman @ubmail.ubait.edu

1. ABSTRACT This paper surveys the end-user training literature. It discusses key training issues in terms of pedagogical approaches, delivery methods, and settings. This paper expands discussion of three emerging training techniques: training through Internet, Performance Support Systems, and Videoconferencing. These training approaches are placed in the larger organizational context focusing on individual characteristics, software categories, and traditional organizational variables. The concept of a training portfolio is introduced with additional consideration of the importance of scaiabiiity of training method and cost/benefit considerations. Research propositions and suggested methodologies are presented to support future research efforts. 1.1 Keywords K-3.1 Computer Uses in Education; K.6.1 Project and People Management lTraining]

2. BACKGROUND “The am&al of the information age has created an increasing number of subjects and individuals requiring training, as computer use by nonprogramming end users continues its rapid growth” (Simon, Grover, Teng, and lVhitcomb, 1996, p. 466-7)

“With the large growth in investments in information technology, it has become critically important to devise

kmkion to make digital/hard copies of al.11 or part of tbis material for personal or classroom use is granted witbout fiz provided that the wpis are not made or distributed for profit or commercial advantage, the copy- &bt not& the title of the publication and its date appear, and notice is given that copyright is by permission oftbe AChL Inc. To copy otherwise, to republish, to post on servers or to rediibute to lists. requires specific permission anndror fee. CPR 9s Boston MA USA Copqtight 1998 0-89791-959-9/98/3..S5.00

Jane Webster Department of Management Sciences

University of Waterloo Waterloo, Ontario, Canada N2L 3G1

jwebstetGmansci2.uwaterloo.ca

ways to improve end-user proficiency. ” (Santhanam and Sein, 1994, p. 378-379)

“Effective training and support are critical to ensure that new information technology is used to best advantage in adding value to the organization’s products and services. ” (Fitzgerald and Cater-Steel, 1995, referencing earlier work by Bostrom, Olfman, and Sein, I990).

Management of human resources has been rated by senior executives as a leading component of successful MIS by senior executives (Brancheau, Janz, and Wetherbe, 1996; Niederman, Brancheau, and Wetherbe, 1991). Information systems practitioners and researchers widely acknowledge that providing appropriate user training is critical to successfully implementing systems, and key to promoting productive use of the technology (Cheney, Mann, & Amoroso, 1986; Katz & Katz, 1995). According to Tumage (1990) , technological failures in organizations are often due to inadequate training rather than to the technology itself. Consequently, organizations must devise ways to leverage investments in information technologies to ensure that they are used properly to avoid costly errors, are used creatively to establish competitive advantage, and are used at all (to avoid wasted expenditures). An obvious target for human resource investment is end-user training. Ultimately, work is performed by individuals. The more knowledge and skills individuals bring to their work, the higher the probability that it will be performed effectively and efficiently. The introduction of a new technology creates a gap between skills possessed by the individual and those needed to fully utilize the new technology. Training is one approach to filling that gap. The level of success of that training depends on such factors as the individual characteristics the employee brings to the training, the method by which the training is delivered, and organizational factors that support or inhibit learning (Baldwin and Ford, 1988). Many traditional methods, such as classroom lectures, are being enhanced or modified through the availability of new computer technologies. These technologies allow for demonstrations of active problem-solving and, where learners have access to computing equipment, interactive problem-solving during class (Leidner and Jarvenpaa,

224

1993). Additionally, computer technologies can allow for novel training methods, such as peer-oriented discussions through the Internet and tutorials through CD-ROhIs. This paper extends past research in several ways. First, it focuses on formal training and learning by examining both traditional and emerging training methods. Second, it highlights those training contingencies (such as organizational characteristics) that have received little research attention in de past. Third, it proposes that researchers study the portfolio of training methods used by organ&&ions, along with their comparative costs and benefits. Finally, it develops a series of research questions to guide future research. The paper proceeds as follows. In the next section, the traditional end-user training literature is reviewed to spotlight research gaps in terms of both training methods and work--lace contingencies that can affect training success. The subsequent section makes the case for an overlooked area of research, that is, an organization’s portfolio of training approaches. The next section proposes a series of research questions to guide future research, and then suggests appropriate methodologies to address these questions.

3. T’RADmONfi END-USER TRAINING RESEARCH The goal of training is to produce a motivated user who has the basic skills needed to apply what has been learned and then to continue to learn on the job. To the information systems practitioner, the over-riding issue becomes: what are the key factors to consider in the management of end-user learning and training? Compeau, Olfinan, Sein, and Webster (1995) highlighted these factors in a special issue of Communications of the ACM on end-user training and learning (see Figure 1). The framework consisted of three main phases: initiation, formal training and learning, and post-training. Additionally it was proposed that contingencies, such as trainee, software, task/job, and organizational characteristics, innuence key decisions made at each phase of the process. The initiation phase includes such issues as the identiiication of training needs and the formation of training groups. Most research in the post-training phase has focused on the evaluation of training immediately

following the training, rather than on transfer of training to the workplace (Baldwin & Ford, 1988). However, managers often are more interested in the longer-term effects of training. Although the initiation and post- training phases represent important considerations in an overall training program, this research will target issues in the formal training and learning phase. (See Wexley and Latham, 1981, for an excellent discussion of identifying training needs -- Chapter 3 - and evaluating training programs - Chapter 5). A key question relating to formal training and learning of computer-based skills for end users focuses on the effectiveness of particular methods. An examination of these methods shows that they can be viewed in terms of pedagogical approach, delivery method, and setting. Researchers studying pedagogical approaches have discovered a variety of approaches that, under at least some conditions, provide value to learners. Examples of these are: hands-on activities, behavior modeling (e.g. Gist, Schwoerer, and Rosen, 1989; Simon, Grover, Teng, and Whitcomb, 1996), cognitive model overview, and exploratory learning (e.g. Davis and Bostrom, 1993; Simon, Grover, Teng, and Whitcomb, 1996). Naturally, hands-on activities refer to opportunities for the learner to directly interact with the computer system (or a training version). Behavior modeling refers to an instructor or peer demonstrating a particular behavior that the learner can then imitate. Cognitive model overview refers to presentation of an abstraction or framework that shows how the various elements of the learning domain fit together. Finally, exploratory learning provides the learner with the opportunity to interact with the computer program in a manner that is not pre-specified. The learner, as a result, will discover tools as they become salient rather than simply because they are next in line. Research has demonstrated that methods incorporating hands-on use, behavior-modeling, good conceptual models, manuals that encourage exploratory learning, and training previews can be effective (Simon, Grover, Teng, & Whitcomb, 1996; Olfman & Mandviwalla, 1994; Santhanam & Sein, 1994; Webster & Martocchio, 1995). Delivery method relates to the manner in which information or tasks are presented to the learner. Examples are from an instructor (either one-way as the instructor presenting information and the learner receiving

225

Initiation ---------> Training and -------> Post-Training Learning

Characteristics Software Task/Job Characteristics Characteristics

Organizational Characteristics

Figure 1: A Framework for the Training Process (adapted from Compeau et al., 1995)

it, or interactively), by videotape/text and other non- interactive media, and computer-based media such as CD- ROM. Setting refers to locations which either have or do not have computing equipment available and to the number of individuals participating as learners. This presents four possibilities (see Table 1). Note that in the computer available condition, learners could receive information from any of the delivery methods and could,

in principle, use any of the pedagogical approachs. In the computer not-available condition, the learner could receive information from either the instructor or video mode and, in principle, use any of the techniques. Leidner and Jarvenpaa (1993) present observations of differing approaches to the use of the computer-supported or electronic classroom.

Table 1: Setting Possibilities

Learner Cardinality 1

MAY

Computer Availability

YeS No

Personal workstation or laboratory cubicle Personal office or viewing room

Computer laboratory or computer equipped classroom Traditional classroom

226

If we consider both pedagogical approach and delivery method (see Table 2), we can see that some of the strengths and weaknesses of these delivery methods are highlighted. The strength of the instructor is in responding to unique situations. A combination of exploratory learning and behavior modeling can be very efficient. The learner may explore until confronted by a new situation, an error, or other challenges. The instructor may demonstrate a known response to dealing with the situation or (perhaps even better) demonstrate responding to new and unknown difficulties. One

strengths of the videotape and computer multimedia modes is in providing highly standardized instruction for example not allowing learners to skip or miss a key element. Another strength is in lending themselves to self-paced study particularly where the learner is working at his or her personal machine rather than working in the lab setting. It also becomes apparent that the delivery method does not determine the pedagogical approach, however, some pedagogical approaches would appear to be easier to use in combination with some delivery methods.

Table 2: Example Combinations of Pedagogical Approach and Delivery Methods

Delivery Methods

Instructor Videotape Computer multimedia/CD ROM

PeaagogicaI Approach

Hmds-Oil Possible if conducted in lab Requires video and Fully supported use computer equipment

Behavior Demonstrations for class in Requires anticipation and Can program/highlight modeling lab setting; one-to-one fihning of desired relevant steps

demonstrations behaviors

Cognitive Can use demonstration, Can include illustrations, Can provide background model graphics, prior reading graphics information, can provide overview links

Exploratory Possible if conducted in lab Weak on feedback and Stronger where pathways learrang error recovery anticipated, or sufficient

pathways for learner wandering

4. ElXERGmG 7IlWIMNG METHODS Emerging training methods, such as Internet-based training, performance support systems, and distance videoconferencing promise the potential for si,anificant improvements in the delivery of training to computer end- users. However, they have received little study to date. Table 3 outlines recent research on selected emerging training methods. Next, we describe each of these methods and discuss them in terms of pedagogical approach, delivery method, and setting. Additionally, we

propose research questions based on the relationship between these techniques and proposed dependent variables. For each of the new techniques we propose consideration of the frequency with which it is found, its effectiveness as an organizational intervention; the conditions under which it may be effective; and how it relates to other variables such as individual differences and organizational contingencies.

227

_ _-,__ . . _ _< ___i __- -_ . - _--~ _-- -AA.-._- -,-_ A--.:zL-- -I-L’.-- --____L-L,_-_ l--. i . __.

Table 3: Selected Research on Emerging Training Methods

Method I Publication

Internet

Performance Support Systems

Distance Video-conferencing

Atkinson & Kydd (in press)

Communications of the ACM (special issue, July 1997)

Webster & Hackley (1997)

4.1 Internet Training Atkinson and Kydd (in press) studied the relationship between individual characteristics and Web usage. However, Internet-based training has received little academic study, and although “it’s been talked about for years at training conferences, Internet-based training is still largely a promise” (Horowitz, 1997). Several organizations experimenting with this technology include the Smithsonian (Heidenrich, 1996) and Booz Allen (Nevins, 1995). In terms of pedagogical approach, it would seem possible that each of the four discussed above could be delivered through Internet. It would be an open question as to which techniques are actually utilized in Internet training as well as which are more effective. In terms of setting, Internet as an educational tool can potentially be used individually or by multiple persons especially in a fully equipped electronic classroom However, Internet still requires computer and telecommunication support. At a more basic level, it remains to be seen how use of Internet is related to the alternative delivery methods. Each of the three methods, instructor contact (in a “text-to-text” rather than face-to-face mode), video/text, or multimedia could be delivered through Internet connections. If the Internet does no more than serve as a conduit for these other delivery methods, then it would be better viewed as an “umbrella” rather than alternative delivery method. However, Internet allows multiple combinations of methods (e.g. multimedia with instructor back-up for deeper exploration of key issues) which may require that it be considered a new delivery method in its own right It should also be noted that use of Internet and intranet technologies enable the potential for an organization to develop a “learning community”. Participants in such a community can both annotate solutions they discover to various problems and profit from comments by others in the coMmmity.

Research Question IA: Will the various pedagogical approaches be utilii equally in Internet training? Research Question 1B: Will the various pedagogical approaches be equally effective in terms of training

outcomes (learning, retention, and transfer) in Internet training? Research Question 1C: What organizational variables will promote or retard the effectiveness of Internet training? Research Question 1D: What characteristics of individual learners will promote or retard the effectiveness of Internet training?

4.2 Performance Support Systems Performance support systems (PSS) are technologies that provide timely support to users as they perform tasks in their work environments. Few empirical studies have addressed the emerging area of PSS. In their review of the PSS literature, Hudzina, Rowley, and Wager (1996) list 38 articles; however most of these are project reports, and none report empirical results. For instance, Nortel Technology has developed and studied a pilot PSS (Bezanson, 1995). The manager for Performance Support Engineering at Nortel Technology told us that appropriate applications for performance support systems include skills- based operational situations (that is, situations in which there is a large volume of business or operational data, and for which learners are either new users or experienced, but infrequent, users). He suggested that PSS are not attractive for rapidly changing jobs or for jobs that need very analytical that PSS are not attractive for rapidly changing jobs or for jobs that need very analytical skills. In terms of pedagogic approach, it is difficult to see how this could be done without hands-on participation. PSS can be organized to support exploratory learning by providing links between tutorial segments such that the learner can explore from one content area to another. Both behavior modeling and cognitive modeling would be available within this context depending on the particular systems developed. In terms of delivery method, much like consideration of Internet, it is not clear if this approach is an alternative to the three traditional methods or is a packaging of the three through potentially creating direct links to experts, providing pre-recorded information, or providing multi- media style linkages. The most likely setting would be use by one individual at a time with computer functionality required for information delivery.

228

Research Question 2A: W?ll the various pedagogical approaches be utilized equally in Performance Support System training?

Research Question 2B: Wii the various pedagogical approaches be equally effective in terms of training outcomes (learning, retention, and transfer) in Performance Support System training?

Research Question 2C: What organizational variables will promote or retard the effectiveness of Performance Support System training?

Research Question 2D: What characteristics of individual learners will promote or retard the effectiveness of Performance Support System training?

43 Distance Videoconferencing Distance videoconferencing, or the use of information and communications technologies for the training process at multiple locations, can provide trainees with access to expertise otherwise unavailable (Webster & Hackley, 1997). Little research has addressed this emerging training method. However, organizations are beginning to explore its use. For example, United Healthcare Corporation and United Technologies are sharing the use of videoconferencing for training users (Hoffman, 1997). In many environments, distance videoconferencing is used to extend the reach of a particular instructor to learners who are geographically separated. It would seem that all pedagogical approaches discussed above would be potentially available for use in this context As with the other emerging training techniques, it is not clear if this procedure serves as an umbrella or substitute for the three identified distribution modes. Clearly, an instructor can communicate with learners in either one-directional or interactive modes (depending on the sophistication of the equipment available). The method can also accommodate pre-designed video/text materials. Combining multimedia and linlmges within the videoconference paradigm can be accomplished through the use of windows. In one scenario, the instructor presents material through one window on screen while the learner uses a second window for participating in an interactive exercise, referencing earlier work (such as text materials), or taking notes. The method can be used by an individual or group at a particular location, and, of course, can be simultaneously used at multiple locations. Research Question 3A: Wii the various pedagogical approaches be utilized equally in Videoconferencing training?

Research Question 3B: Wii the various pedagogical approaches be equally effective in terms of training outcomes 0=-@3, retention, and transfer) in Videoconferencing training?

Research Question 3C: What organizational variables will promote or retard the effectiveness of Videoconferencing training? Research Question 3D: What characteristics of individual learners will promote or retard the effectiveness of Videoconferencing training?

5. TRAINING METHODS IN THE ORGANIZATIONAL CONTEXT Our discussion so far has focused on individual training techniques without consideration of the organizational context in which they are used. Prior literature has focused on learner, software, task, and organizational characteristics. Each of these contingency factors influence decisions about training design, delivery, and effectiveness. Most research on the effects of these characteristics has focused on trainee characteristics; research has provided managers with little direction on the effects of the remaining characteristics on learning and training (Compeau et al., 1995).

5.1 Learner Characteristics Past research has examined such individual characteristics as computer anxiety, playfulness, self-efficacy, visual abilities, and learning styles, and has proposed that these characteristics could be used to place employees into appropriate training groups (e.g., Compeau & Higgins, 1995; Sein, Bostrom, Olfman, & Davis, 1993; Webster & Martocchio, 1992). Research Question 4: How will the individual characteristics of learner influence the frequency of use and the effectiveness of pedagogical approaches and delivery methods?

5.2 Software Characteristics Training research has not compared the training needs for different types of types of II-based tools and applications. However, the director of field technology training in a large organization told us that he sees very different training needs for what he calls “infrastructure technologies” (such as Wmdows97) and “line technologies” (such as a financial planning tool). Clearly this stands to reason. Infrastructure technologies generally will consist of packages from firms that are able to spread the cost of training materials and documentation across customers. Some will be large enough to generate cottage industries of training suppliers. In this context, the firm will often be selecting from among products offered by outside training vendors. In the case of emerging technologies such as Internet and Performance Support Systems, the remaining challenge may be integrating such outside developed materials into a consistent and organized overall training program.

Line technologies, by contrast, will largely be built in- house, by consultants, or customized from more general packages. The firm will be responsible for creating as well as distributing the training and documentation materials.

229

This will provide opportunities for developing features in anticipation of training needs. However, it does not allow for the sharing of cost. Limited time and financial resources may overwhelm opportunities for developing high quality training modules.

Research Question 5: How will differences in the software application (e.g. “line” versus “infrastructure”) the frequency of use and the effectiveness of pedagogical approaches and delivery methods?

5.3 Job/Task Characteristics Additionally, little research has focused on the comparative needs of users based on their job levels (such as executive versus blue-collar employees). Although researchers such as Vandenbosch and Higgins (1996) have examined executives’ learning of executive support systems and Martocchio and Webster (1992) have compared training methods for clerical employees on word processing programs, little research has focused on the unique needs of these different classes of employees. Research Question 6: How will differences in organizational level of the learner influence the frequency of use and the effectiveness of pedagogical approaches and delivery methods?

5.4 Organizational Characteristics Baldwin and Ford’s (1988) meta-analysis included such organizational characteristics as organizational climate and supervisor support We broaden this view by considering such standard variables as firm size and industry, and other variables such as dispersion of learners (centralized versus decentralized). Research Question 7: How will organizational characteristics influence the frequency of use and the effectiveness of pedagogical approaches and delivery methods?

6. ADDITIONAL END-USER TRAINING CONSIDERATIONS In contrast to the relatively well-explored relationships between standard organizational variables and training effectiveness, little prior research has focused on providing end-user computer training in the complexity of real-world settings. While some research has focused on the effectiveness of training methods, particularly at the end of training sessions (Baldwin and Ford, 1988), little has focused on the cost-effective ,delivery of training within a large organization or how particular methods fit into an overall portfolio. Given the rapidity of change in many organizations as they move through generations of computer platforms and business process reengineering programs, the training function is likely to be disbursed through many departments and to address many issues at the same time. Training a classroom full of 30-50 individuals is a very different

organizational issue from organizing training for a sales staff of 10,000 in a dozen countries upgrading from DOS to Windows operating systems. Consideration of the overall training program must address issues such as the scalability of methods and relative costs and benefits.

Consider, for example, the approach of training key users. Training key users involves using non-IT staff for the training and support of fellow users (Fitzgerald & Cater- Steel, 1995). For instance, the Director for End-User Training at a large credit-card company told us that he chose training of key users because of the dispersed nature of the firm’s sales staff He intensively trains 500 key users centrally who return to the field to train and provide on- going support for the remaining users. Fitzgerald and Cater-Steel (1995) found that training key users represented a low-cost strategy, but that it required increased management support. Future research should not only consider the context of training - e.g., the role of evaluation following training -- but also conceptualize an organization’s approach to training across multiple tasks, multiple sites, and large numbers of employees as the development of a “training portfolio”. Clearly, the cost effectiveness of an organization’s training program will include consideration of training methods and their effectiveness, but also must consider how well organizations capitalize on prior investments, coordinate a range of activities, and distinguish under what circumstances a particular method or approach is best applied. Research Question 8: What cost/benefit trade-offs will organizations encounter for each training method? Research Question 9: What, if any, relationship is there between scalability and frequency/effectiveness of us of each training method?

Research Question 10: To what extent will organizations gain from investment in underlying training technology where the supported methods can be used for many projects (e.g. across time and new software installations)? Research Question 11: To what extent will organizations displaying the most effective training be those that most effectively match particular training methods to the educational objectives?

7. SUGGESTED RESEARCH METHODOLOGIES These research questions, targeting both training methods and their use within an organizational portfolio or program, are viewed as multi-dimensional and multi-disciplinary. Further, the portfolio approach in this context has not been investigated in prior research. Thus, future research should utilize a primarily qualitative approach to emphasize the discovery of innovative training methods, as well as refinements or variations of training methods that increase their potential for successful implementation. In-depth

230

case studies will be required to extract the level of detail regarding de blending of training methods into a portfolio, about the evolution of such a portfolio, and about the rationales for particular choices made. Such studies would extend knowledge regarding end-user training methods in terms of breadth of methods discussed and depth of knowledge regarding many of the methods. Organizations should be selected to represent a wide range of industries. Several sources for recruitment are: ComputerWorld’s “Top 25 Corporate IS Training Programs” (Wilson, 1997a) and “Best Places to Work’ (Wilson, 1997b), the American Society for Training and Development (1997), and the Professional Association of Computer Trainers. The collection of cost/benefit data should be collected not only from the proposed case studies, but from secondary data, such as practitioner publications and the Internet. For instance, Desmarais, Leclair, Fiset, and Talbi (1997) and Bastiaens et al. (1997) present a cost-benefit analysis on electronic PSS, and other case studies can be found on the Internet through the EPSS library (1997). As another example, Geisman (1991) conducted a cost/benefit analysis of classroom-based instruction and computer-based tutorials in one organization. He claimed that computer- based tutorials resulted in 50-60s reduction in learning time, 40-50% increase in learner retention, and a 31% decrease in cost. He noted that computer-based tutorials seem to be most appropriate for companies with multiple locations, a large employee population, high attrition rates, and for programs that will be used for at least three years.

Considerable research has been conducted exploring how individuals learn in training environments. Research is be,@nning to emerge regarding new approaches to large- scale organizational training. That research can be informed by differentiating the effects of pedagogical approaches, delivery methods, and setting. Much less attention has been paid to end-user computer training within the larger organizational context. Little has been done in terms definiug the range of training methods available to organizations, the strengths and weaknesses of each method, the circumstances under which each training method is most readily applicable, and the employee population (e.g., executive, white collar, blue collar) for whom each method is most effective. Moreover, little attention has been paid to how an organization can develop an overall program or portfolio of training methods to take advantage of unique organizational properties and to build on prior investments. This paper makes a start towards addressing these gaps in the literature by presenting research questions and suggestions regarding research methods to guide future researchers-

9. EEFERENCES [l] Atkinson, M. and Kydd, C. “Individual Characteristics

Associated with World Wide Web use: An Empirical Study of Playfulness and Motivation,” DATABASE (in press).

[2] Baldwin, T-T. and Ford, J-K., “Transfer of Training: A Review and Directions for Future Research,” Personnel Psychology, (41), 1988, PP. 63-105.

[3] Bastiaens, T. J., Nijhof, W. J., Streumer, J. N., and Abma, H. J. “Working and Leeming With Electronic Performance Support Systems: An Effectiveness Study”, International Journal of Training and Development, (l:l), 1997, pp. 72-78.

[4] Bezanson, W. R. “Performance Support: Online Integrated Documentation and Training,” Proceedings of the ACM SIGDOC-95 Conference, Savannah, GA, October 1995, pp. l-10.

[5] Bostrom, R. P., Olfman, L., and Sein, M. K. “The Importance of Learning Style in End-User Training,” MIS Quarterly, (14:1), March 1990, pp. 101-119.

[6] Brancheau, J.C., Janz, B.D., and Wetherbe, J.C. “Key Issues in Information Systems Management: 1994- 1995 SIM Delphi Results,” MIS QuarterZy, (20:2), June 1996, pp. 225-242.

[7] Cheney, P.H., Mann, R.I., and Amoroso, D.L. “Organizational Factors Affecting the Success of End User Computing,” Journal of Management Information Systems, (3:1), Summer 1986, pp. 65-80.

[8] Compeau, D. R., and Higgins, C. A. “Application of Social Cognitive Theory to Training for Computer Skills,” Information Systems Research, (6), 1995, pp. 118-143.

[9] Compeau, D., O&an, L., Sein, M., and Webster, J. “End-User Training and Learning,” Communications of the ACM, (38:7), pp. 23-26.

[lO]Davis, S.A. and Bostrom, R.P. “Training End Users: An Experimental Investigation of the Roles of the Computer Interface and Training Methods,” MIS Quarterly, (17:1), March 1993, pp. 61-85.

[ll]Desmarais, M. C., Ieclair, R., Fiset, J.-Y., and Talbi, H. “Cost-justifying Electronic Performance Support Systems,” Communications of the ACM, (40:7), 1997,

’ pp. 39-48. [12]EPSS Library (1997). Case Studies.

http://www.epss.com.llbllb~index.htm (accessed July 30,1997).

[13]Fitzgerald, E. P. and Cater-Steel, A. “Champagne Training on a Beer Budget,” Communications of the ACM, (38:6), 1995, pp. 61-68.

[14] Gattiker, U.E. “Computer Skills Acquisition: A Review and Future Directions for Research,” Journal of Management, 18,3, 1992, pp. 547-574.

231

[15] Geisman, J. “Interactive Learning Technologies: A Valuable Addition to Training Curriculums,” Training and Development, (lo), 1991, pp- 17-23.

[16]Gist, M.E., Schwoerer, C., and Rosen, B. “Effects of Alternative Training Methods on Self-Efficacy and Performance in Computer Software Training,” Journal of AppZied Psychology, (74:6), 1989,8&l-891.

[ 171 Heidenrich, M. “Expert Input: Incorporating Internet Can Make In-House Training More Efficient,” Creative Training Techniques, (9:1), 1996.

[18]Hoffman, T. 1997. “Disparate Firms Share in Training,” Computerworld: http://www.computerworld.com/ (accessed August 1, 1997).

[19] Horowitz, A. S. 1997. “Net Train, Net Gain?” Computerworld: http:/lwww.computerworld.coml (accessed August 1,1997).

[20]Hudzina, M., Rowley, K. and Wager, W. “Electronic Performance Support Technology: Defining the Domain,” Perjbrmance Improvement Quarterly, (9: l), 1996, pp. 36-48.

[21]Katz, P. L. and Katz, C. J. 1995. “Choose the Right Training Strategy,” Datamation: http://~~vw.datamation.com/ (accessed in August 1997).

[22]Leidner D. E. and Jarvenpaa, S. L. “The Use of Information Technology to Enhance Management School Education: A Theoretical View,” MIS Quarterly, (19), 1995, pp- 265-291.

[23]Martocchio, J.J. and Webster, J. “The Effects of Feedback and Microcomputer Playfulness on Performance in Microcomputer Software Training,” Personnel PsychoZ~gy, (45:3), 1992, pp. 553-578.

[24]Nevins, M. “Human Resources: World Wide Web Opens Doors to Training Options,” The L&wood Report, (1:6), 1995.

[25]Niederman, F., Brancheau, J-C., and Wetherbe, J.C. “Information Systems Management Issues in the 1990’s,” MIS Quarterly, December 1991.

[26]Olfman, L. and Mandviwalla, “Conceptual Versus Procedural Software Training for Graphical User Interfaces: A Longitudinal Field Experiment,” MIS Quarterly, (1 S), 1994, pp. 405-426.

[27]Sein, M. K., Bostrom, R. P., Olfman, L., and Davis, S. A. “Visualization Ability as a Predictor of User Learning Success,” International Journal of Man-Machine Studies, (39), 1993, pp. 599-620.

[28]Simon, S. J., Grover, V., Teng, J. T. C., and Whitcomb, K. “The Relationship of Information System Training: Methods and Cognitive Ability to End-User Satisfaction, Comprehension, and Skill Transfer,” Information Systems Research, (7:4), 1996, pp. 466-490.

[29]Turnage, J. J. “The Challenge of New Workplace Technology for Psychology, ” American Psychologist, (45), 1990, pp. 171-17s.

[30]Vandenbosch, B. and Higgins, C. “Information Acquisition and Mental Models: An Investigation into the Relationship Between Behaviour and Learning. Information Systems Research, (7:2), 1996, pp. 198-214.

[3 l] Webster, J. and Hackley, P. “Teaching Effectiveness in Technology-Mediated Distance Teleleaming,” Academy of Management Journal, 1997, (in press).

[32] Webster, J. and Martocchio, J.J. “Microcomputer Playfulness: Development of a Measure with Workplace Implications,” MIS Quarterly, (16), 1992, pp. 201-226.

[33]Webster, J. and Martocchio, J. J. “The Differential Effects of Software Training previews on Training Outcomes,” Journal of Management, (21:4), 1995, pp. 757-787.

[34]Wexley, K-N. and Latham, G.P. Developing and Training Human Resources in Organizations, The Scott, Foresman Series in Organizational Behavior and Human Resources, Scott, Foresman and Company, Glenview, Illinois, 1981.

[35] Wilson, L. “Yeam to Learn,” Computer-world: http:l/www.computerworld.coml (accessed in August 1997).

232