Cisco Systems Inc.

eLEARNING: Internet Capabilities Research Project

BYU Research Team David Brinkerhoff Taylor Huckstep

Aaron Moyes Ayoub Sunna

Professors Dr. Douglas Dean

Dr. Markus Gappmaier

Cisco Contacts Michael Astle Patrick Tam

Table of Contents

1. Executive Summary .............................................................................. 3

2. Project Overview................................................................................... 4

2.1 Identify an Area of Research.............................................................. 5

2.2 Research Levels & Methods ............................................................... 7

3. Research Results.................................................................................... 8

3.1 Assessment Instruments...................................................................... 9

3.1.1 Capabilities Assessment....................................................................... 10

3.1.2 Value Assessment ................................................................................. 15

3.2 Corporate Research Results ............................................................. 18

3.2.1 Primary Results.................................................................................... 19

3.2.2 Secondary Results ................................................................................ 23

3.2.3 Tertiary Results.................................................................................... 27

4. End Summary...................................................................................... 31

Appendix...................................................................................................... 32

1. Executive Summary

Project eLearning is a major initiative among corporations today. This research project

revealed that many, companies have saved millions of dollars in costs and gained millions more in synergies through maximizing and exploiting the benefits of Internet enabled eLearning solutions.

Early in this project, based upon coaching from both Dr Douglas Dean and Markus Gappmaier, the project team was able to narrow the scope to eLearning, as opposed to some of the other areas and opportunities for researching various other internet enabled solutions. Research was conducted by interview, use of research data-bases, and through general Internet searches.

The project team created two documents with which to assess the companies to be researched. The Capabilities Assessment was scaled down from a tool already used by Cisco consultants to evaluate the state of their eLearning clients. The Value Assessment was created by the research team with input from Patrick Tam and is a tool for assessing the benefits obtained by companies using eLearning solutions.

Ten companies we contacted to participate in the primary research and many hours were spent doing secondary research.

The resulting research is broken into three areas based on completeness. Level 1 research was obtained for Novell. Level 2 research was obtained for IBM, Lucent, OneBeacon, and PSS/World Medical. Level 3 research was obtained for 25 other companies.

Findings

The Defining factor for evaluating the quality of a complete eLearning solution is the implementation of a Learning Management System. Companies implementing an LMS were better able to measure and take advantage of the benefits of their eLearning solutions.

Companies in rapidly evolving industries tended to realize proportionally greater benefits than companies in more stable industries.

The soft benefits from leading eLearning solutions often out way the hard benefits. Though these benefits may be more difficult to measure, they should not be over looked when considering an investment in eLearning.

2. Project Overview

During the first few meetings we had with Michael to discuss the project, he told us the purpose of this project was to determine the benefits that “leading” companies are gaining from their capabilities in a specific area. Cisco has created assessments to determine whether a company is “leading” or not in specific areas, such as Human Resources, E-learning, or E-learning etc. These assessments ask companies questions about their capabilities in each specific area.

Our group sat down and discussed these areas and determined that we should focus our efforts on e-learning.

Once we selected e-learning, we needed to revise Cisco’s e-learning assessment. We felt that Cisco’s e-learning assessment was too long for our purpose. This assessment needed to be shorter because it was going to be something we would give to managers and executives at various companies we had selected earlier. In general, managers and executives are very time-sensitive and we felt they wouldn’t have the time to fill out a lengthy questionnaire. So we created a scaled-down version of the e-learning assessment. We refer to this assessment as the capabilities assessment.

We also needed a way to measure the value each of the identified companies is receiving from their e-learning capabilities. So we created a value assessment to determine these benefits.

Once we created these assessments, we started to conduct our research. The research we conducted falls into two categories: primary and secondary. Primary research is information that we were able to gain from the actual company. Secondary research is information we gathered online about a company’s e-learning capabilities. Primary research is of more value than secondary research but Michael realized that gathering this information from each of the identified companies would be difficult if not impossible to do. Realistically, he felt that we would need to gather most of our information from secondary research. This secondary research consisted of using the internet to find articles and case studies about each of the identified companies’ e-learning systems. This research, both primary and secondary, comprised the majority of our project.

2.1 Identify an Area of Research

When we first met with Michael Astle we were informed that the project would focus on a solution area within a particular industry. We were given the task to determine what our strengths and interests were in these areas and choose one from each category that we would focus our research efforts on.

The Solution Areas: • Workforce Optimization – Human Resources, E-Learning, IT Excellence • Supply Chain – Order Fulfillment, Product Service, Product Lifecycle

Development • Customer Care – Commerce, Service and Support, Sales Management, Channel

Management, Marketing The Industries:

• Consumer Packaged Goods/Retail • Public Sector – Government Agencies, Schools • Manufacturing

Solution Area Selected (E-Learning) One of our top choices was always E-Learning because of the interest level, our

experience as IT Students, and our Coaches knowledge. Michael agreed that this was a hot area right now and felt it would be of benefit to Cisco for some research to be done in this area. We thus decided to focus our efforts on E-Learning pending a final approval from the other participants at Cisco.

Industry Selected (High Tech Manufacturing) After deciding on E-Learning, we began discussions about which industry would

be the most beneficial. We discussed the necessity of high tech industries to stay on the leading edge of E-Learning. Cisco for example has thousands of employees scattered across the world and they are constantly manufacturing new high tech products that their sales force, technical support staff, and engineers need to be familiar with in order to do their job effectively. It is not always practical or cost effective for all these people to come to San Jose for training, so E-Learning becomes a necessity. We decided to focus generally on high tech and Audio/Video companies in the Manufacturing Industry.

Solution Area Decision-Making Process In considering Workforce Optimization, we were told that Human Resource was a

hot area because it was one that people could easily relate to, and easily identify processes and areas for improvement. This also meant that it was a naturally an area with

a lot of research already done. We finally decided upon E-Learning, and our reasoning for doing so will be discussed later.

IT Excellence was our initial choice because it had the highest interest for us. Michael mentioned that a constant battle for Cisco is to change the view that IT is a cost center, rather than a revenue generating benefit to the organization. He said that Cisco’s approach was to split out IT under departments (accounting, sales, HR, etc) so they are viewed as a strategic benefit to that department. Our biggest reason for not going with this Solution Area was because of a general lack of knowledge among us, and because we thought it would be more difficult to find information about this area through secondary research.

Supply Chain was an area Michael identified early on as having some holes that could use additional research. We decided against this one because of a lack of contacts within the industry and a general lack of experience in this area.

Customer Care was an area that we had a lot more experience in. Commerce, and especially E-Commerce was one we identified as a possible choice. All of us have had Service and Support experience and could work with either Sales Management or Marketing fairly easily…Channel Management a little less so. When we discussed this as a possible Solution Area for us to focus on, Michael Pointed out that most of Customer Care, and especially Commerce, was pretty saturated with information. Also with the state of affairs in the economy it might not be the best choice.

Industry Decision-Making Process

Despite our choice to focus on High Tech Manufacturing, we didn’t want to be limited to one particular industry. We felt that there would be value in creating a comparison across multiple industries. If we compared highly evolving markets with other industries, we hypothesized that we would see a greater Return on Investment among the highly evolving markets. We thus got approval to conduct additional research for E-Learning information on companies across all industries. It should be noted though, that this was not our primary focus, the other industries would be gathered as we ran across them in research on High Tech Manufacturing firms.

2.2 Research Levels & Methods

Research Tools In preparation to conduct primary research, the research team created a standard by which to measure both the capabilities and the recognized value of companies’ eLearning solutions. The research was broken down into three levels and the Capabilities and Value Assessments were used to evaluate the results.

First Level The first level of research was primary research obtained directly through the company. This was done through one of two ways. First, in the case of Novell, we were able to obtain an onsite interview and correspond with our contacts there through email. Second, we attempted to have companies complete the assessments on their own and email us the results. Cisco provided us with a list of companies who they felt might be leaders in eLearning. Using alumni contacts within these companies and other organizations, we attempted to obtain level one research.

Second Level Through the aid of useful third party research, our group was able to obtain a majority of the information necessary to complete the assessment we had created, as well as gain a good understanding of the eLearning systems which they employed. We were able to obtain good second level research on both IBM and Lucent.

Third Level The third level of research consisted of both general internet research and article

database research. We were able to gather sufficient information to see and compare savings and ROI among several other companies. In spite of the fact that we were able to gather substantial information on these companies, it was insufficient to fill out the assessments we had created.

3. Research Results

3.1 Assessment Instruments

3.1.1 Capabilities Assessment

At the beginning of this project, Michael showed us the assessments that Cisco had created for the various areas, such as e-learning, HR, and sales. Cisco has clients fill these assessments out to determine what these client’s capabilities are in that area. These assessments are on average 15-20 pages long, each page consisting of a number of questions. We felt the executives and managers that would need to fill these assessments out would not have the time to do so because of the length of the assessment. (We were asking them to fill these out, they weren’t hiring us to assess their e-learning capabilities---which made the time issue critical.) We needed to come up with a shorter assessment, while still trying to assess whether a company was leading or not. We sat down and revised the original Cisco assessment.

The assessment we created is based on the original Cisco assessment (see Appendix), but is much shorter. The original one gave definitions of various process groups and then asked questions about various processes within each process group. We took these same process group definitions and then asked two questions about each process group: 1. Is the process group internet-enabled? and 2. Is the process group integrated with similar processes? We felt that by asking these two questions about each process group this would give us a fairly good idea as to whether a company was leading or lagging on their e-learning internet capabilities. Thus we drastically reduced the size of the assessment by asking questions about process groups instead of individual processes and we only asked two questions about each process group instead of the original four. The assessment we created is 4 pages long and contains 16 questions. You will find this assessment on the following pages.

E-Learning The following assessment consists of a total of 16 questions. These questions are designed to ascertain whether your e-learning processes are leading or lagging. Following are a list of process groups. We define a process group as: A set of processes that have a natural flow among them and that can be integrated using Internet-based applications. We would like to know whether your process groups are internet enabled and if they are integrated. We define internet enabled as using Internet-based applications to perform processes in a particular process group. We define integrated as Internet-enabled processes that share a common data model and enable the automatic exchange of data between them in real time. We realize that the answers to these questions aren’t necessarily yes or no. For example, some processes in the group may be internet enabled and some might not be. The same with integration, some processes may be integrated with other processes while others may not be. To accommodate this, please rate your process groups on the following scales: Internet Enabled 1. The process group is not internet enabled at all and there are no plans to do so. 2. The process group is minimally internet enabled. 3. 50% of the process group is internet enabled. 4. 75% of the process group is internet enabled. 5. The entire process group is internet enabled. Integrated 1. The processes within the process group are not integrated and there are no plans to do so. 2. The processes in the process group are minimally integrated with each other. 3. 50% of the process group is integrated. 4. 75% of the process group is integrated. 5. The entire process group is integrated.

Managing Competencies Description of a leading Managing Competencies process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for assessing needs by using web-enabled systems and utilizing a common application platform across the enterprise. A training and development department as well as managers can assess and identify their organizations competency gaps through an online application, while employees can assess themselves, see their gaps, and receive the content they need to fill their competency gaps. Please indicate the state of your Managing Competencies functionalities: 1. Is this process group internet enabled? 2. Are the processes in this process group integrated? Procuring Off-the-Shelf Content Description of a leading Procuring Off-the-Shelf Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for selecting and procuring off the- shelf content by using web-enabled systems and utilizing a common approach across the enterprise. A training and development department can search and purchase content on-line and integrate it into its systems. Please indicate the state of your Procuring Off-the-Shelf Content process group: 3. Is this process group internet enabled? 4. Are the processes in this process group integrated? Create Formal Content Description of a leading Creating Formal Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for creating formal, instructionally designed content by using web-enabled systems and utilizing a common application platform across the enterprise. A training and development department can design curriculum, author content, and create tests and assessments using integrated on-line applications. Please indicate the state of your Creating Formal Content process group: 5. Is this process group internet enabled? 6. Are the processes in this process group integrated?

Create Informal Content Description of a leading Creating Informal Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for creating informal, subject matter expert authored content by using web-enabled systems and utilizing a common application platform across the enterprise. Any subject matter expert (SME) can design and author content as well as attach tests to the content using integrated on-line applications. Please indicate the state of your Creating Informal Content process group: 7. Is this process group internet enabled? 8. Are the processes in this process group integrated? Managing Content Description of a leading Managing Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for managing content by using web-enabled systems and utilizing a common application platform across the enterprise. Content creators, authoring either formal or informal content, can catalog both off-the-shelf and custom created content as well as search for and re-use other content through integrated applications. Please indicate the state of your Managing Content process group: 9. Is this process group internet enabled? 10. Are the processes in this process group integrated? Delivering Synchronous Content Description of a leading Delivering Synchronous Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for delivering content synchronously by using web-enabled systems and utilizing a common application platform across the enterprise. A training and development department can convey either formal or informal material through the use of integrated web-based applications. Learners can access live sessions and interact with a class or instructor through these on-line integrated applications. Please indicate the state of your Delivering Synchronous Content process group: 11. Is this process group internet enabled? 12. Are the processes in this process group integrated?

Delivering Asynchronous Content Description of a leading Delivering Asynchronous Content process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for delivering content asynchronously by using web-enabled systems and utilizing a common application platform across the enterprise. A training and development department can make content available to its intended audience 24 hours per day, 7 days per week. Learners can access content at their convenience, and they have the option to take only relevant content , either formal or informal. Please indicate the state of your Delivering Asynchronous Content process group: 13. Is this process group internet enabled? 14. Are the processes in this process group integrated? Manage Learning Description of a leading Managing Learning process group—processes within the group are fully Internet enabled and integrated: Full integration of the processes for managing learning, for both the training and development department and the learner, by using web-enabled systems and utilizing a common application platform across the enterprise. A training and development department can catalog courses, perform cross-charges, generate reports, and track results through integrated applications. Learners can seamlessly search, register and launch courses through on-line integrated applications, while managers can gain visibility to organizational learning activities. Please indicate the state of your Managing Learning process group: 15. Is this process group internet enabled? 16. Are the processes in this process group integrated?

3.1.2 Value Assessment

We knew that one of the most important parts of this project was to ascertain the value being derived from leading e-learning companies. As with the leading/lagging assessment we knew that it would be difficult for us to require hours of analysis from those we would interview. So, to solve this problem we created a list of questions that we felt would be beneficial. We remained in close contact with Michael and Patrick throughout the creation of this document and refined the questionnaire with their help. We feel that the questioned we came up with provide valuable feedback that allows us to analyze what kinds of benefits are being derived from E-Learning Internet Capabilities.

We’ve included the original rough draft of questions as well as the final draft in both printed form and on the included CD in Word and PDF formats.

Value Assessment What is your annual training budget? (Exact numbers if available) $_______________________ The learning mediums include "Instructor Led Training" (ILT), “Web-based Training” (WBT), and “Virtual Classroom” (VC). The characteristics should consider learning/training time, cost (facility, admin, development, delivery, travel & expenses, etc), and measurements of training effectiveness. ILT – Instructor Led Training How much time is spent to develop an ILT course? ________________________ days How long does an average training session last? ______________________ minutes How much is spent on ILT? $___________________________ How much does it cost to develop an ILT course? $___________________________ How much does it cost to deliver an ILT course per person? $___________________________ How much does it cost for travel & related expenses for Instructors? $___________________________ How much does it cost for travel & related expenses for Attendees? $___________________________ Average Cost $___________________________ WBT – Web-based Training How much time is spent to develop a WBT course? ________________________ days How long does an average training session last? ______________________ minutes How much is spent on WBT? $___________________________ How much does it cost to develop a WBT course? $___________________________ How much does it cost to host/support/maintain a VC course? $___________________________ How much does it cost to deliver a WBT course per person? $___________________________ How much does it cost for travel & related expenses for WBT? $___________________________ Rank the effectiveness of WBT compared to ILT? $___________________________ Average Cost $___________________________ VC – Virtual Classroom How much time is spent to develop a VC course? ________________________ days How long does an average training session last? ______________________ minutes How much is spent on VC? $___________________________ How much does it cost to develop a VC course? $___________________________ How much does it cost to host/support/maintain a VC course? $___________________________ How much does it cost to deliver a VC course per person? $___________________________ How much does it cost for travel & related expenses for VC? $___________________________ Rank the effectiveness of VC compared to ILT? $___________________________ Average Cost $___________________________ Have you estimated the savings from E-Learning at your company? Yes No What has been the projected savings of E-Learning? (Monetary and/or Other) ________________________________________________________________________ If E-Learning has been implemented, what have been the achieved benefits of E-Learning?

How do you currently administer your training? Instructor Led Training ____% Traditional instructor-led classrooms ____% of ILT On-the-job training ____% of ILT Handbooks, manuals and self-study Video/Audio tapes/CD’s ____% of ILT Facilitated group activities, seminars ____% of ILT Virtual Classroom ____% Instructor-led broadcast ____% of VC Simulations (ex. Stand-alone flight sims) ____% of VC Computer based training (i.e., CD-Rom) ____% of VC Web-Based Training ____% Instructor-led recordings ____% of WBT Online Lessons ____% of WBT

Other _________________________ ____%

3.2 Corporate Research Results

3.2.1 Primary Results

Novell

Overview Novell, located in Provo Utah, provides software for network solutions. In order

to understand the company’s e-learning capabilities we interviewed an IT manager and a Human resource e-learning coordinator.

Capability Rating After the meeting we filled out the capabilities assessment based on the interview

we had with them. We also had Novell fill out the value assessment. Our conclusions illustrate that based on a scale of internet capabilities ranging from 1-4, 1 being lagging and 4 being leading, Novell is a 2.5. The main deficiency we noticed in Novell’s system is the lack of a LMS (learning management system), which seems to be a defining factor of leading companies. The company, however, is generally improving its e-learning capabilities. Management understands the importance and potential e-learning has for the future. But right now, their separate capabilities and resources are decentralized, which means that each department or division conducts its training separately.

Observations We find it particularly interesting that currently, this high-tech institution is

unable to fully integrate its training programs to effectively and efficiently utilize the benefits of e-learning. They are in the process of establishing a new portal, which will be implemented soon, and this portal is going to raise the level of integration and will connect the separate programs together.

eLearning Solutions Novell’s e-learning software is divided into two groups: Home-grown software

and software provided by outside vendors. Novell uses Skillsoft from Skillscape, which is a skill competency program. Another program the company has in place is Peoplesoft, which is used to provide information to Novell managers. One of the system’s main strengths is an employee manager portal that has a large quantity of information within its well integrated network. Novell also implements some of the training through GroupWise task and through virtual classrooms from Centra and Traceware.

Most of Novell’s training is instructor led training, primarily in a traditional classroom setting. Management rarely brings in third parties to do retail training on their own products. Travel expenses are currently a huge expense for the organization, in fact it is one of the reasons the company has decided to heavily invest in e-learning lately.

Knowing that less than third of the employees are in Provo, Novell understands the potential for savings in travel expenses. These expenses add up to substantial amounts due to the nature of the learning sessions the company conducts. The sessions are divided into groups of 2 hours, 3 days or even 5 days boot camps. In an attempt to educate its workforce Novell contracted with 24/7 books to provide courses and content. This provides employees with on-line access to a research library available to all.

Planned Improvements When Novell first implemented e-learning courses, course completions were low-

--less than 100 in the first year. Right now, the number of course completions is 1000 per year. The level of awareness of the program is currently at 40% and the goal is to reach 80%. This goal seems to be attainable once the new portal is installed. The level of usage (utilization) is currently at 10% and the goal is to reach 50% this year.

As mentioned above the main weakness of Novell’s e-learning systems is integration. This lack of integration results in incurring common costs among departments. If Novell’s e-learning systems were integrated, these costs could be reduced. Furthermore, Novell cannot currently assess the overall benefits of e-learning to the organization. Management just believes it is a necessity for a technology company, to show their commitment to new technology and employee progression and skill development.

Capabilities Assessment Novell seems to be operating efficiently in terms of having internet enabled

capabilities but their capabilities are far from being fully integrated.

Field Internet enabled Integrated Notes

Managing competencies

Semi No Weakness in identifying gaps and recommending content.

Procuring off-the –shelf content

Semi No Key stone, no highly interactive OTS content.

Creating formal content

Yes No Lacks integration, will improve with portal.

Creating informal content

Semi No Low level of emphasis.

Delivering asynchronous content

Yes No Allows administration of testing.

Managing learning Yes Semi with portal Blends in media.

Managing content Yes Yes Industry compliant?

Delivering synchronous content

Yes No No LMS.

Value Assessment Novell’s annual training budget is set at $13,000,000 for all three fields of

learning. The general belief is that all three fields are equally effective assuming similar quality of instruction. However, web based training and virtual classrooms provide significant cost savings when compared to instructor led training. In comparing these three methods, characteristics that should be considered include learning/training time, cost (facility, administration, development, delivery, travel & expenses, etc…), and measurements of training effectiveness.

Field ILT WBT VC

Time to develop

3-5 days for each course day

1-2 days

Avg. training session

1hr to 5 days 2 hrs. 1 HR.

How much is spent?

8,000,000 N/A 30 cents/minute/user

How much it costs to develop course?

250 day/ person

1500 estimate 1500 estimate

How much it costs to deliver a course?

20 day per person

N/A 30 cents/min./user

Travel expenses

500 a day per employee

N/A N/A

Special areas

Traditional classroom

Instructor led recordings

Instructor led broadcast

Usage among departments

50% 30-40% 10-20%

3.2.2 Secondary Results

IBM

Overview IBM is a provider of technology based business solutions. In order to understand

the company’s e-learning capabilities we conducted secondary research online. Research was focused primarily on IBM’s Basic Blue training program for new managers. This is a blended approach to e-learning.

After conducting research on this program, we felt that based on a scale of internet capabilities ranging from 1-4, 1 being lagging and 4 being leading, IBM is a 4. This is because IBM’s online e-learning capabilities have integrated business processes within business process groups.

eLearning Solutions Basic Blue is a 7-8 month training program done primarily online. Only one week

of this program is spent in face to face interaction.

Basic Blue uses Lotus Learning Space 5 to conduct the majority of the training in this program. Lotus Learning Space is a great example of how IBM has integrated business processes. IBM has combined many capabilities/processes into this tool. Some of these capabilities are: video, email, chat, simulators, testing capabilities and course suggestions based on past performance.

Here are some of the highlights of the Basic Blue program:

• Used to train 6400 Managers per year

• 75% of training done through distance training

• Reusable templates reduce costs of content development

• Increased manager productivity. Managers utilize their time better.

• Time to learn new material reduced by 25% (estimate).

• Blended approach leads to sustained behavior changes which leads to significant business performance improvements.

Benefits To analyze some of the benefits IBM is gaining from the Basic Blue program, we

located a case study conducted by Nucleus Research, a well respected, independent research company. This case study revealed the following benefits of the Basic Blue program:

• Direct savings such as reduced program, travel, and manager off-site costs. Because 75 percent of the training is delivered through distance learning and 25 percent is in a traditional classroom, IBM has been able to significantly reduce the costs associated with training while increasing the amount of content taught.

• Reduced the direct cost of content development. In addition to the savings with the Basic Blue project, reusable templates has enabled other IBM divisions to develop content without re-development of the training platform.

• Indirect savings in the form of increased manager productivity. The self-service nature of Basic Blue allows managers to better utilize their time. IBM also estimates managers are able to reduce the time needed to learn new material by 25%.

• IBM also tracks a number of additional benefits from Basic Blue. Measurement of the impact of the enhanced training on managers’ performance has shown that IBM’s blended approach to initial and ongoing training has enabled its managers to make sustained behavior changes which then lead to significant business performance improvements.

3-year Financial Analysis

http://www.nucleusresearch.com/research/b19.pdf

While the Basic Blue program is employed worldwide, it is only used for management training. IBM uses other programs and tools to reach its huge base of employees.

Web Lecture Services is another product developed by IBM that does what its name implies. Some of the highlights of this program include the following:

• Grew out of IBM’s owns successful experience in delivering e-business training to the world wide sales force without resorting to road shows or traditional meetings.

• 25,000 Sales employees reached in 60 days, saving over $20 million

• Overall: 800 lectures created, over 34,000 IBM and Lotus employees have benefited. It’s our number one contributor to e-Learning savings.

We feel that these two examples are indicative of the overall commitment IBM has made to e-learning.

Total Benefits $88,185,000

Total Costs $5,029,712

ROI 1700%

Payback 2 weeks

TCO $1,676,571

A Note on Lucent ROI Numbers

It should be noted that the ROI figures computed in the Nucleus Research Documentation for the Lucent Wireless University were not calculated entirely consistent with the IBM and OneBeacon Insurance figures.. Below is a table that corrects the Lucent ROI numbers. The Lucent ROI numbers were based on the initial outlay. An adjusted ROI is a better indication of the real benefits, and are consistent with the other results. These real benefits are generated based on costs accumulated in years 1-3 and not just the initial costs. The 155,000 will be covered fast but the additional costs in year 1 will delay the recovery of the investments. Year ROI Benefits Costs Net

Benefits Adjusted ROI

Payback

96 155,000 97 809% 2.4 M 1.2 M 1.2 M 98 1542% 4.8 M 1.3 M 3.5 M 99 2302% 7.2 M 1.3 M 5.9 M Net 14.4 M 3.9 M 10.6 M 369% 8 months Year Benefits Expenses ROI Adjusted ROI 99-02 8.4 M 3.9 M 1556% 215%

3.2.3 Tertiary Results

Summary of Selected Second &Third Level Researches

Companies: ROI PaybackPeriod

Travel Savings

Total Training Savings

Training Compression

OneBeacon 199% 6mo 6.8M

PSS/World Med 110% 1.2yrs 3.6M

Wachovia 8mo $160,000

Wyndham 329% 5mo $2M 40%

Sony 100% 12mo 6 Days 66%

Safety-Kleen $200,000 $212,000 25%

First Consulting Group

215% 12mo $700,000

Liberty Group $2000 per training / 80%

50%

*For additional research documents, see the Appendix.

Summaries of Selected Third Level Research

Wachovia Wachovia has already realized big savings by rolling out its program to 1% of its workforce. The company expects exponential returns as they introduce it to more employees. With 900 hours of training now underway, Wachovia has seen these benefits:

• More than $160,000 saved in travel expenses

• Realized a return on investment within eight months of implementation

In addition to these benefits, the company expects huge future savings when the program is rolled out to thousands of employees. And there are other substantial improvements too, including:

• Less time away from the office for the learning specialists and student-participants

• Lowered delivery costs and improved reach to Wachovia’s many diverse locations

• Equal or improved retention of the material

Wyndham Cost Savings: 329% ROI in 1st Year

Results were calculated by including only: employee and instructor travel expenses, facilities costs and ongoing support outlays. The investment paid for itself in 5 months with an ROI of 329%.

Strategic Benefits

In addition to costs savings, Wyndham has identified potential strategic benefits from the Implementation of eLearning solutions. Some key benefits are:

• 2001 savings will top $2M – by using pre-recorded Centra sessions to more effectively reach its large, geographically dispersed employee population, without increasing the training group staff or incurring travel expenses.

• Faster dissemination of information – by using Centra, Wyndham can provide updates to hotel staff for critical guest programs in days, not the weeks required for coordinating travel and meeting planning schedules, improving customer satisfaction.

• Compression of training time – as much as 40% less training time is needed for learners attending Centra sessions, when compared to classroom training.

Liberty Group Some of the benefits that have already emerged for Liberty Group are:

• Faster time to market – Trainers are trained sooner, thus training the sales force sooner, giving a competitive advantage to Liberty.

• Increased productivity and revenues – New agents participating in Centra training have shown a 200% increase in revenue over agents trained in a traditional classroom.

• More frequent collaboration – Senior managers are able to hold eMeetings to address operational issues, saving face-to-face meeting time for strategic matters.

• Decreased travel expenses – Each course delivered with Centra saves over $2,000 in travel expenses.

• Compression of training time – As much as 50% less training time is needed for learners attending Centra sessions, when compared to classroom training.

Sony Cost Savings: Training Costs Reduced by Two-Thirds.

Within one year of deploying Centra, Sony Electronics’ Service Company has converted 100% of its training to virtual classroom delivery. This has resulted in the following benefits:

• Eliminated training-related travel for trainers and technicians. Without the need for travel, billable time for technicians is increased by as much as 6 days a year.

• Delivered more effective training with the ability to show visual content and share ideas online in real-time.

• Delivered product updates faster and more effectively than ever before to the authorized network.

• Increased scheduling flexibility allows training to be delivered closer to new product launches.

Safety-Kleen Safety-Kleen will realize over $200,000 of savings in travel-related expenses and lost wages on the projects currently in the process of migrating to Centra. Highlights of cost savings and strategic benefits which will be enjoyed by Safety-Kleen are the ability to:

Leverage the small training group to reach more employees while reducing travel costs, resulting in:

• Avoiding the cost of delivering a sexual harassment awareness course via traditional classroom training, saving $92,000 in expected travel costs

• Providing a platform to deliver training in a blended learning format, which will result in savings of $43,000 for one course by decreasing the classroom time by one day

• Compressing software application rollout time from four months to six weeks, saving approximately $65,000 in travel expenses and lost wages.

Reduce the costs of conference calls - an estimated $12,000 a year for just six recurring calls involving a very small group of attendees.

Provide a platform for subject matter experts in field offices to share knowledge.

First Consulting Group Cost Savings: 215% ROI in 1st Year

The primary reason for investing in eLearning was to eliminate travel for its learners and instructors. The investment resulted in a breakeven of less than a year and an ROI of 215%.

Strategic Benefits

In addition to cost savings, FCG has identified the following potential strategic benefits from use of Centra:

• Increased billings – as much as $5.6M in incremental revenue by giving Associates more time at the client site

• Reduction in associate turnover – a 1% decrease in turnover will save $500K/year in recruitment expenses alone, while improving client satisfaction

• Service quality and client satisfaction will also be increased with rapid knowledge sharing across the firm

4. End Summary

This research attempted to examine the benefits of e learning to the companies that invested in higher Internet capabilities.

Our findings suggest that the benefits of e learning vary according to capabilities and especially the industry the company operates within. IBM for example is an industry that experiences a lot of change. The computer industry involves coming out with new products and solutions regularly. This nature of the computer industry necessitates continuous training. Cost savings from continuous training is the strength of e learning. By allowing constant updates and cheap modifications, computer and high- tech companies can save a lot on travel expenses and instructor fees. A key element of leading eLearning enabled companies is the implementation of a Learning Management System. Companies that have implemented an LMS as part of their solution have been better able to measure and exploit the benefits of eLearning than companies which have not implemented an LMS.

On the other hand, other industries might not benefit form eLearning as much as technology companies. For example, some industries are characterized by practical practice, which limit the opportunities for training over the web. Furthermore, the workforce might be concentrated in one area; this will reduce and maybe eliminate travel expenses for some organizations.

In our research we examined eLearning savings in different industries and we noticed specific trends. The technology companies primarily Lucent and IBM experienced much higher Returns on investment than smaller companies in different industries. For example, One Beacon the insurance company did indeed benefit form eLearning but the ROI was much smaller than that of the companies mentioned above.

Our research also unraveled a very important characteristic of eLearning. Research on this field tries to point out financial benefits of e-learning as if it was the sole measure of its success. Soft savings are sometimes more important than ROI figures. Companies need not depend on financial measures and ignore savings in productivity and employee retention as examples of soft measures. eLearning provides benefits in a variety of areas, and companies should not overlook the value of soft benefits when considering whether or not to invest in leading eLearning solutions.

We believe that e learning represents a promising field. It is a field in which more companies will invest in more heavily when its benefits are proven for a variety of industries all over the world. The globalization phenomenon we are living right now is probably the strongest advocate of investing in higher Internet capabilities. As the workforces grow to become more dispersed the importance of training on the web will grow and eventually become vital for successful global companies.

Appendix Company Research Documentation IBM......................................................................................................................................1 Lucent Wireless university (1999-2002) .............................................................................6 Lucent Wireless university (1996-1999) ...........................................................................10 OneBeacon Insurance Group .............................................................................................14 PSS/World Medical, Inc. (Nucleus™)...............................................................................18 PSS/World Medical, Inc. (Docent™) ................................................................................22 Wachovia ...........................................................................................................................25 Wyndham Hotels ...............................................................................................................29 Liberty Group.....................................................................................................................33 Sony (LexisNexis™) .........................................................................................................38 Sony (Centra™) .................................................................................................................40 Safety-Kleen ......................................................................................................................44 First Consulting Group (LexisNexis™).............................................................................49 First Consulting Group (Centra™) ....................................................................................51 British Telecommunications ..............................................................................................56 Interwoven .........................................................................................................................61 Accenture ...........................................................................................................................63 B&Q...................................................................................................................................66 Cisco ..................................................................................................................................68 Credit Union National Association....................................................................................70 DoubleClick .......................................................................................................................72 Halliburton .........................................................................................................................74 Harvard Business OnLine ..................................................................................................76 Indus International .............................................................................................................78 Insead .................................................................................................................................80 Kelly Scientific Resources.................................................................................................82 NAM Virtual University ....................................................................................................84 NTL....................................................................................................................................86 Shell Exploration & Production.........................................................................................88 University of Phoenix ........................................................................................................90 U.S. Cellular.......................................................................................................................92 VitesseLearning .................................................................................................................94 Vitro ...................................................................................................................................96 Other Calculating Return On Your eLearning Investment ..........................................................98 Original Capabilities Assessment ....................................................................................103 Rough Draft Value Assessment (Questionnaire).............................................................108