What Have We Learned From the Research on Online

Learning?

Dr. Curtis J. Bonk Professor, Indiana University

President, CourseShare and SurveySharehttp://php.indiana.edu/~cjbonk,

cjbonk@indiana.edu

Tons of Recent Research

Not much of it

...is any good...

Basic Distance Learning Finding?

• Research since 1928 shows that DL students perform as well as their counterparts in a traditional classroom setting.

Per: Russell, 1999, The No Significant Difference Phenomenon (5th Edition), NCSU, based on 355 research reports.

http://cuda.teleeducation.nb.ca/nosignificantdifference/

Online Learning Research Problems (National Center for Education Statistics,

1999; Phipps & Merisotos, 1999; Wisher et al., 1999).

Anecdotal evidence; minimal theory. Questionable validity of tests. Lack of control group. Hard to compare given different

assessment tools and domains. Fails to explain why the drop-out

rates of distance learners are higher.

Does not relate learning styles to different technologies or focus on interaction of multiple technologies.

Online Learning Research Problems

(Bonk & Wisher, 2001)

• For different purposes or domains: in our study, 13% concern training, 87% education

• Flaws in research designs- Only 36% have objective learning

measures- Only 45% have comparison groups

• When effective, it is difficult to know why- Course design?- Instructional methods?- Technology?

Evaluating Web-Based Instruction:Methods and Findings (41 studies)

(Olson & Wisher, October, 2002; International Review of Research in Open and Distance Learning)

Year of Publication

02468

1012

1996 1997 1998 1999 2000 2001

Year

Nu

mb

er

of

Stu

die

s

http://www.irrodl.org/content/v3.2/olsen.html

Wisher’s Wish List Effect size of .5 or higher in

comparison to traditional classroom instruction.

Web Based Instruction

CBIKulik [8]

CBILiao [18]

Average Effect Size

.31 .32 .41

Number of Studies

11 97 46

Evaluating Web-Based Instruction: Methods and

Findings(Olson & Wisher, 2002)

“…there is little consensus as to what variables should be examined and what measures of of learning are most appropriate, making comparisons between studies difficult and inconclusive.”

e.g., demographics (age, gender), previous experience, course design, instructor effectiveness, technical issues, levels of participation and collaboration, recommendation of course, desire to take add’l online courses.

Evaluating Web-Based Instruction: Methods and

Findings(Olson & Wisher, 2002)

Variables Studied:1. Type of Course: Graduate (18%) vs.

undergraduate courses (81%)2. Level of Web Use: All-online (64%) vs.

blended/mixed courses (34%)3. Content area (e.g., math/engineering

(27%), science/medicine (24%), distance ed (15%), social science/educ (12%), business (10%), etc.)

4. Attrition data (34%)5. Comparison Group (59%)

Some of the Research Gaps

(Bonk & Wisher, 2000)

1) Variations in Instructor Moderation2) Online Debating3) Student Perceptions of e-Learning Envir.4) Devel of Online Learning Communities5) Time Allocation: Instructor and Student6) Critical Thinking and Problem Solving

Applications in Sync/Asynchronous Envir7) Peer Tutoring and Online Mentoring: 8) Student Retention: E-learning and

Attrition9) Graphical Representation of Ideas10) Online Collaboration

Compare Higher Ed and Corp

1. Research in Higher Ed

My Evaluation Plan…

Considerations in Evaluation Plan

1. Student

2. Instructor

3. Training

4. Task5. Tech Tool

6. Course

7. Program

8. University or

Organization

Electronic Conferencing: Quantitative Analyses

Usage patterns, # of messages, cases, responses

Length of case, thread, response Average number of responses Timing of cases, commenting,

responses, etc. Types of interactions (1:1; 1: many) Data mining (logins, peak usage, location,

session length, paths taken, messages/day/week), Time-Series Analyses (trends)

Electronic Conferencing: Qualitative Analyses

General: Observation Logs, Reflective interviews, Retrospective Analyses, Focus Groups

Specific: Semantic Trace Analyses, Talk/Dialogue Categories (Content talk, questioning, peer feedback, social acknowledgments, off task)

Emergent: Forms of Learning Assistance, Levels of Questioning, Degree of Perspective Taking, Case Quality, Participant Categories

Student Basic Quantitative Grades, Achievement Test Scores, etc. Number of Posts Overall Participation Computer Log Activity—peak usage,

messages/day, time of task or in system

Attitude Surveys

Student High-End Success

Message complexity, depth, interactivity, questioning

Collaboration skills Problem finding/solving and critical

thinking Challenging and debating others Case-based reasoning, critical

thinking measures Portfolios, performances, PBL

activities

Other Measures of Student Success

(Focus groups, interviews, observations, surveys, exams, records)

Positive Feedback, Recommendations Increased Comprehension, Achievement High Retention in Program Completion Rates or Course Attrition Jobs Obtained, Internships Enrollment Trends for Next Semester

Findings: Learning Improved

(Maki et al., 2000)

Intro to Psych: Lecture vs. Online Online performed better on

midterms. Web-based course students

scored higher since had weekly activities due

Lecture students could put off reading until night before exam.

Findings: Learning Improved

(review by Chang, 2003)

Online outperformed peers in histology (anatomy—plant and animal tissues under microscope) course (Shoenfeld-Tacher et al., 2001)

Web enhancements raised exam performance, grades, & attitudes toward economics Agarwal and Day (1998)

Online business communications students performed better on final exams than on campus (Tucker, 2000)

Integrating Wireless Content Syllabus Magazine, May 13, 2003

Study by Mobile Learning Corp: group of college institutions Digital content helped first-year

college accounting students learn Online interactive exercises

useful to student learning Encouraged independent student

learning, and instructors to adopt coaching role.

Findings: Learning Worse

(Wang & Newlin, 2000)

Stat Methods: Lecture vs. Online No diffs at midterm Lecture 87 on final, Web a 72 Course relatively unstructured Web students encouraged to collab Lecture students could not collab All exams but final were open book

Findings: Learning Worse…Organizational Behavior, IUSE

(Keefe, Educause Quarterly, 1, 2003)

Keefe studied 4 semesters of courses, 6 sections, 118 students Face-to-face more satisfied with course

and instructor Those in online course associated with

lower grades

Learning Improved or Not?

(Sankaran et al., 2000)

Students with a positive attitude toward Web format learned more in Web course than in lecture course.

Students with positive attitude toward lecture format learned more in lecture format.

Contrasting Findings are the Norm

Some courses impersonal, isolating, and frustrating (Hara & Kling, 2001)

Sense of community and lower attrition rates when support interactivity, reflection, and sharing (Harnishfeger, March, 2003)

Problem-Based Learning

Distance Ed, 23(1), 2002

Practical learning issues generated more interactions and higher levels of interaction than theoretical issues

Communities of learners need to negotiate identity and knowledge and need milestones (chat session agreements, producing reports, sharing stories, and new work patterns)

Group development: (1) negotiate problem and timetable, (2) divide work in subgroups, and (3) produce drafts of products

Network Conferencing Interactivity (Rafaeli & Sudweeks, 1997)

1. > 50 percent of messages were reactive.2. Only around 10 percent were truly interactive. 3. Most messages factual stmts or opinions4. Many also contained questions or requests.5. Frequent participators more reactive than low.6. Interactive messages more opinions & humor.7. More self-disclosure, involvement, & belonging.8. Attracted to fun, open, frank, helpful,

supportive environments.

Schallert & Reed, AERA, April 2003

Nonnative students do not participate equally in written discussions

Enthusiastic and frequent contributors do not necessarily make intellectually significant contributions.

Some who seem deeply engaged may be less rigorously engaged in many conversations

Collaborative Behaviors(Curtis & Lawson, 1997)

Most common were: (1) Planning, (2) Contributing, and (3) Seeking Input.

Other common events were:(4) Initiating activities,(5) Providing feedback,(6) Sharing knowledge

Few students challenge others or attempt to explain or elaborate

Recommend: using debates and modeling appropriate ways to challenge others

Dimensions of Learning Process

(Henri, 1992)

1. Participation (rate, timing, duration of messages)

2. Interactivity (explicit interaction, implicit interaction, & independent comment)

3. Social Events (stmts unrelated to content)

4. Cognitive Events (e.g., clarifications, inferencing, judgment, and strategies)

5. Metacognitive Events (e.g., both metacognitive knowledge—person, and task, and strategy and well as metacognitive skill—evaluation, planning, regulation, and self-awareness)

Surface vs. Deep Posts(Henri, 1992)

Surface Processing making judgments

without justification, stating that one shares

ideas or opinions already stated,

repeating what has been said

asking irrelevant questions

i.e., fragmented, narrow, and somewhat trite.

In-depth Processing linked facts and ideas, offered new elements of

information, discussed advantages

and disadvantages of a situation,

made judgments that were supported by examples and/or justification.

i.e., more integrated, weighty, and refreshing.

Level of Cognitive Processing: All Posts

Surface33%

Deep55%

Both12%

Surface

Deep

Both

Critical Thinking (Newman, Johnson, Webb & Cochrane, 1997)

Used Garrison’s five-stage critical thinking model

Critical thinking in both CMC and FTF envir. Depth of critical thinking higher in CMC envir.

More likely to bring in outside information Link ideas and offer interpretations, Generate important ideas and solutions.

FTF settings were better for generating new ideas and creatively exploring problems.

Social Construction of

Knowledge (Gunawardena, Lowe, & Anderson, 1997)

Five Stage Model1. Share ideas,2. Discovery of Idea Inconsistencies, 3. Negotiate Meaning/Areas Agree, 4. Test and Modify,5. Phrase Agreements

In global debate, very task driven. Dialogue remained at Phase I: sharing info

Research on Instructors Online If teacher-centered, students

explore less, engage less, interact less (Peck, and Laycock, 1992)

Informal, exploratory conversation fosters risktaking & knowledge sharing (Weedman, 1999)

Online Teaching Job Varies--Plan, Interaction, Admin, Teaching (McIsaac, Blocher, Mahes, & Vrasidas,

1999)

Three Most Vital Online Teaching SkillsThe Online Teacher, TAFE, Guy Kemshal-Bell (April, 2001)

Ability to engage the learner (30) Ability to motivate online

learners (23) Ability to build relationships (19) Technical ability (18) Having a positive attitude (14) Adapt to individual needs (12) Innovation or creativity (11)

Feelings Toward Online TeachingThe Online Teacher, TAFE, Guy Kemshal-Bell (April, 2001)(Note: 94 practitioners surveyed.)

Exciting (30) Challenging (24) Time consuming (22) Demanding (18) Technical issue (16); Flexibility (16) Potential (15) Better options (14); Frustrating (14) Collab (11); Communication (11); Fun

(11)

Little or no feedback given

Always authoritative Kept narrow focus of

what was relevant Created tangential

discussions Only used “ultimate”

deadlines

Provided regular qual/quant feedback

Participated as peer Allowed perspective

sharing Tied discussion to

grades, other assessments.

Used incremental deadlines

Poor Instructors Good Instructors

Dennen’s Research on Nine Online Courses

(sociology, history, communications, writing, library science, technology, counseling)

Role of Online Teacher(Bonk, Kirkley, Hara, & Dennen, 2001)

Technical—Train, early tasks, be flexible, orientation task

Managerial—Initial meeting, FAQs, detailed syllabus, calendar, post administrivia, assign e-mail pals, gradebooks, email updates

Pedagogical—Peer feedback, debates, PBL, cases, structured controversy, field reflections, portfolios, teams, inquiry, portfolios

Social—Café, humor, interactivity, profiles, foreign guests, digital pics, conversations, guests

Problems and Solutions

(Bonk, Wisher, & Lee, in press)

1. Tasks Overwhelm2. Confused on Web3. Too Nice Due to

Limited Share History

4. Lack Justification5. Hard not to

preach6. Too much data7. Communities not

easy to form

Train and be clear Structure time/dates

due Develop roles and

controversies Train to back up claims Students take lead role Use Email Pals Embed Informal/Social

Benefits and Implications

(Bonk, Wisher, & Lee, in press)

1. Shy open up online2. Minimal off task3. Delayed collab more

rich than real time4. Students can

generate lots of info5. Minimal disruptions6. Extensive E-Advice7. Excited to Publish

Use async conferencing Create social tasks Use Async for debates;

Sync for help, office hours

Structure generation and force reflection/comment

Foster debates/critique Find Experts or Prac. Ask Permission

More Implications Include Variety: tasks, topics,

participants, accomplishments, etc.

Make interaction extend beyond class

Have learners be teachers Find multiple ways to succeed Add personalization and choice Provide clarity and easy

navigation

Ten Ways Online Ed Matches or Surpasses FTF, Mark Kassop, Technology Source, Michigan Virtual Univ, May/June 2003

1. Student-centered learning2. Writing intensity3. Highly interactive discussions4. Geared for lifelong learning5. Enriched course materials6. Online demand interaction and support7. Immediate feedback8. Flexibility9. An intimate community of learners10. Faculty development and rejuvenation

2. Research and Evaluation in Corporate Settings

Collecting Evaluation Data Learner Reaction Learner Achievement Learner Job Performance Manager Reaction Productivity Benchmarks

Forms of Evaluation Interviews and Focus Groups Self-Analysis Supervisor Ratings Surveys and Questionnaires ROI Document Analysis Data Mining (Changes in pre and

post-training; e.g., sales, productivity)

What is Evaluation???

“Simply put, an evaluation is concerned with judging the worth of a program and is essentially conducted to aid in the making of decisions by stakeholders.” (e.g., does it work as effectively as the standard instructional approach).

(Champagne & Wisher, in press)

Meta-Analysis: Recurrent Themes in

E-Learning Reports (Waight, Willging, & Wentling, 2002)

1. 250 e-learning reports from 1999-2001

2. Of those, 100 were sold by private companies for $100-3,000

3. Of remaining 150, 70 outside U.S.4. 15 selected were from government,

bus, and professional associations5. Few studied review existing research

Meta-Analysis: Six Functions of E-

Learning (Waight, Willging, & Wentling, 2002)

Anytime, anywhere Cost effective Global reach Just-in-time Allow personalization Improve collaboration and interactivity

Address learner diversity, learner-centered, and blur working and learning lines

Meta-Analysis: Six Purposes of E-

Learning Reports (Waight, Willging, & Wentling, 2002)

1. Inform investors of opportunities2. Discuss learning in the workforce3. Inform policy makers, educators,

employees, and public4. Identify drivers and players5. Discuss contrib of tech to lrng/perf6. Identity trends and winning

strategies

Overall Blended Learning Results…???

Blended Learning Advantages

1. Course access at one’s convenience and flexible completion

2. Reduction in physical class time3. Promotes independent learning4. Multiple ways to accomplish course

objectives5. Increased opportunities for human

interaction, communication, & contact among students

6. Less time commuting and parking7. Introverts participate more

Blended Learning Disadvantages

1. Procrastination, procrastination, procrastination

2. Students have trouble managing time3. Problems with technology at the

beginning (try too much)4. Can be overwhelming or too novel5. Poor integration or planning6. Resistance to change7. Good ideas but lack of time, money,

& support

Evaluation of E-Learning In Corporate Training: Success Stories and Examples

Success Story #1 (Sitze, March 2002, Online Learning):

EDS and GlobalEnglishCharge: Reduce money on English

trainingGoal: 80% online in 3 monthsResult: 12% use in 12 monthsPrior Costs: $1,500-5,000/studentNew Cost: $150-300/userNotes: Email to participants was

helpful in expanding use; rolling out other additional languages.

Success Story #2 (Overby, Feb 2002, CIO):

Dow Chemical & Offensive Email

Charge: Train 40,000 employees across 70 countries; 6 hours of training on workplace respect and responsibility.

Specific Results: 40,000 passedSavings: Saved $2.7 million ($162,000

on record keeping, $300,000 on classrooms and trainers, $1,000,000 on handouts, $1,200,000 in salary savings due to less training time).

Success Story #3 (Overby, Feb 2002, CIO):

Dow Chemical & Safety/Health

Charge: Train 27,000 employees on environmental health and safety work processes.

Results: Saved $6 million; safety incidents have declined while the number of Dow employees have grown.

Success Story #4 (Overby, Feb 2002, CIO):

Dow Chemical & e-learning system

Charge: $1.3 million e-learning system

Savings: $30 million in savings ($850,000 in manual record-keeping, $3.1 in training delivery costs, $5.2 in reduced classroom materials, $20.8 in salaries since Web required 40-60% less training time).

Success Story #5 (Ziegler, e-learning, April 2002):

British Telecom & sales training

Costs: Train 17,000 sales professionals to sell Internet services using Internet simulation.

Result: Customer service rep training reduced from 15 days to 1 day; Sales training reduced from 40 days to 9 days.

Savings: Millions of dollars saved; sales conversion went up 102 percent; customer satisfaction up 16 points.

Success Story #6. Infusing E-

Learning (Elliott Masie, March 2002, e-learning Magazine)

A manufacturing company transformed a week-long safety program into a three-part offering:

1. One day in classroom

2. Multiple online simulations and lessons.

3. One final day of discussions and exams.

Must accomplish online work before phase 3—

this raised success rate, transfer of skills, and lowered hours away from the job.

Success Story #7. Ratheon, Build Own LMS

(John Hartnett, Online Learning, Summer 2002)

SAP Training Choice: Vendor ($390,000) or Build Internally ($136,000) or Cost of Instructor-led Training ($388,000).

Note Saved $252,000Five Training Components in 18 Weeks (within 6

weeks, 4,000 courses taken by 1,400 students)1. Role-based simulations2. Audio walk-throughs3. Online quick reference system4. Live training support (special learning

labs)5. Online enrollment and tracking

Success Story #8: IBMSpecial E-Learning Issue, April 2001

33,000 IBM managers have taken online courseware.

5 times as much content at one-third the cost.

IBM reported $200 million in savings in one year.

Voided $80 million dollars in travel and housing expenses during 1999 be deploying online learning.

IBM Training of 6,600 New First-Line Managers (Basic Blue)

Phase I: 26 Weeks of Self-paced Online Learning Cohorts of 24 managers Lotus LearningSpace Forum 2 hours/week; 5 units/week 18 mandatory and elective management

topics Need minimum score on mandatory topics 14 real-life interactive simulations LearningSpace tutor guides behavior

Karen Mantyla (2001), ASTD.

IBM Training of 6,600 New First-Line Managers (Basic Blue)

Phase II: In-class 5 day learning lab Experiential higher order learning Bring real-life activities from job Focus on self-knowledge and to understand

their roles as leaders and members of IBM Harvard Business cases, leadership

competency surveys, managerial style questionnaires, brain dominance inventories

Coached by a learner-colleague (teaming impt!)

Less than 1 hour of the 5 days is lecture

IBM Training of 6,600 New First-Line Managers (Basic Blue)

Phase III: 25 Weeks of Online Learning Similar to Phase I but more complex

and focuses on application Creates individual development plan

and organizational action plan Managers reviews and signs off on

these plans

IBM Training Results (Kirkpatrick Model)

Level 1 High satisfaction and enthusiasm for

blended Coaching and climate rated highest

Level 2: 96% displayed mastery in all 15 subject

areas; 5 times as much content covered in this program compared to 5 days of live training

150 Web page requests/learner

IBM Training Results (Kirkpatrick Model)

Level 3 Significant behavior change (in particular in

coaching, styles, competencies, and climate) Graduate had high self-efficacy and believed

that they could make a difference

Level 4 Linkage bt leadership & customer satisfaction Leadership led to teamwork and satisfaction Managers reported improvement on job Improved morale and productivity reported

IBM Training Results (Kirkpatrick Model)

Level 5 Asked graduates to estimate the

impact on their departments in dollars

$415,000 or ROI of 47 to 1. Perceived real and lasting leadership

increases

Updated Success Story #8: IBM CLO, March 2003

5,000 new managers/year Program cost = $5 million, cost

avoidance = $88 million (travel, living expenses, and manager time)

5 times as much content delivered compared to previous new-manager training program

72% delivered through distance, 25% via classroom

Access materials at own convenience.

Blended Learning Advantages for IBM

1. Greater consistency of language, knowledge, and corporate culture across the globe

2. Blended approach to training now replicated in other units

3. Market it’s e-learning design4. Cross functional understanding &

teamwork5. No risk trials and simplicity helps

Success Story #9. Army Three Phases of AC3-DL

I. Asynchronous Phase: 240 hours of instruction or 1 year to complete; must score 70% or better on each gate exam

II. Synchronous Phase: 60 hours of asynchronous and 120 hours of synchronous

III. Residential Phase: 120 hours of training in 2 weeks at Fort Knox

AC3-DL Course Tools Learned faster than in

correspondence course More flexible; could do around full

time work schedules Fit Army small group training model Async for content, sync for

application Need to shorten course modules

and provide sync training earlier to increase retention

Overall frequency of interactions across chat categories (6,601

chats).

On-Task55%Social

30%

Mechanics15%

0%

10%

20%

30%

40%

50%

60%

70%

Month 1,2 Month 3,4 Month 5,6

On-Task Social Mechanics

Success #10: Microsoft Excel Training(Jeff Barbian, Blended Works, Summer 2002, Online Learning)

Group One: 5 scenario-based exercises that offered live use of Excel on real-world tasks, online mentors, FAQs, relevant Web sites, NETg Excel Fundamentals Learning Objects.

Group Two: Same as Group One but without scenarios, but info in 5 scenarios were embedded in the learning objects.

Group Three: No training control.

Success #10: Microsoft Excel Training(Thompson Learning Company Study; Jeff Barbian, Blended Works, Summer 2002, Online Learning)

Group One (the blended group): 30 percent increase in accuracy over Group Two (the e-learning group) and were 41 percent faster

Group Two performed 159 more accurately than Group Three

Groups 1 and 2 relied on the online mentors for support (Note: with these results, Lockhead Martin

became a blended learning convert.

Success #11: NCR: Blended Approaches(Thompson Learning Company Study; Jeff Barbian, Blended Works, Summer 2002, Online Learning)

1. Design of E-Learning (Various methods: Web articles; Synchronous points for team exercises)

2. Field Guide Binders (Web site guidance, live feedback on case studies, live “kick off” that promotes collaboration, hands-on role play)

Over 71 percent of learners were responding to customers more effectively (Kirkpatrick Level 3)

Success #12: Convergys: Blended(Jeff Barbian, Blended Works, Summer 2002, Online Learning)

Leadership Dev, Succession Planning, performance management, etc.

LMS from Knowledge Planet, 3 e-learning libraries, virtual classroom tools to 50 locations in North America & Europe

New managers received: Readings, job aids, meeting checklists, 5 off-the-shelf courses from SkillSoft, virtual classes via LearnLinc (new recruits talk to experienced managers), and a 4 day instructor-led seminar at HQ.

Success #13: Sallie Mae/USA Group (Blended student loan provider program)(Jeff Barbian, Blended Works, Summer 2002, Online Learning)

LEAD (Leadership and Education Development); Groom internal staff to fill supervisory-level positions

4 hours/week in class with internal and external instructors; learn trust, role of managers, etc.

First must complete 3 online management courses from SkillSoft and 6 online project management courses (includes panel presentation by IT Project Team to illustrate how projects are handled in the company’s culture)

Findings: increased temawork, camaraderie, shared understanding of concepts, respect for individual differences, social interaction, and reinforcement for class concepts.

Success #14: Proctor and Gamble(Jeff Barbian, Blended Works, Summer 2002, Online Learning)

1999 = 100,000 employees; 20,000 trained/year

LMS from Saba, live training from Centra CD-based training using Authorware,

CourseBuilder, & Dreamweaver 2002 = 1,200 learning items; 34% Web,

54% CD Global English saved $2.5 million per year Off-the-shelf courses in time

management and managing for success

Proctor and Gamble(Jeff Barbian, Blended Works, Summer 2002, Online Learning)

“Given our learning objectives and needs, should we select Web-based live training, versus classroom, versus video-based, versus CBT, or some blended solution?…It depends, on the resources you have, how far geographically you have to reach, or whether you can get your arm around them and pull them into a classroom.” Art DiMartile, Senior IT Manager, Proctor and Gamble

The Worldwide Expansion of E-Learning!!!

Success #15: Circuit City is training 50,000 employees from 600 stores using customized courses that are “short, fun, flexible, interactive and instantly applicable on the job.”

Success #16: The Army’s virtual university offered online college courses to more than 12,000 students located anywhere in the world in 2001 in the first year of a $42 million e-learning program.

Dr. Sylvia Charp, Editor-in-Chief, T.H.E. Journal, March 2002.

Success #17: Community Health Network of

Indiana; www.ehealthindiana.com (July 15, 2002, American Hospital Association)

Named one of most wired hospitals and most improved hospital system nationwide in the use of technology in health care

Virtual nurse recruitment Web site (live chats with recruiters)

Video streams of nursing leaders Virtual tours of individual nursing units Online application and interactive job-posting

databases Web portal for physicians First in nation to offer live Web cast of in vitro

fertilization procedure Real time clinical data repository

Success #18: Cisco and DigitalThink Course (Cisco vendors)

Most saw significant growth in productivity

74% reported improvement in ability to sell or service clients

Customer satisfaction jumped 50%

Success #18: Cisco and DigitalThink Course (employees)

Sales training self-assessment Ask via survey to estimate how much

time training saved them on the job Ask whether it improved performance Select a percentage for each ROI of 900%; for every $1 spent on

training, Cisco sees a gain of 900% in productivity

Success #19: Kinkos(CLO, May 2003)

1,100 locations in 9 countries Used blended model: Internet +

decentralized instruction, job aids, mentoring, virtual classroom training

Resulted in cost savings Increased staff capability, reduced

time to competence, increased speed to market, and increased compliance and certification

Success #20: Masimo (develops medical signals for vital signs) (CLO, May 2003)

Challenge to keep sales force and OEM distribution partners up-to-date and competent

E-learning has resulted in faster time-to-market and deeper capabilities for adoption of their technology

Increased brand awareness and product awareness among hospital staff

Building communities of uses for future sales

Success #21: Tenet Health System(CLO, March 2003)

By 2010, there will be 21,000 less nurses than today and 40% will be over age 50

Recruitment and retention a major problem

Provided access to 500 hours of online clinical training and learning paths aligned along career trajectories (e.g., RN track a way to recruit)

40% of RNs cited learning opportunities as major reason for taking job

Success #22: Real Estate Company(CLO, March 2003)

Microsoft applications (Outlook, Excel, Word, etc.) training via e-learning

67 percent of costs were non-technical

150 courses completed in 8 months and more than 500 initiated

Success #22: Real Estate Company(CLO, March 2003)

Employee satisfaction and retention up

Time to payback period 5-6 months Faster time to competency and

greater employee productivity Return on investment (ROI) of 22

percent

Success #23: Energy Company(CLO, March 2003)

IT technical training for employees Async, Web-based, self-paced

learning Some employees discussed learning

in virtual classroom In 12 month span, 3,000 courses

completed and another 7,000 partially completed

Success #23: Energy Company(CLO, March 2003)

Payback period of 3-4 months 12 month ROI of 192 percent Faster time to competency Reduced re-work Higher employee retention Higher quality of service Reduced help desk call volume and

costs Less system downtime

Success #24: Defense & Aerospace Company (CLO, March 2003)

Fortune 100 company Trained consultants who built systems Subject matter highly technical & rapidly

changing Cost $100,000 more per year Blended selected—primarily instructor

led with some Web content for self study 60% instructor delivered, 30% web, 10%

individual coaching/mentoring

Success #24: Defense & Aerospace Company (CLO, March 2003)

Payback period of 1-2 months 12 month ROI of 195 percent Faster time to competency Higher employee retention and

customer satisfaction Reduced cycle times Higher quality of service Higher customer loyalty

Some Final Advice…