Adoption and Use of a Presence/Chat Application in Globally-Distributed

Software Development

James HerbslebInstitute for Software Research, International

School of Computer Science1321 Wean Hall

Carnegie Mellon UniversityPittsburgh, PA, USA 15213

+1 412 268-8933jdh@cs.cmu.edu

2

Research Team

• University of Michigan– Tom Finholt– Mark Handel

• Carnegie Mellon University– Alberto Espinosa

• Bell Labs Research– David Atkins– David Boyer– James Herbsleb– Stacie Hibino– Audris Mockus– Dewayne Perry– Larry Votta– Graham Wills

3

Nuremberg

NapervilleSwindon

MalmesburyChippenham

Bangalore

Dublin

Paris

HilversumHuizen

Brussels

Columbus

Tools

Rear View Mirror

CalendarBot

Experience Browser

TeamPortal

Models ofDevelopmentHow to distribute work

across global sites.

BestPractices

Design

Code

Test

Planning Travel xxxxxxxxxxxxxxxxx xxxxxx

Establishing Liaisons xxxxxx xxxxx xxxxxxxxxxxxxx xxxxx xxxx xxxxxxxxxxxxxxxx

Building Trust xxxxxxx xxxxx xxxxxxxxxx

Communication Etiquette xxxxxxxxxxxxxxxxxxx xxxxxxxx xxxxxx

Preventing Delay xxxxxxxx xxx xxxxxx xxxxxxxxxxx xxxxxx xxxxxxx

Using Commercial Tools xxxxxxxxx xxxxxxx xxxx xxxxxxx xxxxxx xxxxxxResearch

TeamResearch

Team

Empirical Studies

New ProductsGlobal Development

Solutions

Collaboratory Project

4

Collaboration Over Distance

• Work items split across sites take much longer

• Many fewer communication paths across distance

• Much less frequent communication across distance

• Less comfortable communicating with remote colleagues

• Much more difficult to identify distant colleagues with needed expertise

• Much more difficult to communicate effectively

• Less likely to perceive themselves as part of the same team

• Common view of priorities – no difference

5

RVM Application Background

• Desire to increase informal communication, context information

• Hoped to create more communication “openness”

• Wanted to increase feeling of “teamness”

• First step: low fidelity prototypes– Users unfamiliar with interactive text– Privacy concerns – surveillance tool?

6

Chat, IM, Presence AwarenessRear View Mirror

GroupChat

Presence Viewer

Option to log in at machine startup.

7

Wireless1 Organization

Swindon

Nuremberg

8

Data Collection

• Study covered 17 months– March 2000 – initial introduction– August 2001 – server shut down

• Qualitative data– Twenty interviews– Two focus groups

• Data from usage logs– Who logged in each day– Group Activity– Group chat messages (not IM messages)

9

Initial DeploymentPlanning, Training

• Targeting Key Users– Identifying key cross-site pairs – Want to achieve critical mass as quickly as possible

• Installation and Training– Hour-long sessions with each user– Installation of RVM (and additional tools) with training and

quick reference cards– Initial training done by two-person teams at each site– Intensive one-week push, followed by one week of follow-up– Trained 15 users in England, 15 in Germany– E-mail and phone support afterwards

10

Initial DeploymentAdoption Curve -- Wireless 1

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug0

10

20

30

40

50

60

70

80

11

Initial DeploymentUse of RVM – Wireless 1

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug

“I Love You” Virus Server Crash Holidays

Reorganizations

12

Adoption

Active users (rolling average)

0.0

1.4

2.8

4.2

5.6

7.0

8.4

9.8

Systems Engineering

Quality

Test

Management 1

Month 1 2 3 4 5 6 7 8 9 10 11 12

13

Initial DeploymentAdoption Issues

• Reliability, usability

• Teams -- key unit for adoption

• Some users -- quick look, duck out style of use

• Individual training sessions – little visible activity

• Privacy versus setup time– Individual permissions very unwieldy

14

Retooling, Rethinking

• Changes in RVM– Bug fixes, testing– Group chat persistence– Group-based security

• Deployment – Team focus

• Targeting users• Training and setup

– Learn how to collaborate, not just how to use the tool

15

Network Team

Columbus

Cary

NapervilleDenver

16

Wireless2 Teams

Dublin

Columbus

17

All GroupsUse of RVM

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug

Wireless 1 (85) Wireless 2 (22) Network (7)

“I Love You” Virus Server Crash Holidays

Reorganizations

18

What Is It Good for?A Pessimistic View . . .

Messages that pop up on screen at an inopportune moment (sometimes from the next cubicle) are destroying workers' concentration. Thoughtless text scrawled and sent in haste can spark online arguments. And in some offices, the question of who is privileged enough to receive certain instant messages is creating the kind of tortured pecking order last seen in high school.

Slatalla, M., The office meeting that never ends. New York Times, Sept. 23, 1999.

19

“Bursty” Chat

250

200

150

100

50

0

Messages and Logins per Day

Time

Chat

Logins

20

Chat Groups

Group Number of Locations

# Chatters (Total Members)

Total messages

Time period (days)

Quality 2 5 (5) 175 235

Research 6 17 (21) 2908 239

Test 1 3 5 (8) 146 224

Kn. Mgt. 3 7 (7) 124 46

Test 2 2 4 (4) 111 116

Arch. 4 4 (8) 104 113

At least 30 days of chatAt least 100 messages

21

Chat CategoriesAvailability Negotiating availability, either now or in the very near future

Non-work topics

Specific non-work content, e.g,. cars, fishing, sports

Work Anything that relates to specific work tasks, processes, or planning; general discussions of business or economics related to work, and discussions about the use of RVM itself.

Greeting Hello, etc., also references to weather or health intended primarily as a greeting. Also, ”closings" such as "Bye!"

Humor Comments clearly intended to be primarily humorous

Other Anything that cannot be categorized elsewhere

Cohen’s kappa = .88Found no flaming.

22

Percentage of Message Type by Group

0%

10%

20%

30%

40%

50%

60%

70%

80%

work availability greeting non-work humor other

Quality

Kn Mgt

Test 1

Architect

Test 2

Research

23

Types of Content by Time of DayPercentage within category

24

Work SubcategoriesCategory % of

MsgsDescription

Technical work

66% Discussion of actual work at hand, e.g., carrying out tests; selecting and defining approaches to the work

Project management

21% Planning the current project, project process issues, project status

Meeting management

8% Planning and running meetings, including the current chat session, as well as future chat and face-to-face meetings, e.g. agenda topics, locations, technology support.

Company 4% Issues affecting the entire company, rather than the current project, e.g.., “You certainly know that discussions [concerning a merger] with [another company] are definitely stopped?”

Other 1% Messages that fit in no other category.

25

Adoption IssuesPerceived Utility

• Interactive text as superfluous

• Is “water cooler” talk real work?

• Users’ perception of causes of multi-site problems– Attribution theory: explain behavior on basis of

personal attributes or situational attributes– Appeared to be tendency to explain “undesirable”

behavior of distant colleagues in terms of personal traits

– If problem is personal traits, is more communication desirable?

26

Groupware’s Critical Mass Dilemma

• Feedback from actual use by groups of real users to get a usable tool– Social and political impacts

• Need a usable tool before you can get critical mass for groups of users– Relatively few innovators, early adopters– Critical mass may not be obtainable

• Possible Solutions:– Extremely tolerant users– Developers use tool– Management pressure– “Progressive” sets of features

27

Research Issues• The role of interactive text in supporting informal

communication in distributed teams– Do remote team members make more person attributions for

“undesirable” behavior?– Does providing context serve to reduce personal

attributions?

• Privacy – How far will group-based model generalize?– What are enduring privacy concerns for different

communities and features?

• Characteristics of RVM Chat that may affect content– Chat participants have known identities (on and off-line)– Chat always “semi-public”– Persists only briefly (unlike b-board)

• Group chat less intrusive than IM?

28

Nuremberg

NapervilleSwindon

MalmesburyChippenham

Bangalore

Dublin

Paris

HilversumHuizen

Brussels

Columbus

Collaboratory Project

Tools

Rear View Mirror

CalendarBot

Experience Browser

TeamPortal

Models ofDevelopmentHow to distribute work

across global sites.

BestPractices

Design

Code

Test

Planning Travel xxxxxxxxxxxxxxxxx xxxxxx

Establishing Liaisons xxxxxx xxxxx xxxxxxxxxxxxxx xxxxx xxxx xxxxxxxxxxxxxxxx

Building Trust xxxxxxx xxxxx xxxxxxxxxx

Communication Etiquette xxxxxxxxxxxxxxxxxxx xxxxxxxx xxxxxx

Preventing Delay xxxxxxxx xxx xxxxxx xxxxxxxxxxx xxxxxx xxxxxxx

Using Commercial Tools xxxxxxxxx xxxxxxx xxxx xxxxxxx xxxxxx xxxxxxResearch

TeamResearch

Team

Empirical Studies

New ProductsGlobal Development

Solutions

We need to finish our preparations for the review!

http://www-spr.research.bell-labs.coCurrent spec.

ConnectIcon

• “Antidote for phone tag”

• Send presence and contact ability to anyone

ConnectIcon

Hi Jim,

We need to talk about the review tomorrow!

Ann Kelly

To check my availability and get my contact information, please click this link:ConnectIcon from Ann Kelly

26 hours ago

23 hours ago

3 days ago

20 hours ago

ConnectIcon

10 minutes ago

Currently in use

Busy