Virtual Windows: Observing Chat Reference Encounters

through Transcript Analysis

Lynn Silipigni Connaway, Ph.D.,Marie L. Radford, Ph.D.Lawrence Olszewski, Ph.D.

19 IAPS International ConferenceSeptember 11-16, 2006

Alexandria, Egypt

Morphing Idea of Library

• Proliferation of digital libraries– Virtual Reference Services (VRS)– Electronic collections

• User Preferences– Sources

• Internet– Electronic sources

• Humans– Parents– Colleagues/Friends– Professors

– Interface design• Google-like • Amazon

Privacy and Confidentiality

• Traditional reference (FtF and Telephone)

– Anonymity and privacy assumed

• VRS– Verbatim transcripts allow unobtrusive

research opportunities– Transcripts provide physical evidence of

session

Privacy and Confidentiality

• Known identity of user – Authenticate and improve service– Identify repeat user– Send follow-up information

• Nature and subject of query– Sensitive questions

• Medical• Legal• Personal situations

– Confidentiality of all queries should be respected

Evaluation of VRS

• Sustainability of VRS– Factors that influence selection and use of

VRS– Behavior of users and librarians in VRS

sessions– User and librarian perceptions of satisfaction

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

• $1,103,572 project funded by:

– Institute of Museum and Library Services $684,996 grant

– Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

Project duration10/1/2005-9/30/2007

Four phases:I. Focus group interviews*II. Analysis of 1,000+ QuestionPoint

transcriptsIII. 600 online surveys*IV. 300 telephone interviews*

*Interviews & surveys with VRS users, non-users, & librarians

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

• Identify what individuals say they do– Focus group interviews– Online surveys– Telephone interviews

• Identify what individuals actually do– Transcript analysis

Phase II:24/7 Transcript Analysis

• Generated random sample– July 7, 2004 through June 27, 2005– 263,673 sessions– 25 transcripts/month = 300 total

• 256 usable transcripts– Excluding system tests and technical

problems

6 Analyses

– Geographical Distribution• Library receiving query• Library answering query

– Type of Library– Type of Questions

• Katz/Kaske Classification

– Subject of Questions• Dewey Decimal Classification

– Session Duration – Interpersonal Communication

• Radford Classification

Librarian Location - Question Received

Other States = 10

United Kingdom = 1

Pennsylvania = 4

Arizona = 4

Kansas = 5

Delaware = 6

Canada = 7

New York = 7

Washington = 8

Utah = 8

North Carolina = 14

Massachusetts = 21

Australia = 36

Maryland = 47

California = 77

0 10 20 30 40 50 60 70 80 90

Number of Questions

n=255

Librarian Location - Question Referred/ Answered

Other states = 12

Germany = 1

Pennsylvania = 4

Michigan= 4

Colorado = 4

Hawaii = 6

Canada = 7

Washington = 7

North Carolina = 7

New York = 8

Connecticut = 9

Massachusetts = 10

Maryland = 35

Australia = 36

California = 88

0 10 20 30 40 50 60 70 80 90 100

Number of questionsn=238

Type of Library Receiving Question

0

10

20

30

40

50

60

70

80

90

Consortium Public University National Law State NotAvailable

K-12

Num

ber

of Q

uest

ions

n=256

Type of Question Asked

Procedural25%

Research2%

Subject Search37%

Inappropriate0%

Holdings6%

Ready Reference30%

n=273

Procedure and Subject

Language1%

Science8%

Technology5%

Arts & recreation4%

Literature5%

History & geography16%

Library procedure & miscellaneous

25%

Philosophy & psychology

1%

Compuer science, information & general

works4%

Social sciences31%

Religion0%

n=273

Dewey Decimal Classification

Arts & recreation5%

Literature6%

History & geography21%

Philosophy & psychology

1%

Compuer science, information & general

works5%

Religion0%

Social sciences42%

Technology7% Science

11%Language

2%

n=273

Service Duration

• Mean Service Duration: 13:53

• Median Service Duration: 10:37

Positive Transcript Example

U Where can I find the leading drug companies in boston doing diabetes treatment / prevention R&D?

L I can probably give you a few sources to get started, but I may wind up referring you to a business and/or medial librarian specialist.

L Let's start witht eNortheastern library web page...

U ok great thanks

L OK. I'm going to try the "co-browse" option -- that might let us see the same information at once...(if it's working!)

U wonderful

L since what you want to find are drug companies, I'll try to get you into a busienss database...

U perfect thank you

L Sorry, I thought there was a way you could search by sic code and get a ranked list of companies in a certian code.

U thats alright, seemed liek you were on the right track

Positive Transcript Example

Negative Transcript Example

U Which way is ur car accelerating when you’re thrown forward after hitting another bumper car?

L Is this a homework question.

L I'm not an expert on driving so I really can't answer that

U can u find a website or something

L I'm not sure what you are asking.

Negative Transcript Example

U …hello?

L I really don't understand how I can answer that for you.

U can i hav another librarian

L The information you gave you me does not help me find any resources to help you.

Focus Group InterviewsReasons for Using VRS

• Convenient• Efficient• More reliable than search engines & free• Allows multi-tasking• Email follow-up & provision of transcript• Pleasant interpersonal experience

– Librarian on first name basis – more personalized• Less intimidating than physical reference desk

– Feel comfortable abruptly ending session

Focus Group InterviewsReasons for not using VRS

• Graduate students– Fear of

• Bothering librarian• Looking stupid & advisors finding out

– Questions may not be taken seriously– Potential technical problems– Bad experiences in FtF influence expectations of VRS

• Screenagers– Virtual stalkers (“psycho killers”)– Not finding a trusted librarian– Unsure of what to expect

Focus Group Interviews Challenges for Users & Non-Users

• Speed and technical problems• Delayed response time• Librarians are not in users’ libraries

– Fear of no subject expertise

• Fear of overwhelming librarian

Focus Group Interviews Suggestions from Users & Non-Users

• Inclusion of multiple languages• Access to subject specialists• Better marketing and publicity

– Information on how to connect and use VRS– Reassurance that users will not bother librarians – the

library wants the service to be used

• Faster technology• Improved interface design

– More color– More attractive

Service Implications

• Sustainability of VRS– Encourage repeat use

• Protect privacy and anonymity– Encryption programs– “Anonymity Button”– Opt-out after registering– Opt-in only for necessary information

– Sales/Homework Help models• Build interpersonal relationships

– Disclose first name - trusted librarian– Positive relational communication

– Trade-offs in service• Personal service vs. personal disclosure

– Follow-up capability vs. anonymity– Amazon-like services vs. protection of personal information

Conclusions

• Current service models do not address privacy issues– Millennial generation wary of virtual

environments• Remote communication poses less

interpersonal risk than FtF• Positive interpersonal communication

imperative for VRS success• Many users appreciate convenience

and immediacy of VRS

Next Steps• Conduct

– Two additional focus group interviews – VRS users– Online survey & telephone interviews with VRS

• Users• Non-users• Librarians

• Analyses– Gender– User Type

• Child/Young adult• Adult• Unknown

End Notes

This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators.

Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center.

Project web site: http://www.oclc.org/research/projects/synchronicity/

Questions

Marie L. Radford, Ph.D. Email: mradford@scils.rutgers.edu www.scils.rutgers.edu/~mradford

Lynn Silipigni Connaway, Ph.D. Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm

Lawrence Olszewski, Ph.D. Email: olszewsl@oclc.org