Seeking Synchronicity: Evaluating Virtual Reference Transcripts
description
Transcript of Seeking Synchronicity: Evaluating Virtual Reference Transcripts
![Page 1: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/1.jpg)
Seeking Synchronicity: Evaluating Virtual
Reference Transcripts
Presented by Lynn Silipigni Connaway
OCLC Members Council February 14, 2006
![Page 2: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/2.jpg)
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
• $1,103,572 project funded by:
– Institute of Museum and Library Services $684,996 grant
– Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions
![Page 3: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/3.jpg)
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
Project duration10/1/2005-9/30/2007
Four phases:I. Focus group interviews*II. Analysis of 1,000 QuestionPoint
transcriptsIII. 600 online surveys*IV. 300 telephone interviews*
*Interviews & surveys with VRS users, non-users, & librarians
![Page 4: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/4.jpg)
Phase II:24/7 Transcript Analysis
• Generated random sample– July 7, 2004 through June 27, 2005– 263,673 sessions– 25 transcripts/month = 300 total
• 256 usable transcripts – Excluding system tests and technical problems
![Page 5: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/5.jpg)
5 Analyses
• Geographical Distribution• Type of Library• Type of Questions
• Katz/Kaske Classification • Subject of Questions• Interpersonal Communication
• Radford Classification• Manual and NVivo coding
![Page 6: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/6.jpg)
110, 41%
12, 4%
105, 38%
40, 14%
8, 3%1, 0%
East
Midwest
West
Australia
Canada
England
Geographical Distribution of Question Origination
![Page 7: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/7.jpg)
Type of Library
88
76
51
38
24
12 10
10
10
20
30
40
50
60
70
80
90
100
Consortium Public University National Law Unclear State K-12
![Page 8: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/8.jpg)
6%0%
25%
2%
30%
37%
Holdings
Inappropriate
Procedural
Research
ReadyReference
Specific Search
Type of Questions
Combined 1st and 2nd Questionsn=273 questions
![Page 9: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/9.jpg)
Subjects of Questions64
35
3027
20
1511 11 11
9 9 8
0
10
20
30
40
50
60
70
1
Procedural
Law
General/Multidiscilinary
Science
History
Business
Education
Literature
Biography
Social Science
Recreation
Geography
Combined 1st and 2nd Questionsn=273 questions
![Page 10: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/10.jpg)
Classification Methodology
• Qualitative Analysis
• Development/refinement of category scheme
• Careful reading/analysis
• Identification of patterns
Time intensive, but reveals complexity
![Page 11: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/11.jpg)
Research QuestionsInterpersonal Communication Analysis
• What relational dimensions are present?
• Are there differences in relational dimensions/patterns of chat users & librarians?
• If so, what are they?• How do users & librarians compensate for lack of nonverbal cues?
• What is the relationship between content & relational dimensions in determining quality?
![Page 12: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/12.jpg)
2 Major Themes
• Relational FacilitatorsInterpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication.
• Relational BarriersInterpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication.
Results Interpersonal Communication Analysis
![Page 13: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/13.jpg)
Manual Coding Results
• 200 Transcripts– 177 Usable Transcripts
![Page 14: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/14.jpg)
Librarian Relational Factors (1-200)
Librarian Relational Facilitators:Manual Analysis
132
8674
64
41
0
20
40
60
80
100
120
140
160
180
200
LF 1 RapportBuilding
LF 2 Deference LF 5 ClosingRitual
LF 4 GreetingRitual
LF 3 Rep. OfNon-Verbal
Cues
n=177 transcripts
![Page 15: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/15.jpg)
121102
62
4027
0
20
40
60
80
100
120
140
160
180
200
CF 1 Deference CF 2 RapportBuilding
CF 3 ClosingRitual
CF 4 Rep. OfNon-Verbal
Cues
CF 5 GreetingRitual
Client Relational Facilitators:Manual Analysis
n=177 transcripts
![Page 16: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/16.jpg)
Comparison Relational Facilitators: Manual Analysis
132
86
41
6474
102121
4027
62
0
20
40
60
80
100
120
140
160
180
200
RapportBuilding
Deference Rep. Of NVCues
Greeting Ritual Closing Ritual
Librarians Clients
n=177 transcripts
![Page 17: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/17.jpg)
3425
0
20
40
60
80
100
120
140
160
180
200
LB 1 Relational Disconnect LB 2 Negative Closure
Librarian Relational Barriers: Manual Analysis
n=177 transcripts
![Page 18: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/18.jpg)
78
33
0
20
40
60
80
100
120
140
160
180
200
CB 1 Closing Problems CB 2 Relational Disconnect
Client Relational Barriers: Manual Analysis
n=177 transcripts
![Page 19: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/19.jpg)
34 3325
78
0
20
40
60
80
100
120
140
160
180
200
Librarians Clients
Relational Disconnect Negative Closure
Comparison of Relational Barriers: Manual Analysis
n=177 transcripts
![Page 20: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/20.jpg)
NVivo Coding
• 100 Transcripts– 79 Usable Transcripts
![Page 21: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/21.jpg)
Librarian Relational Facilitators:NVivo Analysis
n=79 transcripts
3832
63
37
26
0
10
20
30
40
50
60
70
LF1 RapportBuilding
LF2 Deference LF3 NonverbalCues
LF4 GreetingRitual
LF5 ClosingRitual
Relational Facilitators
Nu
mb
er
of
Tra
ns
cri
pts
![Page 22: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/22.jpg)
Client Relational Facilitators:NVivo Analysis
n=79 transcripts
51
11 9
56
25
0
10
20
30
40
50
60
70
CF1 Deference CF2 RapportBuilding
CF3 ClosingRitual
CF5 GreetingRitual
CF4 NonverbalCues
Relational Facilitators
Nu
mb
er o
f T
ran
scri
pts
![Page 23: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/23.jpg)
Comparison Relational Facilitators: NVivo Analysis
n=79 transcripts
63
38 3732
26
5156
9 11
25
0
10
20
30
40
50
60
70
RapportBuilding
Deference Nonverbal Cues Greeting Ritual Closing Ritual
Relational Facilitators
Nu
mb
er o
f T
ran
scri
pts
Librarian Client
![Page 24: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/24.jpg)
Librarian Relational Barriers: NVivo Analysis
n=79 transcripts
2122
0
5
10
15
20
25
30
35
40
45
LB2 Negative Closure LB1 Relational Disconnect
Relational Barriers
Nu
mb
er o
f T
ran
scri
pts
![Page 25: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/25.jpg)
Client Relational Barriers: NVivo Analysis
n=79 transcripts
40
10
0
5
10
15
20
25
30
35
40
45
CB1 Closing Problems CB2 Relational Disconnect
Relational Barriers
Nu
mb
er o
f T
ran
scri
pts
![Page 26: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/26.jpg)
Comparison Relational Barriers: NVivo Analysis
n=79 transcripts
22 21
40
10
0
5
10
15
20
25
30
35
40
45
Negative Closure/Closing Problems Relational DisconnectRelational Barriers
Nu
mb
er o
f T
ran
scri
pts
Librarian Client
![Page 27: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/27.jpg)
NVivo Search TotalsWord Total Percent
sorry 20 25.32
thank you 51 64.56
thanks 34 43.04
thank u 3 3.80
thanx 0 0.00
TY 0 0.00
ty 0 0.00
n=79 transcriptsIncludes words in scripts
![Page 28: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/28.jpg)
NVivo Search TotalsPerson Word Total Percent
Librarian sorry 14 17.72
Client sorry 7 8.86
Librarian thank you 36 45.57
Librarian thanks 18 22.78
Client thank you 28 35.44
Client thanks 24 30.38
Client thank u 3 3.80
n=79 transcriptsIncludes words in scripts
![Page 29: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/29.jpg)
Transcript Reading
• Positive VRS experience– Duration = 1 hour 11 minutes– Academic User– Question – Boston drug company - diabetes– Relational Work– Enthusiastic user– Helpful librarian
• Less than positive VRS experience– Duration = 39 minutes– Middle school or high school student– Question – physics – car acceleration– Poor reference work– Extreme negative closure
![Page 30: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/30.jpg)
End Notes
• This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators.
• Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center.
• Project web site: http://www.oclc.org/research/projects/synchronicity/
![Page 31: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022070404/56813b44550346895da41f51/html5/thumbnails/31.jpg)
Questions
Marie L. Radford, Ph.D.Email: [email protected]/~mradford
Lynn Silipigni Connaway, Ph.D.Email: [email protected]/research/staff/connaway.htm