Seeking Synchronicity: Evaluating Virtual Reference Transcripts
description
Transcript of Seeking Synchronicity: Evaluating Virtual Reference Transcripts
![Page 1: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/1.jpg)
Seeking Synchronicity: Evaluating Virtual Reference
Transcripts
Presented by Marie L. Radford
andLynn Silipigni Connaway
2006 ALISE Conference San Antonio, TX, January 16-19, 2006
![Page 2: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/2.jpg)
Presenters
Marie L. Radford, Ph.D.Associate Professor,Rutgers University, SCILSEmail: [email protected]/~mradford
Lynn Silipigni Connaway, Ph.D.Consulting Research ScientistEmail: [email protected]/research/staff/connaway.htm
![Page 3: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/3.jpg)
Interpersonal Dimensions of Chat Research Stream
S.S. Green Award 44 transcripts (courtesy LSSI)
Maryland AskUsNow! 245 transcripts (courtesy Maryland AskUsNow! Statewide Consortium)
Library Lawline 113 transcripts (courtesy NELLCO Regional Consortium)
Seeking Synchronicity IMLS Grant Rutgers & OCLCToday’s presentation 300 transcripts (24/7, National & International)Additional 1000+ transcripts (QuestionPoint & 24/7)
![Page 4: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/4.jpg)
24/7 Transcript Analysis• Generated random sample
– July 7, 2004 through June 27, 2005– 263,673 sessions– 25 transcripts/month = 300 total
• 256 usable transcripts – Excluding system tests and technical
problems
![Page 5: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/5.jpg)
3 Analyses• Type of Questions
• Katz/Kaske Classification
• Subject of Questions
• Interpersonal Communication • Radford Classification• Manual and NVivo coding
![Page 6: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/6.jpg)
6%0%
25%
2%
30%
37%
Holdings
Inappropriate
Procedural
Research
ReadyReferenceSpecific Search
Types of Questions
Combined 1st and 2nd Questionsn=273 questions
![Page 7: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/7.jpg)
Subjects of Questions64
3530
27
2015
11 11 11 9 9 8
0
10
20
30
40
50
60
70
1
ProceduralLawGeneral/MultidiscilinaryScienceHistoryBusinessEducationLiteratureBiographySocial ScienceRecreationGeography
Combined 1st and 2nd Questionsn=273 questions
![Page 8: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/8.jpg)
Classification Methodology
• Qualitative Analysis
• Development/refinement of category scheme
• Careful reading/analysis
• Identification of patterns
Time intensive, but reveals complexity
![Page 9: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/9.jpg)
Research QuestionsInterpersonal Communication Analysis
• What relational dimensions are present?• Are there differences in relational dimensions/patterns of chat users & librarians?
• If so, what are they?• How do users & librarians compensate for lack
of nonverbal cues?• What is the relationship between content & relational dimensions in determining quality?
![Page 10: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/10.jpg)
•Relational FacilitatorsInterpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication.
•Relational BarriersInterpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication.
Results Interpersonal Communication Analysis
2 Major Themes
![Page 11: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/11.jpg)
Manual Coding Results
• 200 Transcripts– 177 Usable Transcripts
![Page 12: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/12.jpg)
Librarian Relational Factors (1-200)
Librarian Relational Facilitators:Manual Analysis
132
8674 64
41
0
20
40
60
80
100
120
140
160
180
200
LF 1 RapportBuilding
LF 2 Deference LF 5 ClosingRitual
LF 4 GreetingRitual
LF 3 Rep. OfNon-Verbal
Cues
n=177 transcripts
![Page 13: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/13.jpg)
121102
6240
27
0
20
40
60
80
100
120
140
160
180
200
CF 1 Deference CF 2 RapportBuilding
CF 3 ClosingRitual
CF 4 Rep. OfNon-Verbal
Cues
CF 5 GreetingRitual
Client Relational Facilitators:Manual Analysis
n=177 transcripts
![Page 14: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/14.jpg)
Comparison Relational Facilitators: Manual Analysis
132
86
4164 74
102121
4027
62
020
406080
100
120140160
180200
RapportBuilding
Deference Rep. Of NVCues
Greeting Ritual Closing Ritual
Librarians Clients
n=177 transcripts
![Page 15: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/15.jpg)
3425
0
20
40
60
80
100
120
140
160
180
200
LB 1 Relational Disconnect LB 2 Negative Closure
Librarian Relational Barriers: Manual Analysis
n=177 transcripts
![Page 16: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/16.jpg)
78
33
0
20
40
60
80
100
120
140
160
180
200
CB 1 Closing Problems CB 2 Relational Disconnect
Client Relational Barriers: Manual Analysis
n=177 transcripts
![Page 17: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/17.jpg)
34 3325
78
0
20
40
60
80
100
120
140
160
180
200
Librarians Clients
Relational Disconnect Negative Closure
Comparison of Relational Barriers: Manual Analysis
n=177 transcripts
![Page 18: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/18.jpg)
NVivo Coding
• 100 Transcripts– 79 Usable Transcripts
![Page 19: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/19.jpg)
Librarian Relational Facilitators:NVivo Analysis
n=79 transcripts
3832
63
37
26
0
10
20
30
40
50
60
70
LF1 RapportBuilding
LF2 Deference LF3 NonverbalCues
LF4 GreetingRitual
LF5 ClosingRitual
Relational Facilitators
Num
ber o
f Tra
nscr
ipts
![Page 20: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/20.jpg)
Client Relational Facilitators:NVivo Analysis
n=79 transcripts
51
11 9
56
25
0
10
20
30
40
50
60
70
CF1 Deference CF2 RapportBuilding
CF3 ClosingRitual
CF5 GreetingRitual
CF4 NonverbalCues
Relational Facilitators
Num
ber o
f Tra
nscr
ipts
![Page 21: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/21.jpg)
Comparison Relational Facilitators: NVivo Analysis
n=79 transcripts
63
38 3732
26
5156
9 11
25
0
10
20
30
40
50
60
70
RapportBuilding
Deference Nonverbal Cues Greeting Ritual Closing Ritual
Relational Facilitators
Num
ber o
f Tra
nscr
ipts
Librarian Client
![Page 22: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/22.jpg)
Librarian Relational Barriers: NVivo Analysis
n=79 transcripts
2122
0
5
10
15
20
25
30
35
40
45
LB2 Negative Closure LB1 Relational DisconnectRelational Barriers
Num
ber o
f Tra
nscr
ipts
![Page 23: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/23.jpg)
Client Relational Barriers: NVivo Analysis
n=79 transcripts
40
10
0
5
10
15
20
25
30
35
40
45
CB1 Closing Problems CB2 Relational DisconnectRelational Barriers
Num
ber o
f Tra
nscr
ipts
![Page 24: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/24.jpg)
Comparison Relational Barriers: NVivo Analysis
n=79 transcripts
22 21
40
10
0
5
10
15
20
25
30
35
40
45
Negative Closure/Closing Problems Relational DisconnectRelational Barriers
Num
ber o
f Tra
nscr
ipts
Librarian Client
![Page 25: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/25.jpg)
NVivo Search Totals
Word Total Percentsorry 20 25.32thank you 51 64.56thanks 34 43.04thank u 3 3.80
n=79 transcriptsIncludes words in scripts
![Page 26: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/26.jpg)
NVivo Search TotalsWord Total Percentsorry 20 25.32thank you 51 64.56thanks 34 43.04thank u 3 3.80thanx 0 0.00TY 0 0.00ty 0 0.00
n=79 transcriptsIncludes words in scripts
![Page 27: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/27.jpg)
NVivo Search TotalsPerson Word Total PercentLibrarian sorry 14 17.72Client sorry 7 8.86Librarian thank you 36 45.57Librarian thanks 18 22.78Client thank you 28 35.44Client thanks 24 30.38Client thank u 3 3.80
n=79 transcriptsIncludes words in scripts
![Page 28: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/28.jpg)
NVivo 2.0: Advantages• Advantages
– Move subnodes between nodes at any time• Ability to create a hierarchical classification
scheme– Proximity and Boolean searching– Group documents and nodes into sets for use
in searching and analysis– Programmatically calculate descriptive
statistics after coding
![Page 29: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/29.jpg)
NVivo 2.0: Disadvantages• Disadvantages
– User interface does not match standard Microsoft application
• Cannot double click on a document or node to open• Cannot scroll through a document using the mouse scroll
wheel
– Cannot import nodes– Cannot set certain tools to always appear
• Must manually display the Coder bar and coding strips each time for every document/transcript
• NVivo 7 to be released in Feb. 2006– May alleviate disadvantages
![Page 30: Seeking Synchronicity: Evaluating Virtual Reference Transcripts](https://reader036.fdocuments.in/reader036/viewer/2022062315/56815d66550346895dcb70c5/html5/thumbnails/30.jpg)
End Notes• This is one of the outcomes from the project
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives.
• Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center.
• Project web site: http://www.oclc.org/research/projects/synchronicity/