The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a...
-
Upload
stephon-bell -
Category
Documents
-
view
215 -
download
2
Transcript of The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a...
![Page 1: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/1.jpg)
The Impact of Conversational Navigational Guides on the
Learning, Use, and Perceptions of Users of a Web Site
Art Graesser, Tanner Jackson, Matthew Ventura, James Mueller, Xiangen Hu, and Natalie Person
This research was directly supported by contracts from ONR and IDA, and was partially supported from grants by NSF and the DoD.
![Page 2: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/2.jpg)
Outline (The next 20 minutes)
• HURAA
• Experiment
• Results
• Conclusions
![Page 3: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/3.jpg)
What is HURAA?
• HURAA (Human Use Regulatory Affairs Advisor ) is a web-based facility that provides help and training on the ethical use of human subjects in research.
• HURAA is based on documents and regulations from United States Federal agencies.
![Page 4: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/4.jpg)
HURAA (Features)
• Hypertext• Multimedia• Animated navigational guide• Lessons with case-based and explanation-based
reasoning• Help modules• Context-sensitive FAQs (Point & Query)• Glossaries• Archives• Natural language queries
![Page 5: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/5.jpg)
HURAA (Modules)
• Introduction
• Historical Overview
• Lessons
• Explore Issues
• Explore Cases
• Decision Consequences
• Query IRB Documents
![Page 6: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/6.jpg)
![Page 7: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/7.jpg)
Outline• HURAA
• Experiment
• Results
• Conclusions
![Page 8: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/8.jpg)
Experiment• 155 students
• Between subjects:– Navigational Guides
• Full
• Voice
• Text
• None
• Within subjects:– Two Phases
• Acquisition phase
• Test phase
![Page 9: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/9.jpg)
Experiment (Guides)
• Full guide (agent and voice)
![Page 10: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/10.jpg)
![Page 11: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/11.jpg)
Experiment (Guides)
• Full guide (agent and voice)
• Voice guide (synthesized speech only)
![Page 12: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/12.jpg)
![Page 13: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/13.jpg)
Experiment (Guides)
• Full guide (agent and voice)
• Voice guide (synthesized speech only)
• Print guide (text message only)
![Page 14: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/14.jpg)
![Page 15: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/15.jpg)
Experiment (Guides)
• Full guide (agent and voice)
• Voice guide (synthesized speech only)
• Print guide (text message only)
• No guide (no navigational guidance)
![Page 16: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/16.jpg)
![Page 17: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/17.jpg)
Experiment (Predictions)
• If guidance is important:– Full, Print, Voice > None
• Given guidance:– If speech medium is most effective
• Full, Voice > Print
– If text medium is most effective• Print > Full, Voice
– If the agent/persona effect is most effective• Full > Voice
– If the agent/persona effect is distracting• Voice > Full
![Page 18: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/18.jpg)
Experiment (Acquisition phase)
• Introduction - Flash intro used to hook the user and acquaint them with the system.
• Lessons – 4 example cases used to teach/test the seven critical issues.
![Page 19: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/19.jpg)
Lessons 1
![Page 20: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/20.jpg)
Lessons 2
![Page 21: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/21.jpg)
Lessons 3
![Page 22: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/22.jpg)
Experiment (Acquisition phase)
• Introduction – Flash intro used to hook the user and acquaint them with the system.
• Lessons – 4 example cases used to teach/test the seven critical issues.
• Search Task – Users answer 4 specific questions, designed to use different modules in the system (specifically the query documents module).
![Page 23: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/23.jpg)
Search IRB Documents
![Page 24: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/24.jpg)
Experiment (Test phase)
• Memory – Free and cued recall tested core ideas from the Introduction and Lesson material. A Cloze procedure was used to test memory for key words.
• Issue Comprehension – As a transfer test from the Lesson module, users read two sample cases and provided respective ratings of problematic issues.
• Perception Ratings – Users provided ratings on the system (e.g. “You learned a lot about human subjects protections.”).
![Page 25: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/25.jpg)
Outline• HURAA
• Experiment
• Results
• Conclusions
![Page 26: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/26.jpg)
Results (Core ideas)
0
0.2
0.4
0.6
0.8
Free recall Cued recall Cloze recall
FullVoicePrintNone
Table 1. Proportion correct
![Page 27: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/27.jpg)
Results (Problematic issues)
0
0.2
0.4
0.6
0.8
Hits False alarms d’ score
FullVoicePrintNone
Table 2. Proportion correct
![Page 28: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/28.jpg)
Results (Search task)
0
0.2
0.4
0.6
0.8
Full Voice Print None
Table 3. Proportion correct
Table 4. Completion Time (min)
20
22
24
26
28
30
Full Voice Print None
![Page 29: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/29.jpg)
Results (Time on task)
0
5
10
15
20
25
30
Introduction Issues Search
FullVoicePrintNone
Table 5. Mean minutes on task
![Page 30: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/30.jpg)
Results (Perception ratings)
0
1
2
3
4
5
6
Amountlearned
Interest Enjoyment Ease oflearning
FullVoicePrintNone
Table 6. Mean ratings (1 to 6, higher is better)
![Page 31: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/31.jpg)
Outline• HURAA
• Experiment
• Results
• Conclusions
![Page 32: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/32.jpg)
Conclusions
• No significant results from any of the dependent measures.
• This null result is incompatible with any of our previously stated predictions.
• A practical implication of the result is that the animated conversational agent did not facilitate learning, usage, or perceptions of the interface.
![Page 33: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/33.jpg)
Conclusions
• Perhaps, due to the well structured nature of the site, a navigational guide was superfluous.
• An agent could possibly provide help in a more complex environment.
• The value of a navigational guide may increase as a function of the complexity, ambiguity, and perplexity of the system.
![Page 34: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/34.jpg)
Conclusions
• Animated conversational agents have proven to be effective when they deliver information and learning material in monologues and tutorial dialogues.
• Perhaps there are special conditions when a navigational guide of some form will be helpful, whether it be print, voice, or a talking head.
• However, these precise conditions have yet to be discovered and precisely specified in the literature.
![Page 35: The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site Art Graesser, Tanner Jackson, Matthew Ventura,](https://reader030.fdocuments.in/reader030/viewer/2022032517/56649cad5503460f9496f0f3/html5/thumbnails/35.jpg)
The Impact of Conversational Navigational Guides on the
Learning, Use, and Perceptions of Users of a Web Site
Art Graesser, Tanner Jackson, Matthew Ventura, James Mueller, Xiangen Hu, and Natalie Person
This research was directly supported by contracts from ONR and IDA, and was partially supported from grants by NSF and the DoD.