Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of...
-
Upload
jane-alexander -
Category
Technology
-
view
476 -
download
0
description
Transcript of Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of...
![Page 1: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/1.jpg)
Evaluating Digital Technologies
Elizabeth Bolander Meghan Stockdale
![Page 2: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/2.jpg)
Research Questions
• What do visitors think about Gallery One and ArtLens?
• How does Gallery One and ArtLens affect the museum visit experience?
![Page 3: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/3.jpg)
Process
• Literature review conducted
• Iterative process throughout the planning and development
– User testing (usability, content, tone)
– Audience panels
• Ultimately realized we had many different needs that required different approaches
![Page 4: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/4.jpg)
Mixed Methods Approach
• Usability tests of all interactives
– 30 tests per lens, Collection Wall, and ArtLens
• Observations
– Timing and tracking maps recorded for 200 visitors
• Skype conversations with off-site users of ArtLens
![Page 5: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/5.jpg)
Mixed Methods Approach
• Comprehensive in-depth analysis
– Visitor groups placed into one of four test groups
– Pre- and post-visit interviews
• Range of topics explored
– Visit videotaped with flip cameras
• Usually about 2 hours of video was recorded per visit
– Phone interviews completed 2-3 months later
• Testing re-use, memory, perception changes, etc.
![Page 6: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/6.jpg)
Mixed Methods Approach
• User data from Lens stations and ArtLens tracked via Google Analytics
– More difficult to track the Lenses than ArtLens
• Very valuable counterpoint to qualitative data
![Page 7: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/7.jpg)
Research in Action
• Visitor wayfinding concerns
![Page 8: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/8.jpg)
Research in Action
• App labels on walls
![Page 9: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/9.jpg)
Research in Action
• Off-site versus on-site needs
– On-site has the benefit of the actual space
– Off-site users looking for additional contextual information
![Page 10: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/10.jpg)
Strengths and Challenges
• Significant amount of deep, nuanced data
– Learning more than just the impact of technologies, but a comprehensive look at the entire experience
• Data tracking and management can be difficult
• Analysis of overall project and preparing reports for a diverse array of internal teams with different priorities
![Page 11: Exploring Evaluation Methods for Digital Technologies – Elizabeth Bolander, Director of Communications and Research and Meghan Stockdale, Audience Research Associate](https://reader036.fdocuments.in/reader036/viewer/2022081907/547db0005806b5e55e8b45dc/html5/thumbnails/11.jpg)
Let’s Chat!
• What technologies do your museums use? How do you evaluate them?
• What challenges have you experienced? How did you handle them?
• What can we do together to create a community of practice around these types of projects?