A conceptual model for the annotation of audiovisual heritage in a media studies context
-
Upload
liliana-m-melgar-estrada -
Category
Education
-
view
110 -
download
0
Transcript of A conceptual model for the annotation of audiovisual heritage in a media studies context
![Page 1: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/1.jpg)
A conceptual model for the annotation of
audiovisual heritage in a media studies context
Liliana Melgar, Marijn Koolen, Jaap Blom, Eva Baaren, Roeland Ordelman
Workshop “Audiovisual Data And Digital Scholarship: Towards Multimodal Literacy”
Digital Humanities
Krakow, 2016
![Page 2: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/2.jpg)
What do “wish lists” and research have in common?
![Page 3: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/3.jpg)
“Digital humanists are
motivated annotators” (Walkowsky & Barker, 2014)
Annotating is one of
the so-called
“scholarly primitives” (Unsworth, 2000)
![Page 4: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/4.jpg)
A (oral) historian’s case
How cultural
activism is used
in the claim for
identity of “x”
ethnic group?
© AAC Library
© southernfoodways.org
...“My focus is on the
representations of the past as
acts of storytelling (or personal
“narratives”)”
![Page 5: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/5.jpg)
The “Bricolage” of qualitative data analysis
![Page 6: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/6.jpg)
Buster Keaton. 'Sherlock Jr.' (1924)
© Tumblr.com
“Moving images are a blind medium”(Sandom & Enser, 2001)
Added challenges of AV media
![Page 7: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/7.jpg)
Analysis of our research use cases
Media
aesthetics
Media
representations
Cross-media
analysis
Social history of
the media
Social and
cultural history
Fan culture
studies
Genre studies
Discursive
representations
(media linguistics)
Visual
representations
Public debates
War and
conflict
Journalism
(data-driven,
narrative)
Postcolonial
studies
Migration
Desmet case
Reality tv
Memories from
the Moroccan-
Berber activists in
NL/FL
Muslims/islam
representations
Amateur
film/video
Youtube
broadcasts
World War IIIndonesian
decolonisation
Public contention
Refugee crisis
![Page 8: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/8.jpg)
RQ:
How to provide
system support
for manual
annotation tasks
of AV media in
the context of
Digital
Humanities
infrastructures?
Requirements
analysis
Research use
cases
![Page 9: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/9.jpg)
Method
Use case / requirements analysis: one to one interviews with media scholars
Literature review
Analysis of the data models, functionalities and interface features of current
tools that support video annotation
CAQDAS VIDEO ANALYSISPROFESSIONAL
VIDEO EDITORS
DOMAIN SPECIFIC
APPLICATIONS
GENERIC WEB AV
ANNOTATION
➔ Transana
➔ Nvivo
➔ Atlas.ti
➔ Anvil
➔ Advene
➔ Elan
➔ Lignes du temp
➔ Final Cut Pro
➔ Adobe’s
Premiere ProCC
➔ Cinemetrics
➔ Linked TV
➔ ArtTube
➔ Youtube
➔ Synote
![Page 10: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/10.jpg)
Preliminary Outcomes
Study of the information
annotating behavior of
media scholars
Analysis of existing tools for
video annotation
A process model
(use case-driven)
AV media-centered data
model
A concept model
![Page 11: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/11.jpg)
Study of the information
annotating behavior of
media scholars
Analysis of existing tools for
video annotation
A process model of
annotation in a research
workflow (use case-
driven)
A media-centered model
of AV annotation
(data-oriented)
A concept -integrative
model of annotation
Preliminary Outcomes
![Page 12: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/12.jpg)
Conceptual
model
![Page 13: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/13.jpg)
Object:
Media
Document
Actor
Annotation
(output)
Process
Task
Motivation
Context
Conceptual
model,
dimensions
![Page 14: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/14.jpg)
![Page 15: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/15.jpg)
Scholar/
(oral)
historian
Thematization
ManualInterpreting
Themes / Narratives
Oral history interviews
Academic research
![Page 16: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/16.jpg)
Current Open
Annotation Data Model
Current Open Annotation Data
Model for scholarly research
The Open Annotation
Collaboration (OAC)
Project (Hunter et al., 2010)
Adds context and agent = Actor
![Page 17: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/17.jpg)
Method, task, and purpose (in stages)
![Page 18: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/18.jpg)
Previous work has identified stages in the
scholarly research process
Bron et al., 2012. “Overview of the phases in the media studies
research cycle with associated search processes and changes in
the research question (RQ). Arrows indicate possible sequences.”
Traditional scholarly
research process in
the previous use
cases include an
“analysis” phase,
which is mostly done
through manual
annotations, named
as:
-Coding (Grounded
theory analysis)
-Thematization
(narratives)
-Content analysis
Previous work has also identified stages in search process
(Huurdeman & Kamps, 2014; Koolen…)
![Page 19: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/19.jpg)
?RQ
-Storytelling
-Reporting
Exploratory data
analysis
Data
preparationData enrichment
Explore (other)
datasets
Research phases & annotation process
Pre-focused annotation➔ Bookmarking
➔ Open coding (initial coding)-
-tagging
➔ Commenting (memos,
tasks)
Focused annotation➔ Focused coding
➔ Classification/
Thematization /
Narrative
➔ Codebook
➔ Commenting
(analytical memos)
Annotation preparation
➔ Defining coding
schemes (layers)
➔ Select data
enrichment services
per layer
Based on studies about research stages (Bron et al., 2015), search stages (Huurdeman & Kamps, 2014, Koolen et al., 2015); Qualitative data analysis theory (Charmaz, 2006); Concept of
annotation (Agosti et al., 2012; Melgar, 2016); CLARIAH use cases (interviews with media scholars) and discussions during CLARIAH WP5 requirement analysis.
Corpus selection Corpus analysis & enrichment
ExplorationContextualization
(Assembling)PresentationAnalysis
![Page 20: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/20.jpg)
A (oral) historian’s case
How cultural
activism is used
in the claim for
identity of “x”
ethnic group?
Data
preparationExplore
(other)
datasets
Data enrichment
Exploratory data
analysis
Focused
annotation
Corpus selection Pre-Focused
annotation
![Page 21: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/21.jpg)
Conclusion and future work
• We need to approach “annotation” in a broader scope
• We have come up with three models for annotation:
• A conceptual model of annotation
• A process model(s)
• A media-centered, data-oriented model
• Future work includes
• Implementation of our model in the CLARIAH services
• Evaluation with media scholars
• New requirements and further development for each
research use case
![Page 22: A conceptual model for the annotation of audiovisual heritage in a media studies context](https://reader031.fdocuments.in/reader031/viewer/2022030402/587f9acb1a28ab825e8b4db5/html5/thumbnails/22.jpg)
References
Bron, Marc, Jasmijn van Gorp, and Maarten de Rijke. “Media Studies Research in the Data-Driven Age: How Research Questions
Evolve.” Journal of the Association for Information Science and Technology, 2015, n/a-n/a. doi:10.1002/asi.23458.
Huurdeman, Hugo C., and Jaap Kamps. “From Multistage Information-Seeking Models to Multistage Search Systems.” In Proceedings of
the 5th Information Interaction in Context Symposium, 145–154. IIiX ’14. New York, NY, USA: ACM, 2014.
doi:10.1145/2637002.2637020.
Koolen, Marijn, Toine Bogers, Antal van den Bosch, and Jaap Kamps. “Looking for Books in Social Media: An Analysis of Complex
Search Requests.” In Advances in Information Retrieval - 37th European Conference on IR Research, ECIR 2015, Vienna, Austria,
March 29 - April 2, 2015. Proceedings, edited by Allan Hanbury, Gabriella Kazai, Andreas Rauber, and Norbert Fuhr, 9022:184–
196. Lecture Notes in Computer Science, 2015. doi:10.1007/978-3-319-16354-3_19.
Sandom, Christine, and P.G.B. Enser. “VIRAMI: Visual Information Retrieval for Archival Moving Imagery.” Milano, Italy: Archives &
Museum Informatics, 2001. http://www.archimuse.com/publishing/ichim01_vol1/sandom.pdf.
Unsworth, John. “Scholarly Primitives: What Methods Do Humanities Researchers Have in Common, and How Might Our Tools Reflect
This?” London: King’s Collegue, 2000. http://people.brandeis.edu/~unsworth/Kings.5-00/primitives.html.
Walkowski, Niels-Oliver, and Elton T.E. Barker. “Digital Humanists Are Motivated Annotators.” Laussane, Switzerland, 2014.
http://dharchive.org/paper/DH2014/Paper-296.xml