Diane Rasmussen Neal, PhD Assistant Professor Faculty of Information and Media Studies The University of Western Ontario ASIS&T 2011 Panel Social tagging & folksonomies: indexing, retrieving… and beyond?

“emotion [...] is too complex to be easily parsed.”

(The New Yorker music critic Alex Ross, as cited by E. Reynolds, Oct. 3, 2010)

Dissertation (2006) on photojournalism retrieval; “user-assigned descriptors” central

Why should we be limited to finding non-text

online content based on only subject access or bibliographic description?

Advocating a shift from just retrieval to include

exploration

Emotions are especially individual and subjective!

Conclusion: Web 2.0 approach to describing emotion in music is necessary because it is difficult to agree

The Four Seasons (Spring) was the only piece that demonstrated a high level of agreement among participants

higher level of agreement than in Lee & Neal (2007); inverse power law theory held true; but multiple interpretations are still possible

qualitative analysis of author reactions to top last.fm songs with emotion-based tags, in conjunction with music facets from Downie (2003)

highest level of agreement among Happy songs similar patterns in the facets for each emotion: Bernstein’s (1976) claim re:

universality of music?

top-tagged track for each emotion: anger: Alanis Morissette, “You oughta know”

fear: Disturbed, “Fear” sad: Gary Jules, “Mad world” happy: Mika, “Grace Kelly”

disgust (hate): Three Days Grace, “I hate everything about you”

“There are instances where a message expressed in one medium cannot adequately

be transposed to another.” (Svenonius, 1994)

“From the moment of the appearance of the book, the linking of text and image is

frequent…” (Barthes, 1964)

“Pictures are not words, and words are not pictures.”

(O’Connor & Wyatt, 2004)

http://www.flickr.com/photos/shawnxian/322381110/

Visual themes Textual themes

Facial expression Storytelling

Color Jokes

Light contrast Inside story

Symbolic Text-as-image

Inanimate observation

Antithesis

Action Personal opinion

Social norms

H0 was rejected for all three hypotheses, frequently with p=.000

“Happy” pictures are by far the most visited

http://www.visualthesaurus.com

- simple random sample of 100 flickr photos tagged with “happy” - content analysis of photos based on themes identified in CJILS study

"I love that book; it made me cry… I should watch a film tonight."

What is the role of affect-laden comments, tags,

and reviews in cultural product consumption? How is Canadian content perceived globally? How could we develop affect-based applications to

explore and relate online content in new ways?

The researchers acknowledge the support of the Social Sciences and Humanities Research Council of Canada.

Diane Neal

Erica Lenton Pam Saliba Rob Foster

Nadine Desrochers