AWS re:Invent 2016: Getting to Ground Truth with Amazon Mechanical Turk (MAC201)
Mac201 audience research and the industry
-
Upload
rob-jewitt -
Category
Education
-
view
1.327 -
download
0
description
Transcript of Mac201 audience research and the industry
MAC201Audience research and industry approaches
1
• Research how each of the following media sectors measure audience feedback/responses:
– Which institutional bodies conduct the research?
– How do they solicit responses?
– How frequently do they do it?
– Are there things they are overlooking or could be doing better?
– What challenges or opportunities does digital consumption create?
• Audience research gravitates towards being a type of ‘survey’
• Like any survey, it is commissioned with a particular objective and set of material interests in mind
3
• Money and metrics
• Data collection is time consuming and difficult to obtain
• Look for technological solutions and quantitative measures
4
• Consider:– Website visitors– Radio listeners– Television viewers– Video game players– Mobile phone users?
5
6
• Primary provider of television audience measurement in the UK.
• Covers all channels broadcasting across all platforms - terrestrial, satellite and cable in both analogue and digital.
• BARB audience measurement data underpins the trading currency for broadcasters, advertisers and their agencies.
9
• Provide audience data to all parts of television industry, as well as to advertisers and media agencies
• Electronic meters installed in home
• Interested in audience size
• Interested in what the audience thinks of programmes
10
• BARB is a non-profit making limited company, funded by the major players:– BBC, – ITV, – Channel 4, – Five, – BSkyB – IPA (Institute of Practitioners in Advertising). – Other broadcasters and a variety of businesses,
• e.g. research specialists, publishers and advertisers also contribute to the cost of running BARB by subscribing to the service.
11
• Annual subscription: £7,320+• Distributed and represented by region• 5,100 homes fitted with device• 26 million TV owning homes in UK
12
• Establishment Survey
• Undertaken continuously• 53,000 interviews per year• Face-to-face interviews
• Methodological triangulation
13
• ‘All panel household residents and their guests need to register their presence when in a room with a television set on. Each individual does this by pressing a button allocated to them on a dedicated handset similar to a remote control. A screen on the front of the meter confirms that they have registered and periodically provides a reminder as to who has registered. Whenever a panel member leaves a room they need to de-register their presence in a similar way. The metering system monitors all registrations made by each individual for each television in the home’– http://www.barb.co.uk/about/tvMeasurement?_s=4
14
• Ratings
• Audience share
• Reach
17
• Ratings– % figure extrapolated to number of viewers
• Audience share– Total of potential audience for timeslot
• Reach– The amount of viewers who tuned in for a
predetermined time
18
• ‘Series reach isn’t readily available and needs to be calculated from the raw data. If the reach is high, it could mean that relatively large numbers loyally watched all the programmes or it could indicate that people have tried the series and rejected it, with new audiences coming in for each new programme in the series’ – Stoessel, 1999: p.257
19
• Advantages?– =– =– =
• Disadvatages?
20
• Advantages?– Impartial (eg advertising)– Necessary – Mathematically robust
• Disadvatages?
21
• Advantages?– Impartial (eg advertising)– Necessary – Mathematically robust
• Disadvatages? – Power dynamics?– New media?– Mathematically robust?
22
• Nieslon, 1989• David Sarnoff: Research
Institute, Princeton• ‘smart sensing’• Face identification
technology• Eye tracking
23
• “The reason was simple: every time the switchover was made, the networks lost millions of dollars in advertising revenue. This was because the diary system worked for the networks. Diaries rely on people's recall, so the networks scored inaccurately high ratings while the smaller cable stations suffered. Once meters accurately recorded what people were actually watching, the result was always the same: networks lost viewers and cable gained.”– MacKenzie, 2000 on the 1980s switch to meters from diaries
24
• ‘shift from the analysis of what texts do to the audience to what texts mean to them’– (Ruddock, 2001: 116)
25
• What can BARB tell us about the meanings we attach to our consumption?
• How might other approaches to audience research offer different data?
• What barriers are there to audience research into media consumption?
26
• “the object [in this case knowledge about the audience] does not await in limbo the order that will free it and enable it to become embodied in a visible and prolix objectivity; it does not pre-exist itself, held back by some obstacle at the first edges of light. It exists under the positive conditions of a complex group of relations” – Foucault, 1969, The Archaeology of Knowledge
27
• To gauge quality?
28
• To gauge quality?– Hardly
29
• To gauge quality?– Hardly
• Gather advertising revenue?
30
• To gauge quality?– Hardly
• Gather advertising revenue?– That’s more like it
31
• Ratings are:– “a techno-social mechanism that produces things
routinely agreed upon and (almost) never questioned.
32
• Ratings are:– “a techno-social mechanism that produces things
routinely agreed upon and (almost) never questioned.
– Opening this black box might provide us with valuable insight into contemporary culture, the way it represents its audiences, and the way legitimacy is conferred (or not) upon specific cultural artifacts, especially through quantification”
• (Jérôme Bourdon and Cécile Méadel, 2011: 792)
33
• Jérôme Bourdon and Cécile Méadel (2011) “Inside television audience measurement: Deconstructing the ratings machine”Media Culture Society 2011 33: 791
34
• Consider:– Website visitors – Newspapers– Advertising– Radio listeners– Television viewers– Video game players– Mobile phone users?
– Web analytics– N.R.S– ABC/ABCe– RAJAR– BARB– Sales???– Network usage???
35
• Eye-tracking experiments– Usability studies– UX (User eXperience)– Heatmaps
37
• Google’s eye-tracking experiment:– Needs to combine what users
are actually doing and feeling with the eye-tracking data reports.
– Data is just data unless it is meaningful and informative.
38
39
Click-through rates are now averaging less than 0.1%
66% of attention on a normal media page is spent below the fold
• Emotional investment in media• Meta-textual activity• Two screen viewing
40
• Social TV: one screen good, two screens better (Thinkbox)
• 80% of 16-34s engage in online chat when watching TV (Thinkbox)
• Social media now driving live TV viewing
41
• Media consumption is more than just mere exposure and draws on a diverse range of social and symbolic capital.
• Consider what happens when much-loved, but little-viewed, TV series get threatened with cancellation…
42
• NBC’s Emmy Award winning US comedy set in Greendale Community College– Season 1 average viewers = 5.0 million– Season 2 average viewers = 4.48 million
– Season 3 average viewers = 3.65 million
43
• Threat of cancellation led to a number of meta-textual campaigns taking place across a several social media platforms in an attempt to save the show
44
48
49
50
51
• Fans are a powerless elite, structurally situated between producers they have little control over the ‘wider public’ whose continued following of the show can never be assured, but on whom the show’s survival depends– Tulloch, 1995: 145
52
• Farscape cancellation (2003)
53
• Commissioned series 4 and 5
• End of series 4 – abrupt cancellation
• Brascape – female viewers to send in bras to Sci Fi Channel Executive Bonnie Hammer
54
• Farscape cancellation (2003)• http://makikosab.blogspot.com/
2002_09_15_archive.html#81859040 • Fans watching in ‘gaggles’• Nielsen ratings• David Kempster: “So six people somewhere
stopped watching or went somewhere else and we’re no longer a viable show”
55
Cancellation threats
• Star Trek (1964)• Family Guy (2002)• Firefly (2002)• Farscape (2003)• Futurama (2003)• Dollhouse (2009)• Flashforward (2010)• Heroes (2010)• V (2011)
56
Cancellation threats
• Star Trek (1964)• Family Guy (2002)• Firefly (2002)• Farscape (2003)• Futurama (2003)• Dollhouse (2009)• Flashforward (2010)• Heroes (2010)• V (2011)
Resurrections
• Star Trek (1965)• Family Guy (2005)• Serenity (2005)• ?• Futurama (2010)
57
58
59
60
61
• Traditional industry research into audiences was typically interested in size and scale as part of a commercial relationship
• Not so concerned with the meanings attached• However, new and varied approaches as well as
the contribution of real time social-media monitoring (RTSMM) are challenging former assumptions.
• The latter (RTSMM) is also difficult to measure and difficult to make sense of.
62
• Research how each of the following media sectors measure audience feedback/responses:
– Which institutional bodies conduct the research?
– How do they solicit responses?
– How frequently do they do it?
– Are there things they are overlooking or could be doing better?
– What challenges or opportunities does digital consumption create?
• How do ratings prove useful, both from our position as scholars and from the position of programmers and advertisers?
• Think of some circumstances in which ratings might give a misleading or incomplete interpretation of a product’s success (Cult TV, slow starters etc)
64
• You are the social media community manager for the University of Sunderland. You are responsible for managing the Media Dept’s digital presence.– Identify ways in which to increase student
engagement with extracurricular events– How might you improve the gathering of student
feedback (both annually and continually)?– Establish a strategy for ensuring the feedback you
receive is representative65
• Given the problems facing quantitative-based industry studies, try and identify a methodology to gauge the response and reactions of audiences for the following media:– Video games (portable and/or fixed)– Magazines (physical and/or digital)– Smart phones– Tablet computer devices
66