The Guide to Researching AudiencesConcise edition

Dr Rachel Quirk, Martin Olver, Dr Max Hammond & Dr Claire Davies

JISC Content April2009

JISC, British Library, BBC, National Health Service, Becta, and Museums, Libraries and Archives Council working together to fully realise the potential of e-content for all users. For more information on the Strategic Content Alliance, please visit:

www.jisc.ac.uk/contentalliance

Produced and funded by JISC

Curtis+Cartwright Consulting LtdMain Office: Surrey Technology Centre, Surrey Research Park, Guildford, Surrey GU2 7YG T: +44 (0)1483 685020 E: postmaster@curtiscartwright.co.uk W: www.curtiscartwright.co.uk

This document is available in alternative formats which can be found at: www.jisc.ac.uk/contentalliance

Contents PAGE 3

ContentsOverviewofthisguide 4

1. Introduction 51.1 Theconciseedition 51.2 What,why,who…? 51.3 Usingthisconciseedition 71.4 Supportingdocuments 7

2. Describing and defining the target audience 92.1 Theimportanceofthetargetaudience 92.2 Theconceptofthetargetaudiencein

servicedevelopment 102.3 Definingthetargetaudience 102.4 Describingthetargetaudience 11

3. Planning audience research 123.1 Introduction 123.2 Objectivesofaudienceanalysis 123.3 Choosingthemethodology 153.4 Researchingnon-usage 163.5 Recruitingsubjectsforaudienceresearch 163.6 Othersourcesofaudiencedata 17

4. Collecting audience data 184.1 Undertakingresearch:in-houseorthirdparty? 184.2 Overviewofdatacollectionmethods 19

5. Modelling the audience 225.1 Introduction 225.2 Analysingquantitativedata 235.3 Makinguseofqualitativedata 235.4 Personasanduserprofiles 24

6. Making use of audience research 256.1 Introduction 256.2 Audienceresearchincontext 26

Glossary 27

The Guide to Researching AudiencesConciseedition

PAGE 4 Overview of this guide

Overview of this guide

PLANNING AUDIENCE RESEARCH

COLLECTION OF AUDIENCE

DATA

MODELLING THE AUDIENCE

MAKING USE OF AUDIENCE

RESEARCH

Section 3

Setting clear objectives for research activities, and planning activities to meet these objectives

Section 5

Analysing and interpreting the data collected to build up a picture of the audience

Section 4

The methods available for collecting audience data, and deciding how to conduct the research

Section 6

Using the research to meet the needs, wants and expectations of the audience

DESCRIBING AND DEFINING THE TARGET

AUDIENCE

Section 2

Defining who you would like your audience to be,and how to approach segmenting the audience

GLOSSARY

INTRODUCTION

SUPPORTING INFORMATIONCORE GUIDANCE

Section 1

An introduction to the toolkit, including an overview of how it might be used in practice

Annex A

A glossary of the key terms used in the toolkit (eg ethnographic research)

1. Introduction PAGE 5

1. Introduction

1.1 The concise edition1.1.1 Thisisthe‘conciseedition’of‘TheGuidetoResearchingAudiences’(‘theGuide’).1Itisashorterversion

ofthemainGuide,designedtoprovideaneasilyaccessiblesummaryofthekeyprinciplesofaudienceresearchandsomepracticalinformation.

1.1.2 ThefullversionoftheGuidecontainsmoredetailedinformationthanthisconciseedition,buttheyarelaidoutinexactlythesamewaytomakeiteasyforyoutofollow.

1.1.3 Thisconciseeditionalsoincludessnapshotsofthecasestudieswhichsupport‘TheGuidetoResearchingAudiences’(seesub-section1.4below).

1.1.4 TheGuideandsupportingdocumentshavebeenproducedbyCurtis+CartwrightConsultingLtdonbehalfoftheStrategicContentAlliance.

1.2 What, why, who…?

What is in The Guide to Researching Audiences?1.2.1 TheGuidesetsoutthebasicprinciplesofaudienceresearch.Thesecanbefollowedregardlessofthetype

ofserviceoraudience,andwillhelpyoutoconductaudienceresearchmoreeffectively(betterresults)andefficiently(lowereffort),withfewerproblemsandunforeseencomplications.Theyprovidethebuildingblockstoenableyoutodesign,conductandapplyyourownaudienceanalysisresearch.WhatthisGuidewillnotprovideyouwithisaready-madeaudienceanalysisprogrammespecificallydesignedforyourservice.

1.2.2 Itisaguidetocurrentgoodpracticeandastartingpointforfurtherreading.ThereisnothingveryradicalintheGuide–therearemanyotherresourcesavailableonaudienceanalysisandmodelling,someofwhicharereferencedinAnnex D (Bibliography) of the full version of the Guide.

1.2.3 TheGuidehasaspecificfocusononlinedigitalservices.Thedigitalrevolutionhasresultedinanexplosionofpossibilities:newandinnovativeservices;adelugeofcontent;availabilityofadditionalresearchtechniques(egwebstatistics);andpotentialnewaudiences.Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesandnon-digitalservices,andbetweena‘digitalaudience’anda‘non-digitalaudience’areanewandinterestingconsiderationforserviceprovidersandarediscussedinAnnex C (Digital Audiences and Digital Services) of the full version of the Guide.

1 The Guide to Researching Audiences, Curtis+Cartwright Consulting Ltd, v1.0, 5 December 2008.

The Guide to Researching AudiencesConciseedition

PAGE 6 1. Introduction

Why do audience research?1.2.4 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstratethatthey

areusedandvaluedbyanappropriateaudience.TheusesofaudienceresearcharediscussedindetailatAnnex B (Making Use of Audience Research) of the full version of the Guide,andextendfarbeyondsimplydemonstratingacertainnumberofvisitorsorwebsitehitstotickaboxonaservicelevelagreement.Audienceresearchcanbeusedto:

�� Developaproductorservice

�� Evaluatetheservice(egaudiencesatisfaction)

�� Demonstrateaccountabilitytofunders(egvalueformoney,highlyvaluedbyanicheaudience)

�� Informlong-termplanning

�� Attractsponsorshipandmarketing

�� Informbusinessplanning(egtofeedintoabusinesscase)

�� Informthemanagementofcontractualrelationships

�� Supportprofessionaldevelopment

Case study extract… Design of the BBC iPlayer2

Using audience research to design, develop and test a new version of a service

In2008theBBCdecidedtolauncharedesignedversionoftheiriPlayer,tobeknownasBBCiPlayer2,inordertointegratethedeliveryofon-demandTVandradio.Thisredesignreliedonextensiveuserengagementtogaugeuserperceptionsandattitudes–itwasimportanttocheckthatexistingusersofbothserviceswouldnotreactnegativelytothenewcombinedservice.ItwasalsoimportanttofindouthowperceptionsofthedesignofiPlayer2mightaffecttheaudience’sattitudestowardstheBBCasanorganisationandcontentproviderinthe21stcenturydigitalworld.

Separateresearchprojectswereconductedtoinvestigatedesignandusability.Twoqualitativemethodswerechosentoinvestigateaudienceperceptionsofthedesign–moderated‘audiencelabs’atwhichparticipantswereabletotryouttheiPlayer2andsharetheirresponsesandreactionswithotherparticipants,andin-depthindividualinterviews.ParticipantswererecruitedbasedontheirTVandradioconsumptionhabits(egTVonly),withanemphasisonensuringthatradiousersreflectedthefullspectrumofgenres(music,drama/entertainment,comedy,news,documentariesetc).

Whilstanumberof‘radioonly’userswereconcernedabouttheimplicationsthesedesignfeatureshadforthefunctionalityandservicethatradiolistenerswouldreceive,theresponsetothedesignwasgenerallypositive(egthegenerallookofthehomepagewaswellreceived:userslikedthe‘cinematic’feelofthewidescreen).

TheBBCisnowcollectingandanalysingdatafromserverlogfiles,BBCmessageboards,blogshostedelsewhereontheinternet,andisusingaudiencesurveystomonitorthesuccessoftheiPlayer2.Thisdatawillbeusedtoinformfurtherservicedevelopment.

1.2.5 Audienceresearchshouldbeseenasanongoingprocessratherthananoccasional,one-offevent.Evenasmallaudienceresearchprojectisworthwhile.Manytechniquescanbeimplementedquitecheaplyoradaptedtoashoestringbudget.Afrequentlyexpressedconcernisthatapieceofaudienceresearchwon’tbeanyusebecauseitisn’tgoodenough.Someinsightintoyouraudienceisvastlybetterthannoneatallandtheseconcernsshouldnotdeteryoufromundertakingaudienceanalysis.

1.2.6 AglossaryofthekeytermsusedintheGuideisprovidedatAnnex A.

Who is the Guide for?1.2.7 TheGuidehasbeenwrittenforpeopleintheUKpublicsectordeliveringonlinedigitalservices,whowishto

researchtheiraudiences.2

2 Although it is focused on online digital services (ie those accessed via the internet), it will also be useful for researching audiences reached via other channels.

The Guide to Researching AudiencesConciseedition

1. Introduction PAGE 7

1.2.8 Itisprimarilyaimedatnon-expertswhodonothavemarketresearchskillsthemselves,andwhodonothaveaccesstodedicatedmarketresearchteams.Itisthereforelikelytobemostusefulforpeopleinsmallorganisations/services,orthoseconductingsmallprojectsinlargerorganisations.Itmayalsobeusefulforpeopleprocuringserviceswhowanttoknowwhatguidelinestoprovide,andforthosesettingpolicies.

1.2.9 Audienceanalysisresearchshouldbedonewithcommitmentandsupportfromseniormanagement–withoutit,thefullbenefitsoftheresearcharelesslikelytoberealised.Shoulditberequired,thisGuideprovidesyouwithsomeofthebasicinformationneededtowriteaninformalbusinesscaseforaudienceresearchtofacilitatebuy-infromseniormanagement.

1.3 Using this concise edition1.3.1 Thebasicprinciplesarestructuredasstep-by-stepactivitiesinthelifecycleofaudienceresearchandare

representedinFigure1-1onthefollowingpage.3

1.3.2 Howyouchoosetousetheguidancetoplanandconductaprogrammeofaudienceresearchwillbeinfluencedbymanyfactors,includingthematurityoftheservice,thebudgetavailable,theamountofdigitalcontentheldandthebreadthofthetargetaudience.

1.4 Supporting documents1.4.1 TheGuideissupportedbyaseparatedocumentwhichprovidesreal-world case studiesdemonstrating

goodpracticeinaudienceresearch.4Thecasestudiesarefromboththepublicandprivatesectors,andboththeUKandabroad,andareusefultogainapracticalunderstandingofwhatcanbeachievedwhenaudienceanalysisisdonewell.

1.4.2 Anillustrative case studypresentingahypotheticalprogrammeofaudienceresearchtoputthisGuideincontextisavailableasaseparatedocument.5

1.4.3 Anumberofsector-specificandrole-specificbriefingpapersonaudienceresearcharealsoavailablefor:

�� Peoplesettingpolicies

�� Peoplemanagingprogrammesofresearchandservices

�� Peopleprocuringdigitalservicesintheculturalsector

�� Peopleprocuringdigitalservicesintheeducationsector

�� Peopleprocuringdigitalservicesinthehealthsector

�� Peopleexperimentingwithdigitalmedia

3 Note that although these activities are presented as separate activities in a linear process, in practice this is not likely to be the case – it is likely to be an iterative process.

4 The Guide to Researching Audiences: case studies, Curtis+Cartwright Consulting Ltd. v1.0. 5 December 2008.5 The Guide to Researching Audiences: illustrative case study. Curtis+Cartwright Consulting Ltd. v1.0. 5 December 2008.

The Guide to Researching AudiencesConciseedition

PAGE 8 1. Introduction

Figure 1‑1: an outline of the step‑by‑step activities, and structure and content of the Guide

SECTION 3

Planning audience research

SECTION 4

Collection of audience data

SECTION 5

Modelling the audience

SECTION 6

Making use of audience research

SECTION 2

Describing and defining the target

audience

This discusses the importance of knowing who you would like your audience to be, and provides guidance about how to describe and segment it. It will be especially relevant if you are currently designing a new service. However, it will also be relevant if you would like to start from the basics when planning research activities for services that are already in production and have an established audience.

This is the place to start if you have already defined your target audience and wish to start planning your audience research. It addresses the importance of setting clear objectives for audience research activities, and things to consider when planning activities to meet these objectives. It also sets out example questions about the audience which can drive your research.

This provides an overview of the various audience analysis methods available (eg focus groups, web analytics). It also provides some information about the pros and cons of conducting the research in-house and contracting the work out to a market research company to help you decide the best route for your organisation/service.

This focuses on how to analyse and interpret the data that has been collected to address the specific research questions and build up a better understanding of the audience and their interactions with their service.

This briefly discusses using the information to build the service around the needs, wants and expectations of the audience.

2. Describing and defining the target audience PAGE 9

2. Describing and defining the target audience

This chapter discusses the importance of the target audiences and looks at ways of defining the target audiences, and approaches to audience segmentation

Knowing who you would like your audience to be and what service you intend to offer is a vital part of business planning and service development.

Bothformalandinformalapproachestodefiningthetargetaudiencearepossible.

Segmentingtheaudienceintodifferentcategoriescanbehelpfulwhendescribingthetargetaudience.

2.1 The importance of the target audience2.1.1 Knowingwhoyouwouldlikeyouraudiencetobeandwhatservicesyouintendtoofferisvitalforany

service.Itmayappearobvious,butitshouldbepartofbusinessplanningasthereareimplicationsforfundingandpotentialroutestosustainabilityforadigitalservice.

2.1.2 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,thefunctionalityprovided,meansofaccessetc.Itisimportanttorecognisethataservicemayhavemorethanonedistinctaudience.Needsandexpectationsofthedifferentaudiencesarelikelytobeonlypartlyoverlappingandtheimplicationsthishasforservicedeliveryshouldbeconsidered.

Case study extract… oldbaileyonline

A good illustration of audience-focused service development

AprojectwasfundedtodigitisetheOld Bailey Proceedingstoprovideaccesstothetextandscannedimagesoftrialrecords.Theprojectteamwasdeterminedtocreatearesourcethatwouldreflecttheneedsofthedifferenttargetaudiencesegments,andconsultationswiththetargetaudienceprovidedevidenceforitsbidforfunding,guidedthedesignandfeaturesofthewebsite,andguidedthetestingoftheprototypewebsiteandlaterredesign.

ThetargetaudienceswereidentifiedonthebasisofthecontentoftheProceedingsandtheprojectteam’sprofessionalexperienceofworkingintheLondonArchives,forexample‘communitygroups’wereidentifiedasanimportantaudiencesegment.

Followingconsultationwiththetargetaudience,thedesignspecificationincludedfeaturesaimedatmeetingtheneedsofdifferentaudiencesegments.Forexample,itwasdiscoveredthatsomeofthetargetaudienceswerenotfamiliarwith18thcenturyhistory,andthatinthecommunitygroupssegmentschoolswerelikelytobekeyusers.Accordingly,alotofbackgroundmaterialwasprovidedanddedicated‘ForSchools’pageswereincludedtoprovideguidanceforteachersonwaysofusingtheProceedingsinteaching,andresourcestailoredtoschoolstudentsincludingatimeline,glossary,tasksandprimarysource.

Thisaudience-focusedapproachwassuccessful–thesizeoftheaudienceforoldbaileyonlinehasexceededexpectationsandtheserviceisclearlywidelyusedandappreciated.Forfurtherinformationontheoldbaileyonline,seethewebsite(www.oldbaileyonline.org).

The Guide to Researching AudiencesConciseedition

PAGE 10 2. Describing and defining the target audience

2.1.3 Understandingtheaudiencecanbevitaltogiveasenseoffocusandidentitytoaservice.Tryingtobeallthingstoallpotentialusersisdifficult.Differentgroupsofusersmayhaveincompatibleneeds(egexperiencedacademicresearcherswillprioritiseheavyweightsearchandanalysistoolswhenusinganarchive;leisureuserswillexpectaveryuser-friendlyinterfacethatletsthemquicklyexplorethearchiveandgetaflavourofthecontents).Thisdoesn’tmeanthatadigitisedarchiveshouldn’taimtoappealtobothaudiences,butunderstandingthatthesearedistinctaudienceswithdifferentneedsshouldinformthedesignanddevelopmentoftheservice.

2.2 The concept of the target audience in service development2.2.1 Servicedevelopmentmaybeeitherresource-driven(egbecausesomeonehasagreatidea)oraudience-

driven(egifanorganisationwantstoextenditsaudience).Bothapproachesarevalid,butdetailedservicedevelopmentshouldtakeintoaccounttheintendedaudiencefortheservice.Theservicemustappealtothem,sotheirhabits,needsandexpectationswillhaveanimpactonthenatureoftheserviceandthewaysinwhichitispromoted.

Case study extract… Digitisation of the John Johnson Collection

A good example of engaging the target audience to focus service development

TheJohnJohnsonCollection,housedattheUniversityofOxford,iswidelyrecognisedasoneofthemostimportantcollectionsofprintedephemeraintheworld.Tobroadenaccesstothisarrayofrareoruniquearchivalmaterial,aprojectwasconductedbytheBodleianLibrary(UniversityofOxford)andProQuesttoconserve,catalogueanddigitisemorethan65,000itemsoftheCollection.

TheprojectteamconductedsomeinitialaudienceresearchtosupporttheirsuccessfulbidforfundingthroughtheJISCDigitisationProgramme.ThetargetaudienceforthedigitisedCollectionwasdefinedandtheprojectteamdemonstratedtherelevanceoftheCollectiontotheseaudiences.Forexample,lettersofsupportweresolicitedfrompotentialusersinUKuniversitiesandinternationalacademicswithaninterestinephemera,andtheprojectteaminvestigatedwhichhighereducationcoursemodulesmightusethisephemera.

Toensurethatthedigitisationwouldrealisemaximumbenefitforusers,audienceresearchwasanimportantpartofthedesign,developmentandtestingoftheonlinedigitisedCollection.Forinstance,toassistindecidingwhichephemeratodigitise,statisticsforphysicaluseofthevariouscomponentsoftheCollectionoverthepreviousthreeyearswerecompiled.

SomepartsoftheCollectionhavenowbeendigitisedandarefreelyavailabletoallUKuniversities,furthereducationinstitutions,schoolsandpubliclibraries.Uptakeanduseofthesecollectionsarebeingmonitoredusingfocusgroupsandsurveys(usersatisfaction),webstatistics(useractivity),andassessmentofuser-generatedcontentandtagging(userengagement).Thisresearchformsanimportantpartoftheevaluationstrategyforthisproject.

ForfurtherinformationontheJohnJohnsonCollection,seethewebsite(www.bodley.ox.ac.uk/johnson).SomeareasofthewebsitearerestrictedtomembersoftheHEcommunity.

2.2.2 Understandingthetargetaudiencewillbecriticaltothesuccessoftheservice.Audienceresearchwillfocusonfindingoutthesortsofresourcesandservicesthatappealtothetargetaudience,howtheywouldprefertoaccesstheservice,andthetechnologiestheyuse.Backgroundknowledgeaboutattitudes,lifestyle,socialinteractionsetcisalsolikelytobeusefuliftheresourceistobeprimarilyforleisureuseratherthanforformallearning.

2.3 Defining the target audience2.3.1 Definingyourtargetaudiencedoesn’tneedtobeadifficulttechnicalexercise.Agoodstartingpointmay

betopictureanidealaudiencemember–oratypical,actualaudiencemember.Howwouldyoudescribethisperson?Thedescriptorsyouusearegoodstartingpointsforthinkingaboutcategoriesforaudiencesegmentation.

The Guide to Researching AudiencesConciseedition

2. Describing and defining the target audience PAGE 11

2.3.2 Amoreformalapproachtodefiningthetargetaudiencewouldbetodevelopaseriesofuserprofilesorfullyworkedpersonasasanaidtoservicedesignanddevelopment(seesub-section).Negativeuserdescriptions,profilesorpersonascanalsobehelpful:thekind(s)ofpeopleyourserviceisnotintendedfor.

2.4 Describing the target audience2.4.1 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,thefunctionality

provided,meansofaccessetc.Thefeaturesincludedinadescriptionofthetargetaudiencewillvaryaccordingtotheservicebutmayincluderesidentialaddress,occupation,gender,interests,technologicalexpertiseetc.Atargetaudiencemaybeveryinclusive(egallthepeopleinManchester)orhighlyspecific(egqualifiedmedicalprofessionalswitharesearchinterestinOsgood-Schlatter’sdisease).

Audience segmentation2.4.2 Audiencesareoftensegmentedintodifferentcategories.Publisheddata(egtelevisionviewingfigures,

newspaperreadership)oftensegmentstheaudienceonthebasisofdemographiccategories.Academiclibrariesaremorelikelytocategoriseusersaccordingtotheirrolewithinacademia.Themostappropriatewaytodefineatargetaudienceandsegmenttheactualaudiencewilldependontheserviceandtheobjectivesoftheaudienceanalysiswork,andseveraldifferentapproachestosegmentationarepossible.

PAGE 12 3. Planning audience research

3. Planning audience research

This chapter looks at defining research objectives, selecting and combining appropriate methods, researching non-usage and recruiting subjects

Research need not be perfect to be useful – the key to good audience analysis is setting out clear objectives and planning appropriate research to achieve the objectives. Engaging non-users is important but presents special problems.

Objectivescanbebroadlydividedintotwocategories:assessmentofreachanduptake,andassessmentofqualityandimpact.

Startingpointsforplanningresearchinclude:

�� Definingservicedevelopmentobjectives.

�� Definingquestionsaboutthenatureandbehaviouroftheaudienceandnon-users.

Differentaudienceresearchtechniquesaresuitedtodifferentobjectivesandmethodscanbecombined.

3.1 Introduction3.1.1 Planningaudienceresearchshouldstartwiththebasicquestion:What do you want to know about

your audience?Consideringwhy the knowledge is neededandhow it will be put to useisalsoimportant.Understandingtheaudienceisnotanendinitself,butanintegralpartofdevelopingandmaintainingasuccessfulandsustainableservice.Understandingthedriversfor,andusesof,audienceanalysisworkwillhelptoensurethatitisalignedwithwiderservicevisionandobjectives,andmayhelptoobtainbuy-infromseniormanagement.

3.2 Objectives of audience analysis3.2.1 Thekeytogoodaudienceanalysisissettingoutclearobjectivesandplanningappropriateresearch

activitiestomeettheseobjectives.Thereislittlepointinspendingtime,moneyandstaffeffortcollectingdataunlessitisgoingtobeuseful.

Case study extract… London Museums Hub

A good example of setting clear objectives to focus research activities

TheLondonMuseumsHubisapartnershipoffourofthe‘bigplayer’non-nationalmuseumsinLondon:theMuseumofLondon(leadpartner),theLondonTransportMuseum,theGeffryeMuseumandtheHornimanMuseum.TheHubconductedaresearchprojecttoinvestigatethepotentialforajoined-upapproachtoonlineaudiences.

The Guide to Researching AudiencesConciseedition

3. Planning audience research PAGE 13

Theprojecthadawell-definedbrieffortheresearch,whichstatedthat‘theHublacksacoordinatedstrategicapproachtoqualitativeonlineaudienceprofilingandresearch…weaimtobuildamorecompleteunderstandingoftheHub’sonlineaudiences,andusethisknowledgetotakeamorestrategicuser-focusedapproachtoonlinedelivery’.Thisbriefwasunderpinnedbyspecificresearchquestionsabouttheironlineaudiences:Whoareouraudiences?Whydotheyuseourwebsites?Whataretheirexpectationsofthewebsites?Towhatextentaretheseexpectationscurrentlybeingmet?

Theprojectusedamulti-strandedapproachtoaudienceresearch(egcombiningonlinequestionnaires,face-to-faceinterviews,focusgroupsandwebstatistics)togatherinformationtoanswerthesequestions.Thefindingsfromthisprojectwerethenusedtotakeamorestrategicuser-focusedapproachtoonlinedelivery.Forexample,theresearchconfirmedthattheoverwhelmingmajorityofwebsiteusagewasforplanningavisittoamuseum.However,thewebsitewasnotakeyfactorinthedecisiontovisit–thereputationofthemuseuminaparticularfield,locationetcwerethemainfactors.

Followingthisexploratoryresearch,concretestepshavebeenagreedthatwillenabletheHubtomovetowardsacommonapproachtocollectingdataonhowonlineservicesarereceivedbytheaudienceandthebroaderimpactthatonlineserviceshaveonorganisationsasawhole.FormoreinformationontheLondonMuseumsHub,seetheMLALondonwebsite(www.mlalondon.org.uk).

3.2.2 Audienceanalysiscanbedividedintotwobroad,interdependentcategories:assessmentofreach and uptake,andassessmentofquality and impact.Inplanningaudienceresearchitishelpfultodecidehowimportanteachaspectoftheanalysisis.

3.2.3 Researchobjectivesmustbesetaccordingtoindividualserviceprioritiesandcontext.Startingpointsforidentifyingbroadaimsandframingobjectivesincludeservice development objectives(eghowcanweextendthereachofourservice?)andquestions about the audience and non-users(egwhoareouraudience?howdotheyuseourservice?).SeeFigure3-1andFigure3-2forexamples.

Figure 3‑1: understanding non‑users

Usage

Ex-users

Never

NON-USERS

WHY are they not using our service?

Awareness

Not aware

Aware

Reason

Not interested

Happy with alternative

Too slow

WHY do they no longer use our

service?

WHO are they?Which segments of the target

audience are under-represented?

Target audience – existing audience = non-users

WHAT did they think of our

service?

HOW could we make them more

aware?

HAVE they ever used our service?

WHY do they not use our service?

Reason

Not interested

Happy with alternative

Poor public image

WHY are the alternatives preferred?

Reason

Better quality

Better coverage

Easier to access

Audience satisfaction

Aware but cannot accessWHY can they not

access it?

Reason

Technological

Authentication required

Disabilities

Marketing

The Guide to Researching AudiencesConciseedition

PAGE 14 3. Planning audience research

Figure 3‑2: questions about the audience

WHO

are

they

?

EXIS

TIN

G

AU

DIE

NC

E

WH

Y do

they

use

ou

r ser

vice

?

WH

O is

not

usi

ng

our s

ervi

ce?

DOES

the

serv

ice

mee

t the

ir ne

eds

and

expe

ctat

ions

?

Acce

ss

Hom

ePu

blic

Mob

ile

Freq

uenc

y

Regu

lar

Occ

asio

nal

One

-off

Com

petit

ors

Publ

ic

Priv

ate

Inte

rnat

iona

l

Wha

t for

Con

tact

sCo

llect

ions

How

to fi

nd u

s

Firs

t use

Sear

ch e

ngin

eRe

com

men

datio

n

Ref

errin

g si

te

Subs

eque

nt

use

Dire

ct

Sear

ch e

ngin

e

Ref

errin

g si

teD

irect

Rea

son

Qua

lity

Auth

ority

Com

preh

ensi

ve

Clar

ityNo

alte

rnat

ives

Alte

rnat

ives

Dig

ital

Non-

digi

tal

Dem

ogra

phic

Func

tion

Goa

lTe

chno

logy

Do

mai

n

Age

Gen

der

Soci

oeco

nom

ic

cate

gory

Loca

tion

Prof

essi

onal

Rese

arch

er

Acci

dent

al/

impu

lse

Leis

ure

Brow

sing

Defin

ed

obje

ctiv

eSy

stem

atic

ex

plor

er

Expe

rtC

onfid

ent

Nov

ice

Expe

rtSo

me

know

ledg

eNo

kno

wle

dge

HO

W d

o th

ey u

se

our s

ervi

ce?H

OW

do

they

re

ach

our

serv

ice?

Sepa

rate

dia

gram

Audi

ence

sat

isfa

ctio

n

WH

AT a

re th

e lo

nger

term

effe

cts

of th

e se

rvic

e?

Impa

ct a

naly

sis

Qua

lity

Rel

iabi

lity

Con

veni

ence

Opi

nion

Impo

rtan

ce

The Guide to Researching AudiencesConciseedition

3. Planning audience research PAGE 15

3.3 Choosing the methodology

Different approaches to audience analysis3.3.1 Audienceanalysismethodscanbecategorisedindifferentways,butthedistinctionbetweenquantitative

andqualitativemethodsisanimportantone.Quantitativemethods(egsurveys,webanalytics)areusuallymoreappropriatewhenstatisticallyvalidconclusionsneedtobedrawnaboutthesize,compositionandotherpropertiesofanaudience.Qualitativetechniques(egfocusgroups,one-to-oneinterviews)maybemoreappropriateforexploringattitudesandmotivationsindepth,discoveringpatternsofuserbehaviouranddevelopingarelationshipwithaudiencemembers.

3.3.2 Ethnographicorobservationaltechniques(egdiarystudies,accompaniedbrowsing)areincreasinglyusedforstudyingusabilityandpatternsofserviceuse.Ethnographicdataisoftenverydetailedandmayincludecomputercollecteddatawhichisamenabletostatisticalanalysisaswellasaqualitativerecordofuserbehaviour.

Representative data3.3.3 Samplesaresaidtoberepresentativewhenthecompositionofthesampleaudience–withrespecttoall

relevantattributes–isthesameasthatoftheaudienceasawhole.Insomeinstancesitisimportanttoobtaindatafromarepresentativesampleoftheaudience(egtoassessthelevelofuptake,toprovideevidenceforattainmentofkeyperformanceindicators).However,representativedataisnotalwaysvital,particularlywhenresearchispreliminaryorexploratory.

Selecting appropriate methods3.3.4 Thekindofinformationyouwantabouttheaudienceshouldhelptodeterminethemethodsyouuse,but

pragmaticconsiderations(egtime,money,difficultyofrecruitingsubjects,staffavailabilityandexpertiserequired)willinevitablyplayapart.Itiscommontouseacombinationofmethodsinaudienceanalysisprojects,forexamplequantitativesurveysarefrequentlycombinedwithfocusgrouporinterviewwork.Table3-1providesaguidetothemethodsmostlikelytobeusefulinresearchingdifferentaspectsoftheaudience.

Case study extract… Europeana project

A good example of selecting appropriate research methods

Europeanaisa2-yearprojecttodesignandbuildawebportaltomakeavailabletousersawealthofdigitalobjects(egphotographicimages,sounds,paintings,maps,manuscripts,books)frommuseums,librariesandarchivesacrossdifferentEuropeancountries.

A‘maquette’–ananimateddemonstrationwebsite–wasinitiallydevelopedbasedonuserrequirementsanddeterminedinconsultationwithcontentprovidersandtechnicaldevelopers.Anaudienceresearchprojectwasconductedtotestwhetherthismaquetteliveduptotheexpectationsandwishesofrealend-usersintermsoffunctionality,theinterfaceandnavigation.ThiswasanimportantsteptowardsdevelopingafullyfunctionalprototypeEuropeanasite(launchedinNovember2008).

Althoughtheresponsetothedemonstrationsitewasgenerallypositive,anumberofcommonconcernsemerged.Forexample,itwasfoundthatthehomepagewasnotclearenoughanddidnotgiveagoodintroductiontothesite;thesearchresultspagelackedorderinresults,levelofdetailandpresentation;andtherewasconfusionabouthowthesitecouldbebrowsedandsearched.Theseresultswerefedbacktothetechnicaldevelopmentteamworkingonthefullyfunctionalprototype.Forfurtherinformation,pleaseseetheEuropeanawebsite(www.europeana.eu).

The Guide to Researching AudiencesConciseedition

PAGE 16 3. Planning audience research

Table 3‑1: selecting appropriate methods (Key: ‑ unlikely to be suitable; * may be suitable, but of limited use; ** often suitable)

Asp

ect o

f au

dien

ce

Qua

ntit

ativ

e-su

rvey

Web

sta

tist

ics

Ethn

ogra

phic

st

udie

s

Focu

s gr

oups

Indi

vidu

al

inte

rvie

w

Info

rmal

Oth

er

appr

oach

es

Size ** ** - - - - -

Composition ** * - - - * -

Motivations ** * - ** ** * -

Attitudes ** * - ** ** - -

Satisfaction ** * - ** ** - -

Prevalence of non-usage ** - - * - -

Reasons for non-usage - - - ** ** - -

Patterns of usage - ** ** * * - -

Tasks service is used for ** ** **usertracking

** ** - -

Lifestyle * - ** * ** - -

Service usability * * ** * * - Heuristicevaluation6

User requirements - - ** ** ** * -

3.4 Researching non-usage6

3.4.1 Understandingnon-useofaserviceisimportant:publiclyfundedservicesmayhavearesponsibilitytoserviceaparticularaudience(egseniorcitizens,ethnicminoritycommunities).

3.4.2 Engagingwithnon-usersmaybedifficult,time-consumingandpotentiallyexpensive,butitshouldbeviewedasanecessarypartofaudiencedevelopmentwork.Buildingrelationshipswithnon-userswillpaydividends:7

�� Theviewsofnon-usersarenotrepresentedinusersurveysandthedigitalserviceisnotmeetingtheirneeds;thismaybeanimportantfailing

�� Non-userscanprovidevaluableinsightsforservicedevelopment

�� Engagingnon-userscanhelpdemonstrateacommitmenttothataudiencesegmentandincreaseknowledgeaboutmarketingandcommunicationstrategiesthatwillbeeffectiveforthatsegment

3.4.3 Sub-section3.5ofthefull-versionoftheGuideprovidesmoreinformationontheimportanceof,difficultieswith,andpossiblestrategiesforresearchingnon-users.

3.5 Recruiting subjects for audience research3.5.1 Recruitingsubjectsisaperennialproblemandtheeasiesttechniquemaynotalwaysbethebest,andmay

leadtoanunrepresentative,biasedsample.Incentivescanbeusedtoaidrecruitment,butcareneedstobetakentoensurethattheincentiveisappropriatetotheaudienceyouseektorecruit.

6 Heuristic evaluation is where expert evaluators assess the service’s user interface against recognised usability criteria. This method falls outside the scope of this guide but is described briefly in the full-version of the Guide because of its utility in the early stages of service development.

7 Responses to a survey may not provide very much information. Engaging more deeply with non-users through focus groups, discussion fora and interviewing will be necessary to build a richer, more meaningful understanding of non-usage.

The Guide to Researching AudiencesConciseedition

3. Planning audience research PAGE 17

3.5.2 Recruitmentwillbemoredifficultifabigtimecommitmentisrequiredortheresearchisgoingtobeinconvenientforsubjects.Itisalsoimportanttobearinmindhowyourinformationcollectiontechniqueswillinfluenceresponseratesandtherobustnessofthedatacollected.

3.5.3 Userregistrationcanalsobeexploitedeasilybydigitalservicesforaudienceanalysis.8Thisallowstheservicetoassociateusercharacteristicswithuserbehaviour.Registeredusersmaybemorefrequent,moreloyalormorecommittedtotheserviceandthereforemorewillingtoparticipateinresearch.

3.5.4 Anextensionofthisideaistosetupresearchpanels–asampleofaudiencememberswhohavebeenrecruitedspecificallyforthepurposeofregularly(ieonanongoingbasis)respondingtoqualitativeand/orquantitativesurveys.

3.6 Other sources of audience data3.6.1 Don’tneglectthepossibilityofincreasingyourunderstandingofyouraudiencebylookingatresearchdone

byotherserviceprovidersontheiraudienceorbyusingdatafrompopulationsurveyscarriedoutbymarketresearchorganisations,governmentagenciesoracademicbodiesegtheOxfordInternetInstitutebiannualsurveyofinternetusage.9

3.6.2 Inaddition,exploitinginformalknowledgeaboutyouraudience(egobservationsbystaffwhointeractdirectlywiththeaudience,technicalsupportemails)isasmartthingtodoandcanbeparticularlyvaluableifresourcesforaudienceanalysisarelimited.

8 Subject to the provisions of the data protection legislation. 9 Oxford Internet Institute surveys, www.ox.ac.uk/microsites/oxis

PAGE 18 4. Collecting audience data

4. Collecting audience dataThis chapter provides an overview of different methods for collecting audience data and considers outsourcing audience research

The pros and cons of conducing research in-house and commissioning an independent market research agency to undertake it on your behalf are discussed.

Datacollectionmethodscoveredinclude:

�� Quantitativesurveys

�� Interviews

�� Focusgroups

�� Webstatistics

�� Ethnographic(observational)techniques

4.1 Undertaking research: in-house or third party?4.1.1 Technologicaladvancesmakeitincreasinglytemptingtoconductresearchona‘DIY’basis,ratherthan

commissioninganindependentmarketresearchagencytoundertakeitonyourbehalf.However,itisworthconsideringanumberoffactorsbeforedecidingifthein-houseorcommissionedresearchrouteismostappropriate.

Table 4‑1: critique of approaches to undertaking research

Approach Pros Cons

In-house research

�� Canoftenbetheleastexpensiveoption–especiallyifexpertin-houseresourcesareavailable.�� Knowledgeoftheaudience:expertknowledgeand/orexperiencecanhelpinplanningandconductingresearch.�� Greaterlevelofcontrol:amarketresearchagencymighthavedifferentideasabouthowtoresearchanaudience.

�� Lackofexperienceinconductingresearchcanleadtopoorresearchdesignandotherproblemsthatanexperiencedresearchagencywouldhaveanticipatedandplannedfor.�� Questionnairedesignissurprisinglydifficulttogetright.Poorlydesignedquestionsresultinambiguousormisleadingdata.Experienceofquestionnairedesignisimportant.�� Similarly,unlessknowledgeofstatisticalandsamplingtechniquesisavailablein-house,mistakescanbemade–bothattheplanningandattheanalysisstages.

Commission market research company

�� Ifcarefullyselected,aresearchagencywillbringexpertiseandexperiencetobear,resultinginamoreefficient(possiblyevenmorecost-effective)surveyandgreaterclarityofunderstanding.�� Theinvolvementofarespected,independentthird-partybringscredibilitytotheresearch.�� Respondentsaregenerallymorelikelytorespondopenlyandhonestlytoathirdparty.�� Marketresearchagencieshaveaccesstothespecialistresourcesrequiredtoconductresearch(egrecruitersandsurveytools).

�� Commissionedresearchcanbeexpensive.�� Theprocessofidentifyingpotentialresearchagencies,writingaresearchbriefandbriefingtheselectedagencytakestime.�� Evenmarketresearchagenciesexperiencedinyoursectorwilllackyourdetailedknowledge.Bepreparedtospendtimeensuringthattheagencyunderstandsthenatureofyouraudienceandwhatyouaretryingtofindoutaboutit.

The Guide to Researching AudiencesConciseedition

4. Collecting audience data PAGE 19

4.1.2 Ifyoudodecidetouseamarketresearchcompany,‘TheResearchBuyer’sGuide’10publishedbytheMarketResearchSocietyhassomeusefulinformationbothonthetypeofmarketresearchcompanytochoosefromandalsoonthequestionstobearinmindwhenselectingacompany.Itincludesadviceonwhethertosub-contractsomeelementsofaresearchprojecttoconsultants(egquantitativeinterviewingorrecruitmentoffocusgroups),orbuyinthefullservice(whereprofessionalswillhelptodesigntheresearchandcollect,analyseandevaluatedata).

Case study extract… Teens Music and Technology: BBC research

An example of using a research agency and managing the relationship

TheBBCconductedaprojecttoinvestigatetherolethatmusicplaysinthelivesofyoungpeopleandtheextenttowhichyoungpeopleusetechnologytoaccessandperhapssharemusicwiththeirfriends.ThehopewasthataninsightintothebehavioursofthisagegroupwouldleadtonewideasabouthowtheBBCshouldbedevelopingmobileproductsandservicesforthisaudiencegroup.

AresearchagencywascommissionedbytheBBCtoundertakethiswork.TheBBChadveryspecificrequirementsofthework–itwasimportantthattheprojecthadmusicatitscentreratherthanusingtechnologyasthefocus,theresearchmust‘getbeneaththeskin’ofyoungpeopleandofferarealinsightintothepartmusicplaysintheirlives,andtheoutputneededtobea‘designtool’toinspireandstimulatediscussionandsparkideas.Becauseofthesespecificrequirements,itwasnecessaryfortheBBCtoworkverycloselywiththeresearchagencytoensurethattheresearchmettheirrequirements.Acontinualdialoguewasessential,beginningatthepointthattheideafortheresearchwasconceivedandcontinuingrightthroughtofinalreporting.

Anethnographicapproachwasadoptedasthishasthepotentialtoofferrichdetailanddeepinsightsintothebehaviours,practicesandeverydaylivesofpeople.Theresearchagencydesignedthefinalreporttobevisuallystimulating,fullofimagesofthedifferentlifestylesandinterestsofeachoftheparticipatingteenagersandthegroupstheybelongto,andmostimportantly,adocumentthatinvitedreaderstoimmersethemselvesintheworldoftheseteenagers.Thereportcentredonthepersonasoffourfriendshipgroups,representingdifferentinterests,lifestylesandimportantly,tastesinmusic.Becausetheformatofthereportwascriticaltokick-startdiscussionamongstdesignersandproducers,thisledtocloseconsultationwiththeresearchagencyoniterativedraftsofthereporttoensureitlentitselftothispurpose.

Todate,theoutputoftheresearchhasbeenusedinaninformalway.However,itisplannedthataseriesofcreativeworkshopswillbeorganised,challengingdesignerstoreacttothefindingswithnewideas,solutionstosomeoftheissuesthatyoungpeoplehaveidentified,andgenerallyengageinconstructivedebateaboutthewayforwardfortheBBCinthisarea.

4.2 Overview of data collection methods4.2.1 Informationandacritiqueondifferentmethodsofcollectingaudiencedataissummarisedinthefollowing

table.Morepracticalguidanceforusingthesemethodsisprovidedinthefull-versionoftheGuide.Thesetofmethodscoveredisnotintendedtobedefinitiveanddetailedinformationonhowtoimplementeachmethodisnotprovided.

4.2.2 Mostaudienceanalysismethodscanbeusedoradaptedsuccessfullybynon-specialists;however,therearesomecircumstancesunderwhichitmaybeappropriatetoconsideroutsourcingoneormoreelementsofanaudienceresearchproject(seeabove,section4.1).

10 The Research Buyer’s Guide, UK and Ireland (2007); London: Market Research Society.

The Guide to Researching AudiencesConciseedition

PAGE 20 4. Collecting audience data

Case study extract… Arthritis Source

A good example of using complementary research methods to gather information

ArthritisSourceisaninformationalwebsite,developedandhostedbytheUniversityofWashington,providingaccurateandauthoritativeinformationaboutarthritisandrelatedconditions.Afterbeinginserviceforfouryears,aprojectwasconductedtocharacterisetheaudienceoftheserviceandtoredesignthewebsitetomeettheirneeds.

TheresearchprojectusedthreeresearchmethodstogathercomplementaryinformationabouttheaudienceforArthritisSource.Logfileanalysiswasusedtoexplorepatternsofusage–whichpageswerebeingaccessedmostfrequently,andwhenweretheybeingaccessed?Anonlinesurveyofuserswasconductedtocollectabroadrangeofdataaboutusers–whytheyusedthewebsite,whattheirrolewaswithrespecttoarthritis(egmedicalstudent),demographicsandITexpertise.Telephoneinterviewswerethenconductedwithselectedrespondentsoftheonlinesurveytogathermoredetailedinformationaboutusers’knowledgeofarthritisandtheirinteractionswithArthritisSource.

InformationabouttheiraudiencesandtheuseofresourceswasusedbytheteamattheUniversityofWashingtontodiscoverandprioritiseredesignobjectives.Forexample,theresearchrevealedthatnearly20%ofuserswerebasedoutsideNorthAmerica,sobasicchangesweremadetointernationalisethecontent,egbothgenericandbrandnamesfordrugswereincluded.Furthermore,somecommonmisconceptionsaboutarthritisamongthewebsiteaudiencewererevealed,andanonlinequizaboutcommonmythsandmisconceptionswasdesignedtohelpovercomethesemisconceptions.

AudienceanalysisisnowroutineforArthritisSourceandthedesignistweakedwhenthereisevidenceofachangeinuserneeds,ortotakeadvantageofchangesinthetechnologicallandscape,egthesitenowincludesmorevideoclipsandpodcastsashigh-speedinternetaccesshasbecomemorewidelyavailable.Thewebsite’scommitmenttoassessingandrespondingtoaudienceneedsissignalledtotheaudience:webstatisticsarepubliclyavailableandveryshortonlinequizzeshavebeenplacedonmanyofthepagesaskinguserstoratetheirusefulness.

Forfurtherinformation,seetheArthritisSourcewebsite(www.orthop.washington.edu/uw/tabID__3370/Default.aspx).

Table 4‑2 (part 1 of 2): overview of data collection methods

Method Information

Focus groups Description:Focusgroupscanbeorganisedinmanywaysandinmanyformats.However,allformatsarebasedonthepremiseofasmallgroupofrespondentsdiscussingissuesandbeingguidedbyanexperiencedfacilitator(sometimescalledamoderator).

Uses: Forexploringattitudesandmotivationsandforgeneratingnewideas.Focusgroupsareoftenusedastheprecursortosomeformofquantitativesurveyresearch.

Pros: Usefulforcreativediscussionandnewideas,relativelyshorttimescales,well-suitedtointroducingandgainingreactionstostimulusmaterials.

Cons:Reliesonsufficientnumbersofaudiencebeingclusteredinageographicalarea,respondentsmightnotbepreparedtodiscusspersonally,commerciallyoracademicallysensitivesubjectswithothers,requiresexperiencedfacilitators.

One-to-one in-depth interviews

Description:In-depthinterviewsconductedonaone-to-onebasis,bytelephoneorinperson.

Uses:Forunderstandingtherangeofattitudesandmotivationswhichmightexistwithinanaudience.

Pros:Canbeusedtocollectqualitativeorquantitativedata,goodqualitydata,morein-depthinformationcanbecollected,stimulusmaterialcanbeshown.

Cons:Relativelyexpensive,requireahighlevelofrespondentcooperationandcommitment,time-consuming.

The Guide to Researching AudiencesConciseedition

4. Collecting audience data PAGE 21

Method Information

Quantitative surveys

Description:Quantitativesurveysinvolveusingasampleofanaudience/populationasthebasisforproducingastatisticallyreliablepictureofthewhole.Includespostal,email,telephone,andinternetsurveys.

Uses: Forprovidingquantifiableinformationaboutanaudience.Forexample,itsprofileanditspatternsofusageandbehaviour.

Pros: Varydependingonmethod.Ingeneral:cost-effective,offeralargeproportionofrelevantaudienceachancetorespond,lowlevelofcommitmentfromrespondents,internetandemailsurveyscanprovideveryimmediatefeedback.

Cons:Varydependingonmethod.Ingeneral:sampleisoftenself-selecting,noguaranteesofresponserates,methodshouldbechosencarefully(ienotallpotentialrespondentswillhaveinternetaccessorwillbecomfortableusingit),interpretationofquestionscanbesubjective.

‘Traditional’ web statistics11

Description:Webserversgeneratelogfileswhentheyservearequest,whichistypicallydirectlyfroma‘useragent’–usuallyabrowsersuchasInternetExplorerorFirefox.Theselogfilesrecordtherequestwhichwasmade,thestatusoftheresponse(success,failureetc)andsomebasicinformationabouttheuseragent.Thismethodconsidersrelativelysimplewaysinwhichtheselogfilescanbeprocessedtogeneratequantitativeinformation(webstatistics)regardingtheuseoftheservice.

Uses:Forunderstanding,onaggregate,thewaysinwhichanonlineresource(orcollectionofresources)isused,forexampletherelativeusageofdifferentpartsofasite,andhowtheusagevariesovertime.

Pros:Lowcostintimeandeffort,informationcanbeavailableimmediately,providesan‘at-a-glance’viewofhowbusyasiteis,andwhichsectionsofthesitearemostused.

Cons:Limitedinsightintouserbehaviour,verycrudemeasuresofdemographicofaudience,temptingtoinfertoomuchfromtheinformationwhichisavailable.

Web analytics Description:Webanalyticsisamoreadvancedmethodforinvestigatingtheactionsofusers.Webanalyticswillincludetheinformationavailablefromtraditionalwebstatistics,butwhereaswebstatisticsisfocusedontheresource,webanalyticsisfocusedonindividualusers.Theactionsoftheseuserscanbecombinedtoprovidebulkinformationonaservice,butatafarhigherlevelofdetailthanthatavailablefromtraditionalwebstatistics.

Uses:Webanalyticscanbeparticularlyhelpfultosupportsitedesigndecisions,byunderstandingthedecisionsthatuserstakewhenusingaresource.

Pros: Informationcanbeavailableimmediately,providesextensiveinformationonuserbehaviour,canbeusedtosegmentusersbasedonarangeofbehavioursandpossiblyondemographics.

Cons:Commercialwebanalyticspackagesareexpensive,settingupreportingistime-consuminganddemandsgoodbusinessandtechnicalknowledge,insightintouserbehaviourfromwebstatisticsmustbevalidated.

User observation techniques (ethnography)

Description:Ethnographyisatermusedtoencompassawiderangeoftechniquesandapproaches.Atitscoreisadesiretounderstandhumanculturebyobservingbehaviour,butthisbroadprincipleisincreasinglyusedincommercial,publicandsocialresearchcontexts.Itincludesusertrackingstudies(egshadowing,diarystudies)andusetesting(egtask-focusedactivitysessions).

Uses: Ethnographyandobservationaltechniquescanofferaninsightintohowaudiencesuseandinteractwithservicesandresources.Fordigitalservicesverydetaileddataaboutusagecanoftenbecapturedunobtrusivelybycomputerloggingtechniques.

Pros: Varydependingonmethod.Example–usetestingisanexcellentwayofidentifyingproblemswithusability.

Cons:Varydependingonmethod.Generallytime-consumingandresourceintensive.

11

11 In this document, we have divided the measurement of online resources into ‘traditional’ web statistics and web analytics. This is a somewhat artificial boundary, but one that is necessary to consider the full range of opportunities. Traditional web statistics consider bulk information on a resource (ie looks at usage from the service’s perspective), whereas web analytics considers the actions and pathways chosen by individual users (so looks at usage from a user’s perspective).

PAGE 22 5. Modelling the audience

5. Modelling the audienceThis chapter deals with analysis and interpretation of audience data to address specific objectives and improve understanding of the audience

Datafromanaudienceanalysisprojectshouldimproveunderstandingoftheaudienceinseveralways:

�� Answerstospecificresearchquestions–orfurtherevidence

�� Newknowledgeabouttheaudienceanditsinteractionswiththeservice

�� Objectivesforfutureaudienceresearch

Topicscoveredinthischapterincludeapproachestoanalysingquantitativeandqualitativedata,potentialproblemswithdatainterpretationanddevelopingpersonasoruserprofilesforservicedevelopment.

5.1 Introduction5.1.1 Thissectiondealswiththeissuesinvolvedinanalysingandinterpretingthedatatoaddressthespecific

researchquestionsandbuildabetterunderstandingoftheaudienceanditsinteractionswiththeservice.

5.1.2 Theaudiencedatamustbeanalysedappropriately:thetechniquesusedwilldependonthetypeofdataandtheaimsoftheresearch.Thedatashouldprovideevidencethatcanbeinterpretedtodrawconclusionsrelevanttothespecificobjectivesoftheresearch.Itisimportanttobesurethatthedatareallydoprovideevidenceforanystatementsyoumakeandthatyouhaveconsideredallthepossibleinterpretationsofthedata.Fullanalysisandinterpretationofthedatashouldalsoprovidewiderinsightsintotheaudience,anditsbehaviourandrelationshipwiththeservice.

Case study extract… JISC national e-books observatory project

A good example of analysing research data to understand user behaviour

TheJISCnationale-booksobservatoryprojectisa2-yearprojectledbyJISCCollections,whichisinvestigatinguserbehavioursandthedemand,usage,deliveryandbusinessmodelsforcoursetexte-books.Theoutputsofthisresearchwillsupportthefutureprocurementofe-booksinthesector.

Thisprojectisusinganumberofcomplementaryresearchmethodsincludingwebanalytics,userinterviewsandsurveystounderstandhowusersinteractwithelectronicbooks.Forexample,theprojectisusingweblogdatatoprovidequantitativemeasuresofusageandsuggestanumberofpossibleinterpretationsofuserbehaviour.However,thisdatamustbetreatedwithcare(egdoesalownumberofsearchesindicatethatauserhasquicklyfoundtheresourcetheywanted,orthattheyhavegivenupafteroneattempt?).Moredetailedanalysisofusagedata(iewebanalytics)maybeinformative,butinterviewsandsurveysarebeingconductedtohelpvalidateassumptionsandinferencesfromtheweblogdata,andtodevelopanunderstandingofuserbehaviour.

Forfurtherinformation,pleaseseetheprojectwebsite(www.jisc-collections.ac.uk/catalogue/jiscebooksproject).

The Guide to Researching AudiencesConciseedition

5. Modelling the audience PAGE 23

5.1.3 Analysingthedatafromaudienceresearchwillbeeasieriftheresearchwaswellplannedandconducted.Manyproblemsinanalysisandinterpretationcanbeavoidedifdatacollectionproceduresareselectedandimplementedwiththeanalysisinmind.Sometimesatrade-offneedstobemadebetweensimplicity,convenienceandreliabilityofrecording,andconvenienceandpowerforanalysis.Paper-baseddatacollectionmightbeeasiesttoadminister,butdatawillhavetobecodedandenteredontoacomputerbeforeanalysis.

5.1.4 Analysisofthedatafromanaudienceresearchprojectshouldimproveunderstandingoftheaudienceinseveralways:

�� Answerstothespecificresearchquestions–oratleastrelevantevidence

�� Newknowledgeabouttheaudiencemoregenerally

�� Furtherquestionsabouttheaudiencetobeaddressedinfuturework

5.2 Analysing quantitative data5.2.1 Quantitativedatacanbeanalysedstatisticallytoexplorerelationshipsbetweenvariables.Forexample,

toanswerquestionssuchas‘Areregisteredusersmorelikelytodownloadfilesfromaservice?’andtouncoverhomogeneoussegmentsoftheaudience.

5.2.2 Techniquesincluderegression and correlation analysis(fordefiningrelationshipsbetweenvariables)andfactor and cluster analysis(tolookatrelationships amongvariables).

5.2.3 Potentialproblemswithinterpretationofquantitativedatainclude:

�� Poor questionnaire design.Decisionsaboutquestionnairedesigncanhaveimplicationsfortheinterpretationoftheresults,forexampletheexactwordingofstatementsaboutsatisfactionwiththeservicecouldaffecttheproportionofthesamplewhoappeartobecontentwiththeservice

�� Instructions and/or responses are misunderstood.Pilotingaquestionnaireonasmallsampleorusinginterviewerstocollectsurveydatacanhelpavoidtheseproblems

�� The sample of respondents is unrepresentative.Obtainingarepresentativesampleisparticularlydifficultwhensurveysaredistributedtoanunknownsetofpotentialrespondents.Ifthesampleislikelytobeunrepresentativeitisimportanttoconsidertheimplicationsforinterpretationofthedataandcaveattheconclusions

5.3 Making use of qualitative data5.3.1 Interpretingqualitativedataisnecessarilyasubjectiveprocess,soifresourcespermititmaybeworth

involvingmorethanonepersoninanalysisandpresentationofthedata.

5.3.2 Presentationsofqualitativedatawillusuallyattempttosummarisethecontentandimportofthematerial,andtypicallyincludequotationsfromthematerial.Foragroupdiscussionthiswouldusuallyincludeajudgementaboutvariationsofopinionbetweengroupmembersandcommentsonpossiblereasonsfordifferencesofopinion.Abriefsummaryoftheflowofdiscussioncanbehelpfultoelucidatethecontextfortheviewsandattitudesexpressedbygroupmembers.

5.3.3 Itispossibletocodeorcategorisequalitativedatasothatquantitativetechniquescanbeapplied.Codedqualitativedatacanbepresentedintabularorgraphicalformatasasupplementtotextualdigestsofthematerial.

The Guide to Researching AudiencesConciseedition

PAGE 24 5. Modelling the audience

5.4 Personas and user profiles5.4.1 Apersonaisanarchetype,withthepersonaltraits,lifestyle,aspirations,beliefs,needsandgoalsetctypical

ofasegmentoftheaudience.Personascanhelptoprovidefocusforservicedevelopmentworkthathasfairlyhigh-level,genericobjectives,egtomakethesiteeasiertouseor‘makethesitemoreappealingtoschoolagechildren’.Theprocessofdevelopingapersonaisoftenquiteinvolved,includingdetailedinterviewswithpotentialusers,ethnographicstudies,novelactivities–anythingwhichwillhelptofosteragoodunderstandingoftheaudience.

5.4.2 Ifresourcesdonotpermitthecreationoffullyfledgedpersonas,itmaybepossibletoachievesomeofthesamebenefitsbydevelopinguser profiles–descriptionsoftypicalusers,orientedtotheserviceunderconsiderationandfocusingoninteractionswiththeservice,needs,expectationsandattitudestowardstheservice.

6. Making use of audience research PAGE 25

6. Making use of audience research

This chapter reinforces the message that audience analysis should be seen in the context of the service as a whole

Audienceanalysiscanprovidemanybenefits,butanydecisionsonuseofthedatawilldependonthewiderservicecontextandpriorities.

Astrategyforrealisingthefullvalueofaudienceanalysisworkisdiscussed.

6.1 Introduction6.1.1 Audienceresearchisatoolforservicedevelopmentandcanbeusedtoprovidethebenefitsdiscussedin

Section1;however,audienceresearchisanaidtodecision-makingandnotasubstituteforit.Evidencefromaudienceresearchcanhelpserviceprovidersmakeinformeddecisionsaboutspendingondigitisation,newservices,supportfordifferentplatforms,promotingtheservicetonewaudiencesetcbutitisnotasubstituteforbusinessplanningorthepoliticaldecisionsthatneedtobemadeaboutprioritiesforpubliclyfundedservices.

Case study extract… Penn Data Farm

Using audience research to monitor resource usage and organisational performance

TheUniversityofPennsylvaniadevelopedarepositoryofquantitativeinformation,thePennDataFarm,toaidthemeasurementandassessmentoflibraryresourceuseandorganisationalperformance.

TheultimateaimoftheDataFarmistosupportexcellent‘customer’managementandtooptimisethedesignanddeliveryoflibraryresourcesandservices.ExamplesofhowstaffhaveusedtheDataFarmtoachievethisaimincludeanalysinginformationontheuseofelectronicresources(includingfrequencyofusage,costperuseandstatusofusers)todeterminetheoptimumallocationofmoneyforelectronicresources–thisisparticularlyimportantaslibrarybudgetsaresqueezed.TheDataFarmhasalsobeenusedtotracktheuseofresourcesproducedin-houseforUniversityofPennsylvaniamembers(egcourseguides,teachingmaterials,guidestolibraryservicesforasubjectetc)torefinetheirunderstandingoftheUniversitymembers’needsandworkouthowtomeetthem.

TheteamattheDataFarmarealwayslookingforwaystocollectandexploitmoreaudiencedata,buttheyremainveryconsciousofthelimitationsoftheirdata–itisoftendifficulttoknowwhatwebusagestatisticsreallymean,andusabilityanalysisisanimportantcomplementwhendesigningnewtoolsetcforadigitalservice.Forfurtherinformation,seethePennDataFarmwebsite(http://metrics.library.upenn.edu/prototype/datafarm).

6.1.2 Becausedistinctaudiencesegmentshavedifferentrequirements,itisimportanttounderstandtherelationshipbetweenthemandprioritiseallocationofresourcesappropriately.Audienceanalysiscanprovideinformationaboutaudiencesegments,theirrelativesize,andhowtheyuse,valueandaccessaservice,butdecisionsaboutresourceallocationstilldependonservicepriorities.

The Guide to Researching AudiencesConciseedition

PAGE 26 6. Making use of audience research

6.1.3 Decisionsabouthowtodevelopandpromotetheservicearestillbusinessdecisions,notleastbecauseuseoftheservicewilldependonthequalityoftheresource,andthesuccessofthemarketingstrategy.Ofcourse,audienceanalysisshouldinformboththeseactivities.

6.2 Audience research in context6.2.1 Attheconclusionofanaudienceresearchproject,whenthedatahasbeencollected,analysedand

interpreted,andconclusionshavebeendrawn,itisimportanttomakesurethatthefullvalueoftheresearchisrealised.Forexample:

�� Theresultsoftheresearchshouldbedisseminatedtostakeholdersandotherinterestedparties

�� Everyonewhohelpedwiththeprojectshouldbethanked;thisincludesstaffwhohelpedtocarryouttheresearch,participantsandanyoneelsewhocontributed

�� Considertheimplicationsofthefindingsforservicedevelopmentandaudiencedevelopment

�� Lookattheconclusionsoftheresearch–Werethespecificobjectivesmet?Whatlessonscanbelearnedforfuturework?

�� Planfollow-upandfutureresearch

�� Assessthewiderimplicationsofthework(egarethereanyimplicationsforlong-termstrategy?)

6.2.2 Itmaybeappropriatetosharesomeofyourfindingswithpartnersinthesamesector,otherpublicsectororganisationsservingasimilaraudienceetc.

Case study extract… Archival Sound Recordings

Using audience research to develop a service and monitor its use

TheBritishLibrary’sArchivalSoundRecordingswebsite,conceivedasawayofincreasingaccesstotheSoundArchive’sextensivecollection,waslaunchedin2006.

Audienceresearchplayedacentralroleinthedevelopmentofthewebsite.Theinitialphaseofworkrelatedtothechallengeofdevelopingawebsitethatmettheneedsofitsaudienceintermsofcontent,usabilityandaccessibility.Consequently,aspecificationforthewebsitewasdevelopedandthisbecamethefocusoftheinitialaudienceresearchwork.Auserpanelconsistingof10-12individualsfromavarietyofHEandFEinstitutions(amixofacademics,librariansandresearchers)wassetuptorepresenttheneedsofthetargetaudiencethroughoutthedevelopmentandrefinementofthewebsite.Thedemonstratorversionofthewebsitewasalsotestedonnewpeoplewhowerefreshtothesite.Testingwasconductedinadedicated‘usabilitylab’,andledbytrainedfacilitatorswhoshadowedtheusers.

SubsequenttothelaunchoftheArchivalSoundRecordingswebsite,audienceresearchwasusedasawayofensuringthatthewebsitewasfulfillingtheroleforwhichitwasintendedintheHEandFEcommunities.Inaddition,aprogrammeofaudienceanalysiswhichhasexaminedhowandwhenthewebsiteisusedhasbeenundertaken.TheBritishLibraryteamalsoengagedwithuserstodevelopcasestudiesonwaysofmakinguseoftheArchivalSoundRecordings,whichwillbeusedtopromotetheresourcetotheacademiccommunity.

FormoreinformationontheBritishLibrary’sArchivalSoundRecordings,seethewebsite(http://sounds.bl.uk).

6.2.3 Itshouldalsobenotedthataudienceresearchprojectswilloftenresultinchange–whetheritbedirectlytotheserviceoffered,ortotheorganisationitself(egasaresultofachangeincompanystrategy).Changemustbecarefullymanaged,andwillrequirethoughtfulplanningandsometimessensitiveimplementation,andaboveallengagementwiththepeopleaffectedbythesechanges.UsefulfurtherreadingonthemanagementofchangeinaprogrammaticcontextisManagingSuccessfulProgrammes(MSP),OfficeofGovernmentCommerce(OGC),2007edition.

Glossary PAGE 27

GlossaryAudience development

Activitiescarriedouttoincreasetheaudienceforaserviceorresource.

Audience satisfaction Establishingiftheaudienceishappywiththeservicethatisbeingprovided.Forexample,isithappywiththequality/content/accessibility/comprehensivenessoftheservice?Interrogationofaudiencesatisfactionisoftendoneintandemwithaudienceanalysis.

Digital service and digital audience

Inthisguide,digitalisusedtorefertoresourcesand/orservicesexploitingdigitisedcontentordeliveredviaadigitalchannel.Inmostinstancesthismeansaservicewhichisavailableonline,butitalsoincludesdigitisedresourcesdistributedonCDandcouldincludedigitaltelevisionserviceswhichsharesomeofthecharacteristicsofonlinedigitalservices.Digital audiencereferstotheaudienceforadigitalserviceorresource.

Ethnographic research

Researchbasedonobservationsofhumanbehaviour,carriedoutina‘naturalistic’setting.Thiscaninclude‘observation’viaautomatedrecordingdevicesandissometimesextendedtoincludeobservationsofsubjectsmadeinalaboratory–thecrucialpointisthatbehaviourinresponsetotherelevantstimulishouldbenormalandthatthepresenceoftheresearcherortheobservationalequipmentshouldnotinterferewithoraffectthesubject’sbehaviour.

Formative research Inthecontextofservicedevelopmentanddelivery,thistermcoversresearchcarriedoutduringservicedevelopmentwork,aspartofthedevelopmentprocess,toprovidefeedbackonthework.

Front-end research Inthecontextofservicedevelopmentanddelivery,thistermcoversresearchcarriedoutintheplanningandpreparationofaservice.

Focus group Agroupofindividualsselectedandassembledbytheresearchertodiscussandcommentonthetopicunderconsideration.Subjects’personalexperienceofthetopicandinteractionsamongstgroupmembersarethedistinguishingfeaturesofthetechnique.

Impact analysis Analysisofthelonger-termeffectsofaservice(orgroupofservices)onitsaudience.Longer-termevaluationmethodsarerequiredtodetermineiftheimpactofaserviceislasting.

Reach Inthisguide,reachisusedtorefertothenumberofpeopletowhomaserviceorresourceisavailable.Differentfromuptake,whichreferstousageoftheserviceorresource.

Summative research Inthecontextofservicedevelopmentthisreferstoevaluativeresearchassessingtheextenttowhichaservicehasmetspecificcriteriaforsuccess.

Target audience Theintendedaudienceforaserviceorresource.

Segmentation Theprocessofdividinganaudienceintodifferentcategoriesorsegmentsonthebasisofcriteriarelevanttotheserviceorresourceunderconsideration.Segmentsarerelativelyhomogeneousanddistinctfromothersegmentswithrespecttothechosencharacteristics.

Usability Ameasureofthequalityoftheuserexperienceofaserviceorresource.Itgenerallyencompassesfactorssuchaseaseoflearning,efficiencyofuse,memorability,errorfrequencyandseverity,andsubjectivesatisfactionwiththeinterface.

User-centred design Adesignphilosophywhichplacesthegoals,expectationsandskillsofusersatthecentreofthedesignprocess:theserviceorproductisdesignedtofittheuser.Typically,usersareinvolvedthroughoutthedesignanddevelopmentprocess,includingusertestingoftheserviceorproductwhilstitisunderdevelopment

This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/uk/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA

This document is available in alternative formats. For more information: www.jisc.ac.uk/contentalliance

Document No: 547a Version 3.1, December 2009

The Guide to Researching AudiencesConcise edition

Further information about JISC:Web: www.jisc.ac.ukEmail: info@jisc.ac.ukTel: +44 (0)117 331 0789