The Guide to Researching Audiences
Transcript of The Guide to Researching Audiences
The Guide to Researching AudiencesDr Rachel Quirk, Martin Olver, Dr Max Hammond & Dr Claire Davies
JISC Content
JISC, British Library, BBC, National Health Service, Becta, and Museums, Libraries and Archives Council working together to fully realise the potential of e-content for all users. For more information on the Strategic Content Alliance, please visit:
www.jisc.ac.uk/contentalliance
Produced and funded by JISC
Curtis+Cartwright Consulting Ltd Main Office: Surrey Technology Centre, Surrey Research Park, Guildford, Surrey GU2 7YG T: +44 (0)1483 685020 E: [email protected] W: www.curtiscartwright.co.uk
This report is available in alternative formats which can be found at: www.jisc.ac.uk/contentalliance
Contents PAGE 3
ContentsOverview of this Guide 4
1. Introduction 5
1.1 Why,what,who…? 5
1.2 Usingtheguide 6
1.3 Supportingdocuments 9
1.4 Seniormanagementcommitment 9
2. Describing and Defining the Target Audience 10
2.1 Theimportanceofthetargetaudience 10
2.2 Theconceptofthetargetaudienceinservicedevelopment 11
2.3 Definingthetargetaudience 11
2.4 Describingthetargetaudience 12
3. Planning Audience Research 15
3.1 Introduction 15
3.2 Objectivesofaudienceanalysis 15
3.3 Startingpointsfordefiningaudienceresearchobjectives 16
3.4 Choosingthemethodology 22
3.5 Researchingnon-usage 25
3.6 Recruitingsubjectsforaudienceresearch 27
3.7 Othersourcesofaudiencedata 28
3.8 Audienceresearchdoesnotneedtobeperfecttobeuseful 29
4. Collecting Audience Data 31
4.1 Undertakingresearch:in-houseorthirdparty? 31
4.2 Overviewofdatacollectionmethods 33
4.3 Prosandconsofthemethods 43
4.4 Indirectapproachestoaudiencebehaviour 45
5. Modelling the Audience 47
5.1 Introduction 47
5.2 Analysingquantitativedata 48
5.3 Makinguseofqualitativedata 51
5.4 Analysingaudiencesatisfactionforservicedevelopment 52
5.5 Personasanduserprofiles 52
6. Making Use of Audience Research 55
6.1 Introduction 55
6.2 Audienceresearchincontext 56
Appendix A: Glossary 58
Appendix B: Reasons to do Audience Research 60
B.1 Whydoaudienceresearch? 60
B.2 Theusesofaudienceresearch 61
Appendix C: Digital Audiences and Digital Services 64
C.1 Thedigitalrevolution 64
C.2 Audiencesinthedigitalenvironment 65
C.3 Digitalcontentisdifferent 66
C.4 Therelationshipbetweendigitalandnon-digitalworlds 67
C.5 Technologyshouldservetheaudience 67
C.6 Relationshipbetweendigitalandnon-digitalservices 68
C.7 Digitalaudiencesmaybeanalyseddifferently 69
Appendix D: Bibliography 70
D.1 Introduction 70
D.2 Generalaudienceresearch 70
D.3 Planningandanalysis 70
D.4 Methods 71
D.5 Servicedevelopment 72
D.6 Digitalcontext 72
D.7 Evaluationandimpact 73
The Guide to Researching Audiences
PAGE 4 Overview of this Guide
Overview of this Guide
PLANNING AUDIENCE RESEARCH
COLLECTION OF AUDIENCE
DATA
MODELLING THE AUDIENCE
MAKING USE OF AUDIENCE
RESEARCH
Section 3
Setting clear objectives for research activities, and planning activities to meet these objectives
Section 5
Analysing and interpreting the data collected to build up a picture of the audience
Section 4
The methods available for collecting audience data, and deciding how to conduct the research
Section 6
Using the research to meet the needs, wants and expectation of the audience
DESCRIBING AND DEFINING THE TARGET
AUDIENCE
Section 2
Defining who you would like your audience to be, and how to approach segmenting the audience
REASONS TO DO AUDIENCE
RESEARCH
BIBLIOGRAPHY
GLOSSARY
DIGITAL AUDIENCES AND DIGITAL
SERVICES
INTRODUCTION
SUPPORTING INFORMATIONCORE GUIDANCE
Section 1
An introduction to the toolkit, including an overview of how it might be used in practice
Annex A
A glossary of the key terms used in the toolkit (eg ethnographic research )
Annex B
The reasons for doing (and uses of) audience research, eg for service development
Annex C
The differences between digital and non-digital services and audiences
Annex DA bibliography of the information sources used for this toolkit, and suggested further reading
1. Introduction PAGE 5
1.Introduction
1.1 Why, what, who…?
Why do audience research?1.1.1 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstrate
thattheyareusedandvaluedbyanappropriateaudience.Theusesofaudienceresearch
arediscussedindetailatAnnex B, and extendfarbeyondsimplydemonstratingacertain
numberofvisitorsorwebsitehitstotickaboxonaservicelevelagreement.Insummary,
audienceresearchcanbeusedto:
■■ developaproductorservice;
■■ evaluatetheservice(eg audiencesatisfaction);
■■ demonstrateaccountabilitytofunders(eg valueformoney,highlyvaluedbyaniche
audience);
■■ informlong-termplanning;
■■ attractsponsorshipandmarketing;
■■ informbusinessplanning(eg tofeedintoabusinesscase);
■■ informthemanagementofcontractualrelationships;
■■ supportprofessionaldevelopment.
Audienceresearchdoesnotneedtobeperfecttobeuseful
1.1.2 Audienceresearchshouldbeseenasanongoingprocess,ratherthananoccasional,one-
offevent.Evenasmallaudienceresearchprojectisworthwhile.Manytechniquescanbe
implementedquitecheaplyoradaptedtoashoestringbudget.Someinsightintoyour
audienceisvastlybetterthannoneatall.
1.1.3 ThisguidehasbeenproducedbyCurtis+CartwrightConsultingLtdonbehalfoftheStrategic
ContentAlliance.
1.1.4 AglossaryofthekeytermsusedintheguideisprovidedatAnnex A.
The Guide to Researching Audiences
PAGE 6 1. Introduction
Who is this guide for?1.1.5 ThisguidehasbeenwrittenforpeopleintheUKpublicsectordeliveringonlinedigital
serviceswhowishtoresearchtheiraudiences.1
1.1.6 Itisprimarilyaimedatnon-expertswhodonothavemarketresearchskillsthemselves,and
whodonothaveaccesstodedicatedmarketresearchteams.Itisthereforelikelytobemost
usefulforpeopleinsmallorganisations/services,orthoseconductingsmallprojectsinlarger
organisations.Itmayalsobeusefulforpeopleprocuringserviceswhowanttoknowwhat
guidelinestoprovide,andforthosesettingpolicies.
What is in this guide?1.1.7 Thisguidesetsoutthebasicprinciplesofaudienceresearch.Thesecanbefollowed
regardlessofthetypeofserviceoraudience,andwillhelpyoutoconductaudienceresearch
moreeffectively(betterresults)andefficiently(lowereffort),withfewerproblemsand
unforeseencomplications.Theyprovidethebuildingblockstoenableyoutodesign,conduct
andapplyyourownaudienceanalysisresearch.Whatthisguidewillnotprovideyouwithisa
ready-madeaudienceanalysisprogrammespecificallydesignedforyourservice.
1.1.8 Thisisaguidetocurrentgoodpracticeandastartingpointforfurtherreading.Thereis
nothingveryradicalinthisguide–therearemanyotherresourcesavailableonaudience
analysisandmodelling,someofwhicharereferencedinthisguide(seethebibliographyat
Annex D).
1.1.9 Theguidehasaspecificfocusononlinedigitalservices.Thedigitalrevolutionhasresulted
inanexplosionofpossibilities:newandinnovativeservices;adelugeofcontent;availability
ofadditionalresearchtechniques(eg webstatistics);andpotentiallynewaudiences.
Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesand
non-digitalservicesandbetweena‘digitalaudience’anda‘non-digitalaudience’areanew
andinterestingconsiderationforserviceprovidersandarediscussedinAnnex C.
1.2 Using the guide
1.2.1 Thebasicprinciplesarestructuredasstep-by-step activitiesinthelifecycleofaudience
research(alsorepresentedinFigure1-1):2
■■ Describing and defining the target audience (Section 2):ifyouwishtodelvestraight
intothepracticalguidance,thisistheplacetostart.Itdiscussestheimportanceof
knowingwhoyouwouldlikeyouraudiencetobe,andprovidesguidanceabouthowto
describeandsegmentyourtargetaudience.Thissectionwillbeespeciallyrelevantifyou
arecurrentlydesigninganewservice.However,itwillalsoberelevantifyouwouldlike
tostartfromthebasicswhenplanningresearchactivitiesforservicesthatarealreadyin
productionandthathaveanestablishedaudience.
1 Although it is focused on online digital services (ie those accessed via the Internet), it will also be useful for researching audiences reached via other channels.
2 Note that although these activities are presented as separate activities in a linear process, in practice this is not likely to be the case – it is likely to be an iterative process.
1. Introduction PAGE 7
The Guide to Researching Audiences
■■ Planning audience research (Section 3):thissectionistheplacetostartifyouhave
alreadydefinedyourtargetaudienceandwishtostartplanningyouraudienceresearch.
Itaddressestheimportanceofsettingclearobjectivesforaudienceresearchactivities,
andthingstoconsiderwhenplanningactivitiestomeettheseobjectives.Italsosetsout
examplequestionsabouttheaudiencewhichcandriveyourresearch.
■■ Collecting audience data (Section 4):thissectionprovidesanoverviewofthevarious
audienceanalysismethodsavailable(eg focusgroups,webanalytics).Italsoprovides
someinformationabouttheprosandconsofconductingtheresearchin-houseand
contractingtheworkouttoamarketresearchcompanytohelpyoudecidethebestroute
foryourorganisation/service.
■■ Modelling the audience (Section 5):thissectionfocusesonhowtoanalyseandinterpret
thedatathathasbeencollectedtoaddressthespecificresearchquestionsandbuildupa
betterunderstandingoftheaudienceandtheirinteractionswiththeservice.
■■ Applying the information (Section 6):thissectionbrieflydiscussesusingtheinformation
tobuildtheservicearoundtheneeds,wantsandexpectationsoftheaudience.
1.2.2 Howyouchoosetousetheguidancetoplanandconductaprogrammeofaudienceresearch
willbeinfluencedbymanyfactorsincluding:thematurityoftheservice,thebudgetavailable,
theamountofdigitalcontentheldandthebreadthofthetargetaudience.
The Guide to Researching Audiences
PAGE 8 1. Introduction
Figure 1‑1an outline of the step-by-step activities, and structure and content of the guide
SECTION 3
Planning audience research
SECTION 4
Collection of audience data
SECTION 5
Modelling the audience
SECTION 6
Making use of audience research
Setting clear objectives for research activities, and planning activities to meet these objectives
Analysing and interpreting the data collected to build up a picture of the actual audience (and non-users )
Choosing the data collection methods and conducting the programme of audience research
Using the information to build the service around the needs , wants and expectation of the audience
Issues with analysing and interpreting data
Improving understanding of the audience
Audience analysis in context
Realising the full value of the research
Answering specific research questions
Defining research objectives
Combining qualitative and quantitative techniques
Recruiting subjects
Conducting the research in-house or buy it in?
Overview of the methods
SECTION 2
Describing and defining the target
audience
Defining who you would like your audience to be
Defining the target audience
Segmenting the target audience
1. Introduction PAGE 9
The Guide to Researching Audiences
1.3 Supporting documents
1.3.1 Thisguideissupportedbyaseparatedocumentwhichprovidesreal-world case studies
demonstratinggoodpracticeinaudienceresearch.3Thecasestudiesarefromboththe
publicandprivatesectors,andboththeUKandabroadandareusefultogainapractical
understandingofwhatcanbeachievedwhenaudienceanalysisisdonewell.
1.3.2 Anillustrative case studypresentingahypotheticalprogrammeofaudienceresearchtoput
thisguideincontextisavailableasaseparatedocument.4
1.4 Senior management commitment
1.4.1 Audienceanalysisresearchshouldbedonewithcommitmentandsupportfromsenior
management–withoutit,thefullbenefitsoftheresearcharelesslikelytoberealised.
1.4.2 Shoulditberequired,thisguideprovidesyouwithsomeofthebasicinformationneeded
towriteaninformalbusinesscaseforaudienceresearchtofacilitatebuy-infromsenior
management.
3 The Guide to Researching Audiences: case studies, Curtis+Cartwright Consulting Ltd. v1.0.5 December 2008.
4 The Guide to Researching Audiences: illustrative case study. Curtis+Cartwright Consulting Ltd. v1.0.5 December 2008.
PAGE 10 2. Describing and Defining the Target Audience
2.DescribingandDefiningtheTargetAudience
This chapter discusses the importance of the target audiences and looks at ways of defining the target audiences, and approaches to audience segmentation
Knowing who you would like your audience to be and what service you intend to offer is a vital part of business planning and service development.
Bothformalandinformalapproachestodefiningthetargetaudiencearepossible.
Audiencesegmentationcanbehelpfulwhendescribingthetargetaudience.Differentwaysofsegmentingtheaudiencewhicharediscussedinthischapterinclude:
■■ Demographicsegmentation.
■■ Segmentationforaccountabilitytofunders.
■■ Functionalortask-orientedsegmentation.
■■ Objectiveorientedsegmentation.
2.1 The importance of the target audience
2.1.1 Knowingwhoyouwouldlikeyouraudiencetobeandwhatservicesyouintendtoofferisvital
foranyservice.Itshouldbepartofbusinessplanningasthereareimplicationsformethods
offundingandpotentialroutestosustainabilityforadigitalservice.
2.1.2 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,the
functionalityprovided,meansofaccessetc.Itisimportanttorecognisethataservicemay
havemorethanonedistinctaudience.Needsandexpectationsofthedifferentaudiencesare
likelytobeonlypartlyoverlappingandtheimplicationsthishasforservicedeliveryshouldbe
considered.
2.1.3 Understandingtheaudiencecanbevitaltogiveasenseoffocusandidentitytoaservice.
Tryingtobeallthingstoallpotentialusersisdifficult.Differentgroupsofusersmayhave
incompatibleneeds(egexperiencedacademicresearcherswillprioritiseheavyweight
searchandanalysistoolswhenusinganarchive;leisureuserswillexpectaveryuser-
friendlyinterfacethatletsthemquicklyexplorethearchiveandgetaflavourofthecontents).
Thisdoesn’tmeanthatadigitisedarchiveshouldn’taimtoappealtobothaudiences,but
understandingthatthesearedistinctaudienceswithdifferentneedsshouldinformthedesign
anddevelopmentoftheservice.
2. Describing and Defining the Target Audience PAGE 11
The Guide to Researching Audiences
2.2 The concept of the target audience in service development
2.2.1 Servicedevelopmentmaybeeitherresource-drivenoraudience-driven.Bothapproaches
arevalid,butdetailedservicedevelopmentshouldtakeintoaccounttheintendedaudience
fortheservice.Theservicemustappealtothem,sotheirhabits,needsandexpectationswill
haveanimpactonthenatureoftheserviceandthewaysinwhichitispromoted.
2.2.2 Resource-driven service developmentmightoccurbecausesomeonehasagreatidea,or
seesthepotentialforincreasingtheavailabilityanduptakeofaresourcethroughdigitisation.
Thisapproachisentirelylegitimate,butthedevelopmentworkshouldstillbedependenton
identifyingapotentialtargetaudiencewhowillbeservedbythenewresourceandconsider
thelikelyrequirementsofthistargetaudience.
Consider this example….
Ifanarchiveistobedigitised,thedesignoftheservicewillbeinfluencedbythenatureofthetargetaudienceandtheway(s)inwhichitisenvisagedthisaudiencewillusetheresource.Forexample,thetargetaudiencewillinfluence:
■■ thedesignoftheinterface;
■■ thetypesofsearchesenabled;
■■ availabilityofmaterialfordownloadingandre-use;
■■ choiceandpositioningofmetadata.
Audienceanalysisworkduringdevelopmentofthedigitisedarchiveshouldprobablyincludesomeaudienceengagementworkaspartofuserneedsassessment.Designoftheserviceshouldalsoideallyincludeusabilitystudieswithpotentialuserstoensurethattheserviceworksasintendedinthehandsofrealworldusersandmeetstheirexpectations.
2.2.3 Audience-driven service developmenttakesatargetaudienceasthestartingpointfor
servicedevelopment.Anorganisationmightwanttoextenditsofferingtoattractanaudience
forwhomtheirtraditionalnon-digitalofferingsareknowntohavelimitedappeal;alternatively
anorganisationmightwanttobuildonthegoodrelationshipithaswithaparticularaudience
byintroducingnewdigitalresourcestailoredtotheirwishesandexpectations.
2.2.4 Again,understandingthetargetaudiencewillbecriticaltothesuccessoftheservice.
Audienceresearchwillfocusonfindingoutthesortsofresourcesandservicesthatappealto
thetargetaudience,howtheywouldprefertoaccesstheservice,thetechnologiestheyuse.
Backgroundknowledgeaboutattitudes,lifestyle,socialinteractionsetc.isalsolikelytobe
usefuliftheresourceistobeprimarilyforleisureuseratherthanformallearning.
2.3 Defining the target audience
2.3.1 Definingyourtargetaudiencedoesn’tneedtobeadifficulttechnicalexercise.Agood
startingpointmaybetopictureanidealaudiencemember–oratypical,actualaudience
member.Howwouldyoudescribethisperson?Thedescriptorsyouusearegoodstarting
pointsforthinkingaboutcategoriesforaudiencesegmentation.
The Guide to Researching Audiences
PAGE 12 2. Describing and Defining the Target Audience
2.3.2 Amoreformalapproachtodefiningthetargetaudiencewouldbetodevelopaseriesofuser
profilesorfullyworkedpersonasasanaidtoservicedesignanddevelopment.Negativeuser
descriptions,profilesorpersonascanalsobehelpful:thekind(s)ofpeopleyourserviceisnot
intendedfor.
2.3.3 GuidanceondevelopinguserprofilesandpersonascanbefoundinSection5.
2.4 Describing the target audience
2.4.1 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,the
functionalityprovided,meansofaccessetc.Thefeaturesincludedinadescriptionofthe
targetaudiencewillvaryaccordingtotheservicebutmayincluderesidentialaddress,
occupation,gender,interests,technologicalexpertiseetc.Atargetaudiencemaybe
veryinclusive(egallthepeopleinManchester)orhighlyspecific(eg qualifiedmedical
professionalswitharesearchinterestinOsgood-Schlatter’sdisease).
Audience segmentation2.4.2 Audiencesareoftensegmentedintodifferentcategories.Publisheddata(egtelevision
viewingfigures,newspaperreadership)oftensegmentstheaudienceonthebasisof
demographiccategories.Academiclibrariesaremorelikelytocategoriseusersaccording
totheirrolewithinacademia.Themostappropriatewaytodefineatargetaudienceand
segmenttheactualaudiencewilldependontheserviceandtheobjectivesoftheaudience
analysisworkandseveraldifferentapproachestosegmentationarepossible.Themore
personaldatayoutrytocollectaboutusersthemorelikelyusersaretorefuseinformationor
failtocompleteyoursurvey,soprioritisinginformationneedsisimportant.
Demographic segmentation
2.4.3 Thisisaverycommonwayofsegmentingaudienceswhichisusedinpopulationsurveys.
Inconsideringthecategoriestobeusedyoumaywanttobearinmindthatsomeofthe
audiencemaybereluctanttousecertaincategoriesandfailtosupplycategorydata,refuse
toprovidedataaltogetherorprovideincorrectdata.Demographicsegmentationmight
includesegmentationonthebasisofgender,age,educationallevel,income,postcode,
occupationetc.
Segmentation for accountability to funders
2.4.4 Ifaservicelevelagreementspecifiesaparticularpriorityaudience(egparentswithpre-
schoolchildren)thensegmentationshouldincludethisasacategory.Furthersegmentation
mayormaynotbeusefulegfathers,mothers,othercarersforpre-schoolchildren,
responsibleformale,femaleorbothsexesofchildren.Thisadditionalsegmentationmightbe
beneficialiftheserviceprovidersuspectedorknewthattheneedsorexpectationsofthese
sub-segmentsweredifferent,orthoughtthattheservicewasunderusedbyasub-segment.
2. Describing and Defining the Target Audience PAGE 13
The Guide to Researching Audiences
Functional or task‑oriented segmentation
2.4.5 Thismayoftenbemoreusefulthandemographicsegmentationbutitcanbedifficult
todefinenon-overlappingcategories,ortodefinecategoriesclearlyandensurethat
respondentscategorisethemselvesasyouanticipate.Ideallyaudiencesegmentsshould
bedefinedunambiguously;descriptionsofcategorieswhichseemlong,complexoroverly
technicaltorespondentswillbeoff-putting.
Consider this example….
Digitalacademicservicesmightchooseafunctionalschemebasedontheroleoftheuser.Forexample,anacademic researcher:
■■ formallearner,ieastudentenrolledonacourseleadingtoarecognisednationalHEqualification;
■■ independentlearner;
■■ leisureuser.
Butthisschemedoesn’tcapturewhatmaybeimportantdistinctions,forexample:
■■ maturestudentsvs.otherstudents;
■■ graduateresearchersvs.experiencedresearchers;
■■ casualleisureusersvs.committed,knowledgeableleisureusers.
2.4.6 Segmentationshouldcapturethemostrelevantdifferencesbetweenusers.Forexample
ifatheserviceisinterestedincomparisonsbetweenmaturestudentsandotherstudents
becauseitisbelieved(orthereisaudienceresearchtoshow)thatmaturestudentstendto
havelowerconfidenceorskilllevelswhenusingIT,itmightbemoreappropriatetocategorise
usersintermsofself-ratedITskills.Theserviceshouldalsoconsiderwhetheritmattersthat
differentuserswillhaveadifferentideaofwhatconstitutesproficiencyinITskills.
Objective‑oriented segmentation
2.4.7 Undersomecircumstancessegmentationonthebasisofusers’goal(s)orobjective(s)maybe
mostappropriate.Ifyouareinterestedinlearningaboutaudienceneedsandexpectations,
andhoweffectiveyourserviceisinmeetingthoseneeds,itwillbeimportanttoknowwhy
youraudienceusedtheservice,whattheirobjectivewasandhowsuccessfultheywerein
achievingit.
Consider this example….
Object-orientedsegmentationfortheaudienceofamuseum–who is your museum website for?andwhat goals will your users have?
■■ usersplanningavisittothemuseum–seekingbasicinformationaboutopeninghoursandtravel;
■■ userswhohavevisitedthemuseum–wantingtolearnmoreaboutmuseumexhibitsfollowingtheirvisit;
■■ researchers–withaprofessionalinterestinthecollections.
The Guide to Researching Audiences
PAGE 14 2. Describing and Defining the Target Audience
Understanding non-users2.4.8 Understandingnon-useofaserviceisimportant:publiclyfundedservicesmayhave
aresponsibilitytoserviceaparticularaudience(egseniorcitizens,ethnicminority
communities).Wheredigitalservicesareanalternativeorcomplementaryserviceitmaybe
desirabletoensurethattheyreachasegmentoftheaudiencewhofailtousethepre-existing
non-digitalservice.
2.4.9 Highnumbersofsatisfiedusersmaynottellthewholestoryaboutaservice–partsof
thetargetaudiencemayhavebeenunabletoaccessthedigitalservice.Theirviewsare
unrepresentedinusersurveysandthedigitalserviceisnotmeetingtheirneeds;thismaybe
animportantfailing.
2.4.10 Segmentingnon-userscanbeuseful.Animportantdistinctioncanbedrawnbetween
non-userswhoaren’tawareoftheserviceandnon-userswhoareawareoftheservicebut
havechosennottouseit.Non-userswhoareawareoftheservicecouldbefurtherdivided
intoex-users(thosewhohavesampledtheserviceatleastonce)and‘neverusers’(those
whohavenotsampledtheservice).Whatisthereasonfornon-usage(eg tooexpensive,too
difficulttoaccess,happywithanalternative,notinterestedetc.).
2.4.11 Drawingdistinctionsbetweennon-usersimmediatelyraisesfurtherquestionswhichcanbea
usefulstartingpointforaddressingtheproblemofnon-usage:
■■ whyex-usersnolongerusetheservice;
■■ whetherlowlevelsofawarenessindicateaproblemwithdiscoverability;
■■ howaccuratenon-users’perceptionsoftheserviceofferingareetc.
3. Planning Audience Research PAGE 15
3.PlanningAudienceResearch
This chapter looks at defining research objectives, selecting and combining appropriate methods, researching non-usage and recruiting subjects
Research need not be perfect to be useful – the key to good audience analysis is setting out clear objectives and planning appropriate research to achieve the objectives. Engaging non-users is important but presents special problems.
Objectivescanbebroadlydividedintotwocategories:assessmentofreachanduptake,andassessmentofqualityandimpact.
Startingpointsforplanningresearchinclude:
■■ Definingservicedevelopmentobjectives.
■■ Definingquestionsaboutthenatureandbehaviouroftheaudienceandnon-users.
Differentaudienceresearchtechniquesaresuitedtodifferentobjectivesandmethodscanbecombined.
3.1 Introduction
3.1.1 Planningaudienceresearchshouldstartwiththebasicquestion:what do you want to know
about your audience?Consideringwhy the knowledge is neededandhow it will be put to use
isalsoimportant.Understandingtheaudienceisnotanendinitself,butanintegralpartof
developingandmaintainingasuccessfulandsustainableservice.Understandingthedrivers
forandusesofaudienceanalysisworkwillhelptoensurethatitisalignedwithwiderservice
visionandobjectivesandmayhelptoobtainbuy-infromseniormanagement.
3.2 Objectives of audience analysis
3.2.1 Thekeytogoodaudienceanalysisissettingoutclearobjectivesandplanningappropriate
researchactivitiestomeettheseobjectives.Thereislittlepointinspendingtime,moneyand
staffeffortcollectingdataunlessitisgoingtobeuseful.
3.2.2 Audienceanalysiscanbedividedintotwobroad,interdependentcategories:
■■ assessmentofreachanduptake;
The Guide to Researching Audiences
PAGE 16 3. Planning Audience Research
■■ assessmentofqualityandimpact.
3.2.3 Inplanningaudienceresearchitishelpfultodecidehowimportanteachaspectofthe
analysisis.
3.2.4 Analysisofreachanduptakeshouldbeapriorityifthesizeand/orcompositionofthe
audienceisimportant.Forexample,arecentlyintroducedserviceneedstodemonstrate
aminimumaudienceinordertoreceivefurtherfunding.Clearly,ifasegmentofthetarget
audienceisunder-representedoraudiencesizeissmallerthanexpectedquestionsaboutthe
qualityoftheservicewillformpartofthefutureinvestigationsofnon-usage.
3.2.5 Analysisofaudiencesatisfactionmeasureswillbeapriorityifmaintainingorenhancing
theserviceisimportantandtherearefewconcernsaboutthesizeandcompositionofthe
audience,egservicesforaclearlydefined,specialistaudiencewherethereisnodirect
competition.
3.3 Starting points for defining audience research objectives
3.3.1 Itisbeyondthescopeofthisguide,oranyother,toprovideaready-madelistofresearch
objectivesasthesemustbesetaccordingtoindividualserviceprioritiesandcontext.This
sectionprovidessomestartingpointsforidentifyingbroadaimsanddefiningmorespecific
objectives.
Service development objectives can drive audience analysis3.3.2 Oftentheservicedevelopmentcontextprovidesagoodstartingpointforframingquestions:
■■ Howcanweextendthereachofourservice?
■■ Howcanweenhancetheserviceforexistingusers?
■■ Howcanweencourageregularusersofourservicetobecomeregisteredusers?
■■ Howcanweappealtoalargeraudience?
■■ Whatadditionalcontent/features/serviceswouldouraudienceappreciate?
■■ Whatdoesouraudienceexpectfromourservice?
■■ Doesaudienceperceptionoftheservicematchwhatweareattemptingtoprovide?
■■ Isanypartofourserviceobsoleteorredundant?
■■ Isourredesignedserviceasuccessiehavetheobjectivesoftheredesignbeenmet?
3. Planning Audience Research PAGE 17
The Guide to Researching Audiences
Consider this example….
Broadservicedevelopmentobjectivessuchas‘howcanweenhancetheserviceforexistingusers?’canbeusedtodefinespecificaudienceresearchquestionswhichwillprovidetheevidencerelevanttothebroaderquestions.
Specificresearchquestionstohelpanswer‘howcanweenhancetheserviceforexistingusers?’include:
■■ Howdousersratetheexistingservice?
■■ Aretheyawareofallthefeaturesoftheservice?
■■ Whatdotheythinkisbestabouttheservice?
■■ Whatdotheythinkcouldbeimproved?
■■ Whatadditionalcontentorfunctionalitywouldtheyuse?
■■ Whichaspectoftheserviceismostimportanttothem?etc.
Questions about the audience can drive audience analysis3.3.3 Allaudienceanalysisaimstoimproveunderstandingoftheaudience,bothserviceusersand
non-users.Thinkingaboutyouraudienceandwhatyouneedtoknowaboutthemprovides
analternativestartingpoint.Thesequestions(seealsoFigure3-1)arenotintendedasa
definitiveorprescriptiveset,butasastimulusfordiscussion.
Who are our audience?■■ demographics,egage,gender,socioeconomiccategories,geographicallocation,levelof
educationetc.;
■■ functional,egprofessional/researcher/otherspecialist,formallearner,independentlearner,
leisureuser,accidental/impulseuser,maturelearneretc.;
■■ goalorientationegdefinedobjective,browser,systematicexplorer,impulse/accidental
useetc.;
■■ preferredplatformetc.;
■■ domainexpertiseegforusersofElectronicBeowulf,howmuchtheyknowaboutBeowulf,
linguisticsandAnglo-Saxonliterature;
■■ platformortechnologicalexpertiseieexpertise,confidenceandfamiliaritywiththe
platformsandtechnologyfromwhichaserviceorresourcecanbeaccessed.
How do they use our service?■■ meansofaccess;
■■ frequencyofusageegregular,repeated,occasional,one-off;
■■ whattheyuseitfor;
■■ areanyalternativeservicesusedforthesimilarpurposes?
Why do they use our service?■■ inpreferencetoanon-digitalservice?
The Guide to Researching Audiences
PAGE 18 3. Planning Audience Research
■■ isthereanalternative?
■■ quality;authority;comprehensiveness;convenience(speedandeaseofaccess);social
cache.
Does the service meet their needs and expectations?■■ expectations–includingexpectationsofextensionsandenhancement;
■■ needs;
■■ importance;
■■ quality:strengths,weaknesses,suggestedimprovements;
■■ reliability;
■■ authority;
■■ comprehensiveness;
■■ convenience.
How do users discover/reach our services?■■ howoriginallydiscoveredegsearchengine,referringsite,offlinereferral,recommendation
(who?),othermedia(advertisement?),event;
■■ howreachedinsubsequentusage.
3. Planning Audience Research PAGE 19
The Guide to Researching Audiences
Figure 3‑1questions about the audience
WH
O a
re t
he
y?
EX
IST
ING
AU
DIE
NC
E
WH
Y d
o t
he
y u
se
ou
r s
erv
ice
?
WH
O i
s n
ot
us
ing
ou
r s
erv
ice
?
DO
ES
th
e s
erv
ice
me
et
the
ir n
ee
ds
an
d e
xp
ecta
tio
ns
?
Ac
ce
ss
Ho
me
Pu
blic
Mo
bile
Fre
qu
en
cy
Re
gu
lar
Oc
ca
sio
nal
On
e-o
ff
Co
mp
eti
tors
Pu
bli
c
Pri
va
te
Inte
rna
tio
na
l
Wh
at
for
Co
nta
cts
Co
llec
tio
ns
Ho
w t
o f
ind
us
Fir
st
us
e
Se
arc
h e
ng
ine
Re
co
mm
en
dati
on
Refe
rrin
g s
ite
Su
bs
eq
ue
nt
us
e
Dir
ec
t
Se
arc
h e
ng
ine
Refe
rrin
g s
ite
Dir
ec
t
Re
as
on
Qu
ality
Au
tho
rity
Co
mp
reh
en
siv
e
Cla
rity
No
alt
ern
ati
ve
s
Alt
ern
ati
ve
s
Dig
ital
No
n-d
igit
al
De
mo
gra
ph
icF
un
cti
on
Go
al
Te
ch
no
log
y
Do
ma
in
Ag
e
Ge
nd
er
So
cio
eco
no
mic
cate
go
ry
Lo
ca
tio
n
Pro
fes
sio
nal
Re
se
arc
her
Ac
cid
en
tal/
imp
uls
e
Leis
ure
Bro
ws
ing
De
fin
ed
ob
jecti
ve
Sys
tem
ati
c
ex
plo
rer
Ex
pert
Co
nfi
de
nt
No
vic
e
Ex
pert
So
me
kn
ow
led
ge
No
kn
ow
led
ge
HO
W d
o t
he
y u
se
ou
r s
erv
ice?H
OW
do
th
ey
rea
ch
ou
r
se
rvic
e?
Se
pa
rate
dia
gra
mA
ud
ien
ce
sa
tis
fac
tio
n
WH
AT
are
th
e
lon
ge
r te
rm e
ffe
cts
of
the
se
rvic
e?
Imp
ac
t a
naly
sis
Qu
ality
Relia
bilit
y
Co
nve
nie
nc
e
Op
inio
n
Imp
ort
an
ce
The Guide to Researching Audiences
PAGE 20 3. Planning Audience Research
3.3.4 Manyofthequestionsaboutusersarealsorelevanttonon-users,butsomeissuesare
particularlyrelevantwhenconsideringnon-users(seealsoFigure3-2).
Who are they?
■■ whichsegmentsoftargetaudienceareunder-represented?
■■ whichsegmentsofthegeneralpotentialaudienceareunder-represented?(anddoesthis
matter?)
Why don’t they use our service?
■■ awareness:
■− dotheyknowitexists?
■− weretheyabletodiscoverit?
■− weretheyabletoacessit?
■− havetheytriedourservice?
■■ Perceptions:
■− whatdotheythinktheserviceoffersandhowdotheyrateitforquality,valuefor
moneyetc.?;
■− whodotheythinkusestheservice?
■− howcouldtheybepersuadedtousetheservice?
What alternatives to our service do they use?
Why are these alternative services preferred?
■■ speed,easeofaccess,quality,authority,comprehensivecoverageetc.
How well do these alternative services meet their needs and expectations?
3. Planning Audience Research PAGE 21
The Guide to Researching Audiences
Figure 3‑2understanding non-users
Usage
Ex-users
Never
NON-USERS
WHY are they not
using our
service?
Awareness
Not aware
Aware
Reason
Not interested
Happy with
alternative
Too slow
WHY do they no
longer use our
service?
WHO are they?
Which segments of the target
audience are under-represented ?
Target audience – existing audience
= non-users
WHAT did they
think of our
service?
HOW could we
make them more
aware?
HAVE they ever
used our service?
WHY do they not
use our service?
Reason
Not interested
Happy with
alternative
Poor public
imageWHY are the
alternatives
preferred?
Reason
Better quality
Better coverage
Easier to
access
Audience satisfaction
Aware but
cannot accessWHY can they not
access it?
Reason
Technological
Authentication
required
Disabilities
Marketing
The Guide to Researching Audiences
PAGE 22 3. Planning Audience Research
3.4 Choosing the methodology
Different approaches to audience analysis3.4.1 Audienceanalysismethodscanbecategorisedindifferentways,butthedistinctionbetween
quantitativeandqualitativemethodsisanimportantone.Quantitativemethodsareusually
moreappropriatewhenstatisticallyvalidconclusionsneedtobedrawnaboutthesize,
compositionandotherpropertiesofanaudience.Qualitatitivetechniquesmaybemore
appropriateforexploringattitudesandmotivationsindepth,discoveringpatternsofuser
behaviouranddevelopingarelationshipwithaudiencemembers.
Qualitative data
3.4.2 Smallsamplestendtobeusedandfindingscannotbetreatedasastatisticallyreliable
measureoftheaudienceasawhole.Howeverqualitativeresearchcanbeinvaluablein
exploringthecomplexitiesunderlyingaudiencebehaviour,helpingtoexplainwhyaudience
membersbehaveinthewaythattheydo.
3.4.3 Examplesofmethodsproducingqualitativedatainclude:
■■ focusgroups;
■■ one-to-one,in-depthinterviews.
3.4.4 Incertaincircumstances,informationcollectedviaaquantitativesurveycanbequalitativein
nature,egfreetextresponsestoopenquestions.
Ethnographic or observational data
3.4.5 Awidevarietyofdirectandindirectobservationaltechniquesareincreasinglyusedfor
studyingusabilityandpatternsofserviceuse.Ethnographicdataisoftenverydetailedand
mayincludecomputercollecteddatawhichisamenabletostatisticalanalysisaswellasa
qualitativerecordofuserbehaviour.
3.4.6 Examplesofmethodsproducingethnographicdatainclude:
■■ diarystudies;
■■ usergeneratingscenarios;
■■ task-focusedscenarios;
■■ accompaniedbrowsing.
3. Planning Audience Research PAGE 23
The Guide to Researching Audiences
Quantitative data
3.4.7 Quantitativedataprovidesinformationaboutquantitiesandfrequencies.Non-numericaldata
canbecodedorcategorisedtorenderitamenabletoquantitativeanalyticaltechniques.
Quantitativedataisgenerallycollectedfromasamplelargeenoughtopermitstatistical
analysis.Conclusionsfromstatisticalanalysiswillusuallybestatedwithadegreeof
confidence.5
3.4.8 Examplesofmethodsproducingquantitativedata:
■■ telephoneandface-to-facesurveys;
■■ postalsurveys;
■■ emailsurveys;
■■ onlinesurveys;
■■ webanalytics.
Representative data3.4.9 Samplesaresaidtoberepresentativewhenthecompositionofthesampleaudience–with
respecttoallrelevantattributes–isthesameasthatoftheaudienceasawhole.Insome
instancesitisimportanttoobtaindatafromarepresentativesampleoftheaudienceegto
assessthelevelofuptake,toprovideevidenceforattainmentofkeyperformanceindicators
(butseealsosub-section6.8.4).
3.4.10 Usingrepresentativesamplesdemandsaknowledgeofthefrequencyofdifferentsegments
inthetargetoractualaudience.Assessingtherepresentativenessofasamplerequires
informationaboutthesegmenttowhichsubjectsbelong.Itissometimesassumedthat
asampleisrepresentativeifthereisnoobviousevidenceofbias,orifthesamplesizeis
large.Unexpectedlylowresponseratesareapotentialindicationofsamplebiasandshould
generallybeinvestigated.
Selecting appropriate methods3.4.11 Thekindofinformationyouwantabouttheaudienceshouldhelptodeterminethemethods
youusebutpragmaticconsiderations(egtime,money,difficultyofrecruitingsubjects,staff
availabilityandexpertiserequired)willinevitablyplayapart.
3.4.12 Table3-1providesaguidetothemethodsmostlikelytobeusefulinresearchingdifferent
aspectsoftheaudience.
5 For example: based on this sample, 95% of the audience as a whole will fall between these limits; or, as a probability value expressing the likelihood of obtaining the same result by chance.
The Guide to Researching Audiences
PAGE 24 3. Planning Audience Research
Table 3‑1Selecting appropriate methods (Key: – unlikely to be suitable; * may be suitable, but of limited use; ** often suitable)
Asp
ect
of
aud
ienc
e
Qua
ntit
ativ
e-su
rvey
Web
sta
tist
ics
Eth
nog
rap
hic
stud
ies
Focu
s g
roup
s
Ind
ivid
ual
inte
rvie
w
Info
rmal
Oth
er
app
roac
hes
Size ** ** - - - - -
Composition ** *moredetaileddataaboutregisteredusers
- - - * -
Motivations ** * - ** ** * -
Attitudes ** * - ** ** - -
Satisfaction ** * - ** ** - -
Prevalence of non-usage
** - - * - -
Reasons for non-usage
- - - ** ** - -
Patterns of usage - ** ** * * - -
Tasks service is used for
** ** **usertracking
** ** - -
Lifestyle * - ** * ** - -
Service usability * * ** * * - Heuristicevaluation6
User requirements - - ** ** ** * -
Combining methods in audience analysis63.4.13 Itiscommontouseacombinationofmethodsinaudienceanalysisprojects:thismakes
senseforanumberofreasons:
■■ differentsegmentsoftheaudiencemayberecruitedwithdifferentmethods;
■■ differentmethodshavedifferentadvantagesanddisadvantages;
■■ differentmethodsprovidedataondifferentaspectsoftheaudienceandtheirbehaviour.
3.4.14 Inplanninganaudienceanalysisprojectbearinmindhowdatacollectedearlyoninthe
projectcanbeusedtoadjustandimproveplansforthelaterresearch.Itiscommon
tocombinequantitativesurveyswithfocusgrouporinterviewwork.Whichshould
comefirst?Responsestoasurveymayraiseissuesthathadn’tbeenanticipatedand
6 See Section 4.4
3. Planning Audience Research PAGE 25
The Guide to Researching Audiences
suggestappropriatequestionsandthemesformoredetailedexplorationinfocusgroups.
Alternatively,youmayprefertocarryoutasmallnumberofinterviewsorafocusgroupand
distilkeyfindingssothatvaliditycanbetestedwitharepresentativequantitativesurvey.
Talkingtoaudiencemembersindividuallyoringroupsmightsuggesttopicsforinclusionina
survey.
3.4.15 Ifethnographicworkistobecombinedwithanintervieworshortsurveyquestionnairethe
samesubjectscouldbeusedforbothelementsoftheresearch.Itmightbehelpfultofollow
anobservationsessionwithaninterviewsothatyoucandiscusswhatwasobservedandask
aboutgoodandbadaspectsoftheuserexperience.Ontheotherhanditmaybeharderto
recruitusersforastudyrequiringalongertimecommitment.Youcouldaskintervieweesor
surveyrespondentsiftheywouldbewillingtoparticipateinfurtherresearch.Thentheorder
inwhichuserscompletethedifferentelementsofthestudymaybecriticalasyourquestions
mightinfluencetheuser’sbehaviourduringtheethnographicstudyortheuser’sexperience
oftheserviceduringtheethnographicsessionmayaffecttheirattitudeandratingofthe
servicewhenthey’reansweringsurveyquestionsorbeinginterviewed.Theseareissues
whichneedtobeconsideredinplanningtheresearch.
3.4.16 Anotherconsiderationisthedurationoftheresearch:
■■ one-off projectstoprovideasnapshotofanaudienceatasinglepointintime;
■■ repeated surveysieundertakingsimilarsurveysonaregularbasistomonitortrendsor
assesstheimpactofchange;
■■ longitudinal researchiethesamesampleofaudiencemembersistrackedoveraperiod
oftime(egtomonitortheimpactofdevelopmentactivities);avarietyofmeasuresmaybe
used.
3.5 Researching non-usage
The importance of research on non-users 3.5.1 Engagingwithnon-usersmaybedifficult,time-consumingandpotentiallyexpensive,
butitshouldbeviewedasanecessarypartofaudiencedevelopmentwork.Buildingup
relationshipswithnon-userswillpaydividends:
■■ non-userscanprovidevaluableinsightsforservicedevelopment;
■■ engagingnon-userscanhelpdemonstrateacommitmenttothataudiencesegmentand
increaseknowledgeaboutmarketingandcommunicationstrategiesthatwillbeeffective
forthatsegment.
Identifying non-users3.5.2 Reliabledataaboutactualusageofaservicecanbecomparedwithanaccuratedescription
ofthetargetaudiencetoderiveapictureofnon-usage,butthispictureofnon-usagewillonly
beasreliableasthedataaboutusersandtargetaudiencefromwhichitisderived.
The Guide to Researching Audiences
PAGE 26 3. Planning Audience Research
3.5.3 Descriptionsofthetargetaudiencewilloftenbeinsufficientlydetailedegnoinformation
availableaboutthefrequencyorproportionofthetargetsegmenteitherbecausefunctional
segmentationhasbeenchosen,orbecauseappropriatedemographicdataisnotavailable.
Evenifagooddescriptionofthetargetaudienceisavailable,obtainingagoodestimate
ofthenumberandnatureofusersmaybedifficult,particularlyfordigitalservices(multiple
usersforoneIPaddress,thesameuserloggingonatwork,athome,inapublicsettingetc.).
Professionalexpertiseandinformalaudienceresearchmaybethebestcluestopatternsof
non-usage:thesecanbetestedandfollowedupwithmoreformalresearch.
Investigating non-usage
Reasons for non‑usage
3.5.4 Responsestoasurveymaynotprovideverymuchinformation.Engagingmoredeeplywith
non-usersthroughfocusgroups,discussionforaandinterviewingwillbenecessarytobuild
uparicher,moremeaningfulunderstandingofnon-usage.
Recruiting non‑users
3.5.5 Reachingnon-userscanbeproblematic–theiropinions,attitudesandhabitsarenot
representedinserviceusagestatisticsorsurveys.Peoplearelesslikelytobewillingtospend
timeansweringquestionsaboutaserviceorresourcethattheydon’tusethanonetheyvalue
highly.
3.5.6 Onewayofgaininganinsightintohowwidelyyourserviceisusedistoquestionthewholeof
yourtargetaudiencethroughawidelydistributedsurvey.Dependingonthesizeofyourtarget
audienceandhoweasilyyouareabletocontactthemthismaybedifficultandexpensive.
Possiblestrategiesinclude:
■■ Addingyourquestionstoasurveybeingdistributedto,orcarriedouton,thegeneral
population–marketresearchservicescarryoutbroadsurveysandlocalgovernments
mayalsoattempttosurveytheirelectorate.So-calledOmnibussurveyscanbeavery
cheapwayofsurveyinglargepopulations.Theyareaccessedonapay-per-questionbasis
soacoupleofwellconstructedandclearlyfocusedquestionscanbeansweredrelatively
cheaply(forinstance,relatingtoawarenessofaserviceandkeybarrierstouse).
■■ Carryingoutamoregeneralsurveyyourselforpayingamarketresearchcompanytodo
soonyourbehalf.
■■ Targetingaudiencesegmentsyoususpectareunder-representedbychoosingmethods
knowntobeeffectiveinreachingthatsegmentegchoiceofaparticularmedium,buying
acontactlistfromarelevantorganisation,enlistingthehelpofanorganisationwhich
representsorworkswiththesegmentofinterest.
3.5.7 Itmaybehelpfultoenlistthehelpoforganisationswhohavethetrustandconfidenceof
yournon-users.Non-usersarelikelytobemorewillingtoengageifyouareabletoconvince
themthattheirparticipationmayhaveapay-off,thattheirvoicewillbelistenedtoandtheir
commentsmayinfluenceserviceprovisioninthefuture.Directincentivesforparticipationare
usuallyofferedaswell.
3. Planning Audience Research PAGE 27
The Guide to Researching Audiences
3.6 Recruiting subjects for audience research
3.6.1 Recruitingsubjectsisaperennialproblemandtheeasiesttechniquemaynotalwaysbethe
best.Recruitingcommittedmembersoftheaudiencetotakepartinresearchtendstobe
easiestandthisisonepotentialcauseofanunrepresentative,biasedsample.Recruitment
willbemoredifficultifabigtimecommitmentisrequiredortheresearchisgoingtobe
inconvenientforsubjects.Incentivescanbeusedtoaidrecruitment,butcareneedstobe
takentoensurethattheincentiveisappropriatetotheaudienceyouseektorecruit.
Recruiting for quantitative research
3.6.2 Surveyscanbedistributedinanumberofwaysanditispossibletocombinemethodsto
improveresponserate.Eachmethodhasadvantagesanddisadvantages.Surveyscanbe:
■■ placedonawebsite;
■■ emailedtoknownusers,thewholetargetaudienceorasampleofthetargetaudience;
■■ posted;
■■ administeredbyaninterviewerbytelephoneorinperson.
3.6.3 Placingaquestionnaireonawebsiterequiresminimaladministrativeeffort,doesnot
requirecontactdetailsandischeap,butcombiningtheresponsedatawithwebstatisticsis
necessarytoobtainanestimateofresponserate.Responseratestendtobelowandthere
isnowayofcontrollingresponsebiasinthesample;sometypesofuseraremorelikelyto
respondthanothers.
3.6.4 Ifthecompositionoftheaudiencesampleisimportantthensurveysadministeredby
interviewermaybemostappropriateasitispossibletosetquotasforparticularaudience
segments.
3.6.5 Informationaboutresponsebias–systematicdifferencesbetweenrespondentsandnon-
respondents–maybeobtainedifanattemptismadetocontactnon-respondentsviaan
alternativemethodanddiscovertheirreasonsfornotresponding.Thismaybeattemptedifit
isparticularlyimportanttohavearepresentativesampleorwheretheaimwastosurveythe
completeaudience.
3.6.6 Surveydesigncanbeanimportantfactorinresponserateanddrop-outrate(theproportion
ofrespondentswhoceaserespondingpartofthewaythroughthesurvey).Higherresponse
ratesandlowerdrop-outaremorelikelyif:
■■ surveysareshort;
■■ questionsandpossibleresponsesareeasytounderstand;
■■ itispossibletosaveapartiallycompletedsurveyandreturnlater;
■■ surveynavigationiseasyandthesurveyisveryusable;
■■ questionsarevariedintheirstyle,formatandsubject(forexample,avoidrepeatedusage
ofbanksofsimilarlywordedandvisuallyundifferentiatedattitudestatements).
The Guide to Researching Audiences
PAGE 28 3. Planning Audience Research
Recruiting for qualitative and ethnographic research
3.6.7 Recruitingsubjectsforqualitativeresearchpresentsslightlydifferentproblems.Thetime
commitmentrequiredistypicallygreaterandifparticipationintheresearchislimitedto
membersoftheaudiencewhoareavailableduringworkinghoursitwillbedifficulttoobtain
arepresentativeaudiencesample.Recruitmentislikelytobemoresuccessfulifpotential
subjectsarecontactedinperson.
Recruiting registered users
3.6.8 Digitalservicesmayofferusersincentivestoregister;buildingarelationshipwithatleast
partoftheaudienceinthiswaycanpaydividendsforaudienceresearch.Registeredusers
maybemorefrequent,moreloyalormorecommittedtotheserviceandthereforemore
willingtoparticipateinresearch.Ofcoursetheimplicationsofdifferencesbetweenregistered
usersandtherestofthecurrentandpotentialaudienceshouldbeborneinmind,butata
timewhenmoreandmoreorganisationsandaudienceresearchprofessionalsarereporting
‘surveyfatigue’amongstaudiencesthevalueofhavinganeasytorecruitaudiencesample
foraudienceanalysisworkshouldnotbeunderestimated.
3.6.9 Anextensionofthisideaistosetupresearchpanels.Theseconsistofasampleofaudience
memberswhohavebeenrecruitedspecificallyforthepurposeofregularly(ieonanongoing
basis)respondingtosurveys–qualitativeand/orquantitative.Theadvantageofthisapproach
isthatresearchcanbeorganisedrelativelyquicklyandinexpensivelysincethesampleis
alreadyinplace.Insomeinstances,priorknowledgeofandinvolvementinpreviousaudience
researchprojectscanbeadvantageous–althoughtherearealsopotentialdangersrelating
tolackofobjectivity.Familiaritywithissuescanhelpto‘fast-track’panelmemberstoalevel
ofthinkingaboutaservicewhichultimatelyresultsinaddedinsight.Inthepublicsector,
CitizensPanelshavebecomepopularwithlocalauthoritiesandhealthcaretrustsinparticular
butalsotoanextentwithavarietyofcommunityplanningorganisations.Theseare,ineffect,
largescaleresearchpanels.
3.7 Other sources of audience data
Using other people’s data3.7.1 Don’tneglectthepossibilityofincreasingyourunderstandingofyouraudiencebylookingat
researchdonebyotherserviceprovidersontheiraudienceorbyusingdatafrompopulation
surveyscarriedoutbymarketresearchorganisations,governmentagenciesoracademic
bodiesegtheOxfordInternetInstitutebiannualsurveyofinternetusage.7
3.7.2 Ifyouknow–orevenifyoujustsuspect–thatyouraudienceissimilartothatofanother
serviceyoucoulduseinformationaboutthataudience’sattitudesandpreferencesto
informyourownservicedevelopment.Ifyouknowthecompositionofyouraudiencethen
populationsurveyswiththerequiredsegmentationcanprovideausefulguidetoaspects
7 Oxford Internet Institute surveys www.oii.ox.ac.uk/microsites/oxis
3. Planning Audience Research PAGE 29
The Guide to Researching Audiences
ofyouraudienceegdataonmediaconsumptioncouldbeusedformarketingpurposes,
lifestyle-relatedquestionsmightprovidesuggestionsforserviceenhancement.Personasor
userprofilesfromasimilarservicemightbeusefulinservicedevelopmentwork.
3.7.3 Arelatedpointisthattheaudienceforthenon-digitalversionofaservice,whetherornot
theyoverlapwiththeactualortargetaudienceforthedigitalservice,maybeagoodsource
ofdataandthisaudiencemaybeeasiertorecruitforaudienceresearch:alternativesurvey
methodscanbeused,interviewerscanapproachvisitorsface-to-faceetc.
Informal sources of audience data3.7.4 Exploitinginformalknowledgeaboutyouraudienceisasmartthingtodoandcanbe
particularlyvaluableifresourcesforaudienceanalysisarelimited.Thesesourcesmight
include:
■■ callstoahelpdesk;
■■ emailstodigitalservicetechnicalsupport;
■■ observationsbystaffwhointeractdirectlywiththeaudience;
■■ unsolicitedcommentsorsuggestionsfromyouraudience.
3.7.5 Themainproblemisfindingawaytorecordandcollatethesedatasothattheycanbeused
inplanningformalresearch.
3.8 Audience research does not need to be perfect to be useful
3.8.1 Afrequentlyexpressedconcernisthatapieceofaudienceresearchwon’tbeanyuse
becauseitisn’tgoodenough.Someknowledgeabouttheaudienceisverymuchbetterthan
noneatallandtheseconcernsshouldnotdeteryoufromdoingaudienceanalysis.
Representative data are not always vital3.8.2 Acommonconcerninplanningaudienceresearchisthatanaudiencesamplewillbetoo
smallandnotnecessarilyrepresentativeoftheaudienceasawholeorthetargetaudienceas
awhole,butthismaynotalwaysmatter.
3.8.3 Itismore importanttoensurethatyouraudiencesampleiscompleteorrepresentative
when:
■■ reachanduptakearebeingassessedrelativetoaspecifictargetaudience;
■■ bigdecisionsaboutcommitmentofresourcesaredependentonthefindings;
■■ theserviceshouldbeuniversaliereach100%oftargetedaudiencesegments;
■■ findingswillbeusedasevidenceforaccountabilitypurposesegattainmentofkey
performanceindicators.
3.8.4 Itisless importantthatthesampleisrepresentativewhen:
The Guide to Researching Audiences
PAGE 30 3. Planning Audience Research
■■ researchispreliminaryandexploratory;
■■ theaudienceishomogeneous;
■■ variabilityintheaudiencecharacteristicsorbehaviourunderinvestigationislow;
■■ researchwillinformservicedevelopmentforasubsetofthegeneralaudience;
■■ satisfyingacommittedaudienceismoreimportantthankeepingorrecruitingalarge
audience;
■■ alightweight,shortanalysisofamorerepresentativesamplecanbecarriedoutasa
follow-up(egself-completionsurveyratherthanone-to-oneinterviews;webstatistics
provideevidenceonusageofanewfeature;ashortersurveyforwiderdistributionfollows
upimportantorcontentiousissuesraisedbyadetailedsurveyofsmallsample).
4. Collecting Audience Data PAGE 31
4.CollectingAudienceData
This chapter provides an overview of different methods for collecting audience data and considers outsourcing audience research
Theprosandconsofconducingresearchin-houseandcommissioninganindependentmarketresearchagencytoundertakeitonyourbehalfarediscussed.
Datacollectionmethodscoveredinclude:
■■ Quantitativesurveys.
■■ Interviews.
■■ Focusgroups.
■■ Webstatistics.
■■ Ethnographic(observational)techniques.
Otherapproachestobuildingunderstandingoftheaudiencearebrieflydiscussed.
4.1 Undertaking research: in-house or third party?
4.1.1 Technologicaladvancesmakeitincreasinglytemptingtoconductresearchona‘DIY’basis,
ratherthancommissioninganindependentmarketresearchagencytoundertakeitonyour
behalf.However,itisworthconsideringanumberoffactorsbeforedecidingifthein-houseor
commissionedresearchrouteismostappropriate:
The Guide to Researching Audiences
PAGE 32 4. Collecting Audience Data
Table 4‑1In-house research
Approach Pros Cons
In-house research
■■ Canoftenbetheleastexpensiveoption–especiallyifexpertin-houseresourcesareavailable.
■■ Knowledgeoftheaudience:expertknowledgeand/orexperiencecanhelpinplanningandconductingresearch.
■■ Greaterlevelofcontrol:amarketresearchagencymighthavedifferentideasabouthowtoresearchanaudience.
■■ Lackofexperienceinconductingresearchcanleadtopoorresearchdesignandotherproblemsthatanexperiencedresearchagencywouldhaveanticipatedandplannedfor.
■■ Questionnairedesignissurprisinglydifficulttogetright.Poorlydesignedquestionsresultinambiguousormisleadingdata.Experienceofquestionnairedesignisimportant.
■■ Similarly,unlessknowledgeofstatisticalandsamplingtechniquesisavailablein-house,mistakesarelikelytobemade–bothattheplanningandattheanalysisstages.
Table 4‑2Commissioned research
Approach Pros Cons
Commission market research company
■■ Ifcarefullyselected,aresearchagencywillbringexpertiseandexperiencetobear,resultinginamoreefficient(possiblyevenmorecost-effective)surveyandgreaterclarityofunderstanding.
■■ Theinvolvementofarespected,independentthird-partybringscredibilitytotheresearch.Thiscanhelpinachievingseniormanagementbuy-intothefindings;insomecircumstancesfundingbodiesmightlookmorefavourablyonacasefoundedonindependently-conductedaudienceresearch.
■■ Respondentsaregenerallymorelikelytorespondopenlyandhonestlytoathird-partythantheywouldiftheservice-providerasksthequestionsdirectly–especiallyiftheyarecriticalofaspectsoftheservicebutprefernottodamagetherelationship.
■■ Marketresearchagencieshaveaccesstospecialistresourcesrequiredtoconductresearch.Theseinclude:
■■ InterviewersandfocusgrouprecruitersinallareasoftheUK.
■■ Specialistquestionnairedesignandsurveyanalysissoftware.
■■ Commissionedresearchcanbeexpensive.
■■ Theprocessofidentifyingpotentialresearchagencies,writingaresearchbriefandbriefingtheselectedagencytakestime.
■■ Evenmarketresearchagenciesexperiencedinyoursectorwilllackyourdetailedknowledge.Bepreparedtospendtimeensuringthattheagencyunderstandsthenatureofyouraudienceandwhatyouaretryingtofindoutaboutit.
4. Collecting Audience Data PAGE 33
The Guide to Researching Audiences
How do you decide which market research company to use?4.1.2 ‘TheResearchBuyer’sGuide’,8publishedbytheMarketResearchSociety,hassomeuseful
information,bothonthetypeofmarketresearchcompanytochoosefromandonthe
questionstobearinmindwhenselectingacompany.
4.1.3 Itincludesadviceonwhethertosub-contractsomeelementsofaresearchprojectto
consultants(eg quantitativeinterviewingorrecruitmentoffocusgroups),orbuy-inthe
fullservice(whereprofessionalswillhelptodesigntheresearchandcollect,analyseand
evaluatedata).Otheroptionsaretohireagenciestodofieldwork(theinterviewingprocess)
andtabulation(producingdatatablesfromasurvey)–thesetypesofagenciestypicallyhave
ateamofface-to-faceinterviewersand/oratelephoneinterviewingcentre,togetherwith
datacodingandprocessingresources.Therearealsodataprocessingagencieswhichdo
notundertakeinterviewingbutcanprovidecomputerdataprocessingandanalysisresources
andexpertise.
4.1.4 Whenselectinganagency,itwouldbeusefultoconsiderthefollowingquestions:
■■ Whichcompanybestunderstandsyourneedsandprojectbackground?
■■ Whichcompanyhasaddedtoyourthinkingabouthowresearchshouldbeplanned?
■■ Hasthecompanychallengedsuggestedapproaches,highlightingpossibledifficultiesand
solutions?
■■ Doesthecompanyhaverelevantexperience,intermsofindustry,sector,audience,
subject-matterandmethodology?
■■ Doestheapproachsuggestedbythecompanyoffervalueformoney?
■■ Wouldthecompanybecrediblepresentingtoseniormanagementandotherinternaland
externalstakeholders?
■■ Willexperiencedstaffbeallocatedtotheproject(andnotjustappearatthebeginningand
endofaproject)?
■■ Canyouformaproductiveworkingrelationshipwiththeresearchcompany’steam?
4.2 Overview of data collection methods
4.2.1 Informationondifferentmethodsofcollectingaudiencedataissummarisedinthefollowing
tables,providingpracticalguidanceforusingthesemethods.Thesetofmethodscoveredis
notintendedtobedefinitiveanddetailedinformationonhowtoimplementeachmethodis
notprovided.
4.2.2 Mostaudienceanalysismethodscanbeusedoradaptedsuccessfullybynon-specialists;
however,therearesomecircumstancesunderwhichitmaybeappropriatetoconsider
outsourcingoneormoreelementsofanaudienceanalysisproject.
8 Research Buyer’s Guide, UK and Ireland (2007); London: Market Research Society.
The Guide to Researching Audiences
PAGE 34 4. Collecting Audience Data
Web statistics4.2.3 Providingresourcesonlinebringsnewopportunitiestomeasuretheirusage,andto
understandthewaysthatusersinteractwiththem.Inthisdocument,wehavedividedthis
intotwomethods–‘traditional’webstatisticsandwebanalytics.Thisisasomewhatartificial
boundary,butonethatisnecessarytoconsiderthefullrangeofopportunities.Traditional
webstatisticsconsidersbulkinformationonaresource(ielooksatusagefromtheservice’s
perspective),whereaswebanalyticsconsiderstheactionsandpathwayschosenby
individualusers(solooksatusagefromauser’sperspective).
4.2.4 Boththesemethodsarequantitative,butthenumbersgeneratedmustbetreatedwith
care:avastrangeofstatisticsisavailable,buttounderstandwhicharemeaningfulrequires
acombinationoftechnicalandbusinessunderstanding.Forcommercialsites,thereis
oftenacleargoalforusers–aconcludedsale.Akeymeasureformanycommercialsites
istheconversionrate–theproportionofvisitorswhomakeapurchase.Thereisnodirect
equivalentforaresourcewhichisfreeatthepointofuse,anddrawinginsightfromthe
numbersrequirescreativeanalyticalthinkingandcarefulvalidation.
Registered users4.2.5 Increasinglybothdigitalandnon-digitalservicesmayrequireorpermitusersto‘register’
insomeway.Thisallowstheservicetocollectinformationaboutusers–subjecttothe
provisionsofthedataprotectionlegislation.Userregistrationcanbeexploitedeasily
bydigitalservicesforaudienceanalysis:requiringuserstoauthenticateallowsuser
characteristicstobeassociatedwithuserbehaviour.Althoughthiscanbeapowerful
analyticaltechniquetherearepotentialdrawbacks:
■■ unlessusersareauthenticatedbyatrustedauthoritythedatatheyprovideabout
themselvesmaynotbeaccurate;
■■ itisunlikelythatregisteredusersarerepresentativeoftheaudienceasawholeeg
theymaymakemorefrequentormorein-depthuseoftheservice,theymaybemore
committedtotheservice,theymaybemorelikelytobeexpertsinthedomain;
■■ requiringregistrationmaydetersomeofthepotentialaudience.
4. Collecting Audience Data PAGE 35
The Guide to Researching Audiences
Data collection methods
Table 4‑3Focus groups
Focus groups
Description Focusgroupscanbeorganisedinmanywaysandinmanyformats.However,allformatsarebasedonthepremiseofasmallgroupofrespondentsdiscussingissuesguidedbyanexperiencedfacilitator(sometimescalledamoderator).
Uses Usefulforexploringattitudesandmotivationsandforgeneratingnewideas.Focusgroupsareoftenusedastheprecursortosomeformofquantitativesurveyresearch.
Outline of format ■■ ‘Standard’focusgroupstypicallycontain6-10respondentsandeachsessionlastsupto1½hours.
■■ A‘topicguide’–essentiallyanagendadetailingissuestobecovered–isusedbythefacilitatortoguidediscussions.
■■ Asuitablevenueischosen.Thiscanbeameetingroomor,ifobserverswishtoviewfocusgroups,adedicatedviewingfacilitycanbehired(andtherearemanyoftheseacrosstheUK).
Resources and timescales
■■ Relativelyshorttimescales:forexample,aprogrammeoffourfocusgroupsmighttakearound4weekstocompletefrominceptiontodeliveryoffindings.
Other details and considerations
■■ Initialrecruitmentquestionnairesareausefulwayofensuringthatyouobtainthedesiredhomogeneity/heterogeneityofgroups.
■■ Facilitatingfocusgroupsisaskilledtask.Itisthefacilitator’srolenotonlytoensurethatdiscussionsarekeptonsubject,butalsotocontrolthedynamicsofthegroup–forexampleensuringthatstrongercharactersdonotdominatediscussionsandundulyinfluencetheviewsofothers.
■■ Extendedfocusgroupsoftenlasting2½to3hourscanbeusefulforexploringmorecomplexand/ormorediverseissues.However,insessionsofthisdurationitisimportanttoincludenumerousbreaksandotherwaysofdividingdiscussionsintomoremanageablesections.
■■ Variationsonfocusgroupscanbeusefulforsomeaudiences.Forexample,anapproachoftenusedforinterviewingyoungpeopleistousefriendshippairs,triadsormini-groupstoovercomelackofconfidence.
■■ Focusgroupsrelyonasufficientnumberofaudiencemembersbeinglocatedincloseproximitytooneanother.Ifanaudienceiswidelydispersed,focusgroupscannotbeconsidered.
■■ Onlinefocusgroupsarearelativelyrecentdevelopment.Theyhaveseveralbenefits–notleastcost–butlackmanyofthekeybenefitsoftraditionalfocusgroupssuchastheabilitytoobserveinteractionandbodylanguage.
■■ Projectivetechniques–whichhavedevelopedoutoftechniquesusedinpsychoanalysis–aresometimeseffectiveinhelpingtouncoverunconsciousattitudesandmotivations.
■■ Becauseofthelevelofcommitmentdemandedofrespondents,itistypicaltooffer‘incentives’whichcanbeintheformofcash,vouchers,donationstoanominatedcharityetc.
The Guide to Researching Audiences
PAGE 36 4. Collecting Audience Data
Table 4‑4One-to-one in-depth interviews
One-to-one, in-depth interviews
Description In-depthinterviewsconductedonaone-to-onebasis,bytelephoneorinperson.
Uses Incommonwithfocusgroups,one-to-oneinterviewsareusefulinunderstandingtherangeofattitudesandmotivationswhichmightexistwithinanaudience.
Outline of format ■■ One-to-onein-depthinterviewscanbeconductedbytelephoneorface-to-face.
■■ Bothapproachescanbeusedtoobtainqualitativedata–althoughgenerallyface-to-faceinterviewsaremoreeffectiveindeliveringagreaterdepthofinformationandunderstanding.
■■ Face-to-faceinterviewstypicallylastforupto1hour.Telephoneinterviewsareusuallynolongerthan30-45minutes.
■■ A‘discussionguide’orsemi-structuredquestionnaireisusedasthebasisforinterviews.
■■ Inmostcasesrespondentswouldbeinterviewedintheirownenvironment(placeofwork/studyorhomeasrelevant).
Resources and timescales
■■ One-to-oneinterviews,especiallyifconductedinperson,tendtorequiresignificantlylongertimescalesthan,forexample,focusgroups.Geographicallydispersedaudiencescanmeanthatonly2or3interviewsmightbepossibleperday.Consequently,aprojectinvolving30interviewsmighttake6weeksormoretocomplete.
Other details and considerations
■■ One-to-oneinterviewsareespeciallyappropriateforsmall,geographicallydispersedaudiences(eg universitylibrarians).
■■ Theyarealsousefulifthesubjectmatterissensitive–inwhichcaserespondentsmightnotwanttodiscloseinformationtotheirpeerswhichwouldbethecaseinfocusgroups.
■■ Althoughtelephoneinterviewsareeasierandcheapertoarrange,thequalityanddepthofinformationobtainedfromface-to-faceinterviewsisusuallysuperior.
■■ Itisalsoworthrememberingthataface-to-faceformatallows‘stimulusmaterial’tobeused.Thisismoredifficulttoorganisefortelephoneinterviews,althoughmaterialscanbefaxed,emailedorpostedaheadoftheinterview.
4. Collecting Audience Data PAGE 37
The Guide to Researching Audiences
Table 4‑5Quantitative surveys
Quantitative surveys
Description Quantitativesurveysinvolveusingasampleofanaudience/populationasthebasisforproducingastatisticallyreliablepictureofthewhole.
Uses Providingquantifiableinformationaboutanaudience,forexample,itsprofileanditspatternsofusageandbehaviour.
Outline of format Therearemanywaysofcollectingquantitativedataincluding:
■■ Postalsurveys:Audienceand/orprospectiveaudiencemembersaresentaquestionnairewhichtheyareinvitedtocompleteandreturn(usuallyinareply-paidenvelope).
■■ Emailsurveys:Audienceand/orprospectiveaudiencemembersareemailedasurvey(oremailedaURLlink)whichtheyareinvitedtocompleteonline.Contactemailaddressesareneededforthisoption.
■■ Web-basedonlinesurveys:Awebsiteorresourceextensivelyusedbythetargetaudienceisusedtopromotethesurveyandtohostalinkforrespondentstofollow.Thisoptiondoesnotrequirethatemailaddressesareknownbutisaninexactmeansoftargetinganentireaudience.
■■ Telephonesurveys:Audienceand/orprospectiveaudiencemembersarecontactedbytelephone,usingashort,structuredquestionnaire(generallylastingnolongerthan10-15minutes).Dependingonthecompositionoftheaudienceitissometimessensibletoset‘quotas’toensurethatdifferentsegmentsarerepresentedinthesample.
■■ Face-to-facesurveys:Forverylargeconsumeraudiences,on-streetandin-homeface-to-faceinterviewsareoftenappropriate.Thefewercontactsthatneedtobemadetofindeachqualifyingaudiencemember,themorecost-effectivethisapproachbecomes.Forexample,fornon-digitalservicesface-to-faceexitinterviews(egwithlibraryusers)offeracost-effectivemeansofsurveyinganaudiencebutexcludeorunder-representinfrequentandnon-usersofaservice.
Resources and timescales
■■ Telephoneandface-to-faceinterviewersrequireateamof(ideally)experienced,professionally-trainedinterviewers.
■■ Independentfull-serviceresearchagencieshaveaccesstointerviewersinallareasoftheUK.
■■ Theabilityofanorganisationtodevelopandhostonlinesurveysdependsonitsaccesstoappropriatetechnicalresources.
■■ Additionally,ameansofanalysingsurveydataisrequired.Mostindependentresearchagencieswillusespecialistsurveyanalysissoftwarewhichfacilitatesandintegratesallstagesoftheproject;sampleandquestionnairedesign;hostingofonlinesurveys;datacaptureandstatisticalanalysis.
The Guide to Researching Audiences
PAGE 38 4. Collecting Audience Data
Quantitative surveys
Other details and considerations
Sub-sections5.2and5.4provideanoverviewofhowtoanalysequantitativedata,includinglookingatthepotentialproblemswithdatainterpretation,sampling(includingsamplebias),theimportanceofquestionnairedesignandweightingresponses.Thereisalargeamountofinformationavailableinbooksandontheinternettohelpyouunderstandthesetopicsinmoredetail,including:
■■ AquestionnairedesignadvicesheetfromLoughboroughUniversity:www.lboro.ac.uk/library/sklls/Advice/QuestionnaireDesign.pdf
■■ Acomprehensivebookonmarketingresearchincludingusefulsectionsonquestionnairedesign,sampling,analysisandweighting:Marketing Research,Aaker,KumarandDay;JohnWiley&Sons,Inc.;8thed.(2004)
■■ Variousresearchtips,whitepapersandstudiesbyacademicsandpractitioners:www.sysurvey.com/tips/whitepapers.asp
Otherconsiderations:
■■ ‘Self-completion’formats–suchaspostalandonlinesurveys–areattractivesincetheyallowalargeaudiencetobesampledrelativelyeasilyandinexpensively.However,theresultingsampleisself-selecting(ierespondentsmakeaconsciouschoicetobepartofthesample).Aneffectoftenobservedinself-selectingsamplesisthatthepropensitytorespondisnotconsistentacrossallaudiencemembertypes.Inaudienceresearch,morefrequent,committedservice-userstendtobemorelikelytorespondthansporadicandinfrequentusers.Ifthereforetheserviceproviderisaimingtodevelopastrategytoincreaseusageamonginfrequentusers(andthereforeneedstolearnmoreoftheirattitudesandmotivations),self-selectingsurveyformatsarenotusuallyappropriate.
■■ Interviewer-ledformats(egtelephonesurveys)aremoreexpensivetoundertakebuthavetheadvantageofbeingmore‘random’(ietheoreticallyatleast,eachaudiencememberhasamoreequalchanceofbeingselectedforthesurvey).Quotascanalsobesettoensurecoverageofallsegmentswithinanaudience.
■■ Itisimportanttodistinguishbetweenaccuracyandprecision–youmaywellget1000responsestoyoursurveygivingstatisticallyprecisedata,butitcouldbeentirelyinaccurateifyouhavemissedsomekeysegmentsofyourtargetaudience.
4. Collecting Audience Data PAGE 39
The Guide to Researching Audiences
Table 4‑6‘Traditional’ web statistics
‘Traditional’ web statistics
Description Webserversgeneratelogfileswhentheyservearequest,whichistypicallydirectlyfroma‘useragent’–usuallyabrowsersuchasInternetExplorerorFirefox.Theselogfilesrecordtherequestwhichwasmade,thestatusoftheresponse(success,failureetc.)andsomebasicinformationabouttheuseragent.
Thismethodconsidersrelativelysimplewaysinwhichtheselogfilescanbeprocessedtogeneratequantitativeinformation(webstatistics)regardingtheuseoftheservice.
Uses Webstatisticscanhelptounderstand,onaggregate,thewaysinwhichanonlineresource(orcollectionofresources)isused.Forexample,itcanidentifywhichsectionsofthesitearemostused,whereusershavecomefrom(egfromasearchengineoralinkfromarelatedsite),thenetworkfromwhichtheyareaccessingthesite,thetypeofuseragenttheyareusingetc.
Inthecontextofaudienceanalysis,webstatisticshelptounderstandtherelativeusageofdifferentpartsofasite,andhowtheusagevariesovertime.
Outline of format Webserversgeneratelogfileswhichareperiodicallyanalysedtogeneratestatistics.Awiderangeoftoolsareavailabletoundertakethisanalysis,manyofwhicharefreeandopensource.
Differenttoolswillbeabletoundertakeanalysiswithdifferentlevelsofcomplexity.Forexample,abasictoolwillbeabletotellwhichresourcesreceivedthemostaccesses,ortheproportionofuserswhichcamefrom.ac.uknetworks.Morecomplextoolsallowcross-tabulation:dothoseusersat.ac.ukusedifferentresourcesfromthoseat.edu?Whichresourcesarediscoveredthroughsearchengines,andwhicharefoundbynavigatingthroughthesite?
Resources and timescales
■■ Dataiscollectedcontinuouslyduringserveroperation.Typically,statisticsgenerationisundertakenonaperiodicbasis,oftendaily.
■■ Generationofstatisticsisfullyautomated,andcommontoolshandlearchivingofpastinformationandgenerationofusefulreports.
■■ Awiderangeoftoolsareavailableforundertakingthelogprocessing,includingsimplefreeandopensourcetoolssuchasAnalog,AWstatsorWebalizer.Commercialpackagestypicallyoffergreaterfunctionality,andarebetterclassedaswebanalyticstools.
■■ Mostsitesexperiencehourly,daily,weeklyandseasonalvariationsinusage.Thiswillonlybecapturedwithtime.
Other details and considerations
■■ Webstatisticsrecordusageofthewebserver–theydonotfollowtheactivitiesofindividualusers.Eachrequesttoaresourceisessentiallyindependentofallotheractionsofthatuser.
■■ Itisexceedinglydifficulttoestimatethenumberofuniqueusersofasitefromthewebstatistics.Someproxymeasuresarepossible(forexamplethenumberofuniqueIPaddresses),butthesehaveverylowreliability.
■■ Itisusuallynecessarytoexplicitlyfiltersearchengineindexersfromtheresults.
■■ Relatingthenetworklocationofavisitortothecountrytheyarelocatedinispossibletoareasonablelevelofaccuracy,butonlythroughcommercialservices.
■■ Fordynamicwebsites,understandingresourceusageusingbasicwebstatisticspackagesdependsonthesitedesignusingpersistentanduniqueURLs.
The Guide to Researching Audiences
PAGE 40 4. Collecting Audience Data
Table 4‑7Web analytics
Web analytics
Description Webstatisticscanprovideusefulbulkinformationontheutilisationofresources,buthavesignificantlimitationswhenattemptingtounderstandtheusersofaresource.Webanalyticsisamoreadvancedmethodforinvestigatingtheactionsofusers.Webanalyticswillincludetheinformationavailablefromtraditionalwebstatistics,sotheseusesarenotrepeatedhere.
Whereaswebstatisticsisfocusedontheresource,webanalyticsisfocusedonindividualusers.Theactionsoftheseuserscanbecombinedtoprovidebulkinformationonaservice,butatafarhigherlevelofdetailthanthatavailablefromtraditionalwebstatistics.
Webanalyticscanbeparticularlyhelpfultosupportsitedesigndecisions,byunderstandingthedecisionsthatuserstakewhenusingaresource.
Uses Webanalyticscanbeusedtounderstandindetailthebehaviourofusersofasite.Itcansegmentusersmoreclearlythantraditionalwebstatistics,andcanidentifyuniqueuserswithfargreatercertainty.Itcanprovidemeasuresofvisitfrequencyandduration,andanunderstandingof‘clickstreams’whichrecordthepathsthatuserstakethroughthesite.
Thisinformationcanleadtoabetterunderstandingoftheuseofresources.Forexample,itmaybepossibletosegmentusersalonglinessuchasone-offvisitorswho‘land’onaparticularresourcefromasearchengine,andnevervisittherestofthesite,oroccasionalin-depthbrowserswhoarriveatasite,andthenspendsometimenavigatingandusingresourcesoruserswhoreturnoftenbutwhousealimitedsetofresources.
Thiskindofsegmentationcanleadtomakingbetterdecisionsaboutsitedesign,structureandcontent.
Outline of format Arangeofmethodscanbeemployedtotrackthebehaviourofindividualusers.Thekeyoptionsare:
■■ Enhancedloganalysis,wherebythestandardwebserverlogsareanalysedtoattempttoelucidate‘sessions’whichidentifyindividualusers;
■■ Server-sideusertracking,wherethewebserverisadaptedtoidentifyindividualsessionswithinitslogfiles.Thisisoftencombinedwithleaving‘cookies’totrackreturningusers;
■■ Arangeofwebanalyticsservicescanbeusedinan‘out-sourced’manner,wheretheserviceproviderembedscodewithineachpageoftheresource,whichisloadeddirectlyfromtheanalyticsprovider.Thisallowstheanalyticsservicetocollatedatawithoutaccesstotheserviceprovider’swebserverlogs;
■■ Mostcontentmanagementsystemswhichprovide‘library’resourcesonlinewillincorporatesomeusagemonitoringfunctionalitywithintheapplicationitself.Thisshouldincludetrackinganysearchtermsusedwithinthesite.
Followingdatacollection,thedatamustbeanalysed.Initially,thisisoftenbyaniterative,investigativeprocessratherthanbylistingindividualmetrics.Asservicesbecomemoreestablished,itmaybepossibletoelucidatequantitativeKeyPerformanceIndicators(KPIs)fromthewebanalyticsdata.
4. Collecting Audience Data PAGE 41
The Guide to Researching Audiences
Web analytics
Resources and timescales
■■ Asforwebstatistics,dataiscollectedonlineandonanongoingbasis.
■■ Toolscanbeobtainedfornocost(egGoogleAnalytics),onaservicebasis(egWebTrendsOnDemand)orforapurchaseprice(egSawmill,ClickTracks).
■■ Theeffortrequiredtobegincollectingdataistypicallylowandembeddingthetoolswithintheserverworkflowisrelativelyeasy,especiallyforsimplehostingarrangements.
■■ Settingupreportstoanalysethedatacollectedisoftentime-consuming,requiringdetailedconsiderationofthesitestructure,thebehaviourofusers,andagoodunderstandingofthenatureoftheresources.Oncethesehavebeensetup,ongoingmonitoringshouldberoutine.
Other details and considerations
■■ Recordingextensiveinformationaboutthebehaviourofindividualusersraisesprivacyconcerns.Eveniftheservicedoesnotexplicitlyidentifyusers,thisinformationmaywellbepersonaldata(orevensensitivepersonaldata)withinthemeaningoftheDataProtectionAct1998.
■■ Webanalyticshelpstounderstandthebehaviourofusersofasite,butitisnecessarytovalidatefindings.Forexample,ifahighproportionofvisitorsonlyviewonepage,doesthismeanthattheyfoundwhattheywerelookingfor,orthattheydidn’tandtheydon’thavethepatiencetoinvestigatethesitemoredeeply?
■■ Itshouldbenotedwebanalyticsisalsoknownbyothernames.Forexample,‘DeepLogAnalysis’(DLA)isa‘brandname’forresearchundertakenbytheCIBERteamatUniversityCollegeLondon(UCL).DLAreferstotheanalysisofrawserverlogs,andisoftenconductedinconjunctionwithotheraudienceresearchtechniques(egquestionnairesandinterviews),todevelopanunderstandingofuserbehaviour.Althoughapotentiallypowerfultechnique,analysingweblogsinthisdepthisunlikelytobeachievablewithinthelimitedbudgetsofmanypublicsectororganisations,andnorisitalwaysnecessaryorappropriate.
Table 4‑8 User observation techniques (ethnography)9
User observation techniques (ethnography)
Description Ethnographyisatermusedtoencompassawiderangeoftechniquesandapproaches.Atitscoreisadesiretounderstandhumanculturebyobservingbehaviourbutthisbroadprincipleisincreasinglyusedincommercial,publicandsocialresearchcontexts.[Ethnographyis]‘anythingfromhangingaroundaskateboardpark,interviewingteenagersintheirbedrooms,askingrespondentstomakevideodiaries,tointerviewingtheirfriendsandfamilies,orvideoingthemgoingshopping–it’sallethnography.’9
Itisbroadlyacceptedbyacademicsandresearchpractitionersalikethatethnographyintoday’sworldisaboutdifferenttypesofobservation.
Uses Ethnographyissaidtominimise‘researcheffect’byobservingsubjectsintheir‘natural’context(whichmightforexamplemeanattheirdeskatwork,infrontofacomputerscreen,inagalleryormuseumorathome)andbyremovingbiasintroducedbyframingstructuredquestions.Inthecontextofaudienceresearchethnographyandobservationaltechniquescanofferaninsightintohowaudiencesuseandinteractwithservicesandresources.Fordigitalservicesverydetaileddataaboutusagecanoftenbecapturedunobtrusivelybycomputerloggingtechniques.
9 Viewpoint – Ethnography and market research,PhillyDesai;InternationalJournalofMarketResearchvolume49(6);MarketResearchSociety.
The Guide to Researching Audiences
PAGE 42 4. Collecting Audience Data
User observation techniques (ethnography)
Outline of format Therangeofformatsistoobroadtocoverindetailbutpossibleapproachesinclude:
User tracking studies:theaimistocaptureserviceusagebehaviourinarealworldcontext:
■■ subjectsmightbeaskedtokeepaphoto,audioorvideodiaryorweblogrecordingtheirday-to-dayinteractionwiththeserviceoveraparticulartimeperiod;
■■ subjectsare‘shadowed’andbehaviourrecordedbyanobserver,orusingspecialisedequipmentina‘viewinglab’.
Thesemethodscanbecombinedwithinterviewingtoallowaninterviewertoprobereasonsfordecisionsandactions.Interviewingmaytakeplaceaftertheuser-trackingactivityoratthesametimeegaccompaniedinternetbrowsing.
Use testing:aserviceistestedbyusersandthefinedetailofusagebehaviouriscapturedbycomputerorrecordedbyothermeans:
■■ user-generatedscenarios:usersareallowedtoexploreaserviceorresourcefreely,theiractivityandperhapstheircommentsontheuserexperiencearerecordedforsubsequentanalysis.Providesanaturalisticpictureofhowtheaudiencerespondtotheservice;
■■ task-focusedscenarios:usersareaskedtocarryoutapre-definedsetoftasksusingtheservice,theextenttowhichtheirpaththroughtheserviceisdictatedisvariable;aninteractiveformatispossibleietheuserrespondstoinstructionsfroman‘observer’whocanadapttheprotocolinrealtime.Theobservermaybewiththeuserorfollowingtheiractivityremotely.
Againthedetaileduseractivityisrecordedandanalysedtoallowassessmentofhowusabletheserviceisandhowwellitfulfilsitsfunction.
Resources and timescales
■■ Methodscanbecustomisedforsmallerbudgets.
■■ Useofspecialised‘viewinglabs’equippedwitheye-trackingandaudiovisualrecordingisexpensive.
■■ Foracomputer-basedservicedetailedactivitydatacanbecapturedquicklyandcheaplywithappropriateprogramming.
■■ Usetestingcanbeverytimeconsumingbutvariabilityacrosssubjectsislowsosmallsamplescanbeused.
■■ Analysingactivityandobservationaldatarequiressomeexpertise.
Other details and considerations
■■ Digitalservicescanbetestedbyusersatalocationandperhapstimeoftheirownchoosingwhichmaymakerecruitmenteasier.
■■ Serviceevaluationinanaturalisticsettingmayproducedifferentresultsfromtestinginalab.
■■ Ethnographicmethodscanbeadaptedtodifferentcircumstances,valuabledatacanstillbeobtainedifcheaper‘observational’techniquesareused.
4. Collecting Audience Data PAGE 43
The Guide to Researching Audiences
4.3 Pros and cons of the methods
4.3.1 Acritiqueofeachofthemethodsisprovidedinthefollowingtablestohelpyoudecidewhich
techniquesareappropriateforyourresearch.
Table 4‑9Pros and cons of quantitative methods
QUANTITATIVE METHODS
Method Pros Cons
Face-to-face interviews
■■ Goodqualitydata.
■■ Limitvariationsinquestioninterpretation.
■■ Enablemorecomplexquestionstobeasked.
■■ Experiencedinterviewersareabletoproberesponsestoprovidefullerunderstandingofissues.
■■ Stimulusmaterialcanbeshown(egserviceoutlines;informationleafletsetc.).
■■ Relativelyexpensive.
■■ Requireahighlevelofrespondentcooperationandcommitment(mightneedtobeincentivised).
Postal questionnaires
■■ Relativelyinexpensive.
■■ Offeralargeproportionofrelevantaudienceachancetorespond–iftheychooseto.
■■ Demandlowlevelofcommitmentfromrespondents.
■■ Sampleisself-selecting(andthereforenotnecessarilyrepresentative).
■■ Noguaranteesofresponserates.
■■ Timingsaremoredifficulttocontrolandpredict.
■■ Responseratestendtobelow.
■■ Questionnairelengthandcomplexityislimited.
■■ Questioninterpretationcanbesubjective.
Internet/ email surveys
■■ Cost-effective.
■■ Offeralargeproportionofrelevantaudienceachancetorespond–iftheychooseto.
■■ Demandlowlevelofcommitmentfromrespondents.
■■ Canprovideveryimmediatefeedback.
■■ Sampleisself-selecting(andthereforenotnecessarilyrepresentative).
■■ Noguaranteesofresponserates.
■■ Notallpotentialrespondentswillhaveinternetaccessorwillbecomfortableusingit.
■■ Timingsaremoredifficulttocontrolandpredict.
■■ Questionnairelengthandcomplexityislimited.
■■ Questioninterpretationcanbesubjective.
Telephone interviews
■■ Goodbalanceofqualitydataandcost-effectiveness.
■■ Limitvariationsinquestioninterpretation.
■■ Enablemorecomplexquestionstobeasked.
■■ Experiencedinterviewersareabletoproberesponsestoprovidefullerunderstandingofissues.
■■ Stimulusmaterialcannotbeshown.
■■ Someaudiencesarebecomingover-researched–particularlybytelephonesurveys.
The Guide to Researching Audiences
PAGE 44 4. Collecting Audience Data
Table 4‑10Pros and cons of qualitiative methods
QUALITATIVE METHODS
Method Pros Cons
Focus groups ■■ Usefulforstimulatingdiscussionandnewideas.
■■ Consequentlyfocusgroupscanbeaverycreativeapproach.
■■ Thefocusgroupformatiswell-suitedtointroducingandgainingreactionstostimulusmaterials(egideasforserviceframeworks;promotionalandinformationliteratureetc.).
■■ Relativelyshorttimescales.
■■ Reliesonsufficientnumbersofaudiencebeingclusteredinageographicalarea–oratleastwithinaradiusthatrespondentsmightbepreparedtotravel.
■■ Respondentsmightnotbepreparedtodiscusspersonally,commerciallyoracademicallysensitivesubjectswithothers.
■■ Requireexperiencedfacilitators.
■■ Somegroupmemberscanbeundulyinfluencedbytheviewsofothers.
One-to-one in-depth interviews
■■ Easiertoarrangethanfocusgroups.
■■ Interviewercanvisitinterviewee.
■■ Geographicallydiverseaudiencescanberesearched.
■■ Respondentsarenotunderpressurefromotherstorespondinaparticularway.
■■ Arapportcanbebuiltbetweeninterviewerandinterviewee;thisinturncanleadtodisclosureofinformationwhichotherwisemightnothavebeendivulged.
■■ Formattendstobelessdynamicandcreativethanfocusgroups.
■■ Timescalesareusuallylonger(focusgroupscanprovideaccessto40individualswithinacoupleofdays;thesamenumbersinaone-to-oneformatwouldtakeseveralweeks).
Table 4‑11Pros and cons of web statistics
WEB STATISTICS
Method Pros Cons
‘Traditional’ web statistics
■■ Lowcostintimeandeffort.
■■ Informationcanbeavailableimmediately.
■■ Providesan‘at-a-glance’viewofhowbusyasiteis,andwhichsectionsofthesitearemostused.
■■ Limitedinsighttouserbehaviour.
■■ Verycrudemeasuresofdemographicofaudience.
■■ Itistemptingtoinfertoomuchfromtheinformationwhichisavailable.
■■ Basicpackagesdonothandlelibrary-typedynamicsiteswell.
Web analytics ■■ Informationcanbeavailableimmediately.
■■ Providesextensiveinformationonuserbehaviour.
■■ Canbeusedtosegmentusersbasedonarangeofbehaviours,andpossiblyondemographics.
■■ Commercialwebanalyticspackagesareexpensive.
■■ Settingupreportingistime-consuming,anddemandsgoodbusinessandtechnicalknowledge.
■■ Insightintouserbehaviourfromwebstatisticsmustbevalidated.
4. Collecting Audience Data PAGE 45
The Guide to Researching Audiences
Table 4‑12Pros and cons of ethnographic methods
ETHNOGRAPHIC RESEARCH
Method Pros Cons
user tracking: diary studies, shadowing and contextualised interviews
■■ Usefulforbuildingupapictureofserviceusageinabroadcontext.
■■ Abilitytoquestionusersaboutbehaviourallowselucidationofunexpectedusagepatterns.
■■ Participationintheresearchshouldhaveonlyminimaleffectonthesubjects’normalroutine.
■■ Alowcost,loweffortwayofinvestigatingthefinedetailofusagebehaviour.
■■ Methodsofrecordingusagemayinterferewithnormaluserbehaviour.
■■ Diarystudies:qualityofdatacollecteddependsontheskillandcommitmentoftheparticipant.
■■ Shadowing:datacollectionmaybeintrusiveandresourceintensive.
use testing: user-generated activity sessions and task-focused activity sessions
■■ Veryhelpfulinidentifyingunspokenuserrequirements.
■■ Excellentwayofidentifyingproblemswithusability.
■■ Audiencebehaviourmaybeobservedandmeasureddirectly,eliminatinginaccuraciesthatoccurinself-reports.
■■ Capturesthefinedetailofuserbehaviour.
■■ Canprovideanaturalisticpictureofusagebehaviour.
■■ Variabilitybetweensubjectstendstobelowsosmallsamplesizescanbeusedsuccessfully.
■■ Analysinguseractivitydatarequiresconsiderableexpertise.
■■ Testingcanbeverytimeconsuming.
■■ Somesophisticateddatacapturetechniquesrequirespecialisedequipmentsosubjectmustattendatestingfacility.
■■ Toderivemaximumbenefittheremustbegoodcommunicationwithtechnicalsupportteam.
4.4 Indirect approaches to audience behaviour
4.4.1 Audienceanalysishasanimportantroleinservicedevelopmentandvariousethnographic
techniquesareincreasinglyusedtosupportuser-centreddesignandmanyofthese
techniquescanbeadaptedtosuitsmallerbudgets.User-centredservicedevelopmentmay
alsoincorporatemethodswhichdonotinvolvetheaudiencedirectly.Thesemethodsfall
outsidethescopeofthisguidebutaredescribedbrieflybelowbecauseoftheirutilityinthe
earlystagesofservicedevelopmentandasaquickanddirtyalternativetousertestingif
resourcesareseverelylimitedoruserrecruitmentisproblematic.
Heuristic evaluation
4.4.2 Expertevaluatorsassesstheservice’suserinterfaceagainstrecognisedusabilitycriteria.The
exactcriteriausedshouldreflecttheneedsofthetargetaudience,theplatformandtypeof
serviceoffered.
4.4.3 Thismethodofassessingusabilityischeapanddoesnotrequirealargeorrepresentative
sampleofpotentialusersfromthetargetaudience.Themethodcanbeusedevenatavery
earlystageindevelopmentifevaluatorsareabletoworkwithmock-ups.
The Guide to Researching Audiences
PAGE 46 4. Collecting Audience Data
Cognitive walkthrough
4.4.4 Expertevaluatorsplaytheroleofatypicaluser(userprofilesorpersonasmaybehelpful
here)andattempttoperformthetaskstheserviceisdesignedfor.Analysisoftheserviceis
animportantprerequisite:userobjectivesarespecifiedandbrokendowntoshowthesub-
goalsthatmustbeachievedinordertoattaintheobjective.
4.4.5 Thismethodprovidesanideaofhowwelltheservicefunctionsfromtheuserperspective,
butreliesontherole-playingskillsandjudgementsoftheevaluatorsratherthaninvolvingthe
targetaudiencedirectly.Aswithheuristicevaluation,themainadvantagesarethelowcost,
theavoidanceofrecruitmentproblemsandthepotentialtocarryoutevaluationsatanearlier
stageinservicedevelopment.
5. Modelling the Audience PAGE 47
5.ModellingtheAudience
This chapter deals with analysis and interpretation of audience data to address specific objectives and improve understanding of the audience
Datafromanaudienceanalysisprojectshouldimproveunderstandingoftheaudienceinseveralways,byproviding:
■■ Answerstospecificresearchquestions–orfurtherevidence.
■■ Newknowledgeabouttheaudienceandtheirinteractionswiththeservice.
■■ Objectivesforfutureaudienceresearch.
Topicscoveredinthischapterinclude:
■■ Approachestoanalysingquantitativeandqualitativedata.
■■ Potentialproblemswithdatainterpretation.
■■ Gapanalysisofaudiencesatisfaction.
■■ Developingpersonasoruserprofilesforservicedevelopment.
5.1 Introduction
5.1.1 Thissectiondealswiththeissuesinvolvedinanalysingandinterpretingthedatatoaddress
thespecificresearchquestionsandbuildupabetterunderstandingoftheaudienceandtheir
interactionswiththeservice.
5.1.2 Audienceanalysisprojectsshouldstartwiththerecognitionthatfurtherknowledgeabout
theaudiencewouldbehelpfulegforplanningfutureservicedevelopmentoroptimising
marketingstrategy.Thisleadstoframingthebroadobjectivesfortheaudienceanalysisand
settingoutspecificquestionstheresearchshouldaddress.Methodsforcollectingrelevant
datashouldbeselectedandthedetailsofthemethodologyestablished.Datacollectioncan
thenbecarriedout.
5.1.3 Theaudiencedatamustbeanalysedappropriately:thetechniquesusedwilldependon
thetypeofdataandtheaimsoftheresearch.Thedatashouldprovideevidencethatcan
beinterpretedtodrawconclusionsrelevanttothespecificobjectivesoftheresearch.Itis
importanttobesurethatthedatareallydoprovideevidenceforanystatementsyoumake
The Guide to Researching Audiences
PAGE 48 5. Modelling the Audience
egandthatyouhaveconsideredallthepossibleinterpretationsofthedata.Fullanalysisand
interpretationofthedatashouldalsoprovidewiderinsightsintotheaudience,theirbehaviour
andtheirrelationshipwiththeservice.
5.1.4 Analysingthedatafromaudienceresearchwillbeeasieriftheresearchwaswellplanned
andcarriedoutproperly.Manyproblemsinanalysisandinterpretationcanbeavoidedifdata
collectionproceduresareselectedandimplementedwiththeanalysisinmind.Sometimes
atrade-offneedstobemadebetweensimplicity,convenienceandreliabilityofrecording
andconvenienceandpowerforanalysis.Paper-baseddatacollectionmightbeeasiestto
administer,butdatawillhavetobecodedandenteredontoacomputerbeforeanalysis.
5.1.5 Analysisofthedatafromanaudienceresearchprojectshouldimproveunderstandingofthe
audienceinseveralways,providing:
■■ answerstothespecificresearchquestions–oratleastrelevantevidence;
■■ newknowledgeabouttheaudiencemoregenerally;
■■ furtherquestionsabouttheaudiencetobeaddressedinfuturework.
5.2 Analysing quantitative data
5.2.1 Quantitativedatacanbeanalysedstatistically.Detailedexplorationofthemethodsisbeyond
thescopeofthisguideanditisrecommendedthatreadersconsultastatisticalhandbookfor
psychologyorthesocialsciencesegStatistical Methods in PsychologyDavidHowell(2006).
TheuserguidesforstatisticalsoftwarepackagesegSPSSareoftenhelpful.
5.2.2 Statisticaltechniquescanbeusedtoexplorerelationshipsbetweenvariables.Forexample,
toanswerquestionssuchas‘areregisteredusersmorelikelytodownloadfilesfroma
service?’andtouncoverhomogeneoussubsetsorsegmentswithinanaudience.
5.2.3 Quantitativesurveysmayincludequestionsinvitingorrequiringfreetextcommentsorsimply
acatch-allinvitationattheend,forexample,‘Pleaseusethisspacetomakeanyother
commentsaboutourservice’.Datafromtheseitemswillneedtobecodedforquantitative
analysisandmightbemoreappropriatelytreatedasqualitativedata.Thismaymeanmaking
thequalitativecommentsavailableinadifferentformattofacilitatecollationandanalysis.
Regression and correlation5.2.4 Regressionandcorrelationanalysisaremethodsfordefiningrelationshipsbetweenvariables.
■■ Correlation analysislooksatthestrengthandnatureoftherelationshipbetween
variablesegdoesaccesstobroadbandinfluencedownloadingbehaviour,does
goodaccesstobroadbandincreaseordecreasethefrequencywithwhichcontentis
downloaded,doesitincreaseordecreasetheaveragesizeoffiledownloadedetc.
■■ Regression analysisisusedtoderivepredictionsaboutthevalueofonevariable,
basedonanother;multipleregressionanalysisconsidersseveralvariables:however
one–adependentvariable–isgenerallypredictedorexplainedbymeansoftheother
independentvariable(s)andcovariates.
5. Modelling the Audience PAGE 49
The Guide to Researching Audiences
Factor and cluster analysis5.2.5 Boththesetechniqueslookatrelationshipsamongvariables.Theyarenotgenerallyusedin
prediction.Theresearchermustinterprettheoutputoftheanalysistoderiveamodelofthe
audiencewhichbestfitsthedata.
5.2.6 Inverysimpleterms,cluster analysisaimstofindrelativelyhomogeneousgroups
orclusters,themembersofwhichtendtobehaveandthinkinsimilarways.Different
mathematicaltechniquescanbeusedtoidentifyclusterswithinmulti-dimensionaldata.
5.2.7 Factor analysisisastatisticaltechniqueusedtoanalyserelationshipsbetweenalarge
numberofvariablesandtoexplainthesevariablesintermsofcommonunderlyingfactors.In
audienceresearchitisusedtosimplifythenumberoffactorsorcategoriesneededtoexplain
ormodeltheaudience(egfromalargeseriesofstatementsaboutsatisfactionwithaservice
thefollowingfactorsmightemerge:easeofaccess,qualityofcontent,valueformoney).It
isuptotheresearchertoinspectthewayinwhichvariablesaregroupedandassignuseful
namestothefactors.Factoranalysiscanbeusedtodiscoveraschemeforsegmentingan
audience.Ratherthandecidinginadvancewhatthedefiningcharacteristicsofeachsegment
are,factoranalysisletssegmentsemergefromthedata.
Potential problems with interpretation of quantitative data
Instructions and/ or responses are misunderstood
5.2.8 Mostoftheproblemsarisebecausetheaudiencesampledidn’treacttotheresearch
instrument–usuallyasurveyquestionnaire–astheaudienceresearcherintended:
■■ someoralloftherespondentsdidn’tinterpretthequestionsinthewaythattheaudience
researcherintended;
■■ someoralloftherespondentsdidn’tusetheresponseoptionsastheresearcherintended
egscalesusedthewrongwayround,ticksusedinsteadofnumericalpreferenceratings,
multipleresponsecategoriesselectedwhereonlyonewaspermittedetc.;
■■ researcherfindstheresponsesunexpectedlyambiguous;
■■ largenumbersofrespondentsfailtocompleteallthequestions.
Someoftheseproblemscanbeavoidedbypilotingaquestionnaireonasmallsampleor
usinginterviewerstocollectsurveydata.
Questionnaire design is important
5.2.9 Surveysoftenaskrespondentstorateaspectsoftheservice.Sometimesnumericalscales
areused,sometimesverbalorpictorialscalesareused;somescaleshaveaneutralmidpoint,
somequestionswillallow‘notapplicable’or‘don’tknow’asoptions.Thesedecisionsabout
questionnairedesigncanhaveimplicationsfortheinterpretationoftheresults.
The Guide to Researching Audiences
PAGE 50 5. Modelling the Audience
■■ Theexactwordingofstatementsaboutsatisfactionwiththeservicecouldaffectthe
proportionofthesamplewhoappeartobecontentwiththeservice.Respondents
mayuse‘good’tomean‘acceptable,meetingtherequiredstandard’or‘betterthan
acceptable,exceedingtherequiredstandard’.Doyouknowwhatrespondentsmeant
whentheycompletedyourquestionnaire?
■■ Respondentstendtoavoidusingtheendpointsofanyscale,sothatveryfew
respondentswillappear‘extremelydissatisfied’or‘extremelysatisfied’withwhateveris
beingrated.
■■ Respondentswilltendtorespondrelativetowhattheythinkthedefaultoraverageisegif
askedtorateaccessibilityonaonetofivescaletheymightactuallybecomparingyour
servicetowhattheyperceivetobetheaverageforwhatevertheyconsidertobesimilar
services.
The sample of respondents is unrepresentative
5.2.10 Obtainingarepresentativesampleisparticularlydifficultwhensurveysareplacedonlineor
distributedtoanunknownsetofpotentialrespondentsegtoanemailaddresslistincludinga
lotofinvalidaddresses.
5.2.11 Sometimesitiseasytodetectabiasinthesampleegonly10%ofrespondentsarestudents
butyoualreadyhavereliableinformationthatatleast40%ofyouraudiencearestudents,but
iflittleisknownaboutthecompositionoftheaudiencethisismoredifficult.
5.2.12 Thesamplemaybebiasedbyanumberoffactors:
■■ thetypeofincentiveused;
■■ thetimeperiodoverwhichresearchwascarriedout;
■■ themethodofdistributionandpublicity/promotion;
■■ thekindsofquestionsasked.
5.2.13 Ifthesampleislikelytobeunrepresentativeitisimportanttoconsidertheimplicationsfor
interpretationofthedataandmakeclearthecaveatswhichapplytotheconclusions.Ifthere
isnoreasontobelievethatadifferentsamplewouldhaverespondeddifferentlythiscanbe
stated.Ifyouareawareofimportantdifferencesinthecharacteristicsofdifferentaudience
segmentswhichcouldaffectthefindingsthisshouldbeexplained.
Consider this example….
Results from a survey suggest users would prefer money to be spent on making more programmes available to download rather than increasing the number of programmes available with subtitles.
Thismaybebecausethesurveywasmentionedbyapresenteron‘Livingwithhearingimpairment’andoverhalftherespondentssaidtheyconsideredthemselveshearingimpaired.
Itwouldbeusefultostatetheresultsfornon-hearingimpairedrespondentsseparately.
Itisimportanttodetectthebiasinthissamplebecauserelevantdemographicdatawerecollected,butitisoftennotsoobvious.
5. Modelling the Audience PAGE 51
The Guide to Researching Audiences
5.3 Making use of qualitative data
5.3.1 Interpretingqualitativedataisnecessarilyasubjectiveprocess,soifresourcespermititmay
beworthinvolvingmorethanonepersoninanalysisandpresentationofthedata.
5.3.2 Itispossibletocodeorcategorisequalitativedatasothatquantitativetechniquescanbe
appliedegassigningstatementsofattitudetowardstheservicemadeduringfocusgroup
discussionstocategories:verypositive,positive,neutral,somewhatnegative,verynegative.
Howeverdoingthisresultsinlossoftherichdetailwhichmakesqualitativedatasovaluable.
5.3.3 Presentationsofqualitativedatawillusuallyattempttosummarisethecontentandimportof
thematerial.Foragroupdiscussionthiswouldusuallyincludeajudgementaboutvariations
ofopinionbetweengroupmembersandcommentsonpossiblereasonsfordifferencesof
opinion.Abriefsummaryoftheflowofdiscussioncanbehelpfultoelucidatethecontextfor
theviewsandattitudesexpressedbygroupmembers.
5.3.4 Presentationsofqualitativedatawillalsotypicallyincludequotationsfromthematerial.If
quotationsaretobeuseditisimportanttoensurethattheyareaccurate,sointerviewsor
discussionsneedtoberecordedinfull.
5.3.5 Codedqualitativedatacanbepresentedintabularorgraphicalformatasasupplementto
textualdigestsofthematerial.Quantitativeanalysesofqualitativedatafromasmallsample
mightbeworthwhileifyoubelieveyoursampletoberepresentativeoftheaudienceasa
whole,butitwillneverbeasubstituteforarobust,statisticallysoundanalysisofalarger,
morerepresentativesample.
5.3.6 Perhapsamoreinterestingapplicationofquantitativetechniquestoqualitativedatauses
theindividualastheunitofanalysisandlooksatdistributionofbehavioursorcomments
forasingleindividual.Forexamplebrowsingbehaviourcouldbeassignedtocategories
egsearchingasite,reading,downloading,useofsearchengine,enteringdataetc.The
percentageoftimespentoneachactivitycanthenbecalculated.Similarlystatementsmade
duringinterviewcouldbecodedandthefrequencywithwhichaparticularaspectofthesite
orresourcewasmentionedcouldbecalculated.Statementscouldalsobecodedbystrength
offeelingtoassesswhichaspectoftheserviceprovokedmostsatisfactionorfrustration.
5.3.7 Theimportantpointinthistypeofanalysisisthatstatementsaremadeaboutanindividual,
ratherthanthegroupeg‘90%ofsubjectA’sreferencestositecontentweretostoriesabout
sport’.Ifalltheintervieweesreferredtosportmorethanothercontentitmightbealsobetrue
thattheaudienceasawholewouldrefermosttocontentaboutsport,butunlessthesample
sizeislargeenoughtopermitstatisticalanalysessuchaconclusionwouldbeunreliable.An
alternativepossibilityisthatwhilstnotalltheaudiencewouldrefermosttosportsstories,
theymightalltendtohaveafavouritetypeofcontent.Youmightliketoconsiderhow
qualitativefindingslikethesecouldbefollowedupwithquantitativeresearch.
The Guide to Researching Audiences
PAGE 52 5. Modelling the Audience
5.4 Analysing audience satisfaction for service development
5.4.1 Weighted gap analysisisatechniqueforcomparingaudiencesatisfactionwitheachaspect
ofaservice,takingintoaccounttheimportanceattachedtoeachaspect,inordertoderive
prioritiesforserviceimprovement.Inoutline,thetechniqueisusedasfollows:
■■ respondentsarepresentedwithalistoffactorsegcost,valueformoney,easeofaccess,
breadthofcoverageetc.;
■■ respondentsgiveasatisfactionratingandanimportanceratingforeachfactor,usually
accordingtoanumericalscale;
■■ foreachrespondent,theimportancescoreforeachfactorissubtractedfromthe
satisfactionscoreforthatfactor:thisisthegap;
■■ thegapisweightedaccordingtotherelativeimportanceofeachfactor.
5.4.2 Thefollowingtableillustratestheprocess.
Table 5‑1weighted gap analysis
Factor Importance rating(1-10)
Satisfaction rating(1-10)
Gap Importance co-efficient
Weighted gap
Priority
Cost 9 6 3 90% 2.7 3
Value for money 10 8 2 100% 2 5
Ease of access 8 3 5 80% 4 1
24/7 access 5 7 -2(effectively,0)
50% 0 8
Online access 9 8 1 90% 0.9 6
Ease of website navigation
8 4 4 80% 3.2 2
Help and advice 7 4 3 70% 2.1 4
Innovative use of technology
6 5 1 40% 0.4 7
5.5 Personas and user profiles
Personas5.5.1 Personasareessentiallyanaidtoservicedevelopment.Itiseasiertodesignaserviceto
meettheneedsandexpectationsofonespecificuserthantryingtodesigntoabstract
criteria.Personascanhelptoprovidefocusforservicedevelopmentworkthathasfairlyhigh
level,genericobjectivesegtomakethesiteeasiertouseor‘makethesitemoreappealingto
schoolagechildren’.
5. Modelling the Audience PAGE 53
The Guide to Researching Audiences
Consider this example….
The scenario
Anewspaperisredevelopingitswebsiteandneedstoknowwhowilluseitandwhattheywillwantfromit.Theteamemployedtodeveloppersonaslookatresearchonusersofotheronlinenewspapers,demographicdataaboutthenewspaper’sreadershipandpeopleintheregion.Theyalsocarryoutlengthyinterviewswithpeopleintownsandvillagesintheregion,askingabouttheirlifestyle,theirwork,howmuchtheyusecomputers,wheretheygettheirnewsandentertainmentfrom,petgripeswithtechnologyetc.Asaresultofthisprocesstheydevelopfourdistinctpersonas:oneisan‘uncommittedinformationsnacker’calledDan.
Persona #1: the uncommitted information snacker
Danisinhismid-30sandisManagerofHumanResourcesatGobbleandGookinLittleborough.Danisunmarried,butusuallyspend3or4nightsaweekathisgirlfriend’splace.TheygooutfordinneracoupleoftimesamonthandtheirfavouritethingtodoonSundayeveningisgetaDVDandphoneforpizza.ThesedaysDandoesn’tbuyanewspaper–hesometimeslooksatnewspapersonlinebutheoftenusestheBBCsiteashethinksthecoverageislesslikelytobebiased.Helikesbeingabletogetsportsresultsonlineandlikeshowquicklythewebsitecanbeupdated.Hecommunicateswithhismatesbytextmessageandemailshisbrotheraboutonceaweek.Hehasaworkemailaddressandapersonalemailaccount,he’snotthatbotheredaboutspamoraddressspoofing–hedoesn’tdomuchshoppingonlineandreckonstherisksareexaggerated.Heusestheinternetathometocheckfilmtimesandfindnewrestaurants.HeoccasionallybuysbooksfromAmazon.He’saRadioheadfanandthoughtbeingabletopaywhathelikedtodownloadthenewalbumwascool.Danrateshimselfasprettygoodwithcomputers,butisdefinitelynotageek.HeenjoysplayingfootballwithfriendsandhisgoalsinlifearetosaveenoughtobuyahouseandtraveltoSouthAmerica.
Service redevelopment to appeal to Persona #1
ThedesignteamwilldevelopthewebsitetoappealtoDan:
■■ thehomepageofthewebsitewillfeaturemoresportsstories;
■■ thehomepagewillfeaturea‘breakingnews’tickertape–clickingonthiswilltaketheusertothespecificnewsflash;
■■ reviewsforfilmsreleasedonDVDinthelast6monthswillbepostedonlineandthispagewillbelinkedfromboththeFilmpageandtheTVlistings;
■■ itwillbepossibletoviewfilmreviewsaccordingtolocalavailabilityandseescreeningtimesatlocalcinemas;
■■ somenewsstorieswillbetreatedintheroundwithdifferent‘perspectivepieces’toshowwhatthestorywillmeanfordifferentpeopleandcoveragefromothermediasourcestoprovidethe‘balancedcoverage’Danvalues;
■■ thewebsitewillincludea‘today’snewsinpictures’page,withquirkyphotographstoillustratestories–clickingtheimagewilllinktofurthercoverage.
5.5.2 Apersonaisanarchetype,withthepersonaltraits,lifestyle,aspirations,beliefs,needsand
goalsetc.typicalofasegmentoftheaudience.Thekeytosuccessfuluseofpersonasis
developingrichlydetailedpersonasthat:
■■ accuratelyreflectdifferentaudiences;
■■ actasabelievable,realisticstand-inforarealusersothatthedesignteamdevelopa
strongsenseofwhatwillbeappealingandusablefortherealaudience.
The Guide to Researching Audiences
PAGE 54 5. Modelling the Audience
5.5.3 Theprocessofdevelopingapersonaisoftenquiteinvolved,includingdetailedinterviews
withpotentialusers,ethnographicstudies,novelactivities–anythingwhichwillhelptofoster
agoodunderstandingoftheaudience.Interviewingforpersonaresearchisaskilledand
lengthyprocess;theinterviewerneedstobuildagoodrapportwiththeintervieweeinorderto
elicitverydetailedandpersonalinformation.Constructingapersonafromalltheinformation
thathasbeengatheredisoftendoneasaworkshopactivity.
5.5.4 Differentpersonascanbedevelopedtorepresenttheaudiencesegmentswithdifferent
relationshipstoaservice.Thereshouldusuallybeaprimarypersona,representingthehigh
priorityaudienceforthedevelopmentactivitybeingundertaken.Sometimesitishelpfulto
includea‘negativepersona’,anarchetypeforwhomtheserviceisnotintended.
User profiles5.5.5 Ifresourcesdonotpermitthecreationoffullyfledgedpersonasitmaybepossibletoachieve
someofthesamebenefitsbydevelopinguser profiles–descriptionsoftypicalusers,
orientedtotheserviceunderconsiderationandfocusingoninteractionswiththeservice,
needs,expectationsandattitudestowardstheservice.
Consider this example….
Alargeteachinghospitalisevaluatingitslibraryservicesanddevelopsaseriesofuserprofilestoensurethattheservicesmeettheneedsofthedifferentusers.
Oneoftheprofilesisfora‘medicalspecialistintraining’:
■■ needsaccesstospecialistjournalarticlesandtextbooks;
■■ preferselectronicresourcestopaper-basedresources;
■■ willwantaccesstocontentoffsite(forhomestudyandrevision);
■■ needsaccesstoresourcesandservicesoutsidenormalworkinghours;
■■ busyscheduleandchangingworkpatternsmakeflexibility,remoteandmobileaccessakeyrequirement.
6. Making use of Audience Research PAGE 55
The Guide to Researching Audiences
6.MakingUseofAudienceResearch
This chapter reinforces the message that audience analysis should be seen in the context of the service as a whole
Audienceanalysiscanprovidemanybenefits,butanydecisionsonuseofthedatawilldependonthewiderservicecontextandpriorities.
Astrategyforrealisingthefullvalueofaudienceanalysisworkisdiscussed.
6.1 Introduction
6.1.1 Audienceresearchisatoolforservicedevelopmentandcanbeusedtoprovidethebenefits
discussedinSection3,howeveraudienceresearchisanaidtodecisionmakingandnot
asubstituteforit.Evidencefromaudienceresearchcanhelpserviceproviderstomake
informeddecisionsaboutspendingondigitisation,newservices,supportfordifferent
platforms,promotingtheservicetonewaudiencesetc.butitisnotasubstituteforbusiness
planningorthepoliticaldecisionsthatneedtobemadeaboutprioritiesforpubliclyfunded
services.
6.1.2 Becausedistinctaudiencesegmentshavedifferentrequirementsitisimportanttounderstand
therelationshipbetweenthemandprioritiseallocationofresourcesappropriately.Audience
analysiscanprovideinformationaboutaudiencesegments,theirrelativesizeandhowthey
use,valueandaccessaservice,butdecisionsaboutresourceallocationstilldependon
servicepriorities.
6.1.3 Decisionsabouthowtodevelopandpromotethewebsitearestillbusinessdecisions,not
leastbecauseuseofthewebsitewilldependonthequalityoftheresource,andthesuccess
ofthemarketingstrategy.Ofcourseaudienceanalysisshouldinformboththeseactivities.
The Guide to Researching Audiences
PAGE 56 6. Making use of Audience Research
Consider this example….
Amuseumhasdevelopedawebsitewhichprovidesdetailedinformationaboutitemsinthecollections,regularlyupdatedinteractiveonlineexhibitions,ablogaimedatthosewithaprofessionalinterestinthemuseumssectoranddigitisedimagesoftheirmostrenownedexhibits.Audienceresearchshowsthat98%ofvisitorstothesiteonlyviewpageswithtravelinformationandopeningtimes.
Weretheresourcesusedtodevelopthewebsitewasted?Shouldtheexistingwebsitebemaintained?
Theanswerstothesequestionsdependpartlyonwhothetargetaudienceforthewebsiteisandthisisapoliticalandbusinessdecision.Perhapsthemuseumrecognisedthatmostvisitorstothewebsitewouldonlywantbasicinformation,yetstillsoughttoattractasmallaudienceofacademicsandprofessionalswhowouldmakemoreextensiveuseofthesite.
Followupresearchorfurtheranalysisoftheexistingdatamightaddresstheseadditionalquestions:
■■ Hasthesitebeensuccessfulinattractingthisnicheaudience?
■■ Doesthesiteservetheneedsofthisaudience?
Alterationstotheserviceorachangeincommunicationsstrategymightbeneeded.
Otherquestionscannotbeansweredsolelyonthebasisofaudienceresearch:
■■ Shouldthesitebepromotedtovisitorstothephysicalmuseumasapost-visitresource?
■■ Canthemuseumusethewebsitetodevelopanongoingrelationshipwithitsaudienceegallowingthemtovoteforitemstogoondisplay,postcommentsaboutexhibitionsetc.?
6.2 Audience research in context
6.2.1 Attheconclusionofanaudienceresearchproject,whenthedatahasbeencollected,
analysedandinterpretedandconclusionsdrawnitisimportanttomakesurethatthefull
valueoftheresearchisrealised.Forexample:
■■ Theresultsoftheresearchshouldbedisseminatedtostakeholdersandotherinterested
parties.Thismayinvolvepreparingseveraldifferentpresentationsoftheworkfordifferent
audiences,egseniormanagement,funders,generalstaff,communicationsteametc.It
maybeappropriatetoprepareasummaryofthefindingsandfollow-upforparticipants;
peoplearemorewillingtotakepartinresearchiftheyfeeltheircontributionisappropriate
andtheresearchislikelytohaveanimpactontheservice.
■■ Everyonewhohelpedwiththeprojectshouldbethanked;thisincludesstaffwhohelped
tocarryouttheresearch,participantsandanyoneelsewhocontributed.Thiswillpay
dividendswhenfurtheraudienceanalysisworkisundertakenandcontributestoobtaining
buy-inforaudienceanalysisthroughoutaservice.
■■ Considertheimplicationsofthefindingsforservicedevelopmentandaudience
development.
■■ Lookattheconclusionsoftheresearch–werethespecificobjectivesmet?Whatlessons
canbelearnedforfuturework?
■■ Planfollow-upandfutureresearch.
6. Making use of Audience Research PAGE 57
The Guide to Researching Audiences
■■ Assessthewiderimplicationsofthework:
■− istheservicebeingprovidedalignedwiththestatedservicemission,valuesandremit?
■− arethereanyimplicationsforlong-termstrategy?
■− isthebusinessplanorsustainabilitystrategyfortheserviceappropriate?
6.2.2 Itmaybeappropriatetosharesomeofyourfindingswithpartnersinthesamesector,other
publicsectororganisationsservingasimilaraudienceetc.
6.2.3 Itshouldalsobenotedthataudienceresearchprojectswilloftenresultinchange–whether
itbedirectlytotheserviceoffered,ortotheorganisationitself(eg asaresultofachangein
companystrategy).Changemustbecarefullymanaged,andwillentailthoughtfulplanning
andsometimessensitiveimplementation,andaboveallengagementwiththepeopleaffected
bythesechanges.Usefulfurtherreadingonthemanagementofchangeinaprogrammatic
contextisManaging Successful Programmes (MSP),OfficeofGovernmentCommerce(OGC),
2007edition.
PAGE 58 Appendix A: Glossary
AppendixAGlossary
Audience development Activitiescarriedouttoincreasetheaudienceforaserviceor
resource.
Audience satisfactionEstablishingiftheaudienceishappywiththeservicethatisbeing
provided.Forexample,isithappywiththequality/content/accessibility/comprehensiveness
oftheservice?Interrogationofaudiencesatisfactionisoftendoneintandemwithaudience
analysis.
Digital service and digital audienceInthisguidedigitalisusedtorefertoresourcesand/or
servicesexploitingdigitisedcontentordeliveredviaadigitalchannel.Inmostinstancesthis
meansaservicewhichisavailableonlinebutitalsoincludesdigitisedresourcesdistributed
onCDandcouldincludedigitaltelevisionserviceswhichsharesomeofthecharacteristicsof
onlinedigitalservices.Digital audiencereferstotheaudienceforadigitalserviceorresource.
Ethnographic researchResearchbasedonobservationsofhumanbehaviour,carriedoutin
a‘naturalistic’setting.Thiscaninclude‘observation’viaautomatedrecordingdevicesandis
sometimesextendedtoincludeobservationsofsubjectsmadeinalaboratory–thecrucial
pointisthatbehaviourinresponsetotherelevantstimulishouldbenormalandthatthe
presenceoftheresearcherortheobservationalequipmentshouldnotinterferewithoraffect
thesubject’sbehaviour.
Focus group Agroupofindividualsselectedandassembledbytheresearchertodiscuss
andcommentonthetopicunderconsideration.Subjects’personalexperienceofthetopic
andinteractionsamongstgroupmembersarethedistinguishingfeaturesofthetechnique.
Formative research Inthecontextofservicedevelopmentanddeliverythistermcovers
researchcarriedoutduringservicedevelopmentwork,aspartofthedevelopmentprocess,
toprovidefeedbackonthework.
Front end researchInthecontextofservicedevelopmentanddeliverythistermcovers
researchcarriedoutintheplanningandpreparationofaservice.
Impact analysisAnalysisofthelonger-termeffectsofaservice(orgroupofservices)on
itsaudience.Longer-termevaluationmethodsarerequiredtodetermineiftheimpactofa
serviceislasting.
Appendix A: Glossary PAGE 59
The Guide to Researching Audiences
Reach Inthisguidereachisusedtorefertothenumberofpeopletowhomaserviceor
resourceisavailable.Differentfromuptake whichreferstousageoftheserviceorresource.
Segmentation Theprocessofdividinganaudienceintodifferentcategoriesorsegments
onthebasisofcriteriarelevanttotheserviceorresourceunderconsideration.Segments
arerelativelyhomogeneousanddistinctfromothersegmentswithrespecttothechosen
characteristics.
Summative research Inthecontextofservicedevelopmentthisreferstoevaluativeresearch
assessingtheextenttowhichaservicehasmetspecificcriteriaforsuccess.
Target audience Theintendedaudienceforaserviceorresource.
UsabilityAmeasureofthequalityoftheuserexperienceofaserviceorresource.Itgenerally
encompassesfactorssuchaseaseoflearning,efficiencyofuse,memorability,error
frequencyandseverityandsubjectivesatisfactionwiththeinterface.
User-centred design Adesignphilosophywhichplacesthegoals,expectationsandskillsof
usersatthecentreofthedesignprocess:theserviceorproductisdesignedtofittheuser.
Typicallyusersareinvolvedthroughoutthedesignanddevelopmentprocess,includinguser
testingoftheserviceorproductwhilstitisindevelopment.
PAGE 60 Appendix B: Reasons to do Audience Research
AppendixBReasonstodoAudienceResearch
This chapter sets out the reasons for doing audience analysis and some of the uses of the research
Experience and instinct are not a substitute for the information that systematic research can provide.
Theusesofaudienceanalysisinclude:
■■ Servicedevelopment.
■■ Serviceevaluation.
■■ Accountabilitytofunders.
■■ Long-termplanning.
■■ Sponsorshipandmarketing.
■■ Businessplanning.
■■ Managementofcontractualrelationships.
■■ Professionaldevelopment.
B.1 Why do audience research?
B.1.1 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstrate
thattheyareusedandvaluedbyanappropriateaudience.Professionalsoftenfeeltheyknow
‘their’audienceandareconfidentthattheservicetheyprovideismeetingthataudience’s
needs.Experienceandinstinctareoftenasourceofdetailedknowledgeandvaluable
insightsintouserneeds,valuesandbehaviour,howevertheyarenotasubstituteforthemore
completepicturethatsystematicaudienceresearchprovides.
B.1.2 Itisimportanttoremainopen-mindedaboutwhataudienceresearchreveals,andalertto
changesinthenatureandexpectationsoftheaudience.Therapidpaceoftechnological
changemeansthattheaudiencesfordigitalservicesandcontent,andtheexpectationsof
theseaudiences,isalsoevolvingrapidly.
Appendix B: Reasons to do Audience Research PAGE 61
The Guide to Researching Audiences
B.1.3 Itseemsself-evident,butit’sworthstatingunambiguouslythatthe more you understand
about your audience, the better you will be able to meet their needs – and demonstrate that
you are meeting their needs.
B.1.4 Asatisfieduserwillspreadthewordaboutagoodserviceandpromoteaudience
development.Satisfiedandengageduserscanbeavaluableresource:theymaybemore
willingtoshareinformationabouttheirattitudesandhabitsaswellastheirexperienceof
yourservice.Thisfeedbackcanbevaluableindevelopingandenhancingyourserviceinthe
future.
B.1.5 Evenasmallaudienceresearchprojectisworthwhile.Smallerservicesoperatingona
restrictedbudgetshouldnotbeintimidatedbytheconceptofaudienceanalysis.Many
techniquescanbeimplementedquitecheaplyoradaptedtoashoestringbudget.Some
insightintoyouraudienceisvastlybetterthannoneatall.
B.2 The uses of audience research
B.2.1 Theusesofaudienceresearchextendbeyondsimplydemonstratingacertainnumberof
visitorsorwebsitehitstotickaboxonaservicelevelagreement.Someoftheareaswhere
youmayexpecttorealisebenefitsaresetoutbelow.
Product and service developmentB.2.2 Thisisperhapstheareaofaudienceresearchwhichappealsmosttoprofessionalsandstaff
whoengagedirectlywiththeaudience.
B.2.3 Inplanninganewserviceitisimportanttoinvestigatewhoislikelytousetheservice,what
theyexpect,theirpreferredmeansofaccessetc.
B.2.4 Involvingtheaudienceinservicedevelopment–includingtheuser-centred designprocesses
–isincreasinglyviewedasanimportantwayofensuringthataserviceorresourceisusable
andusefulforthetargetaudience:directinvolvementofpotentialusersmakesiteasierto
ensurethattheservicewillmeetexpectationsandthatuserswillbeabletoaccomplishtheir
goalwhentheyusetheservice.
B.2.5 Involvingusersiniterativetestingofprototypesorpilotserviceswilloftenuncovernewand
unexpectedinformationaboutuserbehaviouranduserrequirements,orproblemswith
usabilityandaccessibilitythathadnotbeenanticipated.Gettingthisinformationwhilsta
serviceisbeingdevelopedallowsproblemstoberectifiedandimportantnewfunctionality
tobeadded.Itpotentiallyhelpstoavoidlaunchingaservicewhichisanexpensivefailure
becauseitfailstomeetaudienceneeds.
The Guide to Researching Audiences
PAGE 62 Appendix B: Reasons to do Audience Research
Service evaluationB.2.6 Satisfiedusersaremorelikelytoreturntoawebsite,recommendaserviceorseekoutother
servicesandresourcesfromthesameprovider.Onlyifevaluativeresearchiscarriedoutwill
informationaboutthesize,natureandsatisfactionoftheaudiencebeobtained.Audience
researchonsatisfactionwiththeservice,perceptionsoftheserviceandattitudestowards
itareanimportantcomponentofevaluatingthe‘success’ofaservice.Audienceanalysis
shouldalsoformpartofananalysisofthelongertermorbroader‘impact’oftheservice.The
resultsofevaluativeresearchcanbefedbackintothenextiterationofservicedevelopment.
B.2.7 Collectingandanalysingdataonwhoisactuallyusingtheservice,whetherusersfindit
appealing,informative,valuableetc.isacriticalpartoftheservicedevelopmentcycle.For
example,itshouldpickupanyremainingtechnicalproblemsorconcernsaboutusabilitythat
needtobeaddressed,oritmightrevealthattheaudiencearefindingnovelwaystousethe
servicethatcouldbefurtherexploited.Evaluativeaudienceresearchwillprovideevidence
onhowsuccessfulyourservicedevelopmentprojecthasbeenandhowappropriateand
effectiveanyrelatedcommunicationsandmarketingeffortwas.
Accountability to fundersB.2.8 Publiclyfundedservicesareincreasinglyexpectedtoshowthattheyaredelivering‘valuefor
money’.Audienceresearchshouldprovideimportantevidence:numberofusers,frequency
ofuse,qualityoftheuserexperience,valueor‘impact’oftheserviceetc.Audienceresearch
shouldbetailoredtoshowthataparticularserviceismeetingitsownspecificobjectives.
B.2.9 Ifyouraudienceissmall,youmaynonethelessbeabletoshowthatyourserviceis
worthwhileegifaudienceresearchdemonstratesthatitis:
■■ usedbyahard-to-reachsegment;
■■ highlyvaluedbyanicheaudienceorbyparticularlyinfluentialusers.
B.2.10 Knowledgeoftheseaudiencescanbeusedmorewidelyegtoimproveservicedeliveryfor
thehard-to-reachsegmentorextendingserviceprovisionfornicheaudiencesthroughbetter
understandingoftheirspecialcontentand/ortechnicalrequirements.
B.2.11 Achievinggoodreachanduptakemaybeimportantforpubliclyfundedservices.Information
aboutwhonon-usersareandwhytheyarenotusingyourserviceprovidesusefulevidence
onwhichtobaseastrategytoincreasethereachoftheserviceandcanhelptomakeany
requestforadditionalfundingmorepersuasive.
Long-term planningB.2.12 Awiderangeofaudienceresearchmayberelevant,includingdataondemographicand
socialtrends.Longitudinalsurveydata,iethesameorverysimilarinformationcollected
atseveraltimepoints,isoftenpubliclyavailable.Thiskindof‘audienceanalysis’canbe
usefulforlong-termplanning.Itcanhelpservicesanticipatechangesinaudienceprofile,
lifestyle,habitsandpreferredplatforms(egincreasinguseofmobiletechnologies)andadjust
accordingly.
Appendix B: Reasons to do Audience Research PAGE 63
The Guide to Researching Audiences
B.2.13 Aservice’sownlongitudinaldataareevenmoreusefulforspottingandtrackingemerging
trends.Ifadeclineinusebyaparticularaudiencesegmentisspottedatanearlystagethere
ismuchgreaterscopeforinvestigatingandpotentiallyreversingthedeclinebyadjustingthe
service.Similarly,longitudinaldatarevealingtheemergenceofanewaudiencecanbeused
toplanservicedevelopmenttailoredtotheirneedsifappropriate.Audiencesuggestionsand
responsestoqueriesaboutpotentialserviceenhancementcanbefollowedup.
Sponsorship and marketingB.2.14 Goodevidenceaboutyouraudienceisveryvaluablewhenitcomestoattractingsponsorship
ormarketingyourservice.Apotentialsponsorismorelikelytocommitifyouareabletoshow
thatyouraudienceprofileisalignedwiththesponsor’sclientbaseandvalues.Aneffective
marketingandcommunicationsstrategywilldrawoninformationaboutexistingandtarget
audiencestomakesuremessagesareappealingandpresentedviaappropriatemediaand
platforms.
Business planningB.2.15 Audienceresearch(egbasedonsurveysofatargetaudienceincombinationwithengaging
theexistingaudienceandstakeholders)isofgreatassistanceinassessingthefeasibilityofa
plannednewserviceorserviceenhancement.Informationaboutthelevelofinterestamongst
thetargetaudience,likelyaudiencelevelsandaudienceexpectationsshouldformapartof
thebusinesscaseaswellasinformingservicedesignandmarketing.
Management of contractual relationshipsB.2.16 Wheresomeaspectofserviceprovisionisoutsourced(egredesignofawebsite)itmaybe
helpfultospecifythatacertainsizeofaudience,orlevelofaudiencesatisfactionorimpact,
isachieved.Itwillbeimportanttosetoutclearlyhowaudiencesizeandexperiencearetobe
measuredandwhatthecriteriaforsuccessare.
B.2.17 Thesameprinciplecanbeappliedinternally.Definingobjectivesforaprojectintermsofthe
audienceorincludingaudience-relatedmeasuresofsuccessinevaluationsmaybeuseful
waysofimplementingtheservice-orientedvaluesofapubliclyfundedorganisation.
Professional developmentB.2.18 Professionals’richanddetailedknowledgeaboutserviceusersisvaluableandcanoften
betheinspirationfornewinitiativesandplayavitalpartinservicedevelopment.Formal
audienceresearchshouldbeseenasawayofsupplementingandenlargingprofessional
knowledge.Involvingstaffinaudienceresearchmayhelptofocusresearchandcanbe
empoweringforstaff.
PAGE 64 Appendix C: Digital Audiences and Digital Services
AppendixCDigitalAudiencesandDigitalServices
This chapter sets out some of the differences between digital and non-digital services and audiences, and discusses the implications for audience analysis
The size, composition, attitudes and behaviour of the audience may be different in a digital environment.
■■ Digitalcontentisdifferentfromnon-digitalcontentandthishasimplicationsfortypesofservicethatcanbeoffered.
■■ Therelationshipbetweendigitalandnon-digitalworldsisevolvingandservicesshouldbeawareoftheimplications.
■■ Adoptionofnewtechnologiesshouldbedrivenbyaudienceandserviceneedsandexpectations.
■■ Differenttechniquesmaybeusedforanalysisofdigitalaudiences.
C.1 The digital revolution
C.1.1 Thedigitalrevolutionhasresultedinanexplosionofpossibilities:newandinnovative
services,adelugeofcontent,andpotentiallynewaudiencesandnewusesforoldcontent
mademorewidelyavailablethroughdigitisation.Contentandserviceprovidersareonly
beginningtoexploreandunderstandthescopeandconstraintsofthedigitalenvironment.
Theapproachtodigitalresourcesvariesacrosssectorsandservicesandissubjectto
revisioninresponsetotechnicaldevelopmentsandchangingpatternsofuserbehaviour.
Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesand
non-digitalservicesandbetweena‘digitalaudience’anda‘non-digitalaudience’areanew
andinterestingconsiderationforserviceproviders.
Appendix C: Digital Audiences and Digital Services PAGE 65
The Guide to Researching Audiences
C.2 Audiences in the digital environment
C.2.1 Thetargetaudience(seeSection2)foradigitalserviceneednotbethesameasthe
audienceforanon-digitalequivalent.Adigitalserviceprovidesanopportunitytorecruit
anewaudienceaswellasencouragingsomeoralloftheexistingaudiencetoconvertor
extendtheirusage.Thedigitalenvironmentchangesboththedemographiccompositionof
theaudienceandthewaysinwhichtheyinteractwithaserviceorresource.Usersbehave
differentlyinadigitalenvironmentandhavedifferentexpectations:
■■ Acceptanceandfamiliaritywithdigitalservicesisnotuniversalandthisaffectsthe
effectiveavailabilityofdigitalservicesegelderlypeoplemaybeuncomfortableusinga
computertoaccessservices.
■■ Accesstotheplatformsusedtodeliverdigitalservicesisvariableandthereforeaccessto
digitalservicesisalsovariableegfewerlowincomefamilieshaveaccesstobroadband
internetathome,orlaptopsformobileaccess.
■■ Accesstophysicalservicesisunevenlydistributedacrossthepopulation.
■■ Theaudiencecanaccessdigitalservicesinmultiplecontexts eglaptop,mobiletelephone,
computerathome,computeratwork,computerinapubliclibrary.
■■ Communicatingwiththeaudienceisgenerallyeasierandcheaper.
■■ Socialnormsandattitudestoprivacyaredifferent.
■■ Usersareoftenless‘committed’toaresourcetheychoosetoview–perhapsbecause
accessisrapid,orbecausemultiple,apparentlyverysimilarresourcesrelevanttothe
user’sobjectiveareavailable.
■■ Resourcediscoverystrategiesfordigitalcontentarelesssophisticatedamongstcertain
users.Thisisnotwellunderstood,butstudieshavefoundthatstudentstendtorely
onresourcesthatcanbediscoveredquicklythroughGoogle,10whilstmoreexpert,
experiencedresearchershavehigherexpectationsofresourcequality,whichtheypursue
throughmoresophisticated,multi-layereddiscoverystrategies.
■■ Theaudienceforadigitalserviceisoftenoperatinginanenvironmentwhichmeansthat
supportislessreadilyavailableegathome,outsideofficehours,wherenohelpdeskis
immediatelyavailabletoprovideadviceandguidance.
■■ Concentrationspantendstobeshorterinadigitalcontext11.
■■ Competitionfortheattentionoftheuserisgreaterinanonlineenvironmenteginstead
ofbeingfocusedsolelyonselecting,findingandusingarticlesinaphysicallibrary,the
userissearchingonlineandcanbedistractedbyevents,ormaybeactivelyattemptingto
multi-task.
■■ Physicalconstraintsonaudiencesizenolongerapplyeguserscanaccessanonline
digitisedresourceatanytime,fromanylocation.
■■ Newconstraintsonresourceseglimitationsonbandwidthmayaffectqualityofservice.
■■ Itiseasiertoservenicheaudiences.
10 Information behaviour of the researcher of the future, CIBER briefing paper; Jan 2008
11 What the Internet is doing to our brains – is Google making us stupid? Nicholas Carr, Atlantic Monthly Jul/Aug 2008
The Guide to Researching Audiences
PAGE 66 Appendix C: Digital Audiences and Digital Services
C.2.2 Someusersofdigitalserviceswillembracethefullpanoplyofso-called‘Web2.0’
functionality:12theywillreadandperhapsevencontributetoblogs,sharepersonal
photographsonline,usesocialnetworkingsites,subscribetoRSSfeedsforfavouritesites
anduseandcreatetagsastheybrowsedigitalcontent.However,mostoftheaudiencewill
exploitonlyatinyfractionofthisfunctionalityandsomewillbeactivelyuncomfortableinan
environmentwhichseemstoexpectthesebehaviours:theywillnotfeelpartoftheintended
audiencefortheservice.
C.2.3 Itispossibletotailoradigitalserviceorresourcetomeetthevaryingneedsofdifferent
audiencesegments(seesub-section5.4).Althoughtailoringtheservicemightrequirethe
usertoself-identifyasaparticularclassofuser,itcansometimesbebasedoninformation
abouttheuserobtainedfromathirdparty.13
C.3 Digital content is different
C.3.1 Digitisingcontentpresentsanopportunitytodevelopandextendfunctionalitytoprovidea
muchricherresource,buttheremayalsobepenalties.
■■ Digitalresourcespresentnewresourcediscoveryproblemsegplacingmetadataabove
theauthenticationlevelisimportant;searchengineoptimisationcanhaveabigimpacton
thevisibilityofaresource.
■■ Digitalresourcescanbemoreflexible:powerfulsearching,filteringandordering
functionalitycanbeprovided.
■■ Digitisingresourcesfacilitatescomparisonsbetweendatasets.
■■ Althoughmorepowerfulsearchingtoolsareavailablefordigitalresourcesitisarguably
easiertobrowsenon-digitalcontent.Thisleadstotheslightlycounterintuitivepossibility
thatserendipitousdiscoveryofinterestingcontentmaybemorelikelywhenusingnon-
digitalcontent.14
■■ Digitalcontentmaybeplatformand/orsoftwaredependent.
■■ Digitalcontentcanusuallybeaccessedanytime,anywhere.
■■ Itiseasiertoupdateadigitalresource.
■■ Digitalresourcesaremoreamenabletointeractivefunctionalityegwikis,novel
combinationsofresourcestoprovideaddedfunctionality(iemash-ups).
■■ Undercertaincircumstancesthedigitalenvironmentcanbetreatedasanexperimental
laboratory:smalladjustmentstoresourcesorservicescanbemadeanddataonthe
effectcollectedeasilyandrapidly.Thishasledtothenotionthatwebservicescanbein
‘perpetualbeta’ieadigitalresourcecanundergocontinualevolution.
■■ Itispossibletocreatecertainclassesofdigitalresourcesverycheaply:whenresource
commitmenttoanideaforservicedevelopmentorenhancementislower,thepenaltyfor
failureislowerandamoreexperimentalapproachispossible.
12 What is Web 2.0? Ideas, technologies and implications for education Paul Anderson, JISC Technology and Standards Watch Feb. 2007
13 Developing personalisation for the information environment 2; JISC; 2008.
14 Great minds think (too much) alike? The Economist (17th July 2008).
Appendix C: Digital Audiences and Digital Services PAGE 67
The Guide to Researching Audiences
■■ Licensingandcopyrightmaybedifferentforelectronicversionsofaresource(egaccess
toelectronicjournalarticlesmayberestrictedtoauthenticatedmembersofanHE
institution,althoughaccesstothepaperversionofthejournalispossibleforanyvisitorto
alibrarywhereitisheld).
C.4 The relationship between digital and non-digital worlds
C.4.1 Thedifferencesincontextmeanthatattemptingtoreplicateanon-digitalresourceorservice
inthedigitalenvironmentmaynotbeappropriateoreffective.Simplyreplicatingtheexisting
resourcealsoneglectspossibilitiesuniquetothedigitalenvironment.
C.4.2 Digitalaudiencesdonotnecessarilyoverlapcompletelywithnon-digitalaudiences,even
foranostensiblysimilarresource.Inthedigitalenvironmenttheaudiencehasdifferent
expectationsaboutresourcesandwillexpecttointeractwithresourcesindifferentways.
Consider this example….
Membersofalibrarywhoaccessthesamepapertextmayhaveverydifferentexpectations.Forexampletheymaybe:lookingupaspecificreference,borrowingforrepeatedreferenceorextendedreading,skimreading,searchingforcontentonaparticulartopic,readingfromcovertocover,checkingpublicationdata,examiningillustrations,browsingwhilstwaiting,comparingwithalternativeeditionsetc.Thepapertextisnottheoptimalformatforallthesetasks,buttheusermustadapttotheformat;inthedigitalworldtheresourcecanalsobeadapted.
Adigitalresourcebasedonthetextcouldbecreatedtakingasthestartingpointtherangeoftasksthetargetaudiencewouldwanttocarryout.Themoreflexibledigitalresourcecanbepresentedindifferentwaysandthefunctionalityextendedtofacilitatetheusestowhichtheaudiencewouldliketoputtheresource.
C.4.3 Audienceresearchcanprovideevidenceofthefunctionalitythatuserswouldappreciate.
Consultingthetargetaudiencemayuncoverunanticipatedrequirements,orconverselymay
indicatethatfeaturestheserviceproviderisconsideringintroducingwouldnotbeused,or
wouldnotbevaluedenoughtojustifythecosts.
C.4.4 Digitalresourcesarestillintheirinfancyanditisalsocommontodiscoverthataresource
isbeingusedinwaysorforpurposesnotanticipatedbythecontentorserviceprovider.
Monitoringuseofadigitalresourceusingdifferentaudienceanalysistechniquesallows
serviceproviderstouncoverunexpectedpatternsofusageandtakeadvantageofany
opportunitiesforservicedevelopmentandaudiencedevelopment.
C.5 Technology should serve the audience
C.5.1 Contentandserviceprovidersoftenfeelanobligationtoexploitthenewpossibilitiesofthe
digitalworldbutareunsurehowtodosoeffectively.Technologyshouldbeusedwhereit
canenhancetheserviceandimprovetheaudienceexperience.Ratherthanaskinghowto
incorporateemergingdigitaltechnologies,applicationsetc,serviceprovidersshouldask
whichbenefitsofthenewtoolstheiraudiencemightappreciate.Digitalservicedevelopment
The Guide to Researching Audiences
PAGE 68 Appendix C: Digital Audiences and Digital Services
shouldbedrivenbythemissionoftheservice,ratherthanbyanobligationtoincorporate
emergingapplicationsandformats(egifyouraudiencewantinformationandaforumfor
discussiononhottopicsofthedaybloggingmightbetherightsolution).
C.5.2 Obviouslythisdoesn’tmeanserviceprovidersshouldn’texperimentwithnewpossibilities
(seesub-sectionC.4:experimentationmaybecheaperandeasierinthedigitalenvironment)
butthebusinesscaseshouldstateclearlywhenextensiontoaserviceisofferedonamore
speculativebasisandaudienceresearchwouldbecriticaltodeterminewhetherandhowto
pursuetheidea.
C.5.3 Thereiscurrentlyalotoftalkabout‘Web2.0’,withserviceproviderskeentoknowhow
torespond.RatherthanseeingWeb2.0asasetoftechnologies,itcouldbeviewedasan
approachtoserviceprovisioninthedigitalworld:userscanbecollaboratorsinresource
creation;useofaservicegeneratesdatathatcanbeminedforservicedevelopment;users
participateindesigningtheirexperienceofaservice.
C.6 Relationship between digital and non-digital services
C.6.1 Digitalservicescanbeanalternativeto,complementarytoorsupplementarytoanon-
digitalequivalent.Manyservicesandcontentprovidersnowhaveawebsiteinadditionto
theoriginalphysicalresourceandunderstandingtherelationshipbetweentheaudiences
forthesedifferentservicesisuseful.Ithassometimesbeenassumedthatanonlinesitewill
attractanaudienceforwhomthephysicalresourceisrelativelyinaccessible,butaudience
researchhasshownthatthisisnotnecessarilythecase.
Consider this example….
Someserviceshavefoundthatmostvisitorstotheirwebsiteuseittoobtainsimpleinformationaboutopeninghoursandtraveltoassistinplanningavisit.Thissuggeststhatthewebsiteissimplyanalternativemeansofaccessinginformationalsoavailableonpaperorbytelephone.Butthewebsitemaystillbeattractinguniquevisitorstothemuseumievisitorswhowouldnothavediscoveredthemuseumifthewebsitewerenotavailableorwouldnothavechosentovisitwithouthavingaccessedthewebsite,eveniftheymakelittleuseofthewebsite.Uncoveringthiskindofinformationwouldrequireanintegratedaudienceanalysisstrategy,includingaskingtheaudienceforthephysicalmuseumabouttheiruseofthedigitalmuseumresource.
C.6.2 Servicesmaywanttouseaudienceresearchtoexploretherelationshipbetweentheirvarious
audiencesandtailortheiraudiencedevelopmentandservicedevelopmentworkaccordingly:
■■ Theaudienceforaphysicalresourcecouldbeencouragedtobecomepartofthe
audienceforarelateddigitalservice(egbyofferingfurtherinformationaboutexhibits
online,offeringaccesstomorematerialonline,offeringthechancetoinfluencethe
physicalresourceetc.).
■■ Thedigitalservicecouldbeusedtopromoterelatednon-digitalservices(eg anevent
organiseruseswebsites,perhapsincludingsocialnetworkingsites,toprovideinformation
aboutaneventandattractawideraudience).
■■ Itisrecognisedthatthedigitalserviceservesalargelyuniqueaudienceandtheserviceis
tailoredspecificallytotheirneeds,particularlythosethatcannotbemetthroughanon-
digitalservice.
Appendix C: Digital Audiences and Digital Services PAGE 69
The Guide to Researching Audiences
C.7 Digital audiences may be analysed differently
Methods of audience analysis are discussed in more detail belowC.7.1 Audienceanalysisshouldfollowthesamebasicprinciplesregardlessofthetypeofservice
oraudience,howeveradditionalaudienceresearchtechniqueshavebecomeavailablewith
theadventofdigitalservices:webstatistics,onlinesurveysandamuchwiderrangeof
ethnographictechniques.Webstatisticsareavaluablenewtoolforaudienceanalysisasdata
ispotentiallyavailableonallusersofaresourceratherthanjustasmallsample.
C.7.2 Inthephysicalworldethnographicresearchinvolvesdirectobservationofanaudience
walkingthroughamuseum;askingtheindividualtokeepadiaryofusagetobeusedasa
promptinanintervieworcarryingoutacontextualisedinterview.Datacaptureisindirect:
ahumanbeing,eithertheresearcheroraudiencemember,wouldhavetoobserve,codeor
categoriseandperhapsalsorecallatalaterdatetheusagebehaviourofinterest.Thismeans
thatthefinedetailofusagebehaviourismissed,misrememberedorforgottenaltogether.
Additionally,themoreintrusivetheresearcherrecordingbehaviourthemorelikelyitisthathis
orherpresencewillinfluencetheaudience.
C.7.3 Intheeraofdigitalserviceswebanalyticscaptureuserbehaviourdirectlyanduser
observationstudieshavealsochanged.Studiesmaybecarriedoutinaspecial‘viewinglab’
orremotelywiththeusersinalocationoftheirownchoosing.Userscanbeaskedtocarry
outaparticulartaskwitharesource–eitheraccordingtoagivenprotocolorastheyprefer–
orsimplyinvitedtoexploretheresourcefreely.Users’behaviourastheynavigateawebsite
oruseanotherdigitalresourcecanbecapturedautomatically,accuratelyandtoamuch
greaterlevelofdetailbyspeciallywrittencomputerprograms.Navigationpathsthrougha
resource,searchingbehavioursandtheirsuccessorfailure,downloading,viewingtimesfor
filesandpagesetc.canbecapturedforlateranalysisinconjunctionwithusercommentsand
knowledgeoftheirobjectivesorthetasksattempted.
C.7.4 Webanalyticalmethodsareofonlylimiteduseindeterminingthesizeandcompositionofan
audience.Theyprovide,atbest,onlyanapproximateidentificationofusers:detaileddemographic
informationisnotavailableandanyothersegmentationhastobebasedoninference.
C.7.5 Thevalueofwebanalyticalmethodsisthattheycapturedataabouthowasiteisaccessed
andusedwhichallowsanaudienceanalysttomakeinferencesaboutaudiencebehaviour.It
mustofcourseberememberedthattheseareonlyinferencesandinorderfullytounderstand
whatnavigationpaths,pageviewtimesetc.reallymeanintermsofuserexperienceitis
stillimportanttotalktousersabouttheirexperience.Webstatisticscannottellyouauser’s
motivationsorattitudetoaresource.Forexample:
■■ Arepeatvisitcouldbeusedtoinfersatisfactionwiththeservice,butmightinsteadreflect
frustrationatafailuretofulfiltheoriginalobjective.
■■ Searchbehaviourswhichterminateinadownloadorprintrequestcouldbeusedtoinfer
success,buttheusermaystillbefrustratedthatthesearchtooktoolong;someusers
mayprefertoreadthematerialofinterestdirectlyratherthandownloadingorprintingit.
■■ Longpageviewtimesmightindicateinterestinaresourceorsimplybeanartefactof
tabbedbrowsing.Viewingonlyoneortwopagesonasiteissimilarlysubjecttomultiple
interpretations:thesitewasunappealing;thenatureoftheresourcewasn’twhattheuser
expected;theuserwascalledaway;thesitewasn’taccessiblefortheuseretc.
PAGE 70 Appendix D: Bibliography
AppendixDBibliography
D.1 Introduction
D.1.1 Thereisalargevolumeofpublicationsonaudienceresearch.Herewehavehighlightedthose
wehaveusedasasourceofinformationforthedevelopmentofthisguide,andsomewhich
areusefulforfurtherreading.Wealsoincludereferencestoareasnotcoveredinthisguide,
suchasimpactanalysis.
D.2 General audience research
Audience Research in the Australian Cultural Heritage Sector;EvaReussner,EVRSIG,
MuseumsAustralia;May2003
Understanding your Audiences;MLALondon;www.mlalondon.org.uk/renaissance/index.
cfm?NavigationID=401
User needs assessment for the Adaptive Management Portal; B. Banga,E.Landis,T.Tolle,
L.DecambreandF.Phillips;ACMInternationalConferenceProceedingSeries;Vol129
archive,Proceedingsofthe2002annualnationalconferenceonDigitalgovernmentresearch
Separate or Inseparable? – Marketing and visitor studies;http://archive.amol.org.au/evrsig/
pdf/mprpap.pdf
On the trail of the elusive non-user: what research in virtual reference environments reveals,
L.Connaway,M.RadfordandT.Dickey;BulletinoftheAmericanSocietyforInformation
ScienceandTechnology;Vol34(2),123-135;www.asis.org/Bulletin/Dec‑07
D.3 Planning and analysis
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited
Resources;FreeManagementLibrary;www.managementhelp.org/evaluatn/outcomes.htm
Appendix D: Bibliography PAGE 71
The Guide to Researching Audiences
Knowing your audience and doing market research; www.usa.gov/webcontent/improving/
evaluating/audience.shtml
Statistical Methods in Psychology;DavidC.Howell;WadsworthPublishingCoInc;6thed.
(2006)
Discovering Statistics Using SPSS;AndyField;SagePublicationsLtd.(2005)
Why study users? An environmental scan of use and users of digital resources in humanities
and social sciences undergraduate education;D.Harley,J.HenkeandS.Lawrence;Center
forStudiesinHigherEducation,UniversityofCalifornia,Berkeley(2006)
Classifying response behaviors in web-based surveys;M.BosnjakandT.L.Tuten,Centerfor
SurveyResearchandMethodology(ZUMA);http://jcmc.indiana.edu/vol6/issue3/boznjak.
html
‘Classifying’ physical and online visitors and reflections on ‘flow’[sic];http://amarclk.
blogspot.com/2008/05/classifying‑physical‑and‑online.html
Personas: setting the stage for building usable information sites;A.J.Head;Infotoday.com
(July/August2003);www.onlineinc.com/online/jul03/head.shtml
D.4 Methods
Observing the user experience: a practioner’s guide to user research;MikeKuniavsky;
MorganKaufmannPublishing(2003)
Contextual Design: A Customer-Centered Approach to Systems Designs (Interactive
Technologies);H.BeyerandK.Holtzblatt;MorganKaufmannPublishers;1sted.(1997)
Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and
Techniques (Interactive Technologies),C.CourageandK.Baxter;MorganKaufmann
Publishers(2005)
Usability testing;www.hhs.gov/usability/refine/learnusa.html
Questionnaire design advice sheet;
www.lboro.ac.uk/library/sklls/Advice/QuestionnaireDesign.pdf
Marketing Research;Aaker,KumarandDay;JohnWiley&Sons,Inc.;8thed.(2004)
Technical Implementation of the MINES Survey Methodology;www.libqual.org/documents/
admin/ACRLMINESPlum040705.ppt
Non-visitor research: an important addition to the unknown;HermannSchaefer;Hausder
GerschichtederBundesrepublik,Bonn,Germany
A Guide to Designing and Conducting Visitor Surveys;J.Leones;www.ag.arizona.edu/pubs/
marketing/az1056
The Guide to Researching Audiences
PAGE 72 Appendix D: Bibliography
Oxford Internet Institute surveys;www.oii.ox.ac.uk/microsites/oxis
Knowing your audience and doing audience research;http://usa.gov/webcontent/improving/
evaluating/audience.shtml
Good Practice Guide for Developers of Cultural Heritage Web Services;www.ukoln.ac.uk/
interop‑focus/gpg/Usability
Focus Groups;LyndaKelly;AustralianMuseumAudienceResearchCentre;Museum
Methods(2001)
Viewpoint – Ethnography and market research,PhillyDesai;InternationalJournalofMarket
Research49(6);MarketResearchSociety
Tools for assessing web site usage;S.Anderson,T.Willard,H.CreechandD.Bakker;
InternationalInstituteforSustainableDevelopment(2001)
On interpreting access statistics: Why web usage statistics are (worse than) meaningless
www.goldmark.org/netrants/webstats
D.5 Service development
Customer-focused government: from policy to delivery;LyntonBarker;PublicServices
ProductivityPanel
Managing Successful Programmes (MSP);OfficeofGovernmentCommerce(OGC)(2007)
D.6 Digital context
Digital consumers: re-shaping the information profession; DaveNicholasandIanRowlands;
FacetPublishing(2008)
Visiting the virtual museum: art and experience online;LianneMcTavishinNewMuseum
TheoryandPractice:AnIntroductioned.J.Marstine;BlackwellPublishingLtd.(2006)
My Advice for Incorporation of Web 2.0 into Museums;http://museumtwo.blogspot.
com/2007/04/backwards‑interview‑my‑advice‑for.html
Why bother with digitisation? Users and using digital requirements;W.Kilbride(2004);http://
ahds.ac.uk/creating/information‑papers/why‑bother‑digitising
Do it yourself search engine optimization;www.theinternetdigest.net/archive/diy‑search‑
engine‑optimization.html
Great minds think (too much) alike?TheEconomist(17thJuly2008)
What is Web 2.0? Ideas, technologies and implications for education;PaulAnderson;JISC
TechnologyandStandardsWatch;Feb.2007
Appendix D: Bibliography PAGE 73
The Guide to Researching Audiences
Information behaviour of the researcher of the future;CIBERbriefingpaper;Jan2008
Developing personalisation for the information environment 2;JISC;2008
D.7 Evaluation and impact
The evaluation of public library online services: measuring impact;PeterBrophy;
thepeoplesnetwork;2002WorkshopSeriesIssuesPapersno.1
Public Value Test: Guidance on the conduct of the PVT;BBCTrust;www.bbc.co.uk/bbctrust/
framework/protocols/compliance.html
Digitised Resources: A Usage and Impact Study,JISC;www.oii.ox.ac.uk/research/project.
cfm?id=51
Evaluation of open access online courses: guidance developed by the Massachussetts
Institute for Technology (MIT);http://ocw.mit.edu/OcwWeb/HowTo/Evaluation‑
Measurement.htm
The Guide to Researching Audiences
PAGE 74 Appendix D: Bibliography
This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/uk/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA
This document is available in alternative formats. For more information: www.jisc.ac.uk/contentalliance
Document No: 670 Version 2.1, December 2009
The Guide to Researching Audiences
Further information about JISC:Web: www.jisc.ac.ukEmail: [email protected]: +44 (0)117 331 0789