HSP Proposal v1_4

8/2/2019 HSP Proposal v1_4

1/24

Historical Society of PennsylvaniaUser Study Research

Friday, March 30, 2012

Key imc Research Contact:

AR Elsharawy

Garrett Cobarr

imc Research echnology and User Research

[email protected]

[email protected]

1 206-922-2843

imc Research. Research with Cause Copyright imc Research Custom Research North America 2012. Proprietary and Confdential

imc Research | Custom Research

North America


2/24

imc Research | Custom Research | North America Confdential Page 2 of 26

CONENSBackground & Objectives 3

Project Goals and Objectives 4

Te Research and Interface | User Experience Design Process 6

1. Desk Research 6

2. Qualitative: Phase 1 7

3. UI & UX Prototypes Development: Phase 1 9

4. Qualitative: Phase 2 10

5. UI & UX Prototypes Development: Phase 2 -Final Pass 10

6. Usability testing o prototypes 10

Sample Frame and Recommended Methodology 11

Option 1 (Pop-up Survey) Phase 1 13

Option 1 (Pop-up Survey) Phase 2 15

Option 2 (Email Survey) Phase 1 15

Option 2 (Email Survey) Phase 2 16

Reporting and Analysis 17

Project Milestones and Deliverables to HSP 18

Project Investment 19

Why Choose imc Research for Tis Assignment? 20

References 21

imc Research Project eam Biographies 21

Appendix 23

Key Driver Analysis 23

Key Dissatiser Analysis 23

Key Enhancer Analysis 24

Benets to our KDAP Approach to Identiying Key Drivers 24


3/24


Background & Objectives

Project Background

Te Historical Society o Pennsylvania, hereaer HSP, began to direct attention towards provid-ing new methods o access or its collections in late 2009 with the beginning o the Digital

Center or Americana project series.

Te rst project in this series ocused on the acquisition o equipment, systems and skilledpersonnel to provide the ability to process physical collections or use as digital, online col-lections. Tese investments in resources set the stage to digitize and process 52 Civil War eracollections via signpost and MPLP methodologies.

In summer 2010, the second Digital Center o Americana project began. Tis iteration ocusedon 21 collections representing the breadth o HSPs ethnic an immigrant history holdings.Tese collections were chosen not only due to their high research value and lack o digitalormat, but also to provide comparable test cases or a user study. Tese collections were pro-cessed and digitized in a mixture o processing and digitization levels, which are described

below in the Project Goals and Objectives section.

Project Objectives

Te Historical Society o Pennsylvania, hereaer HSP, seeks to conduct a study to assess thecomparative validity o two digitization processes or acquiring content collections.

MoreProduct,LessProcessmethodology.(MPLP)

Versus

Traditional,FPMmethodology.(FPM)

In relationship to developing an interace and user experience or collection access, it will also:

Assesstheuseofsignpoststosearchandviewcollections.(MPLP)

Versus

Managingthedigitalviewsofafullyprocessedcollection.(FPM)

MPLP with signpost methods are used by libraries, archives and other curatorial institutions toreduce or eliminate the backlogs that oen accompany the digital acquisition o physical col-lections. Tese delays are due to the extensive time and resources required to execute the FPMmethodology.

Te results o this study will be used to:

1. Produce a study researching the comparative validity o MPLP and FPM. Te report

will be released and shared with the global archival community. Te goal o this eort isto help other institutions in making balanced decisions on processing their collections,maximizing sta resources and serving their user constituency.

2. Using the results o the aorementioned research, redesign the end user interaces anduser experience o the online catalog and the digital library (discover.hsp.org and digitalli-brary.hsp.org, respectively)


4/24


Project Goals and Objectives

Tis study will test the eectiveness o dierent digitization and processing methodologies, andthe study ndings will be applied to the current backlog o collections.

Depending on:

Size

Complexity

Condition

Levelofinterestbyresearchers

Each collection is subjected to one of the following:

Basic Processing (MPLP) Full Processing (FPM)

Description

For collections smaller than 1 linear

oot or 3 volumes: collection-level

record only

For larger collections: fnding aid with

brie background and scope/content

notes, box- or older-level inventory

For all collections: fnding aid

with detailed background andscope/content notes, older-level

inventory

ArrangementTo series or sub series level, with rough

arrangement within seriesTo older or item level

HousingReplace non-archival olders only i

damaged or do not ft in box

All unbound materials in archival

olders

Estimated processing rate5 hours per linear oot (2 hours

minimum per collection) 15 hours per linear oot


5/24


Collections are also subjected to one o the ollowing:

MPLPanddigitalsignposting:

A processing methodology that systematically chooses a key piece o content to be digitallyprocessed and placed online as a presentation o a larger, related, collection o content.

Tis process could require the content to be handled more than once, possibly increasingthe risk o damage.

FPMandmassdigitization:

A methodology where entire collections o content are processed en masse. Although thisprocess exposes ragile content to less handling and provides a complete, detailed andhigh resolution content online, it can be resource intensive. Large collections can lan-guish, unseen and remain out o the view o interested users and researchers.

HSP will work with the vendor to develop unctional prototypes based on the aorementionedprocessing and digitization levels. Tese prototypes will then be used in a series o usabilitytests in order to determine the level o processing and description -along with the level o

digitization and description - necessary to adequately meet user needs in terms o inorma-tion retrieval. Tese ndings will then be used to revise the user interaces o our catalog anddigital library.

Te concepts to be validated are as ollows:

ProcessingthiswillyieldinsightintotheeectonusersinregardstousingFPM,MPLP,and/or aspects rom both methods.

Digitizationthiswillyieldinsightintouserbehaviorandacceptancearoundsignpostsversus mass digitization approaches.

DescriptionexpandingfurtheronFPMversusMPLP,wewilldiscoverthelevelofde-scription required to adequately meet user needs, in-house as well as remote. Tis is main-ly or nding aids (documents that describe what is contained within a specic collectionanexamplecanbefoundhere.)Itwilldeterminehowtheinformationispresentedandused.


6/24


The Research and Interace | User Experience Design Process

A snapshot of the researc h and user e xperience sequence of processes.

1. Desk Research

imc Research proceeds through a rigorous set o steps with clients during a research project. Itmight be best describe as a kind o journey that begins with getting to know the client and

the environment they inhabit. One must understand their practice culture, their vocabulary,the challenges they ace and the vision they wish realize or their uture the current industryclimate and competition nature along with currently implements solutions and best practices.

Te path o this journey starts with setting the client scene. In the discovery phase the imc Re-search team, coordinated by their integration leads, ans out to the client constituency askinga myriad o questions. How would you describe what you do? Which areas do you nd mostdicult? What are your expectations o today? What do you want to gain?


7/24


A research paper is assembled identiying the detailed understanding o the project makingsure imc Research has the same level o expertise o HSP into solving the addressed issue inhand.

2. Qualitative: Phase 1

Te proposed study is a qualitative research analysis aimed at providing HSP with a general ap-praisal o the consumer behavior in terms o uses & attitudes, use behavior and requency aswell as the decision making process in order to design a strategy that would be used to betterhelp develop initial UI or the HSP.

Some o the covered objectives, in this stage:

1. Dening the loyalty levels prevalent among respondents towards the current researchsources based on previous experience and usage patterns.

2. Assessment o the expectations prevalent among respondents towards an online cataloginterace.

3. Identication o the possibility o uture usage o online cataloging research, among par-ticipants.

4. Identication o the criteria that would induce uture use o online cataloging research,rom the viewpoint o participants.

5. Identication o the most appropriate suggestions and modications necessary to increaseusability levels to create a repetitive usage pattern towards the online cataloging.

6. Determining participants general image and perceptions held towards the concept oonline cataloging.

7. Acknowledgement o the key attributes that infuence the audience behavior relative tousing a certain library source (Ads, priority, word o mouth, content, infuence, habits,

etc), to create a basic understanding o mass user-ship is driven, this should refect the e-ect o the used processing methodology o eect on preerence o use.

8. Develop an understanding o the consumer usage decision process in order to allocate thedeciders and infuencers o the research habits.

9. Identication o the primary top o mind cerebral messages that drives preerring an on-line catalog research source

10. Assessment o accessibility, likability, and interest levels towards the concept o basic pro-cessing (MPLP) verses FPM.

11. Determining the key attributes that aect the decision o using a certain source

12. Assessment o the websites involvement levels and behavioral predictive abilities.13. Identication o the current market (strengths & weaknesses) o the current interace, as

perceived by the respondents across the current website versus other implemented web-sites both basic processing (MPLP) and FPM.

14. Identication o the reasons or using or not using the each website o the universities andthe competitors.


8/24


15. Identication o the unmet needs, prevalent among respondents towards cataloging.

16. Identication o the perceived target market o catalog research, rom the viewpoint o theparticipants.

17. Assessment o satisaction levels among the purchasers o all identied usage aspects in

terms o:I. Frequency o use

II. Extent o use

III. Reasons o use

IV. Experience (previous usage) and habits

V. Te concept

VI. Winning Probabilities

VII. Others

The Sample and Methodology

In-Depth Face-to-Face Interviews: Face-to-ace interviews will be conducted with the ourresearch segments. Te targeted audience will be sent a notication letter inorming them othe study objectives and nature o topics to be discussed. Interviews will then be called to setup a suitable appointment time or the interview.

Sample Size: Te proposed sample or this study is comprised o a total minimum o 16 respon-dents 4 or each o the main audience segments (split equally MPLP and FPM users ) and thetargeted audience should be:

1. Heavy users: use online resources or research very requently.

2. Process loyalists: Use one or more resource that use either MPLP or FPM users

3. Objective ocused: the use reasoning or the researchers across the segments should beclear and consistent. Tey should not have many reasons or use.

In Depth Interview Matrix:

Phase 1 Sample Structure

Target Sample

MPLP Full Processing

Scholars 1 1

K-12 Educators 1 1

Genealogists and Community Historians 1 1

History and Heritage Organizations 1 1

Total 8 In Depth Interviews


9/24


3. UI & UX Prototypes Development: Phase 1

Tis module be the most critical step in the MPLP validation study / user experience design,orming the anchor or the entire project. Aer a careul and thorough desk research analysishas been completed, taking in the existing HSP websites and comparative web services in ac-

count, then the in-depth interviews o qualitative research takes place with sets o users in keytarget audiences.

Te desk research will orm the basis o the study, the control group. Te qualitative interviewswill provide the needs and desires, that may be hidden rom view, to blend with the compara-tive data. Despite this rich comparative and user based data, this inormation does not leaddirectly to an interace and user experience design.

Considering the user target spread runs rom K-12 educators, through genealogists and com-munity historians, history and heritage organizations, and all the way to scholars, it highlyanticipated that there will be very diverse opinions. An interace element that may be veryhelpul to one constituency, may be a time consuming irritation to another.

How can this be dealt with? Adaptive User Interaces are one solution but can have long devel-opment cycles and be occasionally unpredictable. One could attempt to design to the mostcommon denominator, but with this diverse user audience, we would likely end up with acompromise that no one particularly ound engaging or useul.

Te best solution or the prototypical test, in our experience, would be possibly 2 to 4 interacesbacked up with context sensitive help. Depending on how dierent the needs and opinionsare o the groups we would design an interace or each average o usage. Tis can be a verystrong solution to a complex need set. Te variations among the sub interaces is held togetherwith a strong branding, clear graphics and a consistent global set o controls

It is easy to orget that an interace and user experience is nothing more than a dierent view

o the same data content. Tink o it as pouring water into dierently shaped vases, the fuidtakes on the shape o each container, giving a dierent presentation or the same material.

Matrixing the MPLP signposting and FPM through this solution will give a clear view o thevarious responses to the two dierent data set solutions.

imc Research will be using the wirerame and mockup solution known as Azure. Tis tool willallow us the fexibility to develop the inormation architecture, navigation maps and outputweb based simulations o the potential designs and their behaviors. imc Research will hostthese simulations or the user testing. Azure also has the ability to track and extract user be-havior, as well as, host user discussion and eedback.

Tese prototypes will be rough, but not too rough. Tey must exhibit as much similarity to thenal interaces and UX as can be determined at this initial stage. I they are too ar rom thelast stage then the test may not yield an accurate result. Graphical interace elements will beminimal at this stage but we must determine the division o screen space


10/24


4. Qualitative: Phase 2

Users will be invited to test, evaluate the use, discuss the current driven unctional prototypes,indicate major fows, talk about usability, eel, accessibility, click tale and experience, the cur-rent prototypes. Tey will test both o the process under dierent UIs and decide through a

blind test that is better.Observation Sessions Matrix:

Phase 2 Sample Structure

Target Sample

MPLP Full Processing

Scholars 1 1

K-12 Educators 1 1

Genealogists and Community Historians 1 1

History and Heritage Organizations 1 1Total 8 Observation Sessions

5. UI & UX Prototypes Development: Phase 2 -Final Pass

At this stage o the interace and user experience design, imc Research will be adding theknowledge renements learned rom the qualitative, in depth interviews on the originalprototypes. Assumptions on the rst pass will be set more permanently into position i theyprove correct. Elements and methods will be both removed and added as needed.

Final graphical elements wil l be created and positioned, screen allocations or various interacecomponents will be laid out and run through standard quality assurance testing on the mainbrowsers and platorms.

Also, at this stage, will begin the nalization and specication documents or the web teamHSP chooses to develop the nal website in Collective Access and VuFind. Tese specica-tions and the existing mockup simulations will aid imc Research in the management o theweb development team, providing a set o clear and operational guidelines to work rom.

6. Usability testing o prototypes

imc Research will evaluate the sites using several metrics:

Usability

Websiteusagetracking

Applicanttracking

Usabilitysatisfaction

Contentqualitytesting

Laggingindicatortreeanalysis

Te ultimate goal o the usability is create the best possible website concept.


11/24


o evaluate the site rom the candidates point o view, HSP wishes to commission research thatwill evaluate HSPs sites in two phases:

1. Pre-launchwiththecurrentsiteasabaseline

2. Post-launchmeasuringtheprogressmadebyintroducingthenewUI

Sample Frame and Recommended Methodology

imc Research target audience to be visitors rom the past 2 months who utilized the resourceso the websites they used . Te contact method would most likely be via email. It is possiblethat the details o candidates recall o their experience with HSPs career site will ade soonaer their visit. o help aid with recall o the site, we suggest including static snapshots o thesite in the survey.

Tis approach will only reach those who submitted an application online because HSP is likelyto have captured an email addresses rom an applicant and not all visitors. I it is only thosewho submit an application that HSP would like to monitor through the interviewing and hir-ing process, then we recommend this approach.

imcResearchwouldalsoliketooeranalternativeapproachforyourconsiderationinter-cepting visitors with a pop-up survey upon exit rom the site.

Tis approach oers important benets to HSP:

Morewell-informedevaluationgiventhattheyjustvisitedthesite

Noriskofviolatinganti-spamlawsrequiringthatHSPgainagreementtoopt-inforre-search beore emailing individuals

Apop-upwillallowustosampleboththosewhosubmitanapplicationandthosewhojust visit the site

Te pop-up survey will capture the large majority o the metrics. However, we also realize that

some metrics may require a longer term perspective and cannot be ascertained immediatelyaer visiting the site.

Determining drivers o Positive outcomes or website users to guide the survey design to en-sure HSP objectives are met, imc Research proposes to conduct an audit o existing HSPresearch and data streams. We would like the opportunity to review any existing qualitativeand quantitative research on web site usability, candidate satisaction, and job search processas well as any existing internal metrics i existed, in case non o that ever existed rst threephases should establish this understanding.

An important goal o this research will be to compare usability and development eatures withthe HSP web site experience beore and aer the new UIs launch. Also critical, HSP needs tounderstand what elements o the web site experience matter most. o accomplish this objec-tive, we propose to perorm driver analyses to determine drivers o key positive outcomes:

Usabilityandaccessibilitywiththesite

DecisiontoapplyforpositionatHSP

Candidatesabilitytolocateapositionrelevanttotheirinterestsandresearch

Timeseectiveness


12/24


Attitudinal web site metrics we recommend including are:

Alignmentwithneeds:HowwelldidthesiteliveuptoyourexpectationsofwhataHSPcareer web site should be like?

Frustration/Learnability:euserswillingnesstocontinueorabandontasksbeingper-

formeddidyouaccomplishwhatyousetouttoaccomplishthersttimeyouencoun -tered the web site?

Satisfaction/likability:Howeasywasittousethiswebsite?Intermsoffuntousehowwell would you describe the web site?

Confusion:Whataspectsofthewebsite,ifany,wereconfusing?Intermsofcontentand/or navigation.

VisualAppealHowappealingisthewebsiteintermsofthecolors,layout,graphicsandother visual aspects?

Information:Whatinformationweretheylookingfor?Howeasywasittondspecicinormation on the web site?

Guidance:Didthecontentonthewebsitehelpthemndtherightresearchdatatorinorm their Search?

imc Research utilizes practical, proven, and powerul analytic tools and models that drive im-provement.

imc Research combines the theoretical approaches and models o visitor usability with state othe art behavior processes rom recognized leaders in both academia and the world o busi-ness.

UNDERSTAND:Weoerarobustunderstandingofoverallusabilityandloyaltythatin-corporates the various attitudinal, behavioral, and emotional complexities o stakeholders

relationship with our client. DIFFERENTIATE:Ouruniqueapproachtoidentifyingkeydriversofusabilityandloyal-

ty helps our clients understand the perormance actors that create dissatisaction as wellas long-term loyalty. Our approach clearly dierentiates between drivers o dissatisactionand drivers o loyalty.

EXECUTE:Ourtools,deliverables,andprocessesenableourclientstobettercomprehendwhat clients need and expect rom their product or service, and then to build and executepowerul action plans that improve business results!

Tese approaches are highly relevant to HSPs goal o attracting users who ultimately will ndullling research resources at HSP. Following the completion o each phase o interviewing,

imc Research will conduct Key Driver analysis on pre-UI and post-UI data to identiy action-able drivers o dissatisaction and delight with the web sites. Te ollowing is a conceptualoverview o our analysis. A detailed discussion o our approach to driver analysis is outlinedin the Appendix o this proposal.

In order to provide a more in-depth evaluation o the post-UI, imc Research proposes to linkclick stream data to the attitudinal/survey data to assess:

Functionality:etechnicalcapabilitiesofthewebsite,orhowthewebsitedeliversitscontent.


13/24


Clutter:enumberoflinks,percentofcontent,andotherquantiableitemsonapage.

False Navigation:epropensityfornon-clickableitemswhichareincorrectlyseenaslinks.

InterpretationDependence:Atendencyformultiplemeaningsorspecializedknowledge

unknown to user. FragmentedNavigation:eeaseordicultyexperiencedwithmultiplenavigational

pathways.

Outlined in the pop-up invitation option, imc Research will also highlight and incorporate intothe ndings click stream patterns rom the post-UI data collection. Tis will al low us to de-termine what the web site does well and what can be improved upon. Click stream data canalso be used to help suggest and modiy changes to the post-UI. Obtaining click stream datacan only be accomplished i candidates are intercepting via a pop-up invitation as they enterthe web site.

Key metrics collected in both the pre-UI and post-UI surveys will also be highlighted to pro-

vide an overview o the user experience. Tis overview will be done in total to provide aholistic view, but also on key subgroups including, but not limited to, country, gender, age,status, and education.

Research Design

Outlined below are the two possible research designs or this study. imc Research believes thatboth approaches can successully achieve the objectives or this research; however, recogniz-ing that there could be possible logistical limitations, we have provided background inorma-tionforbothoptionsPhases1and2alongwithpricing.

Option1Pop-upsurveyinvitation+clickstreamdataandfollow-upsurvey.Surveyto

be hosted by Imc Research Option2EmailinvitationsentbyHSPaeraresearchhasbeendone.esurveywill

be hosted by imc Research

Option 1 (Pop-up Survey) Phase 1

Recruitment

Upon entering the career section o the current HSP web site, candidates will receive a pop-up invitation requesting their participation in a ollow-up survey upon exiting the site. HSPdata estimates the trac on the web site to be around 2,300 visitors a month. o establish therequency that the intercept pop-up should be displayed, imc Research suggests conducting adry-run, monitoring web site usage. Te dry-run will be held over a minimum o 35 days toestablish trac patterns weekdays versus weekends, during the morning versus aernoon orevening.

Since we assume that HSP will not allow programming access to their web site, we suggest thatHSP create a DHML pop-up, which is unblockable by standard external window blockingsoware. In the pop-up invitation, no survey questions will be asked, only their agreement toparticipate in research will be requested. Te pop-up will acknowledge that there is a ollow-up survey upon exiting the site.


14/24


Interviewing

Once a candidate has agreed to participate, imc Research will assign a sequence number viaa cookie to the IP Address. Tis will allow us to track the respondent rom the same PCthroughout elding to ensure that they are not intercepted more than once i they were to

enter the site on multiple days or times.In order to link the ollow-up survey inormation to HSPs database we propose two possible

options that we would like to discuss urther with HSP. Te rst approach is to collect re-spondent inormation (name, email address) at the end o the ollow-up survey. Tis willallow us to capture inormation not only or those who researches the website, but also orthose who leave the site and do not submit any research or register on the web site. However,a completed survey would not be dependent upon this, as we cannot orce candidates to pro-

vide this level o inormation in the ollow-up survey.

A second approach is to determine, i upon submitting an application, i HSP assigns an IDnumber to the candidate. Tis ID number can then be instantaneously attached to the URL

that directs the candidate rom the HSP career web site to the imc Research hosted ollow-upsurvey. Te only caution with this approach is that we would then only be able to link thosewho submitted an application and we would exclude visitors rom later linkage.

Upon exiting site, candidates will move to an imc Research hosted browser window to completethe ollow-up survey. Since candidates will have just navigated around the HSP web site, webelieve a reasonable length or the ollow-up interview will be no longer than 10 minutes.Within the ten minute interview, candidates will be asked a series o close-ended questions aswell as one open-end to provide additional insight into the experience.

imc Research, in collaboration with HSP, will develop the questionnaire. imc Research willprogram and host the survey and will provide HSP with regular updates on the status o eld

process. We have included in our costs, ees to translate the questionnaire into any languagesrequired, as well as, to translate the open-end. We didnt assumed that interviews conductedin the US will be in English only. And all open-end responses will be coded.

We have estimated the eld work or this study (not including the dry-run to establish tra-c patterns) to take a minimum o 20 days. Tis will allow imc Research to create quotasor weekdays versus weekends and morning versus aernoon or evening. Tis process willattempt to ensure that we do not skew results towards one type o candidate that has specictime availability or job searches.

Noteworthy,theestimatedcooperationrateforpop-upinvitationsisapproximately1%. However,we eel that a scientic desirability o the website due to its nature will likely increase this slightly, as

researchers may be more apt to agree as they eel it could put them in better standing with HSP.Usingtheconservativecooperationrateof1%alongwithapproximately2,300visitorstothe

siteeachmonthovera35daytimespan,wecanlikelyachieve400interviews. However,sincethe 2,300 monthly does not include international trac, we would like to discuss with HSPthe trac volume rom researchers internationally to ensure that we can align the expectednumber o interviews with the research objectives.

For budgeting purposes, and minimum allowed margin o error we have assumed an average o250 completes per country.


15/24


Option 1 (Pop-up Survey) Phase 2

Recruitment

imc Research has budgeted a Phase 2 separate rom Phase 1 with an entirely new respondentbase using the same popup methodology as in Phase 1.

Interviewing

Te candidate will reely move around the new site selecting the pages and drop down menusnecessary or research. Running behind the scenes, imc Research will be monitoring the clickstream or the respondent. Monitoring the click stream, we will ollow each candidates jour-ney through the web site and track how he/she optimized the route chosen through the website. Additionally we will seek to:

Understandhowusersnavigatethecareerwebsite

Reportonpageviews,clicks,timespent,etc.

Identifypotentialobstaclestotaskcompletion

Determinestrengthsandweaknessesofthesite

Compareusabilityexperiencesacrossdierentsegments

Upon exiting the site they will be prompted to complete a ten minute ollow-up survey withone open-ended question. We have assumed the ollow-up questionnaire in Phase 2 will beapproximately80%similartothequestionnaireusedinPhase1.iswillallowimcResearchto trend key metrics rom the pre-UI to the post-UI ; however, will provide some fexibility toask questions specic to the UI.

imc Research, in collaboration with HSP, will modiy the questionnaire, and imc Research willprogram all changes into our online interviewing package. Again, imc Research will translate

any new questions into Japanese and Russian, and comments rom the one open-ended ques-tion will be translated rom Japanese and Russian into English. Te open-end will be coded.

Also in line with Phase 1, interviewing in Phase 2 will take place over a minimum o 20 days toaccount or trac patterns. Given that the user base may change between now and January2013, we would recommend perorming another dry-run to establish the trac patterns onthe site, so that quotas can be set accordingly.

Option 2 (Email Survey) Phase 1

Recruitment

Recognizing that one crucial piece o inormation likely collected during the application pro-

cess is email address, imc Research recommends using this to recontact applicants. Sinceapplicants were likely not asked initially i they would be open to be contacted at a later timeor research purposes, the invitation to participate, legally, must come rom HSP.

Te process or which we recommend doing this is or HSP to rst supply imc Research withall email addresses or applicants rom the past 6 months. Also included in this le should becountry inormation, so that we can ensure the correct language is displayed to the researcherimc Research will create an invitation link specic to each email address that will allow us totie inormation collected on our hosted survey to the HSP applicant database. imc Research


16/24


will then provide this le back to HSP. HSP would then deploy the invitation to participate toall past applicants. Te study would be branded with the HSP name.

Interviewing

Once deployed and candidates click on the invitation, they will be directed to a imc Research

hosted web site to complete the survey. Te survey will be no longer than 15 minutes andwill include one open-end. Since it may be up to 6 months since the candidate was last on theHSP web site, we recommend using up to two static images o the pre-UI. Tis may help theresearcher to recall dierent aspects: pages, drop down menus, etc., that may cause conusionor help to acilitate the process.

imc Research, in collaboration with HSP, will develop the 15 minute questionnaire. imc Re-search will program and host the survey and will provide HSP with regular updates on thestatus o elding. We have included in our costs, ees to translate the questionnaire, as wellas, to translate the open-ended responses into English. We didnt assumed that interviewsconducted in will be in English only. All open-end responses will be coded.

Option 2 (Email Survey) Phase 2

Recruitment

In March 2013, several months aer the UI launch, imc Research will conduct the Phase 2 othe research. Te recruitment process will be the same, in that HSP will provide imc Re-search with a list o email addresses and country inormation rom recent applicants. imcResearch will create a specic link or each email address and provide the le back to HSP.HSPwillthendeploytheinvitations.Noteworthy,giventhatthereistimebeforetheMarch2013 wave; HSP may consider asking researcher i they would agree to their email addressesbeing used or research purposes. Agreement to this would allow imc Research to email the

applicants directly in Phase 2.Interviewing

Upon clicking on the link, candidates will complete a een minute survey with one open-endquestion.WehaveassumedthequestionnaireinPhase2willbeapproximately80%similartothe questionnaire used in Phase 1. Tis will allow imc Research to trend key metrics rom thepre-UI to the post-UI, but will provide some fexibility to ask questions specic to the UI.

imc Research, in collaboration with HSP, will modiy the questionnaire, and imc Research willprogram all changes into our online interviewing package. Again, imc Research will translateany new questions, and comments rom the one open-ended question will be into English.Te open-end will be coded.

While unknown, we have allowed or similar researchers volumes in 2013. Based on this, imc Re-search has provided costs to complete up to 500 completes per country and again imc Researchhas assumed 20 days to complete interviewing. However, as Phase 2 approaches we would likethe opportunity to explore researchers volumes with HSP to conrm expectations on samplesize based on the Phase 1 experience. We would also like to conrm with HSP appropriate in-terviewing time periods in all countries. Our assumption is that eld work or all countries willbe conduct simultaneously.


17/24


Reporting and Analysis

In addition to the Key Driver Analysis and Click Stream analysis* discussed earlier in thisproposal, imc Research will also provide the ollowing or each phase o research, regardlesso whether the pop-up or email survey methodology is selected. (Click Stream available with

pop up methodology only.)Phase 1

imc Research will provide in Phase 1:

Upto7bannersofdetailedtabulations(onepercountryandoneintotal)thatwillbeused to analyze the results o the data.

Adataleofallrespondentleveldata.

KeyMetricsDeckhighlightingresponsesfromkeyquestionsinPhase1.Ahighlevelkeysubgroup analysis will also be perormed to help direct any modications to the post-UI.

Phase 2

imc Research will provide in Phase 2:

Upto7bannersofdetailedtabulations(onepercountryandoneintotal)thatwillbeused to analyze the results o the data.

Adataleofallrespondentleveldata.

imcResearchwillalsohighlight,basedonclickstreams(click stream analysis will onlybe available in Phase 2 with the pop-up invitation option) what the post-UI does well andwhat can be improved upon. Tis inormation can be used to help suggest and modiychanges to the post-UI. We have assumed or budgeting purposes that the Global CareerSite is used in all target countries and that aside rom language, there are no dierences inwhat researchers will see.

InsightsDeckhighlightingkeydierences(improvements,aws,etc.)betweenthepre-UIand post-UI. Results will again highlight key subgroups including, but not limited to, coun-try, gender, age, status, and education. We have assumed that separate country reports willnot be required and that country level ndings will be included in one Global deck.

ApresentationofkeyndingstoHSPstakeholders.epresentationwillincluderecom-mendations or the post-UI as well as additional research.

imc Research recommends the ollowing ongoing analysis o the Phase 1 and Phase 2 respon-dent data:

Immediatelyfollowingtheinterviewprocess,imcResearchcanlinktheUImetrics.is

may be qualitative eedback, unless HSP has a measuring metric in place. Aerapproximately2yearstime(toachievearobustsamplesize),imcResearchcanlink

the UI metrics to Quality o research.

Aerapproximately2yearstime(toachievearobustsamplesize),imcResearchcananalyze data by unction. Tis will allow HSP to see i the web site attracts and retainsresearcher in dierent segment unctions better or worse.

I HSP is interested in the above uture linkage analysis, imc Research will be pleased to providecost estimates.


18/24


Project Milestones and Deliverables to HSP

An incremental and flexible plan to meet clients needs and spread costs .

Milestones Intervals Month Payment

1 Project authorization T0 April 2012 $33,500

2 Desk Research Presentation T0+1M May 2012

3 Qualitative P1 Presentation T0+3M July 2012

4 Initial Prototypes Presentation T0+4M Aug 2012 $31,720

5 Qualitative P3 Presentation T0+6M Oct 2012

6 Final Prototypes Presentation T0+7M Nov 2012 $14,460

7 Usability testing o prototypes Questionnaire T0+7M+2W Dec 2012 58,000

8 Survey Programming T0+8M Dec 2012

9 Translation (i needed) T0+8M Dec 2012

11 Benchmark feldwork starts T0+9M Jan 2013

12 Benchmark feld work ends T0+10M Feb 2013

13 Analysis complete T0+10M Feb 2013 58,000

14 Website development supervision T0+12M March 2013 5,040

15Questionnaire modifcation or Usability testing othe fnal website

T0+16M April 2013 22,000

16 UI assessment survey feldwork starts T0+16M May 2013

17UI assessment survey feldwork complete allgeographies

T0+17M May 2013

18 Analysis complete T0+17M July 2013 22,000


19/24


Project Investment Costs

Outlined below are cost estimates or Option 1 and Option 2 or both Phases o research.Please note the US only cost includes pricing or design, programming, and project manage-ment which are xed costs associated with this work. I HSP opts to not conduct research in

any other countries but the US the xed costs will need to be allocated to one o the US.Cost estimate details per process module:

Description o Service (US) Type & Number Unit Cost Investment

Desk Research -2 Consultants 38 hours $210 $7,980

Qualitative Phase 1 8 In Depth Interviews $1,650 $13,200

Initial Prototypes

Senior UX Development

Wirerame & MockUp

Manage

Graphical Design

Hours

120

80

12

30

$180

$80

$210

$40

$21,600

$6,400

$2,520

$1,200

Qualitative Phase 2 8 In Depth Interviews $1,540 $12,320

Final Prototypes

Senior UX Development

Manage

Graphical Design

Hours

60

6

60

$180

$210

$40

$10,800

$1,260

$2,400

Quantitative Usability Testing: Prototype 800 Online Interviews $145 $116,000

Manage UX Development 24 Hours $210 $5,040

Quantitative Usability Testing: Final

Website400 Online Interviews $110 $44,000

Market and User Research $193,500

Interace and User Experience $51,220

Total Investment Costs $244,720

Optional Services

Extra Country (English Speaking) $6,900


20/24


Why Choose imc Research or This Assignment?

imc Research has the team, the experience, and the global scope necessary to ensure HSPsbusiness objectives are met:

ProvenTeamUnderstandsHSPandHowtoDriveHighQualityInsights:imcResearch

currently partners with HSP on global programs o strategic importance including theDatabase Usage and Perceptions racking Study, the Windows Mobile Brand EquityStudy, the Windows Client Brand Equity Study, the Business Solutions Satisaction studyor oracle, and the Channel Partners Satisaction study or Google. Our team has beenrecognized or our ability to go beyond the research and provide insights that are settingnew standards or HSP market research projects.

Recent,ApplicableExpertiseinCareerInformation&Services:imcResearchhaspart-nered with the Law School Admission Council (LSAC) on numerous occasions to assessinormation provided by LSAC and approximately 200 law schools in Europe. In thisresearch, we explored how law school applicants make career decisions, and how they de-

termine which schools to apply to and then enroll through their websites. AR Elsharawy,the project lead or this proposal, presented the results o this research to LSAC during itsannual Board meeting and its annual meeting among all law school deans. Tis career-oriented research will provide useul experience upon which to design this survey orHSP.

ProvenApproachestoEvaluatingWebSites:imcResearchhasworkedwithseverallead-ing global organizations to evaluate their web sites, prole visitors, and identiy receptiv-ity to new improvements and oerings. Where appropriate, imc Research also oers cli-ents the ability to link click stream data (behavioral) with attitudinal data (survey-based)to attain a holistic evaluation o the visitors experience on the web site.

imcResearchsUnderstandsLocalMarkets,Cultures,andConsumers:imcisprovidingits research services in over 42 countries; these are wholly-owned imc Research compa-niesnotanetworkofpartners,soyoucanbeassuredanyglobalprojectundertakenbyimc Research will be staed by imc Research trained researchers.


21/24


Reerences

Following are reerences who can comment on imc Research team experiences relevant to thisresearch.

Ajay K. Singh [email protected], MBA, FRCP, MD, ex ocio member and Interim Chair

Chie Academic Ocer (CAO), Harvard Medical School Dubai Center (HMSDC) ExecutiveDirector, Dubai Harvard Foundation or Medical Research

Dr. Tarek Fathey [email protected]

City Hospital Director

Dr. S. Gurumadhva Rao [email protected]

Vice Chancellor o RAK Medical and Health Sciences University

imc Research Project Team Biographies

Garrett Cobarr Integration Lead | User Experience

Has 25 ve years experience in the art and science o user experience and human actors designHe is a specialist in initiative conceptualization, integration, managing catalyst between crossdiscipline teams, productization process, collaboration platorms and execution strategy. Hehas worked with, over 200 startups, prototypes and new product launches.

As a long term interactive multimedia developer and designer, image and color expert in assetacquirement and management, Garret has extensive experience working with archival col-lections, archivists and researchers. Projects as diverse as: worked with YKHC to develop a

nutritional database or the Yupik Eskimos, developed a sel terminating PDF technology orOxfordUniversityMedicalSchoolofNephrologyonpatientcare,developedinteractivetelevi -sion interaces and a PDA interace / user experience or Microso, to name a ew

Garrett is a pioneer in new platorms and has worked extensively in 3D Web technologies,building one o the rst virtual world cities and was the Chie Executive Ocer o SwissopolisGmbH, a Swiss based virtual world project, or 3 years. He is now working on Visualitas, aknowledge sharing and collaboration platorm.

He has a deep and long term experience with internationalization and writes extensively abouttheCulturalInterface.CowroteachapterinthepeerreviewedbookKnowledgeManage-ment:Research&Application.Heisawellknownadvocateandauthorononlineprivacy

andiscurrentlyworkingonthebookproject,IAmNotMyself:PrivacyandIdentityintheAvatarAge.


22/24


Sharmila Das Senior Manager

Sharmila is one o the senior-most managers in the imc Research echnology practice. Sharmi-la has been with imc Research since 2007, and prior to that, Sharmila was a project directorfortheTechnology/TelecomgroupatIntersearch(nowTNS).Asaseniormemberofthe

group, Sharmila has a great deal o experience in working across the imc Research networkand developing strong relationships and teams to execute large international projects, includ-ing the Database racker or HSP.

Sharmila is responsible or leading major echnology accounts, and he brings deep expertise indesigning complex new research programs involving B2B respondents, robust web reportingportals, and customer relationship/loyalty insights.

Sharmila has an MBA with a concentration in Marketing.

Jennier Bendihaj Senior Research Director

Jennier will work with Das and AR during key points in the project to bring her experiencewith technology research, and provide additional support during design and analysis phaseso the project. Jennier has over 10 years o experience in research and has a strong back-ground in quantitative research, and has demonstrated her experience in the Database rack-er study or Google orm Dubai Oce.

Jennier has extensive experience in various types o research methods, including market seg-mentation, customer loyalty, new product development, and market sizing. Tis experienceis leveraged throughout the research process with her involvement in the research design,project direction, data analysis, and report preparation.

Prior to joining imc Research, Jennier earned a B.A. in psychology rom Rutgers Universityand worked as a research assistant in the psychology department conducting research.

AR Elsharawy imc Research National DirectoAR will manage the day-to-day operations and progression o this research study. He will in-

terace with all operations departments to implement each phase o the research process andreport on the survey results and implications. AR has been with imc Research or over ouryears, and has conducted many large scale international projects or leading PC and sowarecompanies; he also has managed a large domestic customer loyalty program or a op-5 tele-coms provider.

Prior to joining imc Research in 2005, AR is in his nal Ph.D. deense rom Dubai University inUAE.


23/24


Appendix

Key Driver Analysis

Our analysis will include identication o key drivers o satisaction or HSP web site visitors.Tis analysis will serve as the base or site improvement eorts. imc Research believes that to

be successul in driving change, the HSP UI must identiy key drivers o visitor delight sepa-ratefromkeydriversofvisitordisappointmentaswellasquantifytheimpactofeachaction-ableperformanceattributeonoveralldissatisfaction.Ourkeydrivertoolsarespecicallydesigned to avoid the common problems associated with regression-based models. We useadvanced game theory analysis to account or attribute collinearity. We eel that our methodo analyzing key drivers dierentiates us rom the competition. Below is an explanation oour model.

imc Researchs Key Driver Analysis is based on the premise that a successul customer or stake-holder satisaction program must help management understand and act upon those thingsthat drive customers away as well as those things that can prevent customers rom being lured

away by competition.In the case o HSPs sites, when researchers eel that they did not receive what they expected

or a core requirement rom a data source site experience, a Key Dissatisers, a reason or thecandidate to lose interest in research HSP site. Te impact o these types o ailures on overallsatisaction, loyalty, and retention is dramatic. Tese are usually the top priority items andthey must be xed beore anything else.

Additionally, when candidates eel that they got more than they expected rom the experienceor certain delight actors, a Key Enhancer is created. Tese are things that can solidiy cus-tomer relationships and prevent visitors rom being lured away by competition, by only aerKey Dissatisers are xed. Tese are the things that create true Loyal Advocates.

imc Researchs Key Driver Analysis uses a combination o techniques to gain a thorough under-standing o these elements:

1. A Key Dissatiser analysis.

2. A Key Enhancer analysis.

Key Dissatisfer Analysis

Key Dissatiser Analysis tells you which attributes explain overall visitor dissatisaction. It notonly identies the attributes that need to be xed in order to increase satisaction, but pro-

vides a prioritization o them.

We use a variation o a URF analysis coupled with a Shapley Value technique to examine how

attributes work in combination with each other and determine which attributes have the mostweight,thusallowingustoprioritizewhatactionstotakerst.

Once we know which attributes are Key Dissatisers, we segment visitors into two groups: thosewhose expectations on any Key Dissatiser are not met versus those whose expectations weremet on all Key Dissatisers. A comparison o key loyalty measures by these two segmentsprovides a clear understanding o the importance o xing ailures within an organization.


24/24

Key Enhancer Analysis

Tis analysis unctions like the Dissatiser Analysis expect that it examines which attributesexplain the highest levels o customer loyaltyi.e., which items are Key Enhancers, etc. Tismodel clearly indicates those areas that can lead to strong Loyalty scores and an enhanced

relationship with the customer. Te Key Enhancers are to be dealt with only aer the KeyDissatisers are addressed.

A key output o the imc Research Key Driver Analysis will be the relative importance o eachattributes impact on overall dissatisaction and the relative importance o each attributesimpact on strong visitor loyalty. ypically the our or ve attributes that are either Key Dis-satisers or Key Enhancers have two to three times more impact than the average o otherattributes (i.e., non key drivers, etc.).

We also plot the impact on dissatisaction versus poor perormance ratings or each attribute ona quadrant, as shown below:

Key Dissatisers plot to the right because they have the greatest impact on overall dissatisac-

tion.etopprioritiesareclearlyshowninthexquadrantattributes4,7,and22havepoor ratings and strong impact on overall customer dissatisaction.

We will also plot the impact o each attribute on strong loyalty versus top box ratings o peror-mance o the attribute on a quadrant chart

Benefts to our KDAP Approach to Identiying Key Drivers

Actionable ResultsIn comparison to other models which can show what is driving overall sitevisitor satisaction, Key Driver Analysis goes a step urther and tells how dierent attributesaect overall satisaction. Te analysis clearly indicates a hierarchy o which attributes aremost to least responsible or poor perormance or ailure.

Te area to ocus on to x the problem is clearly stated, and management can make a judgmentas to where to ocus eorts based on the time and cost it would take to get each repaired.Tus, the approach can aect HSP management choices or short-term versus. long- termsolutions.

Te diagram below oers an example o how imc Researchs Key Driver Analysis could speci-cally help ocus UI site-level improvements to the job search experience in areas that have thehighest level o impact on HSP.

HSP Proposal v1_4

Documents

Transcript of HSP Proposal v1_4