Qualtrics Panels

● What is a Panel?● How we find respondents● Type of respondents available● Use Cases● Best Practices● Why Qualtrics

Overview

But first! Quick example.

1)What is your age?a)Less than 21 b)22-30c)31-45d)45+

1)Which of the following animals do you own?a)Cat b)Dog c)Parakeetd)Fish

1)Have you ever been to Utah (Qualtrics headquarters)?a)Yesb)No (sit down)

What is a Panel?

● Invite only● Email sign up● Social media ● Banner Ads

Recruiting respondents

! Consumers- Moms, Shoppers, College Students. ! Teenagers! International ! Business Professionals! Medical

○ Doctors and professionals○ Consumers with ailments or medical conditions

Types of Respondents

Fordham University Case StudyChallenges

•Difficulty recruiting participants to come into the lab

•Limited subject pool at a small university

•Homogeneous participant demographic consisting primarily of college students

Solutions

•Obtain an appropriate survey panel

•Use a dedicated project manager to help conduct and manage the survey

•Access a diverse, global demographic to improve the quality of research

Results

•Extend survey reach to 11,000 participants in 21 countries

•Reduced time with a dedicated Qualtrics project manager to help manage quotas

•Performed a large international study that would have been extremely difficult to accomplish otherwise

Fordham University Case Study “Qualtrics Panels helped me reach 11,000 participants across 21 countries. We hope our findings can positively influence how the Intergovernmental Panel on Climate Change communicates its findings. I could not have performed this study without Qualtrics

“Our consumers love to share their opinions and Qualtrics captures this

feedback first-hand. This consumer insight is

invaluable to the growth of Yankee Candle and allows us to continually meet our

consumers’ needs.”

Yankee Candle Case StudyChallenges

•Required an in-house research platform to supplement outsourced research

•Wanted to eliminate the manual, time-intensive survey approach

•Needed to expand its consumer insights

Solutions

•Broaden research with feedback from general candle shoppers by using panels

•Test new products and fragrance concepts before investing in development

•Manage in-person fragrance testing to capture immediate feedback

Results

•Saved tens of thousands of dollars in research management costs by reducing the number of

outsourced research projects

•Reduced fragrance testing survey management by 85%

•Increased customer loyalty

Best Practices

Tips: Drafting the Questionnaire

Don’t reinvent the wheel: use a (robust) survey library to jumpstart your questionnaireConsider your respondent’s point of view in terms of fielding considerationsUse your research questions to define your questionnaireOrganize your questionnaire into modules based on question categories and the flow of your surveyAlways test: Test your questionnaire both qualitatively (with human beings) and quantitatively (with test data)

Survey Libraries

A good and methodologically strong survey library

is like having a well-defined blueprint for a house similar to the home you are going to build

Same blueprint, two different houses

Questionnaire Key Considerations

Reduce drop-offs by keeping length of interview (LOI) as short as possible: your questionnaire should be no longer than 15 minutes or you will get higher than normal breakoff rates

Inform your respondents how much time the survey will take them

Explain the purpose of the survey to your respondent (don’t bias your results though)

If you ask sensitive questions, make sure you explain why you need the information

Defining the Research Question

A well-defined research question translates back into a good questionnaire, as nothing extra will be asked, and nothing critical will be missing.

Identify the base: Identify the characteristics of the respondents who should (or should not) answer the research question

Create questions that determine the base: Make sure that your questionnaire has questions that identify respondents who meet these characteristics

Checklist for Transforming the Research Question

Show the ad to the respondent, and track the ad the respondent saw

Putting it all together...

Questionnaire Modules

Step 1: Start with screening questions, so as to screen out respondents as quickly as possibleStep 2: Ask demographic questions – they are both easy and can be sensitive with higher break-off rates

Step 3: Ask quota-specific questions so if a quota group becomes full, the respondent can be screened out at the beginning Step 4: Write a pre-amble after screening giving information about the survey, the topic, the length, and other information that will induce higher response ratesStep 5: Ask general questions that introduce the topic of the surveyStep 6: Ask specific questions

Testing your Questionnaire

Test each question as you write your questionnaire (don’t wait until the end!)

Test entire questionnaire, putting yourself in the place of the respondent

You Other human beings Randomly Generated Data

• Test all new questions, asking for feedback about the questions directly after you ask the question

• Test entire questionnaire, asking two extra questions:

• Was anything confusing?• Did you run into problems

taking the survey?

• Test questionnaire by using randomly generated data and looking at the analysis

Drafting the questions

Tips on Wording Questions

Avoid Leading Words / Questions Give Mutually Exclusive ChoicesAsk Direct Questions Add a “Prefer Not to Answer” OptionCover All Possible Answer Choices Use Unbalanced Scales CarefullyAsk One Question at a Time

Avoid Leading Words / Questions

(Bad) Example: “Quitting smoking is hard. Do you want to quit smoking for good?” Yes/No

Better:“Do you want to quit smoking cigarettes?” Yes / No

Ask Direct and Precise Questions

(Bad) Examples

Don’t leave it up to the respondent to guess what you mean – use precise definitions and err on the side of too much precision. Do not assume that previous questions will give enough context to the respondent.

Add a “Prefer Not to Answer” Option(Bad) Example:

Sensitive questions may cause a respondent to stop taking your survey. Allow the respondent to not provide the information you requested.

Cover All Possible Answer Choices (Bad) Examples:

If someone makes $20,000 exactly – he or she is going to struggle to find the right response to check

What about the person who uses the Internet every couple of weeks?

Ask One Question at a Time

(Bad) Example:

These two questions might not be asking the same question in the mind of the respondent

Tips: Drafting questions

Keep the analysis in mind: will the question type gather the data you are wanting?Don’t use acronyms or industry-specific languageWrite your questions so that a person with a sixth-grade education would be able to understand themFor new questions, use tested questions as templates to help you write the new questions

Key Takeaways

Keep your Survey ShortKeep your Survey SimpleKeep your Survey Well-organizedKeep our Survey’s Analysis in Mind

What do we need to know????

What do we need to know

1) N: (Total number of respondents your looking to reach?)2) Targeting: (Who is your target? What regions? i.e. Wal-Mart shoppers, managers, full-time employees, general population, mothers)3) Screen-Outs: (Will there be any additional screen-out questions or qualifying criteria?)4) Any other quotas or requirements amongst this sample?5) LOI: (Time length. We estimate ~5 multiple choice questions will take 1 minute.)6) When would you want to have data collected by? 7) What are your goals and desired insights in conducting this research?

What is IR???

QUESTIONS???

https://goo.gl/DpbR04