LMAtech 2015 - The Science of Art: Using Data to Inform Design

USING DATA TO INFORM DESIGN

Akhilesh Pant and Jessica DeJong | October 16, 2015 | LMA Tech

@jessdejongOne North Interactive

Art Director

@pant_akOne North Interactive

Strategist

Jessica DeJong Akhilesh Pant

What We’ll Cover Today1. Why you should care about user research

2. Which inquiry methods are most useful to law firms

3. How to gather and analyze data for a range of budgets

4. How to implement that data in a truly innovative solution

5. How to convince your stakeholders to let you try something completely new

What We’ll Cover Today6. And the burning question on everyone’s minds …

How much longer until lunch?

Disclaimers

• We are talking about a mindset shift

• Any research is better than none

• All cat gifs are solely Jessica’s doing

Why Are We Here?

Understanding our users has never been more important.• The legal industry is rapidly evolving.

• Legal marketing needs are rapidly changing.

• Audiences are expanding.

• But this is all we really know.

The Old Way

Assumptions Solution

Common Misconception

Assumptions Existing Data Solution

Optimal Way

HypothesesAudience-Specific

DataIntuitionExisting

Data Solution

What might this look like?

Example: MassMutual

Existing Data: Large majority of life insurance buyers are between 50 and 60.

Challenge: Getting healthy 40-year-olds to buy life insurance.

Example: MassMutual

Research Insight: Members of the target population seem to broadly fall into two categories: 1) financially foresighted, 2) not.

Intuition: We cannot market life insurance to healthy 40-year-olds who are simply not financially foresighted. But we can market financial foresightedness itself.

Example: MassMutual

Solution: An unbranded, comprehensive personal finance education program (Society for Grownups) targeted at this age group.

What do you mean … “user research?”

Let’s look at a simple, non-digital example

If I had asked people what they wanted, they would have said faster horses. — NOT Henry Ford

A lot of times, people don’t know what you want until you show it to them. — Steve Jobs

Ford & Innovation

“Ford’s adherence to his vision of the mass-market car was instrumental in both his early success … as well as his later failure to respond to rapid innovation in the marketplace.”

-Harvard Business Review, August 2011

Apple

Apple does doesn’t do market research, but they do conduct contextual user research.

Apple

Additionally, they have a deeply embedded culture of UX that maintains a clear focus on users and design.

Market vs. User Research

The stories of Ford & Apple highlight the difference between market research and user research, as well as the importance of using an iterative approach and fostering a culture of design.

Data vs Intuition

Not everything that matters can be measured, and not everything that is measured matters. — Elliot Eisner

Role of Intuition“Data has a way of turning a suspicion into a verifiable fact.”

-Nishant Kothary, A List Apart 2015

Data vs Intuition

Data and Intuition

Knowledge is knowing that a tomato is a fruit.

Wisdom is not putting it in a fruit salad.

But why should we care about UX research?

Market differentiation is dependent on innovation.

Innovation requires abandonment of comfort, but not reason.— AKHILESH PANT, PHILOSOPHER

What we aren’t saying

We’re not asking you to do anything ridiculous, like …

Scratch-n-Sniff Websites

Virtual reality office tours

… yet.

Or something even more absurd, like …

Or something even more absurd, like …

Removing awards from attorney bios

Let’s Dive In

Research Validates Assumptions

User Goals Brand & Marketing Goals

Digital Presence

When Research Can Help

Meet the FirmBaker, Little, Atkins & Hill

AnnChief Marketing Officer

Ann’s Stakeholders

GaryManaging Partner

“Our users aren’t millennials.

They don’t scroll.”

Our first reaction

OH YES THEY DO!

Solution: Research

We aren’t criticizing Gary. We love Gary.

But digital marketing and design are constantly changing and legal marketing should keep up.

Our job is to keep Gary (and ourselves!) educated by constantly researching our users.

So who is right?

Neither

We’re both wrong.

Data allowed us to understand the nuanced nature of the “scroll issue” – it’s much more than “everybody scrolls” and “nobody scrolls.”

It’s about engagement.

…and we still may be wrong

Secondary research showed us what general user behaviors are, but we still aren’t sure about law-firm website audiences.

This is where more targeted research can help.

Part of using research means accepting that we may be wrong.

What if we could…

Observe scroll behavior in task-based interviews with client representatives?

and / or

Test engagement on short and long versions of a page that are running simultaneously?

Can I actually do this?

How much does it really cost?

We admit: large-scale user research and testing can be expensive and time intensive.

But there is some good news. It’s nothing like market research; you can start small.

Any amount of research is better than nothing.

Usability Studies: 5-8 people

Source: NielsenNorman Group

To Summarize:

Yes. You can and should be doing user research.

User Research 101

A quick note

The recommendations that follow are specific to legal marketing.

There are a ton of inquiry methods available, but we’ve curated the list to the ones that make the most sense for the legal industry based on our experience in this space.

Types of Research

• Primary vs. Secondary

• Qualitative vs. Quantitative

• Behavioral vs. Attitudinal

• Direct vs. Indirect

User research landscapeBEHAVIORAL

Eye-tracking

Usability Testing

Ethnographic Field Studies

Web Analytics

A / B Testing

Concept Testing

Diary / Camera StudiesCustomer Feedback

Card Sorting

Desirability Studies

Intercept Surveys

Email Surveys

Co-creation exercises

Focus Group

Interviews

QUANTITATIVE

ATTITUDINAL

QUALITATIVE

User research landscapeHIGH VALUE

Eye-tracking

Usability Testing

Ethnographic Field Studies

Web Analytics

A / B Testing

Concept Testing

Diary / Camera Studies

Customer Feedback

Card Sorting

Desirability StudiesIntercept Surveys

Email Surveys

Co-creation exercisesFocus Group

Interviews

LOW COST

LOW VALUE

HIGH COST

Methods

75

User InquiryUsability TestingValidation

76

Email Surveys

Example

Ann’s team sends out a 10-question survey to a strategic set of 200 clients from their CRM system.

Questions are targeted at professional research, firm evaluation and web usage habits.

Email Surveys

Data Gathered

The team is able to analyze a set of (mostly) quantitative data and determine general correlations and hypotheses to test using interviews.

Internal Interviews

Example

Researchers from Ann’s agency of choice (or even a member of the Marketing team) talk to Gary for 45 minutes.

Internal Interviews

Data Gathered

Gary lays out his vision for the firm, the firm’s history, how he uses the firm’s digital properties.

Internal Interviews

Other candidates:• Practice leader

• “Rising star”

• Recent hire

• Recruitment representative

External Interviews

Example

Researcher holds a brief (30-45 minute) conversation with the General Counsel of Initech, one of BLAH’s clients.

External Interviews Data Gathered

• Client’s web and device usage habits

• How they research and evaluate firms

• What content they find most useful (i.e. persuasive)

• Characteristics of their ideal firm – the way such a firm “makes them feel”

External Interviews

Other candidates:• C-Suite

• General Counsel

• Procurement department representative

• Entrepreneur or small business owner

• Head of HR (Labor & Employment issues)


85

Usability Testing

Example

Having gathered insights from basic user inquiry, the team develops an initial prototype.

They recruit another small group of users and ask them to perform tasks on the prototype while speaking aloud about their thought process.

Usability Testing

Data Gathered

The “think-aloud” process allows the team to identify major pain points preventing users from completing basic tasks. It also reveals what is working as intended, and what may even be working better than intended.


88

Web Analytics

Example

Using a research-backed testing plan, the development team tags all major “events” on the site for measurement.

Web Analytics

Data Gathered

The team tracks informed “micro-conversions” to measure site performance, rather than relying on generic indicators (e.g. general traffic, bounce rates).

After 3 months, they identify a number of underperforming elements and pages.

A/B Testing

Example

They design and implement a “version B” of each underperforming element and have 50% of users interact with it, while the remaining 50% see the original version (A).

A/B Testing

Data Gathered

After a month, they measure performance of version A vs. B and implement the “winner” 100% of the time.

That is, till the next innovation cycle!

Bringing Them Together

Bringing them together

We just took you through a single hypothetical scenario, from Inquiry to Validation.

But this process should be repeated as often as possible.

Quantitative Analysis

User Inquiry

DESIGNAnalysis

Inquiry

DESIGN

DESIGN RESEARCHPROCESS

Research Synthesis

Research Outcomes

• Themes

• Personas

• Content Touchpoints

Now …Develop the Solution

Convincing Stakeholders

Identify ChallengeOutline MethodsPresent FindingsShow the Connection

100


101


102


103

What might this look like for BLAH Law?

Challenge 1:

Thought LeadershipMethods Used: Surveys, Interviews, Analytics

Hypothesis: Short-form thought leadership receives higher engagement.

Insight(s) Gathered: Short-form content performed better on mobile devices but long-form thought leadership performed better on desktop.

Solution: Add a “X min to read” indicator at the top of all thought leadership content

Challenge 2:

Social MediaMethods Used: Internal & External Interviews

Hypothesis: Curating content from external sources will drive social media engagement.

Insight(s) Gathered: When curated content is posted on a company page, engagement is not affected significantly.

Solution: Encourage Partners to curate and post relevant articles on their personal social media profiles.

Challenge 3:

Homepage RedesignMethods Used: Usability tests, A/B test

Hypothesis: Users will scroll to the bottom of the homepage if the design indicates there is more content “below the fold”.

Insight(s) Gathered: In our A/B test, 52% of users scrolled to the bottom on option A and 86% of users scrolled to the bottom of the homepage on option B.

Solution: Use option B with the clear “Scroll for More” call-to-action.

So, what should I do next?

Embrace design thinking

Use targeted research to produce powerful data

…and use that data as a rhetorical tool

Design Thinking

We’re talking about more than just testing usability and understanding your users.

We’re advocating for shifting the way you think about your users and digital efforts.

Recommended Approach

Define Design Develop

Plan

Discover

• Interviews• Contextual Inquiry• Ethnographic Field Studies• Secondary Research

Deploy

Optimize

• Heuristic Evaluation• Usability Testing• A/B Testing• Surveys

In Summary

Design has to work, art does not.— Donald Judd

“All service which a [person] can perform for humanity must serve to promote truth.”— Franz Boas

Thank You!

LMAtech 2015 - The Science of Art: Using Data to Inform Design

Education

Transcript of LMAtech 2015 - The Science of Art: Using Data to Inform Design