RWJF.org’s Rollout of Topic TaxonomyThe Project from a Client View

2April 10, 2023

In This Presentation

•About RWJF

•Why do we need a taxonomy?

•The project: development and validation

•The result

•Lessons and next steps

3April 10, 2023

About RWJF

To improve the health and health care of all Americans

•Focus on problems that demand attention

•Passion, perseverance and rigor

•Support programs with measurable impact

•Seek to create meaningful and lasting social change

4April 10, 2023

RWJF By the Numbers

• Total Assets: $10 billion

• Grants and Contracts Awarded in 2006 —928

• Total Active Grants: 2402

• 7 Priorities

5April 10, 2023

Why Taxonomy? A Lot of Content

•4,296 publications, articles, journal articles, webcasts, videos on RWJF.org today

•400 peer-reviewed journal articles and reports added each year

•100 end-of-grant reports added each year

•Does not include (but will soon): news releases, feature stories, speeches, more!

6April 10, 2023

Why Taxonomy? Clarity

Guiding Principles: Be objective, rigorous and transparent and communicate clearly and openly …

2007: 7 Program Areas

2005: 11 Interest Areas

2003: 4 Goals

2001: 8 Priorities

Goals change!

7April 10, 2023

Before The Project: Internal Taxonomy Creation

• Workgroup of grants managers, Web staff, information architect, research team, librarian

• Back-indexing completed in early 2005 and takes place weekly on an ongoing basis

• Approximately 600 terms used to index publications as well as active, pending and turned-down grants

• Categories cover:• RWJF goals and program areas

• What type of work it was (demonstration, research, communications)

• What population benefits from the grant and where, if any

• The subject matter of the project

• Any medical problem or health condition underlying the program

8April 10, 2023

The Project

•Part of a merger of multiple sections into a single section begun in early 2006

•User-focused vocabulary covering conditions, treatment, people, places and particular government and social services

•Permanent home for content as programs change

•Prioritizes frequently searched content

•Must be clear that not all topics are funded

9April 10, 2023

Discovery

•Reviewed consumer health taxonomies, relevant sections of NLM taxonomies

•Reviewed our “competitors” – not much structure to parallel

•Rapid prototype showing a year’s worth of information from the internal taxonomy

• “Wow, that’s a lot of terms”

• “Let’s find out how users want to see this”

10April 10, 2023

Open Card Sort

11 RWJF grantees who also were members of highest priority target audience

3x5 index cards of 100 terms covering the spectrum of content

1 hour to work as individuals to create as many/as few groups as they wanted

11April 10, 2023

Open Card Sort: Results

11 different hierarchies: some flat, some deep (4 levels)

Not a high degree of overlap especially on particular policy issues

Certain areas of agreement (conditions and diseases, health care quality, people and places)

12April 10, 2023

Taxonomy Iteration

•10+ iterations between consultant and RWJF

•First iteration had about 500 topics

•Last iteration had 370 topics

•Top level hovered between 6 and 11 choices

•Hierarchy was between 3 and 5 levels

13April 10, 2023

Iteration Factors

•Remove jargon: “pharmaceutical services” v. “prescription drugs”

•The same word has different meanings in different contexts: “health care system”

•How do hierarchy and visual presentation match up? Launching a browse without browse functionality and as a result went deeper

14April 10, 2023

Closed card sort: “Taxi” (Catalyst Group Design in NY)

•2 days: 7 hourlong sessions per day

•User, interviewer, computer

•Question: Where would you look for an item (for example, asthma in children) in this hierarchy?

•Users saw terms, but not content below terms, and could not go backwards

•Users rated their confidence on a scale of 1-5

•Items developed from search logs and presented in varying order

•Big changes from Day 1 to Day 2

15April 10, 2023

Post-Study Observations

•Top level still does not satisfy a number of users

•Not showing content or allowing back-and-forth really stresses the taxonomy

•Current iteration misses a sizeable number of topics

•Easier to use when looking for health care items and less easy to use for public health items

•Several topics had multiple successful paths

16April 10, 2023

Post-Report Activities

•Relate topics (relate a chronic condition, such as asthma, to chronic care improvement)

•Eliminate redundancy

•Focus on other elements of a relaunch with a hard deadline

17April 10, 2023

The result: taxonomy at launch

•6 top level categories (plus a category of Popular Topics)

•366 terms

•Used in our Publications & Research section and in limited amounts elsewhere

18April 10, 2023

Publications & Research

19April 10, 2023

Long-Term Care

20April 10, 2023

Individual Articles

21April 10, 2023

How’d We Do?

Policy-makers seem to like the usage in popular topics but want keywords used more throughout the site

Twice the proportion of usage as the previous standalone sections (7.5 v. 13.5) (attributable to multiple factors)

Single-page visits down a few points

Still difficult to find things

Need flexible browse interface to account for multiple starting thoughts (a care setting v. a population v. a disease)

22April 10, 2023

Our Lessons

Design hierarchy and interface together

Incorporated more informal feedback with rapid prototyping

Still uncertain as to how to engage internal stakeholders

Assessing usability and user feedback is not enough; need additional content expertise

Look for alternative forms of presentation to suit multiple audience