Optimize Search Relevance in SharePoint 2013 Victor Poznanski Principal Program Manager

Jan Inge BergsethArchitect

SPC145

When You Leave, You Should Know…What:

…our view of relevance is…what tools you can use to manage relevance

When:…to use result sources, query rules, dynamic rules & relevance models

How:…to manage the relevance lifecycle

Challenges for Search

Users and their data

Challenges: Data ComplexityWhat we give to search engines

What most search engines see

1 2 3 4awaycomecontrastivecustomizeforfs14howoptimizingourrelevance (3)

shouldsimplesomess14techniquesview (2)whatwhenwithyou

Author: victor poznanskiTitle: conf relevancy sharepoint Date: 9.4.9Type: pptAuthority: 6Anchor: SPC presentation …

This release adds:• Non-search clicks• Improved clicks• Improved phrases

Challenges: Intent Where is my talk for the

SharePoint conference?

Are SharePoint Conferences always

held at the same place?

I wonder if there are any

presentations from previous

conferences

Different people have different intents

Query Rules help you handle intents

There is rarely a single right answer

SharePoint Conference

Query management tools Relevance management tools

Search Quality ≡ Relevance

Conversational user experience

Query management tools Relevance management tools

Collection Quality

Cust

om

ization

Cust

om

ization

Query management tools Relevance management tools

Configuration in the Conceptual Relevance Flow

For all queries:

Authorities: Level 1: http://networksRanking model: {incorporate user ratings}

Query:BT networkquarterly report

Search Web Part

Query Processing Engine

Document Collection

Thesaurus: BT British TelecomBest bets: BT Network …/BTN.htm

(WORDS BT, British Telecom) AND(WORDS network, networks, networked) AND (WORDS quarterly, quarterlies) AND(WORDS report, reports, reported)

Mixed Results for:• BT Network best bet• BT network quarterly

report• BT network

ContentType=reports

Dynamic Reordering Rules: Quarterly Report {prefer docs from http://reports}

Query Rule: {Terms} Quarterly Report {Terms} ContentType=“reports”

Improving Search Quality…the quality improvement cycle

The Quality Improvement Cycle

Identify the problem queries

Diagnose the

problems

Try to fix

Deploy

1. social features2. user feedback web parts3. analytics reports4. Best Bets

1. eliminate crawl issues2. check permissions3. Test query: item=url4. check freshness

Fixing the Problem: Core Tools

Authorities

• Identify important data hubs

Result Source

• Scope or federate your searches

Fixing the Problem: New Tools

Query Rules: Reformulate the query

Dynamic Reranking Rules: Reorder the Results

Custom Ranking Model: Build relevance from ground zero

Tuning Tool: Incrementally improve existing ranking model

Con

trol

Sim

plic

ity

Using AuthoritiesHelp the ranker to see important hubs

Authorities: SSA-level configuration

Sites that are important

Sites with low intrinsic relevance

Takes ~24hrs to propagate

Authorities: Disconnected

Main Site: Project data

Second Site: Documentatio

n

Third Site: Glossaries

Most Authoritative Second-level Non-authoritative

Authorities: Connected

Authorities: Connected

1

0

1

1

2

4

3

2

∞

4

Setting an authority affects all sites connected through hyperlinks

Sites are weighted

by distance to the authority

Authorities: Connected

1

0

1

1

2

1

1

2

∞

0

Setting an authority can have unexpected side- effects

Authorities: Connected

1

0

1

1

2

1

-

2

∞

0

Non-authorities do not affect connected sites

Using Result SourcesLimiting the scope of queries and federating

Queries are Sent to Sources“Customer

services monthly report”

Results from that sourceSourceUser aims a

query at a source

Protocol/Location

Query Transformation

(scope restriction)

Scoping Your Queries with Result Sources

Scoping a Source I

Scoping a Source II

User’s original queryRestrict to this site

OpenSearch federation demoUsage of OpenSearch federation against Bing (web)Create result block where we edit query to get a specific resultAdding a Bing URL with query as a more linkPin result block to the top.

Query Rules

• Conditionally rewrite the query• Conditionally federate to other

sources

Query Rules: rewriting the query

E.g. for monthly report customer support, prefer:• …results that might be monthly reports• …from particular sites• …with particular authors• …and the most recent date

Query Rules: conditional federationGlaxoSmithKline

bring results from the CRM vertical for this customerBenefits

bring results from the HR vertical

Query Rules: conditions and actions

if query starts with

“what is”

Query Condition

if user segment

is “sales”

ContextCondition

Add results from Wikipedia (without the

“what is”)

Action

What is COGS? [Bing] COGS site:en.wikipedia.org

Query Rules: conditions and actionsPHRASE-BASED MATCHES• Exact match, beginning or end• Ad-hoc or term store dictionary• Match a regex (advanced)

HISTORICAL (LOG-BASED) MATCHES• Is this query more likely aimed at

the following source…?• Do people mostly click on result

of the following type…?

Query ConditionsCURATED (BEST BET)• Show a promoted result

REFORMULATE THE USER QUERY• Show a block of results• Replace the core results

with a different query

Actions

Reformulating a User Query I

monthly report customer support

Action Terms Subject Terms

customer support site:http://reports/regularcontenttype=“MonthlyReport”

Query Transform

Query Rules are Associated with Sources“Customer

services monthly report”

Query Rules associated with that

source

Results from that source + Best Bets +Result Blocks

Local SharePointUser aims a query at a source

Building a Rule (site settings)

Query rule demo• Use Term to create a dictionary with action terms• Use advanced query text match and assign match to

{actionTerms}, remaining terms to {subjectTerms}• Use result block to edit the title• Use query builder to add query terms

Search Verticals And Federation

Simple Search VerticalsSearch Box

Results web part

Nav Entry (all are verticals)

Query Rules

Item Templates

Refiners

Sort Menu

Result source for this site

Federating Results in from a Vertical

Results from the people vertical are federated in to “everything”

Federating using More Likely Queries I

Federating using More Likely Queries II

Vertical demo• From the SharePoint Federate results from the MyRank vertical• Use query condition “Query commonly used in Source” and

select the “MyRank Results (system)”• Add a result block with title: SharePoint Results for

"{subjectTerms}"

Tweaking Relevance

Dynamic Reordering Rules

Dynamic Rules let you reorder results according to a set of criteria• Just like query rules apply to user queries, dynamic rules apply to results

They are configured as part of a query• They translate into a special query syntax called XRANK

Result actions let you promote or demote results that match• In many cases, you can use this instead of Best Bets.

Using the Query Builder

Using the Query Builder

The query template

Using the Query Builder: Sorting

Using the Query Builder: Sorting

This will ensure the freshest reports will come back, but there is no ranking

If the primary sort key is not “rank”, you cannot add dynamic ordering rules

Using the Query Builder: Dynamic Rules

Result conditions and actions

Result Conditions

Result Actions

Promote up or demote down

Manual condition looks like a query

Promotion and Demotion

A promotion-to-top places any result at the top of the results

A promotion of 10 moves a mediocre relevant result to the top

A demotion of 10 places a mediocre revelant result under the most relevant results.

A demotion-to-bottom places any result at the bottom of the results

Dynamic ordering (Xrank) demoAlternatives in O15:

• Edit "Search result" web part• Create a new result source• Query rule without condition

• Create the query rule without condition• Query rule w/o condition and "Change ranked results by

changing the query"

Ranking and Ranking Customization

You can Tune a Custom Ranking Model After…

You’ve considered or tried simpler optionsAuthorities, Thesaurus, Query Rules, Dynamic Rules

You’ve seen ranking could be improved for most queries

A meta-data property should be incorporated into ranking schema, eg: user rating

You’ve configured new managed properties & recrawled:

• Add a new managed property (integers: sortable/refinable, text: searchable)

• Recrawl

How Can You Change Ranking?

ΣRank Score

ωpowerpoint

ωauthor

ωactivity

Title=“XBOX Monthly report”

Type=pptActivity=2M clicksAuthor=Ian RankemRating = 5

Title=“XBOX Monthly report”

Type=pptActivity=2M clicksAuthor=Ian RankemRating = 5

Query: Ian Monthly

ωrating

Transform (normalization)

Document(managed properties)

Tuning Tool demoUse SharePoint list with custom propertyMap Custom property to managed propertyCreate new custom rankmodel using tuning toolPublish new modelUse new model in result source and test

SummarySearch quality is even better in this release

Improved ranking that builds on SS and FS + new featuresImproved linguistics: stemming and thesaurusIdentify hubs: authorities

Manage intent diversityHandle different intents with query rules

Friendly tools for managing relevanceManage the relevance lifecycle using admin UITune ranking models with free tuning tool

Search HOLs and events @ SPCHOL031 – Introduction to Search in SharePoint 2013

HOL034 – Exploring Search Query Rules in SharePoint 2013

HOL032 – Extending the Search experience in SharePoint 2013

HOL033 – People Search in SharePoint 2013

HOL035 – SharePoint Server 2013 Search Connectors and Using BCSMeet a Search SME

Ask questions, meet the community and share knowledge!

Mon-Thu @ Exhibit Hall

Hands on Labs

Daily 10:30am-6:30pm @ HOL Lab Lounge

Ask the Experts

Discuss search!

Wed 6:15PM @ Ask the Experts Lounge

Related Search Sessions @ SPCMon 3:45pm - SPC202 - Search Architecture in SharePoint 2013Speakers: Thomas Molbach, Rune Zakariassen

Tue 9:00am - SPC063 - Customizing Search Experiences in SharePoint 2013Speaker: Kerem Yuceturk

Tue 10:30am - SPC044 - Crawl and Index all Enterprise Content with SharePoint 2013 Search - Speaker: Vaidy Raghavan

Tue 10:30am - SPC125 - Hybrid and Search in the Cloud Speaker: Brad Stevenson

Wed 9:00am - SPC143 - Making Great Search Based Applications with Query Rules in SharePoint 2013 - Speaker: Pedro DeRose

Tue 5:00pm - SPC203 - Search Content Enrichment and Extensibility in SharePoint 2013Speakers: Runar Olsen, Kathrine Hammervold

Evaluate this session now on MySPC using your laptop or mobile device: http://myspc.sharepointconference.com

MySPC

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.