Google Is Just a Two Page SiteRelevant Results with Sitecore.ContentSearch

Martina Helene WelanderTechnical Consulting Engineer, Sitecore

Speaker

• Technical Consulting Engineer at Sitecore• Community and Information Enthusiast• Ecosystem Sites with Dnepropetrovsk Team

Martina Helene Welander

Hi!• Martina Welander• Technical Consulting Engineer• Ecosystem sites• mhwelander.net / @mhwelander

Speaker

• Technical Consulting Engineer at Sitecore• Community and Information Enthusiast• Ecosystem Sites with Dnepropetrovsk Team• @mhwelander / mhwelander.net

Martina Helene Welander

Speaker

In the direction of awesome, that’s where

…let’s do search!

Can haz knowledge?

Google Is Just a Two Page SiteRelevant Results with Sitecore.ContentSearch

Martina Helene WelanderTechnical Consulting Engineer, Sitecore

“Google is simply a search box with a second page of results. And those results are from other sites!”

Lalala hello world

examples lalala ten

items in my tree!

Sitecore.ContentSearch 101

Sitecore 7

Search and index

ALL the items

*

*

Search API(LINQ-based)

Search Technology Provider(DLLs and Configuration)

Search Technology API and Indexes

IEnumerable<DocSearchResult>

var index = Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");

using (var context = index.CreateSearchContext()){

var query = context.GetQueryable<ResultItem>().Where(x => x.Title == "Hej"); var executedResults = query.GetResults(); myModel.myList = executedResults.Hits.Select(x => x.Document).ToList();

}

Where Sitecore adds value• Source content to index to strongly typed object – and back again!• You can actually index anything• Provider model – Solr, Lucene, Elastic Search, Azure Search• Provider-agnostic LINQ-based search API• Highly configurable

Sitecore.ContentSearch is an API

Where should I focus my efforts?

CONFIIIIIG!

Crawlers

Mappers

Converters

Sitecore Field Index Field Object Property

Analyzers

Sitecore Field Searchable Data

Analyzer Wrappers

Back to Plain Ol’ SearchActually kind of difficult

It’s all about the Pentiums analyzers(Tokenizers and Filters)

Tokenizers

Hello my name is Martina

“Hello”, “my”, “name”, “is”, “Martina”

Types of TokenizerStandardTokenizer

“My name is Martina” “My”, “name”, “is”, “Martina”

KeywordTokenizer“My name is Martina” “My name is Martina”

N-Gram Tokenizer (Min 4, Max 5)“sitecore” -> “site”, “itec”, “ecor”, “core”, “siteco”, “iteco” … etc

Filters

Examples of Filters• Standard Filter• (Snowball) Porter Stem Filter• Stop Filter• Synonym Filter• Keep Words Filter• Pattern Replace Filter

ORDER MATTERS!

Indexing Process

Index

Query

Results

“name””Hello”

“Hello, my name is Martina”

“Martina”“my”

Rebuild when analyser changes!

Contains(“Hello, my name is Martina”)

Configuring a custom analyzer

Lucene – What does it look like?

Solr – What does it look like?

<fieldType name="text" class="solr.TextField">  <analyzer>    <tokenizer class="solr.StandardTokenizerFactory"/>    <filter class="solr.StandardFilterFactory"/>    <filter class="solr.LowerCaseFilterFactory"/>    <filter class="solr.StopFilterFactory"/>    <filter class="solr.EnglishPorterFilterFactory"/>  </analyzer></fieldType>

Previewing and help

6492 12:54:21 INFO ExecuteQueryAgainstLucene (sitecore_master_index): content:make~0.7 title:make~0.7 content:new~0.7 title:new~0.7 content:item~0.7 title:item~0.7 - Filter :

Debugging A Lucene-Based ContentSearch In Sitecore- Dan Cruickshank

My Super-Duper Analyzer

…which isn’t very special at all • Standard analyser• Standard filter• Porter Stem Filter• StopWords Filter• Synonym Filter (EXM / ECM, PXM / APS)*• Lowercase filter

The Query

What makes something relevant? (tf.idf)• tf – term frequency • Idf – inverse document frequency • coord - # of terms found in document • fieldNorm – field length

My fields• Title• Text• Byline• Keywords• Product

context.GetQueryable<ResultItem>() .Where(…)

.Filter() vs .Where()

#1 – Find me a match• Equals()• Contains()• StartsWith()

.Where(x => x.ResultsTitle.Contains("scaling"))

.Where(x => x["scaling"].Contains("scaling"))

.Match()

.EndsWith()

#2 – Slop and fuzziness!• Like()• Fuzzy search – fuzziness factor (float)• Phrase search – slop (int)

#3 – I love you, PredicateBuilder Expression<Func<ResultItem, bool>> predicate = PredicateBuilder.True<ResultItem>();

foreach (var word in list) {

predicate = predicate.Or(x => x.Title.Contains(word);}

False for ‘OR’,True for ‘AND’

#4 – Boost

• At query time• At index time (type or field)• Rules-based

BOOST

BOOST

~1000 real items

storageType=“true”

Attempt #1: EVERYTHING

If the title…

• Like phrase (with slop)• Contains phrase• Starts with phrase• Equals phrase

If the content…

• Like phrase (with slop)• Contains phrase• Starts with phrase• Equals phrase

Search: xDB Scaling

Search: Managing engagement plans

Search: Create engagement plans

A couple of important lessons•Whole Phrases vs Individual Terms• Boost()• Contains() / Equals()

Attempt #2: Phrase and terms

“engagement plan setup”OR

“engagement” OR “plan” OR “setup”

“engagement” AND “plan”OR

“engagement” AND “setup”OR

“plan” AND “setup”OR

“engagement” AND “plan” AND “setup”

Needs more boost

Attempt #3: Favouring titles

Sitecore 7 ContentSearch Tips- Matt Burke

“Finding a user’s search term in the title or keywords of a document is probably more relevant than one where the term is only in the body”

My work in progress

If nothing is working, you probably didn’t rebuild your index

Search: xDB Scaling

Search: Manage engagement plans

Search: Create engagement plans

// TODO: On the plane home• Keywords• Location • Pinning exact title matches – “scaling”• Expected search phrases with boost – e.g. “scaling xDB”, “xDB

scaling”, “xDB scaling options”

xDB• Key Behaviour Cache – developer or editor?• Common searches

It’s not all queries and indexes• Vague titles are a bit of a nightmare• Review use of keywords in content• “I would never search for that!” • Continuous user testing and tuning

What I learned• It isn’t magic • Get to know the provider• Content and content structure matter• Search is actually quite hard

OrganizersSponsor

Thanks to our… &…