EXTREME SEARCH SOLUTIONS:

Understanding and Surpassing Joomla! Search

Sully SullivanFor the Austin Joomla! Users’

Group18 February 2013

About Sully• Born (naked) in

Massachusetts• Live in the DC area• Working in Joomla! since

1.0.3• Our company, Terrace

Media, provides white-label development to PR, design and consultant firms

• Hobbies include fine dining, armchair theology, zeppelin racing and messing with Texas.

Blurry and I know

it!

Sully Knows Joomla!

• Lead Community Manager for Joomla!’s Facebook page

• Joomla Social Media Team• Joomla! Marketing Team• Joomla! Community Magazine • Director of Northern Virginia JUG• Organizer of JoomlaDay DC

This Presentation Will:

• Compare Joomla!’s two core search components with external search server options

• Introduce terms and concepts of search and enterprise search servers

• Thumbnail sketch Apache Solr• Be provided as a PDF to you• Contain a lot of Easter eggs and be

fun and awesome like me• Probably mess with Texas

In other words, we have ill-tempered mutant sea bass.

It’s a start.

CMS and Search: IA concepts

• A Content Management System controls, organizes and presents information to the web.

• The process of creating content organization that’s easy to use and effective is called “Information Architecture”

Classic!

I can’t recommend Don’t Make Me Think by Steve Krug highly enough as an introduction to web usability and information architecture.

I’m going to start an analogy from Krug’s book to illustrate the importance of web search in content management.

In a good website, visitors

Get what they want to get from the site

Get what you want them to get from the site

Meet Your Guide, Dorian

• One of my favorite people

• Seventh Grade• Gaithersburg Middle

School• Fluent in English &

French• Hobbies include

violin, ballet, Just Dance 4, hanging out, being awesome, Greek food, world domination

One of Dorian’s favorite places, the Mall

One of Dorian’s favorite places, the mall

Forever 21?

… No, Sears.

Not My Metaphor.Blame Steve Krug.

Sorry, Dorian.

Our Goal

Enter Store

Head Up A Level

Check Signage

Find Our Aisle

Success!

What did we just do?

What did we just do?

What did we just do?

What did we just do?

Success!

Don’t Overthink.What’s an Electrical Hand?

Huh?

Stay Awesome, My Friends.

Don’t Underthink Either.Like say, Ross Dress for Less.

Takeaway #1

… and test to be sure that this taxonomy is

intuitive.

Organize your CMS content in a logical way so that users can find things on

their own

Another Way

Excuse me, umm, Dillon? Hi. Where are the Phillips screwdrivers?

“They’re uhhh, upstairs in Electrical Hand Tools. I’ll show you.”

Success!

What did we just do?

What did we just do?

What did we just do?

Success!

Two Types of Visitors

1. Browsers2. Searchers

Which kind buy screwdrivers?

Both!

Takeaway #2

Work on your search experience as though

it’s your customer service experience. It is.

Asking a store clerk for assistance is asking him to retrieve info from his brain. Using search is asking to retrieve info

from a database.

Vocabulary

Search Form

Vocabulary

Search Term

Vocabulary

Search Term+ Input =Search Query

Vocabulary

SERP:SearchEngineResultsPage

Vocabulary

Result

Results exposedatabase Records

To search, we need a database. In Joomla!:

Visitor inputs search term into search form. (might be a module or component)

Input passed to PHP script (component)

Script queries database via Platform(usually MySQL/MySQLi)

Database query displayed by component as a SERP

Visitor uses SERP, clicks on Result and visits a database Record (Article)

Joomla! has Two Search Engines

Search• AKA “Standard

Search”• Released in 2005

with Joomla! 1.0• Originally built for

Mambo• Matches search

query against the Joomla! database

• Supported by most content components

Smart Search• AKA “Finder”• Released in 2012 with

Joomla! 2.5• Originally an

extension developed further for core inclusion

• Adds new features• Disabled by default• Not all extensions

support Smart Search yet

Smart Search Features

• When you begin to type a search term that is already in the database, you’re offered a suggestion.Vocabulary: Search Suggestion or Autocompletion

• Operands (‘and’, ‘not’ etc) allow for Boolean Search

• Some “Filters” to allow you to use Joomla! metadata (author, date, etc) and this is a limited example of Search Faceting.

Smart Search Limitations

• Not supported by every content component: extension developers need to create support for it.

• Thus, not default in current releases.

• Effective implementation of Smart Search means you have spent some time configuring the component and indexing your site.

Both core search components have Joomla!-based Limitations

• For Search or Smart Search to find a Record, it must be in the database.

• But items stored in the file system are not indexed.– Things that might be in your file system you’d like

people to discover with search include PDFs, images, multimedia.

• Complex content components might require their own search if they have different meta-info to search.– Examples: e-commerce, multimedia, review

components

• Slow performance on really large sites – Because of structural limitations of MySQL’s RDBMS

database type

Good News: It’s Not News.

The Joomla! Development Team has been working to create a Unified Content Model (UCM) which treats all types of content as equivalent database records.

That’s another presentation. See Mitch Pirtle’s presentation at JoomlaDay Italy 2012 if you want to learn more about UCM. Also maybe sacrifice a virgin to the Joomla! volcano and have a six-figure bribe, because the rest of us would like to learn more about it from PLT, too.

Takeaway #3

Learn if your extensions

support it and if so, learn how to

configure it.

Smart Search offers a better Joomla! search experience for

your site

That Was Dry

How can we improve on this Search experience?

Excuse me, umm, Dillon? Hi. Where are the Phillips screwdrivers?

“They’re uhhh, like, sort of upstairs in Electrical Hand Tools. I’ll show you.”

We Can Ask A Better Guide

• Meet Edward Lampert• Chairman, Sears &

Kmart• Dillon’s Boss• Senior Corporate

Executive, so knows lots about screwing things

• Hobbies include cufflink collecting, real estate acquisitions, world domination.

You’re fired, Dorian.

Catch you at Forever 21?

We’re also done with Steve Krug.

You take it from here,

Sully.

Improving Search = Improving Customer Service

• Get us there faster– Dillon didn’t do bad, but I bet Eddie

spares us the “uhh, like, sort of upstairs” part

• Clarify our need– “What kind of screwdriver, a #2

Phillips?”

• Follow-up– “Doing some electrical work? Or

assembling the furniture you bought in the other department?

Improving Search = Improving Customer Service

• Correct Mistakes– “Do you mean an electrical hand tool?”

• Offer suggestions– “You need some screws? We got those

too.”

• Develop Ranges– “How much did you want to spend?”

• Feature Relevant Alternatives– “You really need to see the power

screwdrivers!”

Web Service Options

They’re Like Asking the Mall’s General Info Desk

We Lose ControlDon’t let your visitors ask just anyone for a screwdriver or they’ll end up at the airport drinking vodka and orange juice, which at the airport costs way more than $6.99, I might add.

Just, no.

Keep them in your sight, keep them in your site and control the customer service experience.

Enterprise Search Platform

• Like Joomla!, Free and Open Source Software (FOSS)

• Built on Apache Lucene & Java

• Has the features we want, which are…

Apache Solr features:Uses the Lucene library for full-text search • Faceted navigation • Hit highlighting • Query language supports structured as well as textual search • JSON, XML, PHP, Ruby, Python, XSLT, Velocity and custom Java binary output formats over HTTP • HTML administration interface • Replication to other Solr servers - enables scaling • QPS Distributed Search through Sharding - enables scaling content volume • Search results clustering based on Carrot2 • Extensible through plugins • Pluggable relevance - boost through formula • Caching • Embeddable in a Java Application

Say What?

Solr is not a CMS

Solr Joomla

Get Us There Faster

• Is it faster to find car keys by fishing through a purse or dumping it out in a bin?

• Joomla! has a table-driven database to carry content

• Solr has a table-free environment to find content

Clarify our need

• “What kind of screwdriver, a #2 Philips?”

• Vocabulary:Autocompletion

Autocompletion

Correct Mistakes

• “Do you mean an electrical hand tool?”

• Vocabulary:Search suggestion

• Vocabulary:Spelling correction

Spelling Correction& Search Suggestion

Follow-up

• “Doing some electrical work? Or assembling the furniture you bought in the other department?

• Drilling down into our need with search, just like we did by checking successive navigation tools

• Vocabulary:

Faceting

Faceting

Reasons to Facet

• Complex Content• Keyword Search Not Specific

Enough• Lots of Similar Content• Many more uses!

Develop a Range

• “How much did you want to spend?”

• We can use numeric values such as price, date, age range, etc. to help drill further into our database content

Ranges in Searching

Feature Relavent Alternatives

• “You really need to see the power screwdrivers!”

• Vocabulary:

Search Highlighting

No Search Highlighting

With Search Highlighting

Takeaway #4

If you should, you really should.

Just because you can, doesn’t mean you should.

Solr is complicated.

Complex website content benefit from a tailored user

experience power and performance.

Extreme Customer Service

• Replace navigation and browse by search query URLs using Solr

• Index more than one site into the same Solr instance, to allow searches across multiple websites

Solr on the Web

• http://lucene.apache.org/solr/• Famous brands using Solr

include:– Sears!–WhiteHouse.gov – Instagram– Digg– Zappo’s

Solr in Joomla!

• JSolr developed by Wijiti• http://www.wijiti.com/projects/jsolr• Free

• Installs Indexing component to get content from Joomla! to Solr

• Installs Search component/modules to search Solr from the Joomla! site

Caveats

• Solr is complex software. It takes time to learn.

• Use cases differ. Knowing what is possible is different from knowing how to accomplish it.

• Solr requires extra server resources

• Solr requires an ongoing service commitment

Acknowledgments

• Hayden Young at Wijiti, Solr help• Jen Kramer, presentation advice &

moral support• Miljan Vujosevic at Terrace Media,

presentation advice & immoral support• Dorian at Gaithersburg Middle School,

general awesomeness & spokesmodel services

No harm befell any Sears employees in the making of these slides. I did polish off a vodka bottle, though.

Stay in touch

Sully SullivanTerrace Media Group

5576 Norbeck Rd Suite A-141Rockville, MD 20853

(202) 683-9530sully@terracemedia.com

Skype: TerraceMediaTwitter: @TerraceMedia

Please Like Joomla! on Facebook!