Flax ovum search-across_the_enterprise

Open Source Search for the Enterprise

Charlie HullManaging Director, Flax3rd November 2010OVUM Briefing, Search Across the Enterprise

[email protected]/blog+44 (0) 8700 118334Twitter: @FlaxSearch

mailto:[email protected]

http://www.flax.co.uk/blog

http://twitter.com/#FlaxSearch

Search engine specialists with decades of experience Developers, innovators and strategists Based in Cambridge, UK Technology agnostic – but open source exponents Recently selected as UK Authorized Partner by Lucid

Imagination Customers include Mydeco, NLA, Durrants Ltd, Financial

Times, MediaMiser, MySkreen, Accenture, University of Cambridge Recently asked to present at British Computer Society

and Lucene Revolution conferences

Who are Flax?

“Open-source software (OSS) is computer software that is available in source code form for which the source code and certain other rights normally reserved for copyright holders are provided under a software license that permits users to study, change, and improve the software. […] Some open source software is available within the public domain” (Wikipedia)

What is open source?

It's the work of amateur developers

Myths about open source

It's the work of amateur developers If I use open source, I have to open up my

software/servers/network to all and sundry



software/servers/network to all and sundry Open source software isn't reliable or scalable



software/servers/network to all and sundry Open source software isn't reliable or scalable It's free



software/servers/network to all and sundry Open source software isn't reliable or scalable It's free It's unsupported


Open source search software

Apache Lucene and Solr are trademarks of The Apache Software Foundation

- Flexible licensing- Vector space model- Java and other languages- Well known and supported




- The successor to Muscat- Bayesian probabilistic ranking- C/C++ with language bindings - Highly accurate & scalable





- The successor to Muscat- Bayesian probabilistic ranking- C/C++ with language bindings - Highly accurate & scalable


And more....


Some exampleshttp://www.nla-clipshare.com

Newspaper Licensing Agency – NLA Clipshare20 million newspaper stories6500 usersContent from every major newspaper (and most regionals)Used by journalists, clippings agencies, media monitorsReplacing internal systems at major newspapers

http://www.nla-clipshare.com/

Some exampleshttp://www.nla-clipshare.com

Newspaper Licensing Agency – NLA Clipshare20 million newspaper stories6500 usersContent from every major newspaper (and most regionals)Used by journalists, clippings agencies, media monitorsReplacing internal systems at major newspapersOne of very few ways to search content from all the papers within hours of publication

http://www.nla-clipshare.com/

Some examples

Financial Times – press cuttingsWeb Service for easy integrationXML source dataFaceted searchArea filters (whole article, body, headline, byline or any combination)Synonyms, spelling suggestions

http://presscuttings.ft.com

http://presscuttings.ft.com/

Some examples

Financial Times – press cuttingsWeb Service for easy integrationXML source dataFaceted searchArea filters (whole article, body, headline, byline or any combination)Synonyms, spelling suggestionsBuilt from scratch in a fortnightDesigned as a prototype, scaled to production use without significant change

http://presscuttings.ft.com

http://presscuttings.ft.com/

Some examples

Durrants Ltd. Media monitoring platformThousands of client search profiles Hundreds of thousands of articles per dayComplex publication heirarchyEstablished pipeline

SolutionFlexible query language allows OCR errors, punctuation, fuzzy matching, weightingSupports features of previous engineScalable master-slave architecture

Some examples

Durrants Ltd. Media monitoring platformThousands of client search profiles Hundreds of thousands of articles per dayComplex publication heirarchyEstablished pipeline

SolutionFlexible query language allows OCR errors, punctuation, fuzzy matching, weightingSupports features of previous engineScalable master-slave architecture

Accuracy improved in some cases from 95% rejected to 95% accepted Hardware budget 15% of previous system

Some examples

(Unnamed multinational radio suppliers) Intranet search12 million documentsMultiple formats – Office, PDF, HTML...User and group-based security (LDAP)Faceted searchUsers can 'tag' interesting documents – for

example to identify a 'reference' version

Some examples

(Unnamed multinational radio suppliers) Intranet search12 million documentsMultiple formats – Office, PDF, HTML...User and group-based security (LDAP)Faceted searchUsers can 'tag' interesting documents – for

example to identify a 'reference' versionOpen source chosen because of significant

cost advantage – commercial solutions uneconomic at this scale

A look at Lucene & Solr

Among the top 15 open source projects Installations at over 4,000 companies Downloads have grown nearly 10x over the past three

years Over 7,000 downloads a day.




USA based Employs 9 out of 15 top Lucene committers Offers training, consulting and up to 24x7

support Developing value-add software




USA based Employs 9 out of 15 top Lucene committers Offers training, consulting and up to 24x7

support Developing value-add software Flax are UK partners & resellers

Lucid Works Enterprise

Who are Lucid working with?

Some Lucene & Solr numbers

LinkedIn – 30 million users Internet Archive – a billion indexed pages Salesforce.com – 8 terabytes of searchable data Twitter – a billion queries a day

Why open source search?

Flexible, extendable


Flexible, extendable Powerful & scalable


Flexible, extendable Powerful & scalable Lower cost, especially when planning for growth


Flexible, extendable Powerful & scalable Lower cost, especially when planning for growth Commercial support available as necessary


Flexible, extendable Powerful & scalable Lower cost, especially when planning for growth Commercial support available as necessary

- Freedom to innovate

Looking to the future


More and more content including social media


More and more content including social mediaMultiple delivery platforms


More and more content including social mediaMultiple delivery platforms Search-powered applications


More and more content including social mediaMultiple delivery platforms Search-powered applicationsCloud computing


More and more content including social mediaMultiple delivery platforms Search-powered applicationsCloud computingMore use of entity extraction & sentiment

analysis


More and more content including social mediaMultiple delivery platforms Search-powered applicationsCloud computingMore use of entity extraction & sentiment analysis

Search no longer a bolt-on, but a platform for innovation


More and more content including social mediaMultiple delivery platforms Search-powered applicationsCloud computingMore use of entity extraction & sentiment

analysis

Search no longer a bolt-on, but a platform for innovationOpen source no longer an outsider, but the obvious choice

Thankyou!

Any questions?

[email protected]/blog+44 (0) 8700 118334Twitter: @FlaxSearch

mailto:[email protected]

http://www.flax.co.uk/blog

http://twitter.com/#FlaxSearch

Flax ovum search-across_the_enterprise

Technology

Transcript of Flax ovum search-across_the_enterprise