Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self...
Transcript of Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self...
![Page 1: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/1.jpg)
Thursday, May 24, 12
![Page 2: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/2.jpg)
Who we are
Uri Boness Shay Banon• Co-founder SearchWorkings• @uboness
• Founder of ElasticSearch• @kimchy
Thursday, May 24, 12
![Page 3: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/3.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 4: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/4.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 5: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/5.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 6: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/6.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 7: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/7.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 8: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/8.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 9: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/9.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 10: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/10.jpg)
The Next Hour
How we got here?ElasticSearch
The Distribution ModelA glimpse into the API
Multi TenancyApplications
Q&A
Thursday, May 24, 12
![Page 11: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/11.jpg)
How we got here?
Thursday, May 24, 12
![Page 12: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/12.jpg)
Search - Past
• Traditional “Enterprise” Search
• Federated Search
• Monolithic “do it all” Systems
• Connectors
• Document convertors/processors
• (Enterprise) Security
• oh yeah... and Search
Thursday, May 24, 12
![Page 13: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/13.jpg)
Search - Present• Findablility First
• Free text, faceting, ranking, etc...
• Other top concerns:
• Scale
• Maintenance
• Real time
• Cloud
• DevOps are programmers
• Chef, Puppet, Whirr, Script languages
Thursday, May 24, 12
![Page 14: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/14.jpg)
Search - Future• All about data accessibility & insight
• Real time-ness
• Scale (Big Data)
• Store
• Query/Search
• Analyze
• Familiar & consistent data model and infrastructure
Thursday, May 24, 12
![Page 15: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/15.jpg)
ElasticSearch
• A highly scalable and distributed search engine
• Built on top of Lucene
• Platform & Environment agnostic
• Founded & mainly developed by Shay Banon
• Vibrant community
• Production ready & mature
Thursday, May 24, 12
![Page 16: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/16.jpg)
ElasticSearchAPI
Thursday, May 24, 12
![Page 17: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/17.jpg)
API Design
• Simplicity
• Natural
• Platform friendliness
• Human friendliness
• Consistency
• Extensibility
Thursday, May 24, 12
![Page 18: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/18.jpg)
API Design
• Simplicity
• Natural
• Platform friendliness
• Human friendliness
• Consistency
• Extensibility
REST
Thursday, May 24, 12
![Page 19: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/19.jpg)
REST API Design
vs.
Thursday, May 24, 12
![Page 20: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/20.jpg)
api for all• Why?
• Consistency
• Runtime maintainability
• DevOps are programmers
• What?
• Data (Index, Update, Delete, Search)
• Management & Maintenance
• Monitoring
Thursday, May 24, 12
![Page 21: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/21.jpg)
Dictionary
Thursday, May 24, 12
![Page 22: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/22.jpg)
Dictionary
• Documents & Fields
• Document Type
• Index
• Node
• Cluster
Thursday, May 24, 12
![Page 23: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/23.jpg)
Design Decisions
• Default format: JSON
• Zero Conf. Policy
• System provides defaults for everything
• Enables overriding all defaults
Thursday, May 24, 12
![Page 24: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/24.jpg)
Data API
• Index
• Search
• Query DSL
• Update, Delete
Thursday, May 24, 12
![Page 25: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/25.jpg)
Index
• Index
• Delete (by id / query)
• Update
• Bulk API (not covered here)
Thursday, May 24, 12
![Page 26: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/26.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1PUT
Thursday, May 24, 12
![Page 27: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/27.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1PUT
Thursday, May 24, 12
![Page 28: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/28.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1
indexPUT
Thursday, May 24, 12
![Page 29: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/29.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1
index typePUT
Thursday, May 24, 12
![Page 30: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/30.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1
index type idPUT
Thursday, May 24, 12
![Page 31: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/31.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1
index type idPUT
Thursday, May 24, 12
![Page 32: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/32.jpg)
Indexing - Addhttp://localhost:9200/goto-adam/session/1
index type idPUT
Thursday, May 24, 12
![Page 33: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/33.jpg)
Indexing - Delete
DELETE http://localhost:9200/goto-adam/session/1
OR
DELETE http://localhost:9200/goto-adam/session/_query
Thursday, May 24, 12
![Page 34: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/34.jpg)
Indexing - UpdateLet’s track the number tweets mentioning this talk:
POST http://localhost:9200/goto-adam/session/1/_update
Thursday, May 24, 12
![Page 35: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/35.jpg)
Indexing - UpdateLet’s track the number tweets mentioning this talk:
That’s better... from now on we just update the count
POST http://localhost:9200/goto-adam/session/1/_update
Thursday, May 24, 12
![Page 36: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/36.jpg)
Search
• Query DSL
• Simple query
• filtered query
• facets (terms & date histogram)
• Other supported search features
Thursday, May 24, 12
![Page 37: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/37.jpg)
Query DSL• Programming language friendly
• Tool friendly
• Self explanatory
• Fully supports all Lucene search constructs
• All Lucene query types and filters
• Additional query types (e.g. Geo, Parent/Child, Nested, and more)
• Easily extensible
• Plug-in your own query types with their own custom DSL
Thursday, May 24, 12
![Page 38: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/38.jpg)
Queries
Thursday, May 24, 12
![Page 39: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/39.jpg)
Basic QueryPOST http://localhost:9200/twitter/tweet/_search
Thursday, May 24, 12
![Page 40: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/40.jpg)
Basic QueryPOST http://localhost:9200/twitter/tweet/_search
Thursday, May 24, 12
![Page 41: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/41.jpg)
Rich Boolean Queries
Thursday, May 24, 12
![Page 42: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/42.jpg)
Filtered Queries
Thursday, May 24, 12
![Page 43: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/43.jpg)
Query Types
• text, query_string, field
• term, range, prefix
• bool, dis_max
• custom_score, custom_filters_score
• ...
Thursday, May 24, 12
![Page 44: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/44.jpg)
Filter Types
• term, range
• geo (distance, bbox, polygon)
• bool, and, or, not
• ...
Thursday, May 24, 12
![Page 45: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/45.jpg)
Facets
examples
Thursday, May 24, 12
![Page 46: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/46.jpg)
Terms Facets
Thursday, May 24, 12
![Page 47: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/47.jpg)
Terms Facets
Thursday, May 24, 12
![Page 48: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/48.jpg)
Date Histogram
Thursday, May 24, 12
![Page 49: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/49.jpg)
Date Histogram
Thursday, May 24, 12
![Page 50: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/50.jpg)
More Available Facets
• Histogram
• Statistical
• Terms Stats
• Range
• Geo Distance
• Filter
Thursday, May 24, 12
![Page 51: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/51.jpg)
Other Features• Pagination & Scrolling
• Sorting
• Highlighting
• Script Fields
• Realtime GET
• Multiple search types
• Min score filtering
• Named filters
• And much more...
Thursday, May 24, 12
![Page 52: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/52.jpg)
Management API• Indices
• Create & Delete
• Topology
• Update Settings
• Mapping
• Put & Delete
• Aliases & “Views”
• Refresh, Flush, Optimize
• Cluster
• Node shutdown
• Update Settings
Thursday, May 24, 12
![Page 53: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/53.jpg)
Monitoring API• Index Level
• State
• Stats
• Segments Info (Low level Lucene)
• Cluster Level
• Health
• State
• Nodes stats
Thursday, May 24, 12
![Page 54: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/54.jpg)
Distribution Model
Thursday, May 24, 12
![Page 55: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/55.jpg)
index - shards and replicas
Node Node
Client
curl -XPUT localhost:9200/test -d '{ "index" : { "number_of_shards" : 2, "number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 56: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/56.jpg)
index - shards and replicas
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test -d '{ "index" : { "number_of_shards" : 2, "number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 57: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/57.jpg)
indexing - 1
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test/type1/1 -d '{ "name" : { "first" : "Shay", "last" : "Banon" } , "title" : "ElasticSearch - A distributed search engine"}'
• Automatic sharding, push replication
Thursday, May 24, 12
![Page 58: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/58.jpg)
indexing - 2
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test/type1/2 -d '{ "name" : { "first" : "Shay", "last" : "Banon" } , "title" : "ElasticSearch - A distributed search engine"}'
• Automatic request “redirection”
Thursday, May 24, 12
![Page 59: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/59.jpg)
search - 1
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test/_search?q=test
• Scatter / Gather search
Thursday, May 24, 12
![Page 60: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/60.jpg)
search - 2
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test/_search?q=test
• Automatic balancing between replicas
Thursday, May 24, 12
![Page 61: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/61.jpg)
search - 3
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 0(replica)
Shard 1(primary)
Client
curl -XPUT localhost:9200/test/_search?q=test
failure
• Automatic failover
Thursday, May 24, 12
![Page 62: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/62.jpg)
adding a node
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 1(primary)
Shard 0(replica)
• “Hot” relocation of shards to the new node
Thursday, May 24, 12
![Page 63: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/63.jpg)
adding a node
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 1(primary)
Node
Shard 0(replica)
• “Hot” relocation of shards to the new node
Thursday, May 24, 12
![Page 64: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/64.jpg)
adding a node
Node
Shard 0(primary)
Shard 1(replica)
Node
Shard 1(primary)
Node
Shard 0(replica)
• “Hot” relocation of shards to the new node
Shard 0(replica)
Thursday, May 24, 12
![Page 65: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/65.jpg)
node failure
Node
Shard 1(primary)
Node
Shard 0(replica)
Node
Shard 0(primary)
Shard 1(replica)
Thursday, May 24, 12
![Page 66: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/66.jpg)
node failure - 1
Node
Shard 1(primary)
Node
Shard 0(primary)
• Replicas can automatically become primaries
Thursday, May 24, 12
![Page 67: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/67.jpg)
node failure - 2
Node
Shard 1(primary)
Node
Shard 0(primary)
• Shards are automatically assigned, and do “hot” recovery
Shard 0(replica)
Shard 1(replica)
Thursday, May 24, 12
![Page 68: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/68.jpg)
dynamic replicas
Node
Shard 0(primary)
Node
Shard 0(replica)
Client
curl -XPUT localhost:9200/test -d '{ "index" : { "number_of_shards" : 1,
"number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 69: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/69.jpg)
dynamic replicas
Node
Shard 0(primary)
Node Node
Shard 0(replica)
Client
Thursday, May 24, 12
![Page 70: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/70.jpg)
dynamic replicas
Node
Shard 0(primary)
Node Node
Shard 0(replica)
Client
Shard 0(replica)
curl -XPUT localhost:9200/test/_settings -d '{ "index" : {
"number_of_replicas" : 2 }}'
Thursday, May 24, 12
![Page 71: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/71.jpg)
transaction log• Indexed / deleted doc is fully persistent
• No need for a Lucene IndexWriter#commit
• Managed using a transaction log / WAL
• Full single node durability (kill dash 9)
• Utilized when doing hot relocation of shards
• Periodically “flushed” (calling IW#commit)
Thursday, May 24, 12
![Page 72: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/72.jpg)
Multi Tenancy
Thursday, May 24, 12
![Page 73: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/73.jpg)
multi tenancy -indices
Node Node Node
Client
curl -XPUT localhost:9200/test1 -d '{ "index" : { "number_of_shards" : 1, "number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 74: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/74.jpg)
multi tenancy -indices
Node
test1 S0(primary)
Node Node
test1 S0(replica)
Client
curl -XPUT localhost:9200/test1 -d '{ "index" : { "number_of_shards" : 1, "number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 75: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/75.jpg)
multi tenancy -indices
Node
test1 S0(primary)
Node Node
test1 S0(replica)
Client
curl -XPUT localhost:9200/test2 -d '{ "index" : { "number_of_shards" : 2, "number_of_replicas" : 1 }}'
Thursday, May 24, 12
![Page 76: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/76.jpg)
multi tenancy -indices
Node
test1 S0(primary)
Node Node
test1 S0(replica)
Client
curl -XPUT localhost:9200/test2 -d '{ "index" : { "number_of_shards" : 2, "number_of_replicas" : 1 }}'
test2 S0(replica)
test2 S1(primary)
test2 S1(replica)
test2 S0(primary)
Thursday, May 24, 12
![Page 77: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/77.jpg)
multi tenancy - indices
• Search against specific index
• curl localhost:9200/test1/_search
• Search against several indices
• curl localhost:9200/test1,test2/_search
• Search across all indices
• curl localhost:9200/_search
• Can be simplified using aliasesThursday, May 24, 12
![Page 78: Thursday, May 24, 12 · Query DSL • Programming language friendly • Tool friendly • Self explanatory • Fully supports all Lucene search constructs • All Lucene query types](https://reader034.fdocuments.in/reader034/viewer/2022043006/5f910571a7566f78251c4975/html5/thumbnails/78.jpg)
Applications• Unstructured search functionality
• typical free text query (text analysis)
• Structured search functionality
• Query DSL (mainly Filters)
• Data Aggregation & Analytics
• Facets (stats, histograms)
• Alerts
• Percolation
Thursday, May 24, 12