Java clients for elasticsearch
-
Upload
florian-hopf -
Category
Technology
-
view
1.665 -
download
0
Transcript of Java clients for elasticsearch
Java Clients for
Florian Hopf@fhopf
elasticsearch
Elasticsearch?
„Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
Elasticsearch?
„Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
Elasticsearch?
Installation
# download archivewget https://artifacts.elastic.co/downloads/
elasticsearch/elasticsearch-5.0.0.zip
unzip elasticsearch-5.0.0.zip
# on windows: elasticsearch.batelasticsearch-5.0.0/bin/elasticsearch
Accessing Elaticsearch
curl -XGET "http://localhost:9200"
{ "name" : "LI8ZN-t", "cluster_name" : "elasticsearch", "cluster_uuid" : "UvbMAoJ8TieUqugCGw7Xrw", "version" : { "number" : "5.0.0", "build_hash" : "253032b", "build_date" : "2016-10-26T04:37:51.531Z", "build_snapshot" : false, "lucene_version" : "6.2.0" }, "tagline" : "You Know, for Search"}
Indexing
curl -XPOST "http://localhost:9200/food/dish" -d'{ "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 }}'
Indexing
curl -XPOST "http://localhost:9200/food/dish" -d'{ "food": "Ayam Penyet", "tags": ["chicken", "indonesian"], "spicy": true}'
Search
curl -XGET "http://localhost:9200/food/dish/_search?q=chicken"...{"total":2,"max_score":0.3666863,"hits":[{"_index":"food","_type":"dish","_id":"AVg9cMwARrBlrY9tYBqX","_score":0.3666863,"_source":{ "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 }}},...
Search using Query DSL
curl -XPOST "http://localhost:9200/food/dish/_search" -d'{ "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } }}'
Elasticsearch?
„Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
Distributed
Distributed
Recap
● Java based search server● HTTP and JSON● Search and Filtering Query-DSL● Faceting, Highlighting, Suggestions, …● Nodes can form a cluster
Transport-Client
Transport-Client
dependencies { compile group: 'org.elasticsearch.client', name: 'transport', version: '5.0.0'}
Transport-Client
TransportAddress address = new InetSocketTransportAddress(
InetAddress.getByName("localhost"), 9300);
Client client = new PreBuiltTransportClient(Settings.EMPTY) addTransportAddress(address);
Search using Query DSL
curl -XPOST "http://localhost:9200/food/dish/_search" -d'{ "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } }}'
Search using Search Builders
SearchResponse searchResponse = client.prepareSearch("food")
.setQuery(boolQuery().
must(matchQuery("_all", "rice")). filter(
termQuery("tags.keyword", "chicken"))) .execute().actionGet();
assertEquals(1, searchResponse.getHits().getTotalHits());
SearchHit hit = searchResponse.getHits().getAt(0);String food = hit.getSource().get("food").toString();
Indexing
XContentBuilder builder = jsonBuilder() .startObject() .field("food", "Roti Prata") .array("tags", new String [] {"curry"}) .startObject("favorite") .field("location", "Tiong Bahru") .field("price", 2.00) .endObject() .endObject();
IndexResponse resp = client.prepareIndex("food","dish") .setSource(builder) .execute() .actionGet();
Indexing
Transport-Client
● Connects to an existing cluster● Uses the binary protocol also used for inter
cluster communication
Transport-Client
Sniffing
Sniffing
● State of the cluster can be requested from elasticsearch
● Client side loadbalancing
TransportAddress address = new InetSocketTransportAddress(
InetAddress.getByName("localhost"), 9300);
Settings settings = Settings.builder().put("client.transport.sniff", true).build();
Client client = new PreBuiltTransportClient(settings) addTransportAddress(address);
Transport-Client
● Full API-Support● Efficient communication● Client side loadbalancing
Transport-Client Gotchas
● Compatibility between elasticsearch versions● Elasticsearch dependency
REST-Client
Elasticsearch 5 REST-Client
Elasticsearch 5 REST-Client
dependencies { compile group: 'org.elasticsearch.client', name: 'rest', version: '5.0.0'}
Elasticsearch 5 REST-Client
+--- org.apache.httpcomponents:httpclient:4.5.2+--- org.apache.httpcomponents:httpcore:4.4.5+--- org.apache.httpcomponents:httpasyncclient:4.1.2+--- org.apache.httpcomponents:httpcore-nio:4.4.5+--- commons-codec:commons-codec:1.10\--- commons-logging:commons-logging:1.1.3
Elasticsearch 5 REST-Client
RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200, "http"), new HttpHost("localhost", 9201, "http")).build();
Elasticsearch 5 REST-Client
HttpEntity entity = new NStringEntity( "{ \"query\": { \"match_all\": {}}}", ContentType.APPLICATION_JSON); // alternative: performRequestAsync Response response = restClient.performRequest("POST",
"/_search", emptyMap(), entity); String json = toString(response.getEntity()); // ...
Elasticsearch 5 REST-Client
● Less dependent on elasticsearch version ● Clean separation network application/cluster● Minimal dependency● Sniffing● Error handling, timeout config, basic auth,
headers, …● No query support (for now)
Jest – Http Client
Jest
dependencies { compile group: 'io.searchbox', name: 'jest', version: '2.0.0'}
Client
JestClientFactory factory = new JestClientFactory();factory.setHttpClientConfig(new HttpClientConfig .Builder("http://localhost:9200") .multiThreaded(true) .build());
JestClient client = factory.getObject();
Searching Jest
String query = jsonStringThatMagicallyAppears;
Search search = new Search.Builder(query) .addIndex("library") .build();
SearchResult result = client.execute(search);assertEquals(Integer.valueOf(1), result.getTotal());
Searching Jest
JsonObject jsonObject = result.getJsonObject();JsonObject hitsObj = jsonObject.getAsJsonObject("hits");JsonArray hits = hitsObj.getAsJsonArray("hits");JsonObject hit = hits.get(0).getAsJsonObject();
// ... more boring code
Searching Jest
public class Dish {
private String food;private List<String> tags;private Favorite favorite;
@JestIdprivate String id;
// ... getters and setters}
Suche mit Jest
Dish dish = result.getFirstHit(Dish.class).source;
assertEquals("Roti Prata", dish.getFood());
Jest
● Alternative HTTP implementation● Queries as Strings or via Elasticsearch-Builder● Indexing and searching Java beans● Node Discovery
Spring Data Elasticsearch
Spring Data
● Abstractions for different data stores● Speciality of each store available● Dynamic Repository implementations● Popular modules
● Spring Data JPA● Spring Data MongoDB
Dependency
dependencies {compile group: 'org.springframework.data',
name: 'spring-data-elasticsearch', version: '2.0.4.RELEASE'}
Entity
@Document(indexName = "spring_dish")public class Dish {
@Idprivate String id;private String food;private List<String> tags;private Favorite favorite;
// more code
}
Repository
public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> {
}
Configuration
<elasticsearch:transport-client id="client" />
<bean name="elasticsearchTemplate" class="o.s.d.elasticsearch.core.ElasticsearchTemplate"> <constructor-arg name="client" ref="client"/></bean>
<elasticsearch:repositories base-package="de.fhopf.elasticsearch.springdata" />
Indexing
Dish mie = new Dish();mie.setId("hokkien-prawn-mie");mie.setFood("Hokkien Prawn Mie");mie.setTags(Arrays.asList("noodles", "prawn"));
repository.save(Arrays.asList(hokkienPrawnMie));
Searching
Iterable<Dish> dishes = repository.findAll();
Dish dish = repository.findOne("hokkien-prawn-mie");
Repository
public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> {
List<Dish> findByFood(String food);
List<Dish> findByTagsAndFavoriteLocation(String tag, String location);
List<Dish> findByFavoritePriceLessThan(Double price);
@Query("{\"query\": {\"match_all\": {}}}")List<Dish> customFindAll();
}
Recap
● High level abstraction● Entity beans● dynamic repository implementation● HTTP support in the making● Slower feature development
Recap
● Transport-Client● Full API support● Elasticsearch dependency
● REST-Client● Uses HTTP● currently lacking search API
Recap
● Jest● Http client ● Support for Java beans
● Spring-Data-Elasticsearch● High level abstraction● Dynamic repositories
Links
● http://elastic.co● https://github.com/searchbox-io/Jest● https://github.com/spring-projects/spring-
data-elasticsearch● https://github.com/fhopf/singajug-examples● http://blog.florian-hopf.de