EzDevInfo.com

Java HTTP Client for ElasticSearch

I'm trying to connect from Java to ElasticSearch but I can only connect over HTTP. I can't use the TransportClient. Is there a Java client wrapper around the ElasticSearch REST APIs? If so, how do I use it?

How to get a list of indices from elasticsearch using jest

I'm trying to retrieve a list of indices using Jest, but I just got as far as:

        Stats statistics = new Stats.Builder().build();
        result = client.execute(statistics);

How can i retrieve the list of indices from the result? Do I have to use something else than Stats? It would also help if someone could show me a detailed documentation of Jest. The basics are really well documented, but with the different kinds of builders I'm really lost at the moment.

how to get type names of elastic search index in internal java api or jest api

I have a index with the name of demo and It contains different types. I'm using elastic search java internal api and rest api jest both of them in my app. Basicly I want to make this request

curl -XGET 'http:localhost:9200/demo/_mapping'

Is there any way to do that especially in jest api? There is no documentation to get mapping for rest client api. What is your suggestion ?

Completion suggester builder for Jest

Is there a built in solution in jest for sending completion suggester requests to elasticsearch?

Or do I need to implement my own io.searchbox.AbstractAction for that?

Switching between the Nodes in ElasticSearch using JEST client

I have 3 Nodes in my ElasticSearch Cluster. eg. 10.10.0.1, 10.10.0.2, 10.10.0.3

Now i am trying to connect to the Cluster using Jest Client API and giving the IP(10.10.0.1) of one of the above nodes. Now if the Node(10.10.0.1) is down, so does that mean that i would not able to acess the custer ?

How to handle this kind of problem where we have multiple nodes and we try to connect to the culster from the code using one IP address ?

Or

Should I go and create other Connection giving the other IP address ? which I feel is very Cude way of doing it because, what if we have 100 node, In that case we would end up trying to connect to 100 nodes one after other.

Need urgent assistance.

Thanks in advance, MIK

Not able to Query alphanumeric fields from ELASTIC SEARCH using TERMS QUERY

I am trying to query Alphanumeric values from the index using TERMS QUERY, But it is not giving me the output.

Query:

{
  "size" : 10000,
  "query" : {
    "bool" : {
      "must" : {
        "terms" : {
          "caid" : [ "A100945","A100896" ]
        }
      }
    }
  },
  "fields" : [ "acco", "bOS", "aid", "TTl", "caid" ]
}

I want to get all the entries that has caid A100945 or A100896

The same query works fine for NUmeric fields.

I am not planning to use QueryString/MatchQuery as i am trying to build general query builder that can build query for all the request. Hence am looking to get the entries usinng TERMS Query only.

Note: I am using Java API org.elasticsearch.index.query.QueryBuilders for building the Query.

eg: QueryBuilders.termQuery("caid", "["A10xxx", "A101xxx"]")

Please help.

Regards, Mik

OR and AND Operators in Elasticsearch query

I have few json document with the following format :-

    _source: {
            userId: "A1A1",
            customerId: "C1",
            component: "comp_1",
            timestamp: 1408986553,
     }

I want to query the document based on the following :-

(( userId == currentUserId) OR ( customerId== currentCustomerId) OR (currentRole ==ADMIN) )  AND component= currentComponent)

I tried using the SearchSourceBuilder and QueryBuilders.matchQuery, but I wasnt able to put multiple sub queries with AND and OR operators.

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchQuery("userId",userId)).sort("timestamp", SortOrder.DESC).size(count);

How we query elasticsearch using OR and AND operators?

Service using Jest is blocking on threadpool, why?

I've a Java + Spring app that will query Elasticsearch using Jest client (poor choice because it is poorly documented). Elasticsearch has response times of about 8-20 ms with 150 concurrent connections, but my app goes up to 900 -1500 ms. A quick look at VisualVM tells me that the processor usage is below 10% and profiling it tells me that 98% of the time all that the app does is wait on the following method

org.apache.http.pool.PoolEntryFuture.await()

that is part of Apache HttpCore and a dependency of Jest. I don't have a limitation in terms of threads that can run on tomcat (max is 200 and VisualVM says that the maximum number of thread during the experiment was 174). So it's not waiting free threads.

I think that the latency increase is excessive and I suspect that Jest is using an internal threadpool that has not enough threads to cope with all the requests... but I don't know.

Thoughts?

Elasticsearch Jest update a whole document

I have an elasticsearch server which i'm accessing via a java server using the Jest client and i was looking for the best way to update multiple fields of a document each time.

I have looked to the documentation so far, and i have found that there are two way for doing it :

1. Partial update via a script : i don't think it is suitable for multiple field update (because i don't know the modified fields).
2. Whole document update: via re-indexing the whole document.

My question is how could i update the whole document knowing that Jest provide only update via a script?

Is it the best way to delete a document and indexing the updated version?

Thanks in advance,

Is it possible to perform an ElasticSearch upsert with Jest

After looking through the documentation, I am still unsure - If an upsert can be performed against elasticsearch with the Jest java client library?

Implement spellcheck in elasticsearch with the help of jestclient in spring mvc

I would like to implement an elasticsearch application with jestclient using springmvc framework in that i wish to search the content with spell check option like google is there any possibility to implement it. Is possible then tell me how to achive it.

How can we use the ElasticSearchJestClient to send a gzip compressed POST request to Elastic Search

We are trying to send many 10s of MB worth of Base64 encoded documents to Elastic Search over a (sometimes) fairly saturated network.

Is it possible to get the Java Elastic Search JestHttpClient to gzip compress the POST request body? It looks like Elastic Search can cope with compressed requests.

Troubles creating an index with custom analyzer using Jest

Jest provides a brilliant async API for elasticsearch, we find it very usefull. However, sometimes it turns out that resulting requests are slightly different than what we would expect.

Usually we didn't care, since everything was working fine, but in this case it was not.

I want to create an index with a custom ngram analyzer. When I do this following the elasticsearch rest API docs, I call below:

curl -XPUT 'localhost:9200/test' --data '
{
  "settings": {
    "number_of_shards": 3,
    "analysis": {
      "filter": {
        "keyword_search": {
          "type":     "edge_ngram",
          "min_gram": 3,
          "max_gram": 15
        }
      },
      "analyzer": {
        "keyword": {
          "type":      "custom",
          "tokenizer": "whitespace",
          "filter": [
            "lowercase",
            "keyword_search"
          ]
        }
      }
    }
  }
}'

and then I confirm the analyzer is configured properly using:

curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens

in response I receive multiple tokens like exp, expe, expec and so on.

Now using Jest client I put the config json to a file on my classpath, the content is exactly the same as the body of the PUT request above. I execute the Jest action constructed like this:

new CreateIndex.Builder(name)
            .settings(
                    ImmutableSettings.builder()
                            .loadFromClasspath(
                                    "settings.json"
                            ).build().getAsMap()
            ).build();

In result

• Primo - checked with tcpdump that what's actually posted to elasticsearch is (pretty printed):

  {
    "settings.analysis.filter.keyword_search.max_gram": "15",
    "settings.analysis.filter.keyword_search.min_gram": "3",
    "settings.analysis.analyzer.keyword.tokenizer": "whitespace",
    "settings.analysis.filter.keyword_search.type": "edge_ngram",
    "settings.number_of_shards": "3",
    "settings.analysis.analyzer.keyword.filter.0": "lowercase",
    "settings.analysis.analyzer.keyword.filter.1": "keyword_search",
    "settings.analysis.analyzer.keyword.type": "custom"
  }
  

• Secundo - the resulting index settings is:

  {
    "test": {
      "settings": {
        "index": {
          "settings": {
            "analysis": {
              "filter": {
                "keyword_search": {
                  "type": "edge_ngram",
                  "min_gram": "3",
                  "max_gram": "15"
                }
              },
              "analyzer": {
                "keyword": {
                  "filter": [
                    "lowercase",
                    "keyword_search"
                  ],
                  "type": "custom",
                  "tokenizer": "whitespace"
                }
              }
            },
            "number_of_shards": "3"   <-- the only difference from the one created with rest call
          },
          "number_of_shards": "3",
          "number_of_replicas": "0",
          "version": {"created": "1030499"},
          "uuid": "Glqf6FMuTWG5EH2jarVRWA"
        }
      }
    }
  }
  

• Tertio - checking the analyzer with curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens I get just one token!

Question 1. What is the reason that Jest does not post my original settings json, but some processed one instead?

Question 2. Why the settings generated by Jest are not working?

Calling CreateIndex with settings without using ImmutableSettings

In the Jest readme there is an example of creating index with settings but it uses the ImmutableSettings class from ElasticSearch library. Unfortunately due to some conflicts, we can't include ElasticSearch library in our project. Is there a way to use CreateIndex.Builder to create index with settings without using ImmutableSettings class from ElasticSearch library?

String settings = "\"settings\" : {\n" +
            "        \"number_of_shards\" : 5,\n" +
            "        \"number_of_replicas\" : 1\n" +
            "    }\n";
client.execute(new CreateIndex.Builder("articles").settings(ImmutableSettings.builder().loadFromSource(settings).build().getAsMap()).build());

Thanks, Len

maxTotalConnection and defaultMaxTotalConnectionPerRoute

In HttpClientConfig, if we configure Builder with 3 servers and we don't define any explicit routes, and we want to have maximum of 20 connections per server, we should set maxTotalConnection to be 20, i.e does a pool get created for each server? Also, do we need to explicitly set defaultMaxTotalConnectionPerRoute even though we are not defining any routes explicitly, i.e. we should also set defaultMaxTotalConnectionPerRoute to 20?

Regards, LT