elastic4s
Non blocking, type safe DSL and Scala client for Elasticsearch
I'd need to disable index refreshing for the course of a batch indexing (of gigabytes) and set it back again after it's done. But from the source code of elastic4s I can't find a way to do it other than at index creation time... Is it possible? Or is there a workaround for this?
In java client :
client
.admin
.indices()
.prepareUpdateSettings()
.setSettings(settings)
.setIndices(indexName)
.execute()
.actionGet()
Natively :
curl -XPUT 'localhost:9200/my_index/_settings' -d '
{
"index" : {
"refresh_interval" : -1
}
}
'
Source: (StackOverflow)
This is a small code I am using to do a simple search:
import com.sksamuel.elastic4s.{ElasticsearchClientUri, ElasticClient}
import com.sksamuel.elastic4s.ElasticDsl._
import org.elasticsearch.common.settings.ImmutableSettings
object Main3 extends App {
val uri = ElasticsearchClientUri("elasticsearch://localhost:9300")
val settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").build()
val client = ElasticClient.remote(settings, uri)
if (client.exists("bands").await.isExists()) {
println("Index already exists!")
val num = readLine("Want to delete the index? ")
if (num == "y") {
client.execute {deleteIndex("bands")}.await
} else {
println("Leaving this here ...")
}
} else {
println("Creating the index!")
client.execute(create index "bands").await
client.execute(index into "bands/artists" fields "name"->"coldplay").await
val resp = client.execute(search in "bands/artists" query "coldplay").await
println(resp)
}
client.close()
}
This is result that I get:
Connected to the target VM, address: '127.0.0.1:51872', transport: 'socket'
log4j:WARN No appenders could be found for logger (org.elasticsearch.plugins).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Creating the index!
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
Disconnected from the target VM, address: '127.0.0.1:51872', transport: 'socket'
Process finished with exit code 0
Creation of an index and adding document to this index is running fine but a simple search query is giving no result. I even checked this on Sense.
GET bands/artists/_search
{
"query": {
"match": {
"name": "coldplay"
}
}
}
gives
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "bands",
"_type": "artists",
"_id": "AU21OYO9w-qZq8hmdTOl",
"_score": 0.30685282,
"_source": {
"name": "coldplay"
}
}
]
}
}
How to solve this issue?
Source: (StackOverflow)
I am using elastic4s 1.5.10 and trying to build up a query that I prepared on elasticsarch REST endpoint. Now trying to rewrite to elastic4s dsl.
POST /index_name/type/_search
{
"_source":{"include":["name","surname"]},
"query": {
"bool": {
"must": [
{
"more_like_this": {
"fields": ["desc"],
"ids": ["472825948"],
"min_term_freq":0,
"max_query_terms":200
}
},
{
"match": {"city": "London"}
},
{
"match": {"operation": "add"}
}
]
}
}
}
The goal of this query is get similar items to 472825948 which has the same operation (add) in the same city (London).
My attempt in elastic4s follows:
es_client.execute{
search in s"$storage_folder/${typ.name()}" query bool(
must(
morelike id adId in s"$storage_folder/${typ.name()}" fields("desc") minTermFreq(0) maxQueryTerms(200),
matchQuery(field="city",value = "London"),
matchQuery(field="operation",value = "add"))
)sourceInclude("name","surname")
}
}
"morelike" doesn't work in this context. Either the query as it is in raw json doesn't make sense or elastic4s doesn't support this or ...
Could someone help?
Thx
Source: (StackOverflow)
I want create index with dynamic template and turn analyzing off for string fields. I have created query for elastic search, but how to translate it into elastic4s statments? (version elastic4s 1.3.x is preffered)
The statement is:
PUT /myIndex
{
"mappings": {
"myType": {
"dynamic_templates": [
{
"templateName": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"index" : "not_analyzed",
"omit_norms" : true
}
}
}
]
}}}
P.S.
May be it is possible to create this index by executing this "raw" request, but I did not find how to do that with elastic4s 1.3.4 :(
Source: (StackOverflow)
I'm using ElasticSearch to index some of my models but I see that only one field updated gets indexed ;
I first create a mapping like;
client execute {
create index "places" mappings(
"shop" as (
"location" typed GeoPointType,
"available" typed BooleanType,
"posted" typed DateType,
"updated" typed DateType
)
)
}
And then, in the method Shop.save, I do the following;
posted = new Date
updated = new Date
super.save
// index in ES
client execute {
index into "places" -> "shop" id id fields {
"location" -> GeoPoint.parseFromLatLon(lat.toString + "," + lon.toString)
"available" -> true
"posted" -> posted // field in the object
"updated" -> updated // field in the object
}
}
But then, when I go to host:9200/places/shop/1, I only see:
{
_index: "places",
_type: "shop",
_id: "1",
_version: 1,
found: true,
_source: {
updated: "2014-09-11T13:52:40.072Z"
}
}
What am I doing wrong?
Edit
I'm using: elastic4s 1.3.2
elasticsearch 1.3.2
and Scala with the Play Framework (2.3.4)
Source: (StackOverflow)
I'm using elastic4s in my scala project to communicate with ElasticSearch. For development I started a local node and everything is working fine. For production, I would like to use Amazon Elasticsearch Service. I've configured that service and allowed access to it via the ip of my ec2 instance. I can verify that it works by ssh-ing into ec2 and doing:
curl search-blabla-blabla.us-east-1.es.amazonaws.com/_cluster/health
However, I am having trouble connecting elastic4s to that ES instance. I'm trying:
ElasticClient.remote("search-blabla-blabla.us-east-1.es.amazonaws.com", 9300)
which results in:
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
Reading the docs, it appears that elastic4s can only connect via TCP [1] and Amazon Elasticsearch Service does not support TCP[2]: The service supports HTTP on port 80, but does not support TCP transport.
Could someone confirm that elastic4s and Amazon ES really don't work together? Because that would mean I have to re-write all of my ES code.
Source: (StackOverflow)
I want to implement the following query in Elastic4s. Don't see any way to implement the matched_fields clause in the highlighter. Any help?
{
"query": {
"multi_match": {
"type": "most_fields",
"query": "hello world",
"fields": [ "text", "text.human" ]
}
},
"highlight": {
"order": "score",
"fields": {
"text": {
"matched_fields": [ "text", "text.human" ],
"fragment_size": 100,
"number_of_fragments": 10
}
}
}
}
Source: (StackOverflow)
how I can create the query with client elastic4s scala?
I call using marvel/sense
GET /business/_search
{
"query": {
"function_score": {
"query": {
"match": {
"name": "my text"
}
},
"script_score": {
"script": "_score + log(doc['reviews'].value + 1 )",
"lang": "groovy"
}
}
},
"facets": {
"industry": {
"terms": {
"fields": ["type", "industry"]
}
}
},
"size": 10
}
But how I can create the query with elastic4s?
Source: (StackOverflow)
I tried to run an example code of elastic4s, as follows,
import com.sksamuel.elastic4s.ElasticClient
import com.sksamuel.elastic4s.ElasticDsl._
object hw extends App {
val client = ElasticClient.local
client.execute(create index "bands")
client.execute { index into "bands/artists" fields "name"->"coldplay" }.await
val resp = client.execute { search in "bands/artists" query "coldplay" }.await
println(resp)
client.close
}
The program correctly prints the results, but it does not exit itself. I don't know if there are problems with my code or environment.
Source: (StackOverflow)
I'm trying to implement a service in my play2 app that uses elastic4s to get a document by Id.
My document in elasticsearch:
curl -XGET 'http://localhost:9200/test/venues/3659653'
{
"_index": "test",
"_type": "venues",
"_id": "3659653",
"_version": 1,
"found": true,
"_source": {
"id": 3659653,
"name": "Salong Anna och Jag",
"description": "",
"telephoneNumber": "0811111",
"postalCode": "16440",
"streetAddress": "Kistagången 12",
"city": "Kista",
"lastReview": null,
"location": {
"lat": 59.4045675,
"lon": 17.9502138
},
"pictures": [],
"employees": [],
"reviews": [],
"strongTags": [
"skönhet ",
"skönhet ",
"skönhetssalong"
],
"weakTags": [
"Frisörsalong",
"Frisörer"
],
"reviewCount": 0,
"averageGrade": 0,
"roundedGrade": 0,
"recoScore": 0
}
}
My Service:
@Singleton
class VenueSearchService extends ElasticSearchService[IndexableVenue] {
/**
* Elastic search conf
*/
override def path = "test/venues"
def getVenue(companyId: String) = {
val resp = client.execute(
get id companyId from path
).map { response =>
// transform response to IndexableVenue
response
}
resp
}
If I use getFields() on the response object I get an empty object. But if I call response.getSourceAsString I get the document as json:
{
"id": 3659653,
"name": "Salong Anna och Jag ",
"description": "",
"telephoneNumber": "0811111",
"postalCode": "16440",
"streetAddress": "Kistagången 12",
"city": "Kista",
"lastReview": null,
"location": {
"lat": 59.4045675,
"lon": 17.9502138
},
"pictures": [],
"employees": [],
"reviews": [],
"strongTags": [
"skönhet ",
"skönhet ",
"skönhetssalong"
],
"weakTags": [
"Frisörsalong",
"Frisörer"
],
"reviewCount": 0,
"averageGrade": 0,
"roundedGrade": 0,
"recoScore": 0
}
As you can se the get request omits info:
"_index": "test",
"_type": "venues",
"_id": "3659653",
"_version": 1,
"found": true,
"_source": {}
If I try to do a regular search:
def getVenue(companyId: String) = {
val resp = client.execute(
search in "test"->"venues" query s"id:${companyId}"
//get id companyId from path
).map { response =>
Logger.info("response: "+response.toString)
}
resp
}
I get:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test",
"_type": "venues",
"_id": "3659653",
"_score": 1,
"_source": {
"id": 3659653,
"name": "Salong Anna och Jag ",
"description": "",
"telephoneNumber": "0811111",
"postalCode": "16440",
"streetAddress": "Kistagången 12",
"city": "Kista",
"lastReview": null,
"location": {
"lat": 59.4045675,
"lon": 17.9502138
},
"pictures": [],
"employees": [],
"reviews": [],
"strongTags": [
"skönhet ",
"skönhet ",
"skönhetssalong"
],
"weakTags": [
"Frisörsalong",
"Frisörer"
],
"reviewCount": 0,
"averageGrade": 0,
"roundedGrade": 0,
"recoScore": 0
}
}
]
}
}
My Index Service:
trait ElasticIndexService [T <: ElasticDocument] {
val clientProvider: ElasticClientProvider
def path: String
def indexInto[T](document: T, id: String)(implicit writes: Writes[T]) : Future[IndexResponse] = {
Logger.debug(s"indexing into $path document: $document")
clientProvider.getClient.execute {
index into path doc JsonSource(document) id id
}
}
}
case class JsonSource[T](document: T)(implicit writes: Writes[T]) extends DocumentSource {
def json: String = {
val js = Json.toJson(document)
Json.stringify(js)
}
}
and indexing:
@Singleton
class VenueIndexService @Inject()(
stuff...) extends ElasticIndexService[IndexableVenue] {
def indexVenue(indexableVenue: IndexableVenue) = {
indexInto(indexableVenue, s"${indexableVenue.id.get}")
}
- Why is getFields empty when doing get?
- Why is query info left out when doing getSourceAsString in a get request?
Thank you!
Source: (StackOverflow)
I am using elastic4s to implement elastic search. I am trying to enable ttl but I couldn't figure it out how ? I mean yes _ttl enabled is going to be true but within the code where and how should I be implementing it ? I am using the latest version 1.3.2 for elastic4s
Source: (StackOverflow)
After using Scala for a while and reading about all over the place and especially here
I was sure I know when to use curlies. as a rule of thumb if I want to pass a block of code to be executed i will use curly braces.
how ever this nasty bug surfaced using elastic4s DSL
using curly braces:
bool {
should {
matchQuery("title", title)
}
must {
termQuery("tags", category)
}
}
compiles to:
{
"bool" : {
"must" : {
"term" : {
"tags" : "tech"
}
}
}
}
while using parenthesis:
bool {
should (
matchQuery("title", title)
) must (
termQuery("tags", category)
)
}
gives the correct result:
{
"bool" : {
"must" : {
"term" : {
"tags" : "tech"
}
},
"should" : {
"match" : {
"title" : {
"query" : "fake",
"type" : "boolean"
}
}
}
}
}
This was compiled using scala 2.11.6 - Even more confusing is that evaluating the expression in intellij debugger gives correct result no matter what I am using.
I noticed only the last expression was being evaluated why is that?
Source: (StackOverflow)
i'm trying to index some data to elastic search by using the
elastic4s API and play framework
I'm basically calling this method from the controller
def test(){
val client = ElasticClient.local
client.execute { create index "bands" }
client execute { index into "bands/singers" fields "name"->"chris martin" }
client.close()
}
I didn't get any error on play or in the elastic search log,
Then i checked with the Sense plugin if the data is indexed and i got
{
"error": "IndexMissingException[[bands] missing]",
"status": 404
}
It looks like the query didn't go to the server?? ...
Source: (StackOverflow)