Questions tagged [elasticsearch]

13064 questions
1

votes
2

answer
741

Views

Loading Wikipedia Dump into Elasticsearch

I want to load a XML Wikipedia dump like: http://ftp.acc.umu.se/mirror/wikimedia.org/dumps/enwiki/20171001/enwiki-20171001-pages-articles.xml.bz2 into Elasticsearch (5.6.4). However, all tools and tutorials I found are outdated and not compatible with my Elasticsearch version. Can anyone explain wha...
Schroder22
1

votes
1

answer
503

Views

Understanding Decay function and its parameters in ElasticSearch

I was reading about decay functions in ElasticSarch in order to promote more recent results If I define decay function as following: 'DECAY_FUNCTION': { 'FIELD_NAME': { 'origin': '2013-09-17', 'scale': '10d', 'offset': '5d', 'decay' : 0.5 } } How the score will be affected within the Offset...
Simple Code
1

votes
3

answer
2.2k

Views

How to find all duplicate documents in ElasticSearch

We have a need to walk over all of the documents in our AWS ElasticSearch cluster, version 6.0, and gather a count of all the duplicate user ids. I have tried using a Data Visualization to aggregate counts on the user ids and export them, but the numbers don't match another source of our data that i...
user1469297
1

votes
1

answer
253

Views

How to get the search History in Elasticsearch?

There is any way to get the search history in Elasticsearch. In this I want to get the search queries which are searched by customers in last one month.
Tomas
1

votes
1

answer
664

Views

ElasticSearch: analyzer on field must be set when search_analyzer is set

I have read about previous version of ES (< 2) where the 'token_analyzer' key needs to be changed to 'analyzer'. But no matter what I do I am still getting this error: 'type': 'mapper_parsing_exception', 'reason': 'analyzer on field [email] must be set when search_analyzer is set' Here is what I am...
Nate Uni
1

votes
1

answer
459

Views

Elasticsearch how to support transaction involving multiple documents

i use elasticsearch,and Denormalizing Data,like PUT /my_index/user/1 { 'name': 'John Smith', 'email': '[email protected]', 'dob': '1970/10/24' } PUT /my_index/blogpost/2 { 'title': 'Relationships', 'body': 'It's complicated...', 'user': { 'id': 1, 'name': 'John Smith' } } but the problem is that Elasti...
Amitābha
1

votes
1

answer
23

Views

Should as filter context (OR without scoring in Elasticsarch possible?)

I am trying to setup most efficient way to build an OR without having a scoring, since I want to order my results by business values afterwards. Unfortunately i don't get it done. :( What I need: COLOR=X AND ( title = Y OR description = Z) What I tried (but it is malformed): { 'query': { 'bool': {...
Phil
1

votes
2

answer
29

Views

What is wrong with my Elastisearch query?

I need to make multi_match and bool query at once, but query below doesn't work :( When I use them separatley, they working perfect. { 'query': { 'multi_match': { 'query': 'kotlety*', 'fields': [ 'name' ] }, 'bool': { 'filter': { 'term': { 'status': 2 } } } }, 'size': 24 } response is: { 'error': {...
Paul
1

votes
2

answer
36

Views

How to secure elasticsearch API's

I wonder how people normally secure their elasticsearch API's, when being accessed from browser side? If we access elastic server from server side, even then a simple DELETE request is sufficient to delete a whole index, which might contain priceless information. I've read that they provide RBAC f...
Ramesh Pareek
0

votes
0

answer
4

Views

cakephp 3.X elasticsearch QueryBuider

Following the manual of the QueryBuilder from the ElasticSearch plugin on https://github.com/cakephp/elastic-search/blob/master/src/QueryBuilder.php , the search for entries containing 'milk' or 'cheese' works as expected using the 'in' operator. $this->loadModel('Archief', 'Elastic'); $query = $thi...
Joost
1

votes
1

answer
79

Views

Set filter in Kibana 7/6.7.2 for latest _id in DSL with size 1

I want the filter to be set on dashboard which will fetch only latest record So Used below query in DSL { 'query': { 'match_all': {} }, 'size': 1, 'sort': [ { '_id': { 'order': 'desc' } } ] } This is not working in Elasticsearch Kibana 7.0 and I have checked in 6.7.1 also , it is not working 'size'...
happy
0

votes
0

answer
5

Views

Elastic search index only searchable fields

I would like to create an index in which only some fields are indexed. I created a template with the enabled property set to false. So no field is indexed by default. https://www.elastic.co/guide/en/elasticsearch/reference/6.4/enabled.html Then I defined the fields I want to index with dynamic templ...
Sydney
0

votes
0

answer
6

Views

Get sorted results after aggregation in elastic search

I am getting aggregated result for nested object (Minimum price for each nested object) All of my search results are sorted and now I need minimum value for each result. Result: Response from aggregation is not sorted, as it is in my search result (min, some_field) Question: Is there any solution...
Marium Malik
1

votes
1

answer
794

Views

Elasticsearch: add a synonym filter on my english analyser

I've got an index with a field description which is analysed like that: 'description':{ 'analyzer' : 'english', 'type' : 'string' } I have defined an synonyms dictionnary in a file synonyms.txt which contain: ipod, i-pod, i pod => i-pod I would like to add this synonym dictionnary to my analyzer, bu...
mel
1

votes
2

answer
2k

Views

Elasticsearch: should + minimum_should_match vs must

I test with these 2 queries Query with must { 'size': 200, 'from': 0, 'query': { 'bool': { 'must': [ { 'match': { '_all': 'science' } }, { 'match': { 'category': 'fiction' } }, { 'match': { 'country': 'us' } } ] } } } Query with should + minimum_should_match { 'size': 200, 'from': 0, 'query': { 'boo...
plchia
1

votes
1

answer
964

Views

Partial update in ElasticSearch with JEST

I'm trying to use JEST to implement CRUD actions to ElasticSearch, and I've surveryed some articles, the method I found to do 'Update' is 'Index' another document with the same '_id', but I'll replace the whole document in that way. I want to know is JEST capable to do 'Partial Update'? Edit : I fou...
Alanight
1

votes
2

answer
1.4k

Views

Kibana and Docker-Compose: Automatically load in saved object

Goal: Load in Kibana saved object when starting Elastic Stack from Docker-Compose. I have an 'export.json' object from Kibana, containing my default index, 10+ visualizations, and dashboard setup. I am able to successfully start the Elastic Stack from Docker-Compose and manually load the object, but...
Alan
1

votes
2

answer
449

Views

What is the right way to snyc/import tables from a postgres DB to elasticsearch?

I want to import some tables from a postgres database into Elastic search and also hold the tables in sync with the data in elastic search. I have looked at a course on udemy, and also talked with a colleague who has a lot of experience with this issue to see what the best way to do it is. I am surp...
Dan
1

votes
1

answer
234

Views

What are benefits of running ELK stack on Docker over running it on VM

I'm learning ELK stack. I wonder, why would people run it on Docker? If I understand everything correctly, it would have to have some directory of a host OS mapped to be persistent over resets of the image. Meanwhile, running just VL with installed docker would be persistent anyway. Why should I use...
Kowalski Paweł
1

votes
1

answer
17

Views

Elastic Search bulk operation

I really can't understand why when I run a bulk insert I lost the previous data in the same collection without executing any delete operation? this is weird. any idea? var client = new elasticsearch.Client( { hosts: [ 'http://localhost:9200/' ] }) . . . InserTweets: function (arrayobj, callbac...
Gelso77
1

votes
2

answer
140

Views

ElasticSearch - Failed to parse content to map - Unexpected character ('\"'

Wondering if you can help, I'm having issues with my code in Elasticsearch that is giving out an error. My code is below and the output is also below, any help is greatly appreciated. curl -X PUT 'localhost:9200/_mapping/jdbc' -H 'Content-Type: application/x-ndjson' -d ' { 'mappings': { 'jdbc' :...
Mark Staples
1

votes
3

answer
32

Views

How can I send multiple documents to elastic with one API call

I'm new to Elastic and I need a way to push multiple documents to Elastic by only making one call to POST http://localhost:9200/myindex/mytype/. The body schema looks like this: { 'docs': [ { '_source': {'message':'message1'} }, { '_source': {'message':'message2'} } ] } I tried with the ingest...
Agus
2

votes
1

answer
60

Views

Elasticsearch: FVH highlights with multiple pre and post tags marking tokens incorrectly?

I'm querying my index using boolean query with two match terms. For each term I have a separate set of pre- and post- tags. Using highlights I would like to obtain the documents in which both terms exist and see which tokens were matched as each of them. The index contains documents in Polish analyz...
mrapacz
1

votes
0

answer
17

Views

how to sync data from mongodb to elastic search with python

I am currentlt using Monstache connector for sync mongodb elastic search. is there any other recommended connector for sync. Architecture: Angular7 Elastic Search Mongo DB Phython Pycharm. is there any other soultion apart from this. https://rwynn.github.io/monstache-site/.
Mohamed Sahir
1

votes
0

answer
175

Views

ElasticSearch max and min score

I want to search the fields and apply a function to the doc score that includes the max and min score of the search result. I can use the script score but how I get the max and min scores in script?
gg gg
1

votes
1

answer
253

Views

Run Elastic Search on pdf and ppts

I am new to elastic search. I have read its tutorials. But need guidance on my problem: I have a collection of pdf documents and power point files on my system. I need to build a system using elastic search where I can retrieve these files on the basis of keywords present in this file. Can someone p...
Astha Sachdev
1

votes
2

answer
1.2k

Views

How to monitor Elastic Stack without X-Pack?

Can we monitor the elastic stack 6.0 and above(like elastic search..) without using the X-Pack?As we know many of the Features like security, machine learning, graph APIs don't be supported under BASIC(free Licence). So I want to know if there are any APIs, without Licence limitation, can be used t...
Jason
1

votes
0

answer
296

Views

nested type mapping in elasticsearch with python

I am trying to map elasticsearch index, but I keep getting an error with nested type. request_body = { 'setting': { 'number_of_shards': 3, 'number_of_replicas': 1 }, 'mappings': { 'examplecase': { 'properties': { 'id': {'index': 'not_analyzed', 'type': 'string'}, 'source': {'index': 'analyzed', 'typ...
Extria
1

votes
0

answer
147

Views

Basicauth setup on Elasticsearch 6 dot release

I am trying to test few plugins related to user access for test user (custom access based doc filtering) on ES and I need to have basic auth to validate it. Is there a user manual for setting up basic auth setup at elasticsearch (6 dot release) at server end. Any information will be great help.
Ankur Saran
1

votes
0

answer
155

Views

how to paginate in Elastic Search Autocomplete (search as you type in index time)

I am facing a road-block in elastic search autocomplete feature. I am using custom analyser to a autocomplete field & it is working like a charm as expected . Now I want to paginate the result so that my client (angular app) does not need to deal with high volume of data . Precisely , user types 'MA...
spondon majumdar
1

votes
0

answer
41

Views

Offline time report using Kibana and ElasticSearch

Our devices are sending pings to ElasticSearch, every 5 minutes, in the following format: { deviceId: 34567, timestamp: 1514889787 } (We may alter the stored timestamp format if needed). If the device is offline, the ping is not send at all (ElasticSearch is storing only pings for devices being onli...
Sfisioza
1

votes
1

answer
302

Views

How do you convert a Django model with ManytoMany field to an elasticsearch_dsl DocType class?

I'm using elasticsearch_dsl to help with interfacing with elasticsearch and this is the model I currently have and I'm trying to recreate as a DocType: class HouseIndex(DocType): house_type = String() #people sold = Boolean() built_datetime = Date() #alerts # associated_locations hash = String() cla...
Stupid.Fat.Cat
1

votes
1

answer
189

Views

Elasticsearch 6.1 Java API search issue

I am using ES - 6.1 and corresponding java version of the api. I was able to index the data using the java API. I validated this via kibana. All the mappings were correctly saved but an error comes up when I try to search. Even the most basic search returns me an empty collection. // client is a ref...
ajit2210
1

votes
1

answer
800

Views

How to validate Elasticsearch Painless scripts?

We use many ScriptQuery and ScriptField in our project, and provide inline scripts for them in painless language. My question is, how can we validate these painless scripts? In other words, how to make sure that they will compile? The approach we use today is to test them out locally through Kibana...
Sunil Purushothaman
1

votes
2

answer
631

Views

get distinct values of string field using elasticsearch query Client java API

I'm using elasticsearch 5.4 I need to get distinct values of fields using elasticsearch query I'm trying this query but is not wokring GET /index/index_type/_search { 'size': 0, 'aggs': { 'distinct': { 'terms': { 'field': 'status' } } } thank you
YK mar
1

votes
1

answer
484

Views

ElasticSearch 6 High Level Rest Client preparePutMapping

I am trying to transition from ElasticSearch 2 to either 5 or 6. I think that I want to jump straight to 6.1.1 and use the RestHighLevelClient, since it is closer to the existing transport client that I am using than the low level rest client. However, I am running across a problem As part of my i...
C Dorman
1

votes
0

answer
119

Views

Error connecting to elasticsearch Java CLient API 5.4.3

I'm developping a web application using spring websphere and elasticsearch I want to conenct my java client api with elasticsearch but I got this error thanks for helping [05/01/18 19:54:31:660 WET] 00000054 SystemErr R ERROR StatusLogger No log4j2 configuration file found. Using default configu...
YK mar
1

votes
1

answer
489

Views

mongoosastic [mapper_parsing_exception] No handler for type [string] declared on field [type]

Here is the error { Error: [mapper_parsing_exception] No handler for type [string] declared on field [type] at respond (/home/jilu/Documents/project/backup/assignment_ok/node_modules/elasticsearch/src/lib/transport.js:289:15) at checkRespForFailure (/home/jilu/Documents/project/backup/assignment_ok/...
Gurudath Putannkar
1

votes
1

answer
447

Views

Elasticsearch date range check for nested array of date ranges

all devs, this is my first question here please ignore my mistakes. I am working on a project where I have a list of booking. This list has start and end dates, and for elasticsearch I already have declared and indexed them as nested 'Bookings':[{ 'StartDate':'2017-01-10T10:00:00+11:00', 'EndDate':...
Ravian
1

votes
0

answer
14

Views

How can we frame ES query for select from a group of ids

Here is the SQL query I am looking to write in ES select * from es where sale_id in (select * from ES group by sale_id) I have done a group by of the query as follows: { 'size': 200, 'aggs': { 'group_by_state': { 'terms': { 'field': 'sale.sale_id', 'size': 10 } } } } Explanation This code is used t...
Sumeet Kore

View additional questions