Questions tagged [elasticsearch]
13064 questions
1
votes
2
answer
741
Views
Loading Wikipedia Dump into Elasticsearch
I want to load a XML Wikipedia dump like:
http://ftp.acc.umu.se/mirror/wikimedia.org/dumps/enwiki/20171001/enwiki-20171001-pages-articles.xml.bz2
into Elasticsearch (5.6.4).
However, all tools and tutorials I found are outdated and not compatible with my Elasticsearch version.
Can anyone explain wha...
1
votes
1
answer
503
Views
Understanding Decay function and its parameters in ElasticSearch
I was reading about decay functions in ElasticSarch in order to promote more recent results
If I define decay function as following:
'DECAY_FUNCTION': {
'FIELD_NAME': {
'origin': '2013-09-17',
'scale': '10d',
'offset': '5d',
'decay' : 0.5
}
}
How the score will be affected within the Offset...
1
votes
3
answer
2.2k
Views
How to find all duplicate documents in ElasticSearch
We have a need to walk over all of the documents in our AWS ElasticSearch cluster, version 6.0, and gather a count of all the duplicate user ids.
I have tried using a Data Visualization to aggregate counts on the user ids and export them, but the numbers don't match another source of our data that i...
1
votes
1
answer
253
Views
How to get the search History in Elasticsearch?
There is any way to get the search history in Elasticsearch. In this I want to get the search queries which are searched by customers in last one month.
1
votes
1
answer
664
Views
ElasticSearch: analyzer on field must be set when search_analyzer is set
I have read about previous version of ES (< 2) where the 'token_analyzer' key needs to be changed to 'analyzer'. But no matter what I do I am still getting this error:
'type': 'mapper_parsing_exception',
'reason': 'analyzer on field [email] must be set when search_analyzer is set'
Here is what I am...
1
votes
1
answer
459
Views
Elasticsearch how to support transaction involving multiple documents
i use elasticsearch,and Denormalizing Data,like
PUT /my_index/user/1
{
'name': 'John Smith',
'email': '[email protected]',
'dob': '1970/10/24'
}
PUT /my_index/blogpost/2
{
'title': 'Relationships',
'body': 'It's complicated...',
'user': {
'id': 1,
'name': 'John Smith'
}
}
but the problem is that Elasti...
1
votes
1
answer
23
Views
Should as filter context (OR without scoring in Elasticsarch possible?)
I am trying to setup most efficient way to build an OR without having a scoring, since I want to order my results by business values afterwards.
Unfortunately i don't get it done. :(
What I need:
COLOR=X AND ( title = Y OR description = Z)
What I tried (but it is malformed):
{
'query': {
'bool': {...
1
votes
2
answer
29
Views
What is wrong with my Elastisearch query?
I need to make multi_match and bool query at once, but query below doesn't work :( When I use them separatley, they working perfect.
{
'query': {
'multi_match': {
'query': 'kotlety*',
'fields': [
'name'
]
},
'bool': {
'filter': {
'term': {
'status': 2
}
}
}
},
'size': 24
}
response is:
{
'error': {...
1
votes
2
answer
36
Views
How to secure elasticsearch API's
I wonder how people normally secure their elasticsearch API's, when being accessed from browser side?
If we access elastic server from server side, even then a simple DELETE request is sufficient to delete a whole index, which might contain priceless information.
I've read that they provide RBAC f...
0
votes
0
answer
4
Views
cakephp 3.X elasticsearch QueryBuider
Following the manual of the QueryBuilder from the ElasticSearch plugin on https://github.com/cakephp/elastic-search/blob/master/src/QueryBuilder.php
, the search for entries containing 'milk' or 'cheese' works as expected using the 'in' operator.
$this->loadModel('Archief', 'Elastic');
$query = $thi...
1
votes
1
answer
79
Views
Set filter in Kibana 7/6.7.2 for latest _id in DSL with size 1
I want the filter to be set on dashboard which will fetch only latest record
So Used below query in DSL
{
'query': {
'match_all': {}
},
'size': 1,
'sort': [
{
'_id': {
'order': 'desc'
}
}
]
}
This is not working in Elasticsearch Kibana 7.0 and I have checked in 6.7.1 also , it is not working
'size'...
0
votes
0
answer
5
Views
Elastic search index only searchable fields
I would like to create an index in which only some fields are indexed. I created a template with the enabled property set to false. So no field is indexed by default.
https://www.elastic.co/guide/en/elasticsearch/reference/6.4/enabled.html
Then I defined the fields I want to index with dynamic templ...
0
votes
0
answer
6
Views
Get sorted results after aggregation in elastic search
I am getting aggregated result for nested object (Minimum price for each nested object)
All of my search results are sorted and now I need minimum value for each result.
Result:
Response from aggregation is not sorted, as it is in my search result (min, some_field)
Question:
Is there any solution...
1
votes
1
answer
794
Views
Elasticsearch: add a synonym filter on my english analyser
I've got an index with a field description which is analysed like that:
'description':{
'analyzer' : 'english',
'type' : 'string'
}
I have defined an synonyms dictionnary in a file synonyms.txt which contain:
ipod, i-pod, i pod => i-pod
I would like to add this synonym dictionnary to my analyzer, bu...
1
votes
2
answer
2k
Views
Elasticsearch: should + minimum_should_match vs must
I test with these 2 queries
Query with must
{
'size': 200,
'from': 0,
'query': {
'bool': {
'must': [ {
'match': {
'_all': 'science'
}
},
{
'match': {
'category': 'fiction'
}
},
{
'match': {
'country': 'us'
}
}
]
}
}
}
Query with should + minimum_should_match
{
'size': 200,
'from': 0,
'query': {
'boo...
1
votes
1
answer
964
Views
Partial update in ElasticSearch with JEST
I'm trying to use JEST to implement CRUD actions to ElasticSearch, and I've surveryed some articles, the method I found to do 'Update' is 'Index' another document with the same '_id', but I'll replace the whole document in that way.
I want to know is JEST capable to do 'Partial Update'?
Edit :
I fou...
1
votes
2
answer
1.4k
Views
Kibana and Docker-Compose: Automatically load in saved object
Goal: Load in Kibana saved object when starting Elastic Stack from Docker-Compose.
I have an 'export.json' object from Kibana, containing my default index, 10+ visualizations, and dashboard setup. I am able to successfully start the Elastic Stack from Docker-Compose and manually load the object, but...
1
votes
2
answer
449
Views
What is the right way to snyc/import tables from a postgres DB to elasticsearch?
I want to import some tables from a postgres database into Elastic search and also hold the tables in sync with the data in elastic search. I have looked at a course on udemy, and also talked with a colleague who has a lot of experience with this issue to see what the best way to do it is. I am surp...
1
votes
1
answer
234
Views
What are benefits of running ELK stack on Docker over running it on VM
I'm learning ELK stack. I wonder, why would people run it on Docker?
If I understand everything correctly, it would have to have some directory of a host OS mapped to be persistent over resets of the image.
Meanwhile, running just VL with installed docker would be persistent anyway.
Why should I use...
1
votes
1
answer
17
Views
Elastic Search bulk operation
I really can't understand why when I run a bulk insert I lost the previous data in the same collection without executing any delete operation?
this is weird.
any idea?
var client = new elasticsearch.Client( {
hosts: [
'http://localhost:9200/'
]
})
.
.
.
InserTweets: function (arrayobj, callbac...
1
votes
2
answer
140
Views
ElasticSearch - Failed to parse content to map - Unexpected character ('\"'
Wondering if you can help, I'm having issues with my code in Elasticsearch that is giving out an error. My code is below and the output is also below, any help is greatly appreciated.
curl -X PUT 'localhost:9200/_mapping/jdbc' -H 'Content-Type: application/x-ndjson' -d '
{
'mappings': {
'jdbc' :...
1
votes
3
answer
32
Views
How can I send multiple documents to elastic with one API call
I'm new to Elastic and I need a way to push multiple documents to Elastic by only making one call to POST http://localhost:9200/myindex/mytype/.
The body schema looks like this:
{
'docs': [
{ '_source': {'message':'message1'} },
{ '_source': {'message':'message2'} }
]
}
I tried with the ingest...
2
votes
1
answer
60
Views
Elasticsearch: FVH highlights with multiple pre and post tags marking tokens incorrectly?
I'm querying my index using boolean query with two match terms. For each term I have a separate set of pre- and post- tags. Using highlights I would like to obtain the documents in which both terms exist and see which tokens were matched as each of them. The index contains documents in Polish analyz...
1
votes
0
answer
17
Views
how to sync data from mongodb to elastic search with python
I am currentlt using Monstache connector for sync mongodb elastic search. is there any other recommended connector for sync.
Architecture: Angular7 Elastic Search Mongo DB Phython Pycharm.
is there any other soultion apart from this. https://rwynn.github.io/monstache-site/.
1
votes
0
answer
175
Views
ElasticSearch max and min score
I want to search the fields and apply a function to the doc score that includes the max and min score of the search result. I can use the script score but how I get the max and min scores in script?
1
votes
1
answer
253
Views
Run Elastic Search on pdf and ppts
I am new to elastic search. I have read its tutorials. But need guidance on my problem:
I have a collection of pdf documents and power point files on my system. I need to build a system using elastic search where I can retrieve these files on the basis of keywords present in this file. Can someone p...
1
votes
2
answer
1.2k
Views
How to monitor Elastic Stack without X-Pack?
Can we monitor the elastic stack 6.0 and above(like elastic search..) without using the X-Pack?As we know many of the Features like security, machine learning, graph APIs don't be supported under BASIC(free Licence).
So I want to know if there are any APIs, without Licence limitation, can be used t...
1
votes
0
answer
296
Views
nested type mapping in elasticsearch with python
I am trying to map elasticsearch index, but I keep getting an error with nested type.
request_body = {
'setting': {
'number_of_shards': 3,
'number_of_replicas': 1
},
'mappings': {
'examplecase': {
'properties': {
'id': {'index': 'not_analyzed', 'type': 'string'},
'source': {'index': 'analyzed', 'typ...
1
votes
0
answer
147
Views
Basicauth setup on Elasticsearch 6 dot release
I am trying to test few plugins related to user access for test user (custom access based doc filtering) on ES and I need to have basic auth to validate it. Is there a user manual for setting up basic auth setup at elasticsearch (6 dot release) at server end. Any information will be great help.
1
votes
0
answer
155
Views
how to paginate in Elastic Search Autocomplete (search as you type in index time)
I am facing a road-block in elastic search autocomplete feature.
I am using custom analyser to a autocomplete field & it is working like a charm as expected .
Now I want to paginate the result so that my client (angular app) does not need to deal with high volume of data .
Precisely , user types 'MA...
1
votes
0
answer
41
Views
Offline time report using Kibana and ElasticSearch
Our devices are sending pings to ElasticSearch, every 5 minutes, in the following format:
{
deviceId: 34567,
timestamp: 1514889787
}
(We may alter the stored timestamp format if needed).
If the device is offline, the ping is not send at all (ElasticSearch is storing only pings for devices being onli...
1
votes
1
answer
302
Views
How do you convert a Django model with ManytoMany field to an elasticsearch_dsl DocType class?
I'm using elasticsearch_dsl to help with interfacing with elasticsearch and this is the model I currently have and I'm trying to recreate as a DocType:
class HouseIndex(DocType):
house_type = String()
#people
sold = Boolean()
built_datetime = Date()
#alerts
# associated_locations
hash = String()
cla...
1
votes
1
answer
189
Views
Elasticsearch 6.1 Java API search issue
I am using ES - 6.1 and corresponding java version of the api. I was able to index the data using the java API. I validated this via kibana. All the mappings were correctly saved but an error comes up when I try to search. Even the most basic search returns me an empty collection.
// client is a ref...
1
votes
1
answer
800
Views
How to validate Elasticsearch Painless scripts?
We use many ScriptQuery and ScriptField in our project, and provide inline scripts for them in painless language.
My question is, how can we validate these painless scripts? In other words, how to make sure that they will compile?
The approach we use today is to test them out locally through Kibana...
1
votes
2
answer
631
Views
get distinct values of string field using elasticsearch query Client java API
I'm using elasticsearch 5.4
I need to get distinct values of fields using elasticsearch query
I'm trying this query but is not wokring
GET /index/index_type/_search
{
'size': 0,
'aggs': {
'distinct': {
'terms': {
'field': 'status'
}
}
}
thank you
1
votes
1
answer
484
Views
ElasticSearch 6 High Level Rest Client preparePutMapping
I am trying to transition from ElasticSearch 2 to either 5 or 6. I think that I want to jump straight to 6.1.1 and use the RestHighLevelClient, since it is closer to the existing transport client that I am using than the low level rest client.
However, I am running across a problem As part of my i...
1
votes
0
answer
119
Views
Error connecting to elasticsearch Java CLient API 5.4.3
I'm developping a web application using spring websphere and elasticsearch I want to conenct my java client api with elasticsearch but I got this error thanks for helping
[05/01/18 19:54:31:660 WET] 00000054 SystemErr R ERROR StatusLogger No log4j2 configuration file found. Using default configu...
1
votes
1
answer
489
Views
mongoosastic [mapper_parsing_exception] No handler for type [string] declared on field [type]
Here is the error
{ Error: [mapper_parsing_exception] No handler for type [string] declared on field [type]
at respond (/home/jilu/Documents/project/backup/assignment_ok/node_modules/elasticsearch/src/lib/transport.js:289:15)
at checkRespForFailure (/home/jilu/Documents/project/backup/assignment_ok/...
1
votes
1
answer
447
Views
Elasticsearch date range check for nested array of date ranges
all devs, this is my first question here please ignore my mistakes. I am working on a project where I have a list of booking. This list has start and end dates, and for elasticsearch I already have declared and indexed them as nested
'Bookings':[{
'StartDate':'2017-01-10T10:00:00+11:00',
'EndDate':...
1
votes
0
answer
14
Views
How can we frame ES query for select from a group of ids
Here is the SQL query I am looking to write in ES
select *
from es
where sale_id in (select * from ES group by sale_id)
I have done a group by of the query as follows:
{
'size': 200,
'aggs': {
'group_by_state': {
'terms': {
'field': 'sale.sale_id',
'size': 10
}
}
}
}
Explanation
This code is used t...