Questions tagged [elasticsearch]
13040 questions
1
votes
0
answer
175
Views
ElasticSearch max and min score
I want to search the fields and apply a function to the doc score that includes the max and min score of the search result. I can use the script score but how I get the max and min scores in script?
1
votes
1
answer
253
Views
Run Elastic Search on pdf and ppts
I am new to elastic search. I have read its tutorials. But need guidance on my problem:
I have a collection of pdf documents and power point files on my system. I need to build a system using elastic search where I can retrieve these files on the basis of keywords present in this file. Can someone p...
1
votes
2
answer
1.2k
Views
How to monitor Elastic Stack without X-Pack?
Can we monitor the elastic stack 6.0 and above(like elastic search..) without using the X-Pack?As we know many of the Features like security, machine learning, graph APIs don't be supported under BASIC(free Licence).
So I want to know if there are any APIs, without Licence limitation, can be used t...
1
votes
0
answer
296
Views
nested type mapping in elasticsearch with python
I am trying to map elasticsearch index, but I keep getting an error with nested type.
request_body = {
'setting': {
'number_of_shards': 3,
'number_of_replicas': 1
},
'mappings': {
'examplecase': {
'properties': {
'id': {'index': 'not_analyzed', 'type': 'string'},
'source': {'index': 'analyzed', 'typ...
1
votes
0
answer
147
Views
Basicauth setup on Elasticsearch 6 dot release
I am trying to test few plugins related to user access for test user (custom access based doc filtering) on ES and I need to have basic auth to validate it. Is there a user manual for setting up basic auth setup at elasticsearch (6 dot release) at server end. Any information will be great help.
1
votes
0
answer
155
Views
how to paginate in Elastic Search Autocomplete (search as you type in index time)
I am facing a road-block in elastic search autocomplete feature.
I am using custom analyser to a autocomplete field & it is working like a charm as expected .
Now I want to paginate the result so that my client (angular app) does not need to deal with high volume of data .
Precisely , user types 'MA...
1
votes
0
answer
41
Views
Offline time report using Kibana and ElasticSearch
Our devices are sending pings to ElasticSearch, every 5 minutes, in the following format:
{
deviceId: 34567,
timestamp: 1514889787
}
(We may alter the stored timestamp format if needed).
If the device is offline, the ping is not send at all (ElasticSearch is storing only pings for devices being onli...
1
votes
1
answer
302
Views
How do you convert a Django model with ManytoMany field to an elasticsearch_dsl DocType class?
I'm using elasticsearch_dsl to help with interfacing with elasticsearch and this is the model I currently have and I'm trying to recreate as a DocType:
class HouseIndex(DocType):
house_type = String()
#people
sold = Boolean()
built_datetime = Date()
#alerts
# associated_locations
hash = String()
cla...
1
votes
1
answer
189
Views
Elasticsearch 6.1 Java API search issue
I am using ES - 6.1 and corresponding java version of the api. I was able to index the data using the java API. I validated this via kibana. All the mappings were correctly saved but an error comes up when I try to search. Even the most basic search returns me an empty collection.
// client is a ref...
1
votes
1
answer
800
Views
How to validate Elasticsearch Painless scripts?
We use many ScriptQuery and ScriptField in our project, and provide inline scripts for them in painless language.
My question is, how can we validate these painless scripts? In other words, how to make sure that they will compile?
The approach we use today is to test them out locally through Kibana...
1
votes
2
answer
631
Views
get distinct values of string field using elasticsearch query Client java API
I'm using elasticsearch 5.4
I need to get distinct values of fields using elasticsearch query
I'm trying this query but is not wokring
GET /index/index_type/_search
{
'size': 0,
'aggs': {
'distinct': {
'terms': {
'field': 'status'
}
}
}
thank you
1
votes
1
answer
484
Views
ElasticSearch 6 High Level Rest Client preparePutMapping
I am trying to transition from ElasticSearch 2 to either 5 or 6. I think that I want to jump straight to 6.1.1 and use the RestHighLevelClient, since it is closer to the existing transport client that I am using than the low level rest client.
However, I am running across a problem As part of my i...
1
votes
0
answer
119
Views
Error connecting to elasticsearch Java CLient API 5.4.3
I'm developping a web application using spring websphere and elasticsearch I want to conenct my java client api with elasticsearch but I got this error thanks for helping
[05/01/18 19:54:31:660 WET] 00000054 SystemErr R ERROR StatusLogger No log4j2 configuration file found. Using default configu...
1
votes
1
answer
489
Views
mongoosastic [mapper_parsing_exception] No handler for type [string] declared on field [type]
Here is the error
{ Error: [mapper_parsing_exception] No handler for type [string] declared on field [type]
at respond (/home/jilu/Documents/project/backup/assignment_ok/node_modules/elasticsearch/src/lib/transport.js:289:15)
at checkRespForFailure (/home/jilu/Documents/project/backup/assignment_ok/...
1
votes
1
answer
447
Views
Elasticsearch date range check for nested array of date ranges
all devs, this is my first question here please ignore my mistakes. I am working on a project where I have a list of booking. This list has start and end dates, and for elasticsearch I already have declared and indexed them as nested
'Bookings':[{
'StartDate':'2017-01-10T10:00:00+11:00',
'EndDate':...
1
votes
0
answer
14
Views
How can we frame ES query for select from a group of ids
Here is the SQL query I am looking to write in ES
select *
from es
where sale_id in (select * from ES group by sale_id)
I have done a group by of the query as follows:
{
'size': 200,
'aggs': {
'group_by_state': {
'terms': {
'field': 'sale.sale_id',
'size': 10
}
}
}
}
Explanation
This code is used t...
1
votes
1
answer
42
Views
Spring boot elastic not giving expected result
I am querying elastic through spring boot.
case:- 1
There is some data in my elastic like region = 'REGION 1'
I want to get all the data where 'REGION 1', but getting an empty result.
case 2:-
There is some data in my elastic like region = 'REGION1'
I want to get all the data where 'REGION1', I am g...
1
votes
0
answer
64
Views
How is the BalancedShard Algorithm of elasticsearch is implemented
Hi i am trying to learn how elastic search balances the shard allocation in the cluster.I tried to learn from the BalancedShard java implementation that is provided in the github but i am not able to get a hold on it.Could anyone give me an overview of how the BalancedShard algorithm works.
Github l...
1
votes
2
answer
242
Views
How to set up autosearch nodes in Elasticsearch 6.1
I have created cluster of 5 nodes in ES 6.1. I am able to create cluster when I added line with all ip addresses of other nodes into configuration file elasticsearch.yaml as discovery.zen.ping.unicast.hosts. It looks like this:
discovery.zen.ping.unicast.hosts: ['10.206.81.241','10.206.81.238','10.2...
1
votes
0
answer
593
Views
Elasticsearch painless get string field (which contains XML) into a variable
I'm trying to parse an Elasticsearch string field (named Request.Body) which contains XML. This field contains a SOAP request string, like this :
myData
myData
myData
myData
false
My goal is to extract the value of the search tag in a scripted field (in Kibana) using the painless language.
I tried t...
1
votes
1
answer
680
Views
Elasticsearch combining multiple buckets and aggregations
Let's assume we're looking at data that's reasonably simple -- each document in our index has this structure:
{
'Time': '2018-01-01T19:35:00.0000000Z',
'Country': 'Germany',
'Addr': 'security.web.com',
'FailureCount': 5,
'SuccessCount': 50
}
My question essentially boils down to how I might go about...
1
votes
1
answer
199
Views
Using reverse token with a custom normalizer
Apparently when I try to run the following mapping
'normalizer': {
'my_normalizer': {
'type': 'custom',
'char_filter': [],
'filter': [
'lowercase'
]
},
'my_reverse_normalizer': {
'type': 'custom',
'char_filter': [],
'filter': [
'lowercase',
'reverse'
]
}
}
I get:
Custom normalizer [my_reverse_norma...
1
votes
1
answer
349
Views
How to create bulk parent child relationship(join) in Elastic Search PHP while creating mapping and indexing
I want to create documents with parent child relationship.
I have data like below,
Parent Document data with parent_id = null
{
'id': 1,
'workflow_name': 'Diwali',
'list_name': 'number',
'list_id': '798',
'msgType': 'Promotional - National',
'sender_id': 'MANISH',
'submit_date': '201...
1
votes
0
answer
54
Views
Rails 5 - How to search a string as per the order which we enter?
In rails 5, I am using searchkick gem for the first time, I want a search result for auto-complete list as per the letter which I enter.
'Cleanliness', 'Taste', 'Service' these are data which I want to show as a suggestions(list)
Now the current search is working like,
When misspellings: false, if I...
1
votes
1
answer
653
Views
Not able to find curator.yml (elasticsearch-curator) in linux
Official site of elasticsearch says the default config file exists in /home/username/.curator/curator.yml
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/command-line.html
But there is no such folder.
Also, I tried creating curator.yml and give path using --config option. But, i...
1
votes
0
answer
50
Views
Write a query which sort term aggregations buckets based on inner sorted top_hits results
I have trouble to write a specific query in elasticsearch.
The context:
I have an index where each document represents a “SKU”: a declination of a product (symbolized by pId).
For example, the first 3 documents are declinations in color and price of product 235.
BS is for “Best SKU”: for a...
1
votes
2
answer
293
Views
Cast or convert mongodb _id object to string in Logstash pipeline
I am working on creating a pipeline to get data from MongoDB to ElasticSearch using Logstash.
I am using dbschema mongodb jdbc drivers. I am able to connect to database using driver but I am facing issue with _id . As in MongoDB its of type object So I am getting issue with converter. Here is error...
1
votes
0
answer
160
Views
Logstash with elasticsearch input and output keep looping results
I would like to reindex and filter my log again. What I get the information from Internet is using the logstash to filter the data again.
I tried and it can really split my data into different fields, however, the data keeps looping. That is, I have 100,000 logs but after filtering and output to ela...
1
votes
1
answer
266
Views
Elasticsearch delete a list of indices with the Javascript client failing with “No Living connections”
I am using the official Elasticsearch Javascript client with version 6.0 and trying to delete a list of indices.
The code is:
const client = new elasticsearch.Client({
'hosts': [
'https://the-es-host-here.us-west-2.es.amazonaws.com'
],
'apiVersion': '6.0',
'sniffOnStart': true
});
const indices = [...
1
votes
2
answer
109
Views
String sort for buckets in aggregation elastic search
I want to sort the result by field that has string value, for example, I have responded like this
'aggregations': {
'group_agg': {
'doc_count_error_upper_bound': -1,
'sum_other_doc_count': 82,
'buckets': [
{
'key': '38:16:33:6A:71:FF',
'doc_count': 1,
'group_agg_top_hit': {
'hits': {
'total': 1,
'm...
1
votes
1
answer
118
Views
Get elastic search query from spring data Jpa
In my code base, I have two code for search on elastic.
One is through ElasticSearchTemplte from there I build the elastic query and query on elastic.
In the second one, I am querying through spring-data-jpa.
In the first case, I can see my elastic search query and can directly query on ES.
My quest...
1
votes
0
answer
43
Views
Elasticsearch: Ordering of Highlighted results does not work
I am trying to order the highlighted results returned by Elasticsearch. As per the documentation, here is how I do so:
enter code here res=es.search(
index='my-index',
size=30,
body=
{
'query':
{
'multi_match':
{
'fields':['chapter_name','chapter_id','subchapter_name','subchapter_id','range_name','r...
1
votes
0
answer
30
Views
Relation based search with elasticsearch / kibana
I'm running a software where a client-application communicates with a backend-service. In order to be able to track what the client does I want to analyze my logs in the backend. Therefore I've setup an ELK stack in the backend.
Setup and application behaves as follows:
a) filebeat delivers logfiles...
1
votes
1
answer
143
Views
Sonarqube failing to start even after clearing the $SONAR_HOME/data/es folders
I am getting the below error message in the sonarqube logs and sonarqube web page is not accessible. I found out the solution that clearing the $SONAR_HOME/data/es/ would fix it. But it didn't. Please let me know if there are any other alternative ways.
2017.11.30 08:58:51 WARN es[o.e.index.engine...
1
votes
0
answer
21
Views
Elasticsearch: Why can't I use “5m” for precision in context queries?
I'm running on Elasticsearch 5.5
I have a document with the following mapping
'mappings': {
'shops': {
'properties': {
'locations': {
'type': 'geo_point'
},
'name': {
'type': 'keyword'
},
'suggest': {
'type': 'completion',
'contexts': [
{
'name': 'location',
'type': 'GEO',
'precision': '10m',
'path'...
1
votes
0
answer
30
Views
How to query compare between rows in collection ElasticSearch
I have data in Elasaticsearch like this:
{'number1':1 ,'result':0 , 'date':'2018-01-17T01:00:57Z'}
{'number1':5 ,'result':0 , 'date':'2018-01-17T02:00:57Z'}
{'number1':6 ,'result':0 , 'date':'2018-01-17T03:00:57Z'}
{'number1':4 ,'result':0 , 'date':'2018-01-17T04:00:57Z'}
I want to query result:
{'n...
1
votes
0
answer
77
Views
Why matchPhrasePrefix query does not work with underscored text fields?
In my elastic index, I've a document with a text field with underscore separated word like 'test_upload_document.txt'. I need to perform a full text search on this index and I am using a MatchPhrasePrefixQuery in the following way
BoolQueryBuilder qb = new BoolQueryBuilder();
qb.must(QueryBuilders...
1
votes
1
answer
115
Views
How to store the content of the website in the Status Index using StormCrawler?
We are trying to store the content of the webpage in the Status Index, along with url, status and metadata information.
We tried to edit the ES_IndexInit.sh and add the next property in the Status' mapping section:
'content': {
'type': 'text',
'index': 'true',
'store': true
}
but we can't see anyth...
1
votes
0
answer
109
Views
Elasticsearch can't handle multiple requests without dramatically decrease its performance
I have a two node cluster hosted in ElasticCloud.
Host Elastic Cloud
Platform Google Cloud
Region US Central 1 (Iowa)
Memory 8 GB
Storage 192 GB
SSD Yes
HA Yes
Each node has:
Allocated Processors 2
Number of processors 2
Number of indices 4*
Shards (p/ index) 5*...
1
votes
1
answer
37
Views
How to Create more than one type(table) in single Index(db) using Java API in Elasticsearch
I am in the process of learning elasticsearch using java api
I want to create two or more types with different mappings in elasticsearch using the java api.
how to create more than one type in same index in elastic search in java api ?
the code i tried is :
To create the index named 'alpha'. the bel...