Questions tagged [elasticsearch]
8428 questions
1
votes
1
answer
2.7k
Views
Elasticsearch Query String Query with @ symbol and wildcards
I defined a custom analyzer that I was surprised not built-in.
analyzer": {
"keyword_lowercase": {
"type": "custom",
"filter": [
"lowercase"
],
"tokenizer": "keyword"
}
}
Then my mapping for this field is:
"email": {
"type": "string",
"analyzer": "keyword_lowercase"
}
This works great. (http://.../_...
1
votes
2
answer
416
Views
Logstash - Elesticsearch - Kibana : Store logs over tcp-input and show it in Kibana
I'm trying to setup logstash, elasicsearch and Kibana for visualization the logs.
The log should be send through TCP to logstash, filtered, output to ES-index and than shown in kibana.
The message i'm sending to logstash :
msg_to_tcp="id=1324 type=error name=system_name"
logstash.conf:
input{
tcp {...
1
votes
3
answer
1.3k
Views
ElasticSearch QuadPrefixTree vs GeohashPrefixTree
I am new to ElasticSearch and I want to understand the difference between using geohashes and quadtree. In the reference it is written:
GeohashPrefixTree - Uses geohashes for grid squares. Geohashes are base32 encoded strings of the bits of the latitude and longitude interleaved. So the longer the h...
0
votes
0
answer
4
Views
Elastic Search Interval in Date Range Query
I have a requirement where my query should be like this:
{
"size": 0,
"_source": [
"dateCreated"
],
"query": {
"bool": {
"must": {
"range": {
"date_published": {
"from": "2019-01-01", //Fixed date
"to": "--15 days from given 'from' date--" //Cannot find solution for this
}
}
}
}
}
}
I could not find...
1
votes
2
answer
1k
Views
How to use Spring Data Elasticsearch @Query
When I use the following code as Spring docs said eclipse always show Syntax error on token.
public interface BookRepository extends ElasticsearchRepository {
@Query("{"bool" : {"must" : {"field" : {"name" : "?0"}}}}")
Page findByName(String name,Pageable pageable);}
1
votes
1
answer
356
Views
Elasticsearch Realtime GET support
When I index a document in ES, I am trying to access the same document within in the refresh interval has passed and the search is not returning the result. Is there a Realtime GET support which allows to get a document once indexed regardless of the "refresh rate" of the index. I tried reducing the...
1
votes
1
answer
463
Views
How to query a specific shard for a document
I can use the preference to query ES shard #2 for tweet document with id 1 with URL like this:
GET twitter/tweet/1/_search?preference=_shards:2
How can I do this with Java API?
EDIT
This:
GET /bookshop/book/_search?preference=_shards:0
lists the document while the both below returns NPE:
GET /booksh...
0
votes
0
answer
3
Views
Kibana chart drill down facility
When we google for kibana dashboard charts in old posts it has been mentioned as interactive, so drill down is not possible. Is there any version or plugin available for drill down in kibana dashboard.
1
votes
1
answer
1.5k
Views
Mapping definition for [location] has unsupported parameters: [geohash : true] : Elasticsearch 5.X
I'm using elasticsearch 5.2, but when setting up index mapping with [geohash:true] for a geo_point field I'm getting the following error
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [location] has unsupported parameters: [geohash : true]"
}
],...
1
votes
2
answer
3.2k
Views
Elasticsearch 5.x donot start. max size virtual memory for user [elastic] is too low, increase to [unlimited]
I am moving from elasticsearch 2.x to 5.x and facing this problem while starting up.
[2017-02-28T14:38:24,490][INFO ][o.e.b.BootstrapChecks ] [node1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-02-28T14:38:24,494][ERROR][o.e.b.Bootstrap...
1
votes
1
answer
5.7k
Views
Index JSON files in elasticsearch using Python?
I have a bunch of JSON files(100), which are named as merged_file 1.json, merged_file 2. json and so on.
How do I index all these files into elasticsearch using python(elasticsearch_dsl) ?
I am using this code, but it doesn't seem to work:
from elasticsearch_dsl import Elasticsearch
import json
impo...
0
votes
0
answer
3
Views
First time starting an API, spark-java and elasticsearch
I've been given a challenge to build a RESTful API in Java, using Spark and elasticsearch. I've never made an API, or using frameworks before so I'm having trouble even getting to the coding. I'll try to ask this as best as I can, but I not sure what to ask honestly. What I "think" I want to know is...
12
votes
1
answer
153
Views
TOO_MANY_REDIRECTS error when iFraming Kibana dashboard using cookies
I'm attempting to embed password protected Kibana dashboards inside an iFrame to my Node powered Express application, based on information in this thread. Kibana is protected using X-Pack and requires users to login in order to see their visualised data.
This currently requires the user to log in t...
0
votes
0
answer
5
Views
sonarqube 6.7.6 LTS - unresponsive ES - always need to reindex ElasticSearch
We have upgraded from SonarQube 5.6 LTS version (Community) to SonarQube 6.7.6 LTS version with an Enterprise License.
We followed the instructions from here https://docs.sonarqube.org/display/SONARQUBE67/SonarSource+Editions and have upraded our VMs according to the new spec here: https://docs.sona...
0
votes
0
answer
2
Views
ElasticSearc - unable to get nested object from multilevel nested object
I have multilevel nested object. the first nested object is category and inside category there is one more nested object is group.
So I want to get distinct category along with nested groups using aggregation query.
am successful to get distinct category but unable to get group detail with it.
Mappi...
0
votes
0
answer
3
Views
Passing items/keywords to SelectedFilters while using them as tags in ReactiveList
I tried to replicate this example on a local machine with the current version of ReactiveSearch (v3 Web). Everything works fine except for one function (no error shown in console). It's about the "topics" in this example, which are used as tags. In the Codesandbox-example you can click on a tag (or...
2
votes
1
answer
69
Views
Reuse/cache script fields to use other parts of the query
I am working on a project where we need to show distance the listings are from a user location. To show the distance, when lat/lon is given in the input, we calculate distance using a script field called "distance"
"script_fields" : {
"distance" : {
"script" : {
"lang": "painless",
"source": "Math.r...
2
votes
0
answer
17
Views
What is the nest way to bulk index(around 40 k files of type .docx) using ingest-attachment?
I am fairly familiar with the ELK stack and currently using Elastic search 6.6. Our use case is content search for about 40K .docx files
(uploaded by Portfolio managers as research reports.
Max file size allowed is 10 MB, but mostly file sizes are in few Kb).
I have used the ingest attachment plu...
0
votes
0
answer
5
Views
Elasticsearch - Terms + Date Range + Top Hits
I need to run a terms aggregation, then filter it by date range and then finally use a top hits aggregation to get extra fields.
For just date ranged nested within a terms, its returning doc_count 0 for some reason and then with the top hits, its returning nothing.
Here is the query without the to...
0
votes
0
answer
3
Views
Is it a right option to choose Time Series Databases over an RDBMS Transaction Table for storing and retriving Application specific transactions
Currently we have Transaction Table which keeps track of Visitors in and out of Visitor Management System (SaaS), Which is doing great. Recently I am exploring in where else we can create a Transaction table other than current RDBMS, I am thinking of doing Quick POC using Elastic & Time series DB (W...
1
votes
2
answer
77
Views
'now' not working when using range query for seaching
{
"size": 0,
"query": {
"range": {
"LogTime": {
"gte": "now-1d",
"lte": "now",
"include_lower": true,
"include_upper": true,
"boost": 1.0
}
}
},
"aggregations": {
"top_aggs": {
"date_histogram": {
"field": "LogTime",
"interval": "1h",
"offset": 0,
"order": {
"_key": "asc"
},
"keyed": false,
"min_doc...
0
votes
0
answer
5
Views
Difference in ElasticSearch between scraped data and default set value (via Scrapy) in a text field
I found something really weird in ElasticSearch and I can't get my head around it. Let me first describe how I feed data into ElasticSearch:
I use Scrapy to crawl data
I use ScrapyElasticSearch as a pipeline to ES
In my example I created an index in ES that contains 3 text fields: title, title_alt a...
1
votes
1
answer
54
Views
Using AWS4 Signature via Postman for CRUD Elastic operations
I'm trying to post data to Elasticsearch managed by AWS using AWS4 signing method. I would like to achieve this via postman pre-script. I tried using below script which worked perfectly for GET operation of Elastic search but its not working for POST or PUT or DELETE operation & keep giving me error...
-4
votes
1
answer
115
Views
Embedding Elasticsearch in Desktop Application
I'm developing a C++ desktop application and want to have complete set of search capabilities like stemming, customized ranking algorithm, etc for some huge text content.
previously I used ElasticSearch in some web applications. but is it possible to embed ElasticSearch in a desktop application and...
1
votes
1
answer
9
Views
Storing 128bit numbers in elastic or mongodb
I have design requirement to be able to save an IPv6 address in a decimal format. In java I have BigInteger that can hold 40 digit number and Oracle takes it via JDBC driver just fine.
How can do this in Elastic or Mongo DB. From what I read, max numbers supported in elastic or mongodb are 64bit bi...
0
votes
0
answer
6
Views
Wiremock/Elasticsearch migrating from Java8 to Java9 / Java11
I am currently migrating app from openJDK 8 to openJDK 9/11.
Project uses Maven 3.5.2 (also tried with 3.6).
I am using Ubuntu 18.04 as primary OS.
My IDE is IntelliJ IDEA 2018.3.2
After adding module-info.java most of the maven dependencies worked fine with
requires ${module.name}
I have proble...
1
votes
0
answer
8
Views
How should I filter blocked user contents?
I've created little social website which has user generated content in it. And users have option to block each other if they don't like what they post.
So I'm using postgresql and I have three simple tables.
Table User
id (pk) | user_name
1 | a
2 | b
3 | c
Table Post
id (pk) |...
0
votes
0
answer
5
Views
Elasticsearch: Data server not discovering master
I have a 1 data - 1 master es cluster. (using 6.4.2 on CentOS 7)
On my master01:
==> /opt/elasticsearch/logs/master01-elastic.my-local-domain-master01-elastic/esa-local-stg-cluster.log
1
votes
1
answer
450
Views
What are concrete indices
In relation to elasticsearch, what are concrete indices.
The elasticsearch docs mention them hundreds of times, but I can't find a definition anywhere.
For example:
count - allowNoIndices:
Whether to ignore if a wildcard indices expression resolves into no ~concrete indices~.
1
votes
2
answer
516
Views
How to search for exact order matching of words in elasticsearch?
I need to make a match for an exact sentance. Here is the query Im using
{
"query": {
"match": {
"description": "void names error"
}
}
}
But the above query is returning me not only the exact matching documents but also many other partial matches too. How to make an exact match of the above sentance...
1
votes
3
answer
1.3k
Views
Elasticsearch Bulk API : Cannot post more than one record
I am trying to post the following using the bulk api. I have ES 2.2.0
{"index":{"_index":"junktest","_type":"test"}}
{"DocumentID":"555662","Tags":["B","C","D"],"Summary":"Summary Text","Status":"Review","Location":"HDFS","Error":"None","Author":"Abc Mnb","Sector":"Energy","Created Date":"2013-05-23...
2
votes
1
answer
58
Views
Has anyone been able to use elasticsearch xpack sql with Spark?
Using PySpark I'm trying to read data from elasticsearch. Normally I would set the query to something along the line (see query below) and set the es.resource to index such as "my_index/doc" and I'm able to read data into spark:
q ="""{
"query": {
"match_all": {}
}
}"""
However recently I've trie...
1
votes
1
answer
1.4k
Views
Trouble setting request specific timeout in Elasticsearch DSL
I'm trying to set a timeout for a specific request using elasticsearch_dsl. I've tried the following:
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search, F
...
def do_stuff(self, ids):
client = Elasticsearch(['localhost'], timeout=30)
s = Search(using=client,
index= 'my_ind...
1
votes
1
answer
1k
Views
Search in elasticSearch using alias name
I have an alias pointing to two indices. I would like to use the alias name to search the documents rather than passing two index names and getting the results.
Example :
Like this : SearchResponse response = client.prepareSearch("alias")
Not like this : SearchResponse response = client.prepareSea...
0
votes
0
answer
4
Views
Kibana machine learning. How to increase max csv file size import?
I'm using ELK since a few days and I try to import a csv file. When i import a csv file with 2000 lines or less in maching learning everything goes well. But when i try to import a csv file with 5000 lines or more (when the size > 1 mb), the window "Analyzing data" is endlessly loading. I searched o...
1
votes
2
answer
1.1k
Views
Kibana Link multiple dashboards
I am working with the elastic stack (logstash, kibana) and I would like to know if it is possible to have a "link" between two different dashboards.
Explanation :
I have a dashboard "Movies" and in it I have all the movies in my library. I would like click on "let's say pie chart" , I am expecting...
1
votes
1
answer
608
Views
using MultiMatchQueryBuilder for 'and' keyword query search
I have a bunch of documents stored in elasticsearch with fields title and abstract. I have to search documents for queries like 'word1 word2 ..'. Currently I am using spring data
MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder(query, "abstract", "title");
Iterable result...
1
votes
1
answer
1.8k
Views
Cannot use GeoJSON with Elasticsearch 5 and Kibana 5
I do not manage to index GeoJSON data in Elasticsearch and create Tile Map with them.
--> Sending GeoJSON to ES:
curl -XPUT "http://localhost:9200/datas/data/3617" -d
'{
"geometry": {
"coordinates": [
7.040691666666667,
43.626736111111114
],
"type": "Point"
},
"properties": {
"createdAt": {
"$date"...
1
votes
1
answer
293
Views
Kibana - How to create a visualization to show a duration per user?
I have four columns :
timestamp
username
action
session_id
I would like to show / visualize for each user the average duration for all sessions.
That means that I have multiple lines for one session and that I have to get the duration per session to compute the average.
An example of data :
User...
0
votes
0
answer
4
Views
Elasticsearch APM server too many active handles
We have recently put apm agent for one of our high traffic apis written in Nodejs Restify. While monitoring using pm2 monit i see that the number of active handles has increased drastically which eventually ends up making API unavailable. This happened after we attached the apm agent to it. What can...