Find all ID where ID are not in my blacklist

Refresh

November 2018

Views

1.4k time

3

After many lectures , I cannot say if this kind of query is possible with elasticsearch , I found the "getting started" really excellent but the rest of guide have a lack of examples (from my point of vue ).

See my structure below, I need to retrieve all id who are not in my blacklist. My blacklist is some reference id. For this example I am the id 1 with the firstname "me" . Here in the structure we see I blacklisted "bob" , so the bob id (2) is in my blacklist array because I don't want to find bob in my search result.. :)

Is it possible to only retrieve (dynamically for sure) all id who are not in my blacklist in one query? If you come from SQL, the same logic could be :

SELECT id FROM index WHERE id NOT IN (SELECT * FROM blacklist WHERE id = 1)

I would like to avoid the 2 step query , if my schema is bad and should be reconsidered , please I'm totally open for advice or suggestions.

Here is the structure :

{
    "id: 1,
    "balance": 16623,
    "firstname": "me",
    "blacklist" : [2,1982,939,1982,98716,7611,983838, and thousands others ....],

}
{
    "id: 2,
    "balance": 16623,
    "firstname": "bob,
    "blacklist" : [18,1982,939,1982,98716,7611,983838, and thousands others ....],

}
{
    "id: 3,
    "balance": 16623,
    "firstname": "jhon",
    "blacklist" : [18,1982,939,1982,98716,7611,983838, and thousands others ....],

}

1 answers

0

Вы можете использовать использовать поиск терминов фильтра вместе с не фильтруют следующим образом .

Я создал индекс с тремя документами, которые вы перечислили:

DELETE /test_index

PUT /test_index

PUT /test_index/doc/1
{
    "id": 1,
    "balance": 16623,
    "firstname": "me",
    "blacklist" : [2,1982,939,1982,98716,7611,983838]
}
PUT /test_index/doc/2
{
    "id": 2,
    "balance": 16623,
    "firstname": "bob",
    "blacklist" : [18,1982,939,1982,98716,7611,983838]
}
PUT /test_index/doc/3
{
    "id": 3,
    "balance": 16623,
    "firstname": "john",
    "blacklist" : [18,1982,939,1982,98716,7611,983838]
}

Затем создать запрос , который отфильтровывает документы , которые находятся в черном списке для "me":

POST /test_index/doc/_search
{
   "filter": {
      "not": {
         "filter": {
            "terms": {
               "id": {
                  "index": "test_index",
                  "type": "doc",
                  "id": "1",
                  "path": "blacklist"
               }
            }
         }
      }
   }
}
...
{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "test_index",
            "_type": "doc",
            "_id": "1",
            "_score": 1,
            "_source": {
               "id": 1,
               "balance": 16623,
               "firstname": "me",
               "blacklist": [2,1982,939,1982,98716,7611,983838]
            }
         },
         {
            "_index": "test_index",
            "_type": "doc",
            "_id": "3",
            "_score": 1,
            "_source": {
               "id": 3,
               "balance": 16623,
               "firstname": "john",
               "blacklist": [18,1982,939,1982,98716,7611,983838]
            }
         }
      ]
   }
}

Если вы хотите , чтобы отфильтровать пользователь , чей черный список используются, вы можете настроить несколько более сложный фильтр , используя или :

POST /test_index/doc/_search
{
   "filter": {
      "not": {
         "filter": {
            "or": {
               "filters": [
                  {
                     "terms": {
                        "id": {
                           "index": "test_index",
                           "type": "doc",
                           "id": "1",
                           "path": "blacklist"
                        }
                     }
                  },
                  {
                     "term": {
                        "id": "1"
                     }
                  }
               ]
            }
         }
      }
   }
}
...
{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 1,
      "max_score": 1,
      "hits": [
         {
            "_index": "test_index",
            "_type": "doc",
            "_id": "3",
            "_score": 1,
            "_source": {
               "id": 3,
               "balance": 16623,
               "firstname": "john",
               "blacklist": [18,1982,939,1982,98716,7611,983838]
            }
         }
      ]
   }
}

Вот код, который я использовал:

http://sense.qbox.io/gist/0b6808414f9447d4f7d23eb4c0d3e937ec2ea4e7