Elastic Search搜索数据Terms聚合返回的不正确的问题

项目中使用Elastic Search做搜索，在聚合产品的标签的时候，我发现标签的种类有很多超过了10个（这一点很重要），但是聚合标签出来的数据只有10个，缺少了很多标签，查询的语句如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
{
  "from" : 0,
  "size" : 15,
  "query" : {
    "bool" : {
      "must" : [ {
        "terms" : {
          "categoryId" : [ 1046, 1045, 1044 ]
        }
      }, {
        "terms" : {
          "tagList.id" : [ 1063, 1138, 1115, 1142 ]
        }
      } ]
    }
  },
  "aggregations" : {
    "categoryId" : {
      "terms" : {
        "field" : "categoryId"
      }
    },
    "tagList.tagContent" : {
      "terms" : {
        "field" : "tagList.tagContent"
      }
    }
  }
}

百思不得其解，看文档发现自己的查询和Terms的聚合使用都是没有任何问题的，但是出来的数据为什么会少呢？于是昨天上午花了将近一个小时的时间才找到问题的所在，下面是文档上对返回数据的一个解释：
By default, the terms aggregation will return the buckets for the top ten terms ordered by the doc_count. One can change this default behaviour by setting the size parameter.

Size
The size parameter can be set to define how many term buckets should be returned out of the overall terms list. By default, the node coordinating the search process will request each shard to provide its own top size term buckets and once all shards respond, it will reduce the results to the final list that will then be returned to the client. This means that if the number of unique terms is greater than size, the returned list is slightly off and not accurate (it could be that the term counts are slightly off and it could even be that a term that should have been in the top size buckets was not returned).

原来Elastic Search对数据聚合默认返回10个，我聚合的数据术语桶超过了10个，可以自行设置size来返回不同数量的术语桶。于是乎，查询的语句修改成下面的样子就成功返回了所有的数据：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
{
  "from" : 0,
  "size" : 15,
  "query" : {
    "bool" : {
      "must" : [ {
        "terms" : {
          "categoryId" : [ 1046, 1045, 1044 ]
        }
      }, {
        "terms" : {
          "tagList.id" : [ 1063, 1138, 1115, 1142 ]
        }
      } ]
    }
  },
  "aggregations" : {
    "categoryId" : {
      "terms" : {
        "field" : "categoryId",
        "size" : 100
      }
    },
    "tagList.tagContent" : {
      "terms" : {
        "field" : "tagList.tagContent",
        "size" : 100
      }
    }
  }
}

其实我还有一个问题，在使用Range范围聚合的时候，发现聚合的数据不准确，就是说10-20本来就30条记录，但是聚合显示这个范围的数据只有10个，这个问题我暂时还没有找到原因，有人知道的话，我很高兴有人能告诉我！

2018/10/12更新：
对于上面的Range范围聚合不正确的原因已经找到，发现还是自己太过粗心，人家提供的开发文档没有仔细的看，终于我在2018/10/11的时候重新看了一下Elasticsearch Reference 5.5 » Aggregations » Bucket Aggregations » Range Aggregation,最开始的一段话就是问题的答案！

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/*
A multi-bucket value source based aggregation that enables the user to define a set of ranges - each representing a bucket. During the aggregation process, the values extracted from each document will be checked against each bucket range and "bucket" the relevant/matching document. Note that this aggregation includes the from value and excludes the to value for each range.

Example:
*/

{
    "aggs" : {
        "price_ranges" : {
            "range" : {
                "field" : "price",
                "ranges" : [
                    { "to" : 50 },
                    { "from" : 50, "to" : 100 },
                    { "from" : 100 }
                ]
            }
        }
    }
}

上面的统计的含义是：

  • 统计价格小于50的总数
  • 统计价格大于等于50，小于100的总数
  • 统计价格大于等于100的总数

总之，就是统计包含下限，不包含上限。

  • 作者: Sam
  • 发布时间: 2018-08-30 23:37:03
  • 最后更新: 2020-10-24 11:13:01
  • 文章链接: https://ydstudios.gitee.io/post/39992895.html
  • 版权声明: 本网所有文章除特别声明外, 禁止未经授权转载，违者依法追究相关法律责任!