Aggregation Query filtering on results

OpenSearch
1

I have this query:

GET user_info,user_auth_cards_info/_search
{
  "size": 0,
  "aggs": {
    "sorted_user_id": {
      "terms": {
        "field": "user_id",
        "size": 15
      },
      "aggs": {
        "filtered_names": {
          "top_hits": {
            "size": 2,
            "_source": {
              "includes": [
                "e_name",
                "cards_user_name",
                "status",
                "del_status",
                "user_id",
                "frog_id",
                "telphone"
              ]
            }
          }
        }
      }
    }
  }
}

user_info has e_name
user_auth_cards_info has cards_user_name

Issue I’m having is that I want to filter out data where e_name and cards_user_name don’t start with a particular string ie “test” but if one of them do then they both should be in the aggregation. I can’t figure out how I should be doing this

2

I think the way to go is to put a Filter Aggregation within your terms aggregation. Something like:

sorted_user_id
filter_out_test <---- this would be your filter agg
filtered_names

This way the original counters would show up in your terms aggregation (sorted_user_id), but then only the interesting ones will be filtered at the filter_out_test level. The top hits aggregation will simply surface top N of these results.

4

With the filter out by test what would be the approach to do so because I still can’t find any valid approach to do it

5

Ah, sorry, my post wasn’t formatted so it’s confusing. This is what I meant:

"aggs": {
    "sorted_user_id": {
      "terms": {
        "field": "user_id",
        "size": 15
      },
      "aggs": {
        "filtered_names": {
          "filter": {
              .... insert query here that filters your data
          "aggs": {
            "filtered_top_hits": {
              "top_hits": {
                "size": 2,
                ....
6

Hey, thanks for the response. Still having a couple issues with filtering out the data. So this is what I got so far:

GET user_info,user_auth_cards_info/_search
{
  "size": 0,
  "aggs": {
    "user_id": {
      "terms": {
        "field": "user_id"
      },
      "aggs": {
        "filter_out_test": {
          "filter": {
            "bool": {
              "should": [
                {
                  "match": {
                    "e_name.autocomplete": "yellow"
                  }
                },
                {
                  "match": {
                    "cards_user_name.autocomplete": "yellow"
                  }
                }
              ]
            }
          },
          "aggs": {
            "top_names": {
              "top_hits": {
                "size": 2,
                "_source": {
                  "includes": [
                    "e_name",
                    "cards_user_name",
                    "status",
                    "del_status",
                    "user_id",
                    "frog_id",
                    "telphone"
                  ]
                }
              }
            }
          }
        }
      }
    }
  }
}

So for example if cards_user_name matches yellow but e_name doesn’t it should still return both the result.

So I have this data in user_info
{“e_name” : “test”, “user_id” : 1}

then in user_auth_cards_info
{“cards_user_name” : “yellow”, “user_id” : 1}

Now when I run the search on yellow it is only returning the user_auth_cards_info but I still need it to return both. I thought by doing the should if it matched to one it would return both but obviously not.

Do you know how I can fix this, thanks.

7

I’m a bit confused about what you’re trying to accomplish.

Your filter there will “catch” documents that either have yellow in e_name or in cards_user_name. So the second doc should match, but the first one shouldn’t. If you want to match both, I don’t see the point of the filter, you can just skip it. But that’s your original query, which doesn’t work for you.

8

I need the filter because then it would get irrelevant data that doesn’t match either.

For example say if I had

So I have this data in user_info
{“e_name” : “hello”, “user_id” : 1}

then in user_auth_cards_info
{“cards_user_name” : “dev”, “user_id” : 1}

and searched for test

then I would get:

1 → [{“e_name” : “hello”, “user_id” : 1}, {“cards_user_name” : “dev”, “user_id” : 1}]

which isn’t correct because none of them contain test, whereas the filter I’m trying to implement will have at least one matching

9

If you search for “test” (in a match query, for example), none of them should show up because they don’t contain the word “test”, no?

Also, where would that search go? In the “query” part of the request, correct?

10

Yeah query part and none would show up. But the problem is if one of them matches test it will only save that one whereas I’m trying to get it to save both. Just struggling to get it working