Getting extra string in aggregations object as key like sterms# or date_range#

OpenSearch
, , , ,
1

OpenSearch 2.8.0

Hi team
I am using OpenSearch 2.8.0 version with OpenSearch rest high level client 2.8.0 and whenever perform search operations I am getting extra substrings attached to key of aggregations object in json response. for example sterms# and date_range#. i am attaching opensearch.yml and search response screenshots for your reference.

opensearch.yml

node.name: 192.168.0.199

network.host: 127.0.0.1
discovery.type: single-node

#use mmapfs insted of niofs
index.store.type: mmapfs

#limit the fielddata cache
indices.fielddata.cache.size: 40%

#Do not allow automatic index creation while inserting document
action.auto_create_index: false

#settings for REST
http.port: 31014
path:
logs: C:\opensearch_logs
data: C:\opensearch-data-2.8.0

#Disabling the Disk space checks to avoid flood stage disk watermark 95% exceeded errors
cluster.routing.allocation.disk.threshold_enabled: false

#try to lock the process address space into RAM, preventing any Opensearch memory from being swapped out.
#bootstrap.memory_lock: true

#Maximum number of aggregation buckets allowed in a single response. (Defaults to 10,000)
#This is a like a safety check against bad search requests which can degrade the system performance.
search.max_buckets: 100000

plugins.security.disabled: true

date_range_in_search_resposne
date_range_in_search_resposne1085×633 69.5 KB

Thanks

2

Could you please share the search request with aggregations? Thank you.

3

Hi Reta,

here is my search request query.

{
“size”: 0,
“query”: {
“bool”: {
“must”: [
{
“match_all”: {}
}
],
“filter”: {
“bool”: {
“must”: [
{
“exists”: {
“field”: “i18n.en”
}
},
{
“term”: {
“common.is_last_version_uVtqOXUi”: true
}
},
{
“term”: {
“common.is_visible_msgciepY”: true
}
},
{
“bool”: {}
}
],
“should”: [
{
“term”: {
“common.container_type_raw_gtlxxWEH”: “1”
}
},
{
“term”: {
“common.container_type_raw_gtlxxWEH”: “0”
}
}
]
}
}
}
},
“aggregations”: {
“checkin_date”: {
“date_range”: {
“field”: “common.checkin_date_RlpCtbMV”,
“ranges”: [
{
“key”: “TODAY”,
“from”: “now/d”,
“to”: “now+1d/d”
},
{
“key”: “YESTERDAY”,
“from”: “now-1d/d”,
“to”: “now/d”
},
{
“key”: “TOMORROW”,
“from”: “now+1d/d”,
“to”: “now+2d/d”
},
{
“key”: “THIS_WEEK”,
“from”: “now/w”,
“to”: “now+1w/w”
},
{
“key”: “LAST_WEEK”,
“from”: “now-1w/w”,
“to”: “now/w”
},
{
“key”: “LAST_7_DAYS”,
“from”: “now-7d/d”,
“to”: “now”
},
{
“key”: “NEXT_WEEK”,
“from”: “now+1w/w”,
“to”: “now+2w/w”
},
{
“key”: “THIS_MONTH”,
“from”: “now/M”,
“to”: “now+1M/M”
},
{
“key”: “LAST_MONTH”,
“from”: “now-1M/M”,
“to”: “now/M”
},
{
“key”: “LAST_30_DAYS”,
“from”: “now-30d/d”,
“to”: “now”
},
{
“key”: “NEXT_MONTH”,
“from”: “now+1M/M”,
“to”: “now+2M/M”
},
{
“key”: “THIS_YEAR”,
“from”: “now/y”,
“to”: “now+1y/y”
},
{
“key”: “LAST_YEAR”,
“from”: “now-1y/y”,
“to”: “now/y”
},
{
“key”: “NEXT_YEAR”,
“from”: “now+1y/y”,
“to”: “now+2y/y”
},
{
“key”: “OLDER”,
“to”: “now-1y/y”
},
{
“key”: “NEWER”,
“from”: “now+2y/y”
}
]
}
}
},
“_source”: ,
“script_fields”: {}
}

thanks

4

Thank you, do you use typed_keys with your search request like /_search?typed_keys?

5

yes we are using , when search request executes then internally it is calling /_search?typed_keys? as you can see in the logs attached below.

16:55:30.232 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager - Connection [id: http-outgoing-0][route: {}->http://127.0.0.1:31014] can be kept alive indefinitely
16:55:30.232 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 127.0.0.1:2349<->127.0.0.1:31014[ACTIVE][r:r]: Set timeout 0
16:55:30.232 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.PoolingNHt tpClientConnectionManager - Connection released: [id: http-outgoing-0][route: {}->h ttp://127.0.0.1:31014][total kept alive: 1; route allocated: 1 of 10; total allocated: 1 of 30]
16:55:30.232 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalIODispatch - http-outgoing-0 [ACTIVE] [content length: 2626; pos: 2626; completed: true]
16:55:30.235 [main] DEBUG org.opensearch.client.RestClient - request [POST http://127.0.0.1:31014/design/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true] returned [HTTP/1.1 200 OK]

Thanks

6

This is why you have names like that, see please documentation [1] (Elasticsearch but this part works the same in OpenSearch):

 	
The aggregation type, histogram, followed by a # separator and the aggregation’s name, my-agg-name.

[1] Aggregations | Elasticsearch Guide [8.11] | Elastic

7

Hi Reta,

i have gone through the above document that shared, but there is nothing related to disabling typed_key using rest high level client. Do we have any documentation or link that mention how to disabled typed_keys or how to set typed_keys=false in java rest high level client search request.

Thanks

8

Hey @maohammad.aamir ,

This is indeed not configurable at the moment and is always set by default:

        params.putParam(RestSearchAction.TYPED_KEYS_PARAM, "true");

Please open an issue on Github [1], it should be a simple fix.
Thank you.

[1] Issues · opensearch-project/OpenSearch · GitHub