Searching does not work in one of the index

OpenSearch Dashboards
1

(relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch version : 2.3.0
Opensearch Dashboard version : 2.3.0
Broswer : Chrome ( Tested with Mozilla, Brave, safari )

Describe the issue:

Searching was working good with out any issues but 2 days earlier it started to throw below exception

Bad Request

null_pointer_exception
Cannot invoke "org.opensearch.search.aggregations.InternalAggregations.getSerializedSize()" because "reducePhase.aggregations" is null

Error: Bad Request
at Fetch._callee3$ (https://hidden.net/4104/bundles/core/core.entry.js:15:584612)
at tryCatch (https://hidden.net/4104/bundles/plugin/queryWorkbenchDashboards/queryWorkbenchDashboards.plugin.js:2:2179)
at Generator._invoke (https://hidden.net/4104/bundles/plugin/queryWorkbenchDashboards/queryWorkbenchDashboards.plugin.js:2:1802)
at Generator.next (https://hidden.net/4104/bundles/plugin/queryWorkbenchDashboards/queryWorkbenchDashboards.plugin.js:2:2954)
at fetch_asyncGeneratorStep (https://hidden.net/4104/bundles/core/core.entry.js:15:577704)
at _next (https://hidden.net/4104/bundles/core/core.entry.js:15:578020)

Attached screen shot below, can you please help me what the error meant

Thank you!

Configuration:

The search term is “objects”

Relevant Logs or Screenshots:

image
image886×550 49.4 KB

2

@raj1209 Could you check the free disk space on your data nodes?

3

I’m having the same problem, and I have plenty of free space on all nodes

4

Its not the disk space issue and I have enough or more free space.

This is the error I am getting in Dev Tool
{
“error” : {
“root_cause” : [
{
“type” : “query_shard_exception”,
“reason” : “failed to create query: field expansion matches too many fields, limit: 1024, got: 1276”,
“type” : “search_phase_execution_exception”,
“reason” : “”,

"caused_by" : {
  "type" : "null_pointer_exception",
  "reason" : "Cannot invoke \"org.opensearch.search.aggregations.InternalAggregations.getSerializedSize()\" because \"reducePhase.aggregations\" is null"
}

},

5

Search works when you search with field name but not when just the value

Example 1 :

type : “objects” - search works fine ( note : type is just one of the field inside logs )

Example 2 :

“objects” - This search does not work and throws the above exception.

Is it because search is going through/indexing the documents above the limit set ?
i came across the below 2 forums while searching for solution

Forum1 : Error since 7.4 upgrade - field expansion matches too many fields - Elasticsearch - Discuss the Elastic Stack

Forum 2 - Elastic Search: failed to create query: field expansion matches too many fields, limit: 1024, got: 1399 - myVesta

In forum it says to increase the count of the parameter in opensearch.yml file “indices.query.bool.max_clause_count: 4096”

I’m not sure if the above line works or safe to do
Can you please suggest on this.

Thank you

6

I solved the issue for me. the solution was to update the following setting on the indices in question:

PUT /index_pattern-*/_settings
{
	"index.max_docvalue_fields_search" : "200"
}

I figured this out by running the request as a curl, which gave me a useful error, that gave me the solution. I did this by going to to chrome dev tools, going to the discover page in opensearch, and selecting the offending index pattern. If you go to the network tab, you can see the request that’s used for the discover query. right click, and copy as curl. Run that in a terminal, and you might get a more helpful error.

7

@raj1209 According to OpenSearch documentation indices.query.bool.max_clause_count: has value 4096 assigned by default.

Searching all eligible fields could be a resource-intensive operation. The indices.query.bool.max_clause_count search setting defines the maximum value for the product of the number of fields and the number of terms that can be queried at one time. The default value for indices.query.bool.max_clause_count is 4,096.

However, in the code SearchModule.java the default is set to 1024

As per documentation, increasing the max value of that setting would impact the search performance.

8

So is it advised to search with field name instead of searching only value or messages
This does not happens in all the index though, I’m currently facing with one index now.
This is how it concludes or is there anything you advise on this.

Thank you