Running OpeanSearch 2.11.1 as a search engine for Graylog logging tool, I have a query that sorts data by an integer field named “myField” after first grouping by a keyword field, and then shows the result in a table. If I run it for 6 hours worth of data, it runs fine. If I run it for one day, it errors out within a second, printing this:
Unable to perform search query: OpenSearch exception [type=illegal_argument_exception, reason=Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [myField] in order to load field data by uninverting the inverted index. Note that this can use significant memory.].
So it gives a rather misleading error message that the field is not well configured, when in reality it’s the volume of data that is to blame. Opensearch is running on 3 nodes, with each process having 4GB of heap space.
Can someone please suggest a solution here? Does it make sense to add “fielddata=true” to the keyword field or myField?