Opensearch Query Very Slow

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 3.0

Describe the issue: Hello, my cluster is composed by 9 datanote with 12TB of Storage and 64 GB RAM and 3 master node, with 16 GB RAM.

At the moment my datanode have 30% allocation, around 2.5 TB of data.

When i try to run a query from Dev tool Opensearch Dashboard, the results are very slowly.
For example running GET _cat/allocation -> results after 87000ms . It is not normal.

I’have alreasy tuned jvm options for my opensearch node with half of the total RAM. (32 on datanodes ,8 on masters) and on the 3 kibana nodes i have node.option -> --max-old-space-size=4096

What can i do?

2

@abarocco is this a new cluster or was this running fine prior to recently?

You should check your GC logs to get more details.

You can also check thread_pools using the following command:

GET _cat/thread_pool?v

To be able to troubleshoot this better I would recommend enabling slow logs