Info about normal_transfer_queue_consumer Thread Pool in OpenSearch

OpenSearch
1

We are deploying OpenSearch in a Kubernetes environment with three pods — one each for the client, manager, and data roles.

Currently, we are not ingesting any data into OpenSearch, and there are no indices present in the system.

image

However, we’ve observed that a thread pool named normal_transfer_queue_consumer is active, with 24 threads running on each pod.

image

Here normal_transfer_queue_consumer is always active and it’s count is 24. Will there be any performance impact ? Any reason why the thread count is 24.

Below is the resource configuration used for opensearch pods

manager:
replicas: 1
resources:
limits:
#cpu: “1”
memory: “2Gi”
requests:
cpu: “500m”
memory: “1.5Gi”
java_opts: “-Xms1g -Xmx1g”

data:
replicas: 1
resources:
limits:
#cpu: “1”
memory: “4Gi”
requests:
cpu: “500m”
memory: “3Gi”
java_opts: “-Xms2g -Xmx2g”

client:
replicas: 1
resources:
limits:
#cpu: “1”
memory: “4Gi”
requests:
cpu: “500m”
memory: “3Gi”
java_opts: “-Xms2g -Xmx2g”

Is there any parameter to changes this number of thread normal_transfer_queue_consumer during startup?

  1. There is any parameter to changes this number of thread normal_transfer_queue_consumer during startup?

  2. If we reduce number of normal_transfer_queue_consumer threads or disable it (changes to 0), will it have any impact to the system?

Regards,
Varun S

2

@Varun_Srinivasa Take a look a this pool request. This option has been introduced to support the upload timeouts solution.