we decided to increase the RAM limits (kubernetes) and also increased indexing_pressure.memory.limit from 10% to 15%, it helped to fix the issue. But After some researching it seems like we did workaround but not really the fix? can anyone explain what is the core issues that can lead to memory pressure hit during relocations? we have default concurrency and other index operation settings.
@bakili if there was a lot of rebalancing happening during indexing this can occur by design. There are some configurations that are available to work around this, such as:
(lowering these from the default 2, to 1, would be a good start). More details can be found in the docs
It might also be a good idea to increase the refresh period for the busy index, using the following:
PUT my-hot-index/_settings
{ "index.refresh_interval": "30s" }
This will have a delay on visibility of course.
If you are experiencing a heavy index period, it would be a good idea to pause the rebalancing until the pressure has ceased, using cluster.routing.rebalance.enable configuration.
Also it would be useful to see if there are any underlying issues causing reallocations, using the following API:
thank you @Anthony , I wonder do you know the details or is there any guide explaining details like under what circumstances that memory pressure would be hit? like how it builds up what are other parameters affects that memory usage pressure other than cluster_concurrent_rebalance?