Circuit_breaker_triggered: True and real-time data upload

Plugins k-NN
1

Hi All!
If knn.memory.circuit_breaker.enabled: true and have more data than the amount of memory (to avoid node crash) then circuit_breaker_triggered sets “true”, and data loading stops. If knn.memory.circuit_breaker.enabled: false, then indexes take up all the memory and the machine breaks down. Is it possible to configure the cluster so that we can avoid a crash and still upload real-time data (except for the option to allocate a lot of memory and option and option to evict the index from memory (= set knn.cache.item.expiry.enabled: true))?

Thanks!

2

Hi @doc113

Is there any data that you no longer need in memory? For instance, will some of your data no longer need to be searched as new data is added in?

Jack