Pause cluster deployed by opensearch-k8s-operator

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 2.17.0
OpenSearch operator 2.6.0

Describe the issue:
I deploy opensearch cluster via GitHub - opensearch-project/opensearch-k8s-operator: OpenSearch Kubernetes Operator and I want to pause cluster for some period of time and then restart. By pause I mean to scale down all pods (nodepools) to zero and leave only PVC and then start them again returning to initial state. Something like this Restart or pause the cluster - Percona Operator for MongoDB functionality

The only idea for me is to manually scale down operator to zero and scale down corresponding statefulsets to zero and then scale up operator and let it return cluster to initial state via recovery feature.

Maybe there is better approach?

1 Like

@SlavaUtesinov I’m not aware of any automated process. I think it should be kept manual to keep the order of scale down and scale up process.

What is the use case for this scenario?
The only thing I would add to this process is regular snapshots in case of failure during the scale-up process.

Hi, @pablo. Use case, for example, when I am not going to use opensearch cluster for a long time and don’t want to waste kubernates resources: ram and CPU. So I want to kind of freeze it but with possibility to return it back to state right before freeze. At the same time persistence volumes are preserved so I don’t need to have a deal with snapshots at the first glance.

PS What do you think about my approach?

@SlavaUtesinov I’ve tested it on my side a couple of times and it worked whenever I was bringing it back. I’ve created a few indices and they were all there after the full restore of the cluster.
I would still advise snapshots before scaling down the environment. Unless you don’t care about the existing data.

But if main data or something else have been corrupted snapshots won’t help because they are incremental artifacts with respect to former and depend on them unlike full backups?

@SlavaUtesinov Snapshots are full backups of your indices and cluster state. The incremental part regards the way the subsequent backups are made.

Ok, thanks