How big is your environement? Is this just a lab setup or is this for production? Do you have access to virtual machines or perfer K8’s /Docker/EC2?
What I have done in my lab, with one Virtual machine is before sending any data I created my Index templates, then any ISM needed for the indices sets. Once data arrives the templates and ISM configurations will take care of number of shards, replica’s & any index sets needed. It saves time going that route.
Thanks @Gsmitt. Right now, it’s simply a lab with hopes of implementing in production. Production would need to follow more devops principles where I can
manage the configurations and reproduce in different environments. Helm seemed like a good fit to stand up Opensearch clusters using Opensearch charts but I wanted to be able to
configure cluster settings (CCR), templates, and policies as well. A push button approach if I am to keep using buzz words. So right now I’m settling for a few helm deployments. One for each tier of Opensearch storage and one for a container to make REST API calls to the cluster configuring templates and policies. The templates and policies are in a local git repo.
Understood, As for Helm and/or K8’s i havent play with them much so I wont be much help there.
It seams like a good approach to what you want to do thou. I mainly work off VM’s from Hyper-v and Proxmox.
Perhaps someone else here might have a better idea with the kind of setup you wish to have.
Sorry I cant be more help.
I am familiar with your problem. One approach I have heard from a colleague was to create a jenkins pipeline or ansible playbook that gets triggered by a git hook and syncs the settings, templates, policies, etc. So something similar to your container.
The problem obviously is that you have to implement the logic yourself, so this is kinda a big oversight imo from ES and OpenSearch.
Though it really depends on your use case. If you treat your instance like your production database, this isn’t an issue really, as you will (hopefully) never lose data. If however you operate in a more micro-service type environment, where you want to recreate a specific config regularly from scratch or maybe have multiple users who want to manage their own templates, but don’t want to give them admin access, then this really is an issue.
If you’re in k8s, readiness probes come to mind (Configure Liveness, Readiness and Startup Probes | Kubernetes). Before an OpenSearch node becomes ready, it can check whether your index templates, ISM policies and maybe other cluster settings are already there. If not, put them.
Thanks @mgelszinnis. Ansible is a good call using their URI module in a play. Gives me idempotence which is the word I should have initially used describing this Opensearch configuration management dilemma. However, Ansible will not work in production in the long run so I’m still on the lookout. Appreciated nonetheless.