How to use searchable snapshots effectively from ILM standpoint

I have a cluster that ingests 6-10TB each month and would like to have 6 months worth of data. Keeping all of that data in the cluster is somewhat prohibative from a storage costs standpoint and from a startup time standpoint. Searchable snapshots is the ideal solution for this.

What is the best way to snapshot indexes older than let’s say 1 month, delete the index and restore from snapshot with the remote_snapshot storage type. Moving snapshots from 1 storage tier to another works fine, but I can’t find this option?

I can script this myself, but it would be better to use the default opensearch functionality.

Hey @neographikal

If you wanted to keep snapshots for a long period of time, what I have done was create a repository on a /mnt/remote_ storage/opensearch_repo
then configure Snapshot policies using cron expression for Snapshot schedule.Just an idea.

Yeah, that’s not the problem. I can do snapshots via the snapshot management, no problem. From what I’ve read the workflow for searchable snapshots should be:

create snapshot of index (fine)
delete index (fine)
restore snapshot with remote_storage: no clue at all

I can script it myself through the management api’s but I don’t think that’s the way to go, at least for the user friendly solution.

hey @neographikal

I assume you configure this? and is your remote_storage part of that configuration?

To use a shared file system as a snapshot repository, add it to opensearch.yml:

path.repo: ["/mnt/snapshots"]

Rest should be done throu the Web UI

Depending on your setup you could use mount.

mount -t nfs

Then adjust you fstab file to mount on boot. dont forget permission, example here

Hey @neographikal, unfortunately ISM does not yet support searchable snapshots.

Gmsitt, thanks for the suggestions but that’s not the problem. I’m afraid andross is right and the options are not there yet.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.