How to use searchable snapshots effectively from ILM standpoint

[neographikal]

I have a cluster that ingests 6-10TB each month and would like to have 6 months worth of data. Keeping all of that data in the cluster is somewhat prohibative from a storage costs standpoint and from a startup time standpoint. Searchable snapshots is the ideal solution for this.

What is the best way to snapshot indexes older than let’s say 1 month, delete the index and restore from snapshot with the remote_snapshot storage type. Moving snapshots from 1 storage tier to another works fine, but I can’t find this option?

I can script this myself, but it would be better to use the default opensearch functionality.

[Gsmitt]

Hey @neographikal

If you wanted to keep snapshots for a long period of time, what I have done was create a repository on a /mnt/remote_ storage/opensearch_repo
then configure Snapshot policies using cron expression for Snapshot schedule.Just an idea.

[neographikal]

Yeah, that’s not the problem. I can do snapshots via the snapshot management, no problem. From what I’ve read the workflow for searchable snapshots should be:

create snapshot of index (fine)
delete index (fine)
restore snapshot with remote_storage: no clue at all

I can script it myself through the management api’s but I don’t think that’s the way to go, at least for the user friendly solution.

[Gsmitt]

hey @neographikal

@neographikal

restore snapshot with remote_storage: no clue at all

I assume you configure this? and is your remote_storage part of that configuration?

To use a shared file system as a snapshot repository, add it to opensearch.yml:

path.repo: ["/mnt/snapshots"]

Rest should be done throu the Web UI

EDIT:
Depending on your setup you could use mount.
Example:

mount -t nfs 192.168.0.42:/share/public/mnt/opensearch_repo

Then adjust you fstab file to mount on boot. dont forget permission, example here

[andrross]

Hey @neographikal, unfortunately ISM does not yet support searchable snapshots.

[neographikal]

Gmsitt, thanks for the suggestions but that’s not the problem. I’m afraid andross is right and the options are not there yet.