Universal snapshot policy for all indexes

Plugins Index Management
1

Hi

Is there a way to create a snapshot policy which creates snapshot name depending on index it implemented to?

In details, the idea is to create an ism policy to create hourly snapsot and assign it to every index created in elsasticsearch cluster. It seems higly desirable that snapshot name SHOULD be the same as index name. So, it seems there should be a way to parametrize snapshot name field in the policy. If there any otpions to do that?

There is an example of snapshot creation policy. The “snapshot” action has the required parameter
“snapshot”, witch represents snapshot name. The actual snapshot is created using this parameter + %timestamp

{
“policy”: {
“policy_id”: “snapshot_at30_min”,
“description”: “get snapshot”,
“default_state”: “newdata”,
“states”: [
{
“name”: “newdata”,
“actions”: ,
“transitions”: [
{
“state_name”: “backup”,
“conditions”: {
“cron”: {
“cron”: {
“expression”: “30 * * * *”,
“timezone”: “America/Los_Angeles”
}
}
}
}
]
},
{
“name”: “backups”,
“actions”: [
{
“snapshot”: {
“repository”: “hourly_snapshot_repo”,
“snapshot”: “hourly_backup”
}
}
],
“transitions”: [
{
“state_name”: “newdata”
}
]
}
]
}
}

1 Like
2

At the moment, snapshot name is a required attribute in the policy and is needed to be set - ref

Having said that I believe we can update this requirement to default to index name if no name is provided. Let me circle back with you on this.

1 Like
3

any update on this?.
we are also facing a similar issue

1 Like
4

@curiousmind
I have a PR to enable this feature, after the PR is released users can specify the snapshot name as a script which will be resolved during runtime.
This enables naming created snapshot based on the index the policy is attached to, by defining policy as snapshot: {{ctx.index}}.

The created snapshots will continue to have the timestamp as suffix.

PR: Making snapshot name a scripted field by thalurur · Pull Request #425 · opendistro-for-elasticsearch/index-management · GitHub

1 Like
5

@thalurur
I wonder if a similar change would be useful in the ISMRollup’s targetIndex field. Currently it is not really useful to make a policy with an ISM Rollup if you intend to apply the policy to a group of indices like you might with daily log indices because the hard-coded target index simply overwrites the previous one.

6

Is it possible to create a policy that will be create a snapshots and automatically delete it if the snaphot is more than N-days old?
Something like ES:

  "retention": {                    
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }

Thanks for answer.
Version: opendistro-for-elasticsearch:1.13.1