Opensearch setting heirarchy (cluster vs index template vs SMP)

OpenSearch
,
1

Hey!

We’re trying to manage default settings for a fairly large migration via snapshots from our old ES cluster.

Ideally we’d like the old indexes to be stored on our cold nodes, as well as create a default policy for all indexes which puts them in cold storage unless explicitly specified via SMP.

New indexes / incoming data will have a hot/cold management pipeline which is pretty standard.

My question:

  1. How do the allocation settings work when both are specified? Does the index setting take precedence?
  • cluster.routing.allocation.include.<attribute>
  • index.routing.allocation.include.<attribute>

  1. If we set the allocation via lifecycle actions, does this overwrite the index
    template? i.e. the template specifies hot, but the lifecycle has an action after 30 days that moves the index to a cold node. EDIT: Can confirm that lifecycle overwrites templates. Templates appear to only be set once upon index initialization.

  2. I’ve figured out how to force snapshots to be restored onto cold nodes by adding index.routing.allocation.include.<attribute> to the index settings when restoring, but I can’t seem to apply index templates or lifecycle policies to a snapshot automatically based on index patterns. Is this just impossible, or is there a way to force snapshots to use a template / policy?

I’d love to hear any thoughts on whether there’s a better way to manage this.

Thanks for the help!

  • List item
3

Yes, the index level setting take precedence.

Is this just impossible, or is there a way to force snapshots to use a template / policy?

Do you mean when restoring a snapshot, index template and lifecycle policy can be applied to the newly restored indexes automatically? Both index template and the lifecycle policy in OpenSearch named ISM policy can define index patterns, they will take effect if the index name matches.

4

Hey, thanks for the response!

I can’t seem to get an ISM policy or Index Template to apply to restored snapshots.

i.e.

  • snapshot including index with name “test”
  • state management policy with index pattern applying to ‘test’
  • index template with index pattern applying to ‘test’

Restoring the snapshot does not apply either the template or state management policy to this index.

5

Something worth noting is that these snapshots were restored from an old version of ElasticSearch, and came with some bizarre behaviour around index settings. There were several index settings prefixed with ‘archived.*’ which prevented any changes to other settings until they were removed. I was able to remove these when restoring the snapshot by using ‘ignore index settings’, but templates still do not apply when restoring the snapshot.