ISM policy for Remote indices

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 3.6

Describe the issue: I want to change the policy attached to a remote or restored index and getting an exception. The index is in a failed state due to an invalid state “initial_state”. I tried to change the policy using DEV tools and I get a response “updated 1300 indices”. When I go back and look at one of those indices, it still has the old policy attached.

Configuration:
API to update/change the policy.

POST _plugins/_ism/change_policy/remote-90d-*
{
  "policy_id": "csg_default_cleanup_policy",
  "state": "intial_state"
}

I tried to remove the policy using this API.

POST _plugins/_ism/remove/remote-90d-.ds-logstash-csg-wkbv_web_vztime-000009

Relevant Logs or Screenshots -

This is the exception I am seeing for the remove API

{
  "error": {
    "root_cause": [
      {
        "type": "index_management_exception",
        "reason": "Failed to remove policy because ISM auto_manage setting update requests failed with exception:"
      }
    ],
    "type": "index_management_exception",
    "reason": "Failed to remove policy because ISM auto_manage setting update requests failed with exception:",
    "caused_by": {
      "type": "exception",
      "reason": "java.lang.Exception: Failed to remove policy because ISM auto_manage setting update requests failed with exception:"
    }
  },
  "status": 500
}

Thanks

Murali

2

Hi @muraliv,

I reproduced both behaviors on OpenSearch 3.6.0.
change_policy is asynchronous, so updated_indices only means the change was recorded, not that the new policy has already been applied. If the policy change is successfully applied, the index should then be managed by the new policy rather than the old one.
To check whether the change is still pending or has failed:

GET _plugins/_ism/explain/remote-90d-.ds-logstash-csg-wkbv_web_vztime-000009?show_policy=true

You may also want to double-check the state name in the request, as it is written as intial_state.

On OpenSearch 3.6.0, I was able to reproduce that _ism/remove cannot complete on a remote_snapshot index because the remove flow attempts to update the auto_manage setting, and that update fails on the read-only index.

3

Hi @muraliv,

As an additional update, I did some further testing in my OpenSearch 3.6.0 lab and found two possible workarounds, depending on what you are trying to achieve with _ism/remove:

  1. If the goal is to stop further ISM operations, changing the remote index to a policy with no actions or transitions successfully prevents further ISM actions.The index remains ISM-managed, so this does not remove the policy.
  2. If the goal is to have a completely unmanaged remote index, another option is to restore a new remote index from the source snapshot without attaching an ISM policy. I verified that the newly restored remote index remains unmanaged, provided that no matching ism_template attaches a policy automatically.

The second option does not remove ISM from the existing remote index in place; it creates a newly restored remote index that is unmanaged from the beginning.

4

Hi @Yanlin

Thank you, I will keep this solution in mind for the future.

Our goal was to remove the current cleanup policy and reattach the updated cleanup policy. We ended up deleting the remote index.

Thanks

Murali