Unable to upgrade to 2.3

OpenSearch
, ,
1

Versions
Current version OpenSearch 1.3

Describe the issue:

Unable to upgrade from OpenSearch 1.3 to 2.3

Relevant Logs or Screenshots:

This is the error I am seeing

The following indices are not compatible with the target Elasticsearch version OS_2.3 - smartmeters. Please refer to our documentation Upgrading Amazon OpenSearch Service domains - Amazon OpenSearch Service for tips to address this issue

CleanShot 2023-11-15 at 17.59.31@2x
CleanShot 2023-11-15 at 17.59.31@2x970×312 34.2 KB

Upon checking these are the following requirements

  • The type parameter was removed from all OpenSearch API endpoints in version 2.0. For more information

  • If your domain contains any indexes (hot, UltraWarm, or cold) that were originally created in Elasticsearch 6.8, those indexes are not compatible with OpenSearch 2.3.

Before you upgrade to version 2.3, you must reindex the incompatible indexes. For incompatible UltraWarm or cold indexes, migrate them to hot storage, reindex the data, and then migrate them back to warm or cold storage. Alternately, you can delete the indexes if you no longer need them.

We are not using type parameter, but our index was created in Elasticsearch 6.x, what is the correct way to reindex?

Is it possible to reindex smartmeters without creating any new indexes?

Should I create new index like ?

PUT smartmeters_new
{
   "mappings":{
      "properties" : {
        "city" : {
          "type" : "keyword"
        },
        "state" : {
          "type" : "keyword"
        },
        "zipcode" : {
          "type" : "integer"
        },
      }
    },
    "settings":{
      "index" : {
        "number_of_shards" : "5",
        "number_of_replicas" : "0"
      }
    }
}

and then run

POST _reindex
{
   "source":{
      "index":"smartmeters"
   },
   "dest":{
      "index":"smartmeters_new"
   }
}

I tried this but I am getting timeouts so wanted to know if this is the correct approach.

Size of current index CleanShot 2023-11-15 at 18.08.23

Apologies in advance I was not able to find a solution, not sure if someone has posted similar query, apologies if this is a silly question.

2

@thrijith Have you tried wait_for_completion=false with _reindex API?
When you use that option OpenSearch Dashboards will return a task number as a result. Then you can monitor the task status.

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "security-auditlog-2023.11.15"
  }
  , "dest": {
    "index": "test2"
  }
}

i.e.

{
  "task": "adUwvZ06Rjav6v1Abkw0aQ:157051"
}

GET _tasks/adUwvZ06Rjav6v1Abkw0aQ:157051

{
  "completed": true,
  "task": {
    "node": "adUwvZ06Rjav6v1Abkw0aQ",
    "id": 157051,
    "type": "transport",
    "action": "indices:data/write/reindex",
    "status": {
      "total": 12,
      "updated": 0,
      "created": 12,
      "deleted": 0,
      "batches": 1,
      "version_conflicts": 0,
      "noops": 0,
      "retries": {
        "bulk": 0,
        "search": 0
      },
      "throttled_millis": 0,
      "requests_per_second": -1,
      "throttled_until_millis": 0
    },
    "description": "reindex from [security-auditlog-2023.11.15] to [test2]",
    "start_time_in_millis": 1700053189408,
    "running_time_in_nanos": 327735539,
    "cancellable": true,
    "cancelled": false,
    "headers": {
      "X-Opaque-Id": "251f33f7-e6f4-4c3f-bf98-8959c732ed71"
    }
  },
  "response": {
    "took": 321,
    "timed_out": false,
    "total": 12,
    "updated": 0,
    "created": 12,
    "deleted": 0,
    "batches": 1,
    "version_conflicts": 0,
    "noops": 0,
    "retries": {
      "bulk": 0,
      "search": 0
    },
    "throttled": "0s",
    "throttled_millis": 0,
    "requests_per_second": -1,
    "throttled_until": "0s",
    "throttled_until_millis": 0,
    "failures": []
  }
}
3

No I haven’t tried that but will do that and see how it goes, but based on Reindex - OpenSearch documentation it looks like wait_for_completion is default false?

image
image1920×1122 137 KB