Migrating from ElasticSearch 7.10+ to OpenSearch 1.x

General Feedback
1

Hi,

What are our options for migrating from ElasticSearch 7.10+ to OpenSearch 1.x? What would be the quickest way to ingest say 1TB of data? For large clusters, this really is a pain point in moving them over to OpenSearch.

My understanding is that as it stands, the only possible way is to reindex the data as index compatibility with OpenSearch breaks after ElasticSearch 7.10.2. Is this correct?

Thanks,
James

2

Hey @james.hunter

I just did this with two node cluster, maybe my steps might help.

Disable shard replication in your Elasticsearch cluster:

curl -X PUT "http://hostame-of-an-OpenSearch-node:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{

  "transient" : {

     "cluster.routing.allocation.enable" : "primaries"

  }

}

'

Shutdown Elasticseacrch

sudo curl -SL https://artifacts.opensearch.org/releases/bundle/opensearch/1.x/opensearch-1.x.repo -o /etc/yum.repos.d/opensearch-1.x.repo

sudo yum clean all

sudo yum install opensearch-1.3.x

sudo rsync -avP /var/lib/elasticsearch/* /var/lib/opensearch/

a. elasticsearch.yml:

node.data: true 

node.master: false

b. opensearch.yml:

node.roles: ['data']

a. elasticsearch.yml:

node.data: false 

node.master: true

b. opensearch.yml:

node.roles: ['master']

opensearch.yml

action.auto_create_index: false

cluster.name: test-cluster1

discovery.seed_hosts: node2,node3

cluster.initial_master_nodes: node1,node2,node3

node.name: node1

path.data: /var/lib/opensearch

path.logs: /var/log/opensearch

plugins.security.disabled: false/true


curl -XGET 'http://hostame-of-an-OpenSearch-node:9200/_nodes/_all?pretty=true'

green and all nodes are using the new version, re-enable shard allocation:

curl -X PUT "http://hostame-of-an-OpenSearch-node:9200/_cluster/settings" -H 'Content-Type: application/json' -d'

{

  "transient" : {

     "cluster.routing.allocation.enable" : "all"

  }

}

'

That worked for myself. Not usre if you seen this here is the documentation for upgrading.

2 Likes
3

Which version of ElasticSearch did you try with? Compatibility breaks after 7.10.2, I don’t think those steps work with ElasticSearch 7.17 for example.

See under the support matrix on this blog

Moving from open source Elasticsearch to OpenSearch

Thanks

4

You said 7.10+, could you specify the exact version you’re running?
There are quite a few hostile changes made by Elastic Co. since about 7.13+, and moving from basic license in -7.13 depends mostly on whether or not you used features in the basic license or only features also found in the OSS version.

Officially however, support is only up until the latest Open Source version of Elasticsearch and no later.

1 Like
5

Hey,

Sorry I didnt realize you on 7.17, I was going off your statement:

1 Like
6

Apologies @Gsmitt and others for not being clearer about which version nor responding sooner.

Version - ElasticSearch 7.17

My question isn’t specific to ElasticSearch 7.17 though, it is more specifically for versions after the OpenSearch fork.

What are the migration paths from ElasticSearch 7.17 to OpenSearch 1.x?

Thanks

7

Hey @james.hunter

You may want to look at this link, I believe your Elasticsearch version is higher then what the documentation requires to upgrade to Openseasrch 1.x.

8

Thanks @Gsmitt I’ve read that bit of documentation a few times. Confusion comes from the use of 7.x to describe valid migration paths (i.e. rolling upgrade) as it assumes knowledge that rolling downgrades are not possible and doesn’t highlight compatibility breaks after 7.10.2 although I’m not sure exactly which versions and which paths it breaks. For example, I think 7.12 indices may be compatible with OpenSearch 1.x.

As indexing the data takes a lot of time, how does one migrate from ElasticSearch 7.17 to OpenSearch 1.3.9 without downtime?

9

Hey,

Total understand, but from Opensearch and/or Graylog. ES-OSS-7.10 you good for Opensearch 1.x.

i would run that in a dev lab see if it works before Production. Sorry man, I havent tried that yet. BUt i would be curious if you did get that to work. Sharing is caring :smiley: