Reindexing consuming much time from upgrading from 1.3.0 to 2.7.0

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

openserach 1.3.1 —> 2.7.0 upgrade

Describe the issue:
I am upgrading opensearch from 1.3.0 to 2.7.0, I had logs of 10GB in pre-prod it took around 1hr downtime to reindex the logs before showing new logs on dashboard but the thing is now I have 600GB of logs in prod but can’t have 5 days downtime :smiling_face_with_tear:
I need some suggestions to upgrade prod opensearch with zero or may be 1-2 hr downtime only!!!

2

Hey @mouryasatyam

I have an idea but you may not like it.

Create a new instance OS 2.7.0, transfer the old server snapshot to the new OS server.

Once completed and you have all your Settings/Dashbaords, etc… then re-route all nodes to new Opensearch sever and leave the old one running until the log retention ends ( i.e, 30 days, 60 days, etc…). No down time down time, you still have access to old data. Just an idea

1 Like
3

Thankyou for the suggestion
I was thinking for the same way as I have log retention period of 7 days so just thinking of this strategy:

  1. Take snapshot of disk
  2. Install old version with the snapshot
  3. Delete all logstash indexes
  4. Upgrade the version as no logstash it would take lesser time
  5. Let it run for 7 days
  6. Then delete the old cluster and divert everything to new one
1 Like
4

I assume these are Virtual machines? if so are you using Proxmox or Hyper-v ?

EDIT With hyper-v you can just clone your VM and add new ip address, etc…

5

I am using kubernetes for openserach with aks can’t directly clone it I think!!

6

Oh gotcha, Unfortunately I dont use K8’s.

7

ok thanks for help!!
will figure out something to achieve this!!

1 Like