How to copy data from OpenSearch 1.3.13 to OpenSearch 2.13.0 in k8s env

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch 1.3.13 [k8s version: 2.21.x]
OpenSearch 2.13.0 [k8s version: 2.28.9]

Describe the issue:

I want to copy indices from OpenSearch 1.3.13 to OpenSearch 2.13.0.
Is there any work around for that?

I can see the data in the pvc [OpenSearch 1.3.13] after mounting the pvc by mounting it to a demo pod.

How ever I am not sure if manual copy of indices to new cluster inside the new pvc [Path=/usr/share/opensearch/data/nodes/0/indices] will display the old data.

I don’t have any special rules for indexing other than logstash-*
I am using fluent-d with OpenSearch.

Configuration:

Both the clusters are running in k8s hosted cluster

Relevant Logs or Screenshots:

N/A

The version 1.3.13 is compatible with 2.13.0, so you can copy the whole directory data from the node 1.3.13 to the empty node 2.13.0, then start the node 2.13.0, this behavior just like upgrade a node from 1.3.13 to 2.13.0, but I don’t think you can copy the segments files to a running 2.13.0 node because the cluster state is not consistent between the two nodes.

Thank you @gaobinlong for your reply.

I am able to import the data but time stamp is not retained I think.

For example, in my old pvc, time stamp for following indice is 02 May = _3vG8nkMSgOo909tcMj07g [42MB size]

But after importing this indice in to new pvc, the date is 23 May i.e. today’s.
If I try to search logs for 02 May from opensearch dashboard, I am getting zero results.

I restarted opensearch-dashboard after above activity and I notice following logs =

Disabling OpenSearch Security Dashboards Plugin
Removing securityDashboards...
Plugin removal complete
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"dataSourceManagement\" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]"}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"applicationConfig\" is disabled."}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"cspHandler\" is disabled."}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"dataSource\" is disabled."}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"cpu.cgroup.path.override\" is deprecated and has been replaced by \"ops.cGroupOverrides.cpuPath\""}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"cpuacct.cgroup.path.override\" is deprecated and has been replaced by \"ops.cGroupOverrides.cpuAcctPath\""}
{"type":"log","@timestamp":"2024-05-23T08:52:54Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"opensearch.requestHeadersWhitelist\" is deprecated and has been replaced by \"opensearch.requestHeadersAllowlist\""}
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
{"type":"log","@timestamp":"2024-05-23T08:52:55Z","tags":["info","plugins-system"],"pid":1,"message":"Setting up [51] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,securityAnalyticsDashboards,savedObjects,home,apmOss,dashboard,mlCommonsDashboards,assistantDashboards,visualizations,visTypeVega,visTypeTimeline,visTypeTable,visTypeMarkdown,visBuilder,visAugmenter,anomalyDetectionDashboards,alertingDashboards,tileMap,regionMap,customImportMapDashboards,inputControlVis,visualize,ganttChartDashboards,searchRelevanceDashboards,reportsDashboards,indexManagementDashboards,management,indexPatternManagement,advancedSettings,console,notificationsDashboards,dataExplorer,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,observabilityDashboards,queryWorkbenchDashboards,discover,savedObjectsManagement,bfetch]"}
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
{"type":"log","@timestamp":"2024-05-23T08:53:03Z","tags":["info","savedobjects-service"],"pid":1,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations..."}
{"type":"log","@timestamp":"2024-05-23T08:53:04Z","tags":["info","savedobjects-service"],"pid":1,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2024-05-23T08:53:05Z","tags":["info","savedobjects-service"],"pid":1,"message":"Detected mapping change in \"properties.homepage\""}
{"type":"log","@timestamp":"2024-05-23T08:53:05Z","tags":["info","savedobjects-service"],"pid":1,"message":"Creating index .kibana_3."}
{"type":"log","@timestamp":"2024-05-23T08:53:05Z","tags":["info","savedobjects-service"],"pid":1,"message":"Migrating .kibana_2 saved objects to .kibana_3"}
{"type":"log","@timestamp":"2024-05-23T08:53:06Z","tags":["info","savedobjects-service"],"pid":1,"message":"Pointing alias .kibana to .kibana_3."}
{"type":"log","@timestamp":"2024-05-23T08:53:06Z","tags":["info","savedobjects-service"],"pid":1,"message":"Finished in 1307ms."}
{"type":"log","@timestamp":"2024-05-23T08:53:07Z","tags":["warning","cross-compatibility-service"],"pid":1,"message":"Starting cross compatibility service"}
{"type":"log","@timestamp":"2024-05-23T08:53:07Z","tags":["info","plugins-system"],"pid":1,"message":"Starting [51] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,securityAnalyticsDashboards,savedObjects,home,apmOss,dashboard,mlCommonsDashboards,assistantDashboards,visualizations,visTypeVega,visTypeTimeline,visTypeTable,visTypeMarkdown,visBuilder,visAugmenter,anomalyDetectionDashboards,alertingDashboards,tileMap,regionMap,customImportMapDashboards,inputControlVis,visualize,ganttChartDashboards,searchRelevanceDashboards,reportsDashboards,indexManagementDashboards,management,indexPatternManagement,advancedSettings,console,notificationsDashboards,dataExplorer,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,observabilityDashboards,queryWorkbenchDashboards,discover,savedObjectsManagement,bfetch]"}
{"type":"log","@timestamp":"2024-05-23T08:53:11Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
{"type":"log","@timestamp":"2024-05-23T08:53:15Z","tags":["info","http","server","OpenSearchDashboards"],"pid":1,"message":"http server running at http://0.0.0.0:5601"}

Any ideas’s how to fix this issue?
Thanks.

Regards,
Amey.

I don’t think coping data directly can change the timestamp because the segments file on the disk is immutable, you can check the document ids between the two indices, if the copy process works well, each document in 2.13.0 should have same ID with the document in 1.3.13.