AWS Opensearch slow performance

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): AWS Opensearch 2.7

Describe the issue:

Recently, we’re migrating ElasticSearch 7.3.10 self-hosted on EC2 to AWS OpenSearch 2.7 and meet some performance issue.

For example: During the peak hour, our ElasticSearch can index the data ~5000 request/s, but with OpenSearch there is just arround ~1000 request/s

Configuration:

ElaticSearch is running in 2 regions, each region has 3 c5.2xlarge nodes

Opensearch is running in the same 2 regions, each region has 3 c5.2xlarge.search data nodes with cross-cluster replication

They are using the same EBS volume configuration with default gp3 500GB

They are using the same index, same size, refresh_interval is 60s with 1 primary and 2 replicas per cluster.

Is there any difference between ElasticSearch & OpenSearch which causing the performance issue with the same instance type. And please help to advice what I can do to optimize the OpenSearch cluster?

Anyone please help

opensearch is not the same as elasticsearch - Elasticsearch vs. OpenSearch: Performance and resource utilization analysis | Elastic Blog
also CCR might give you some additional performance penalty.

do you have master nodes enabled?
check such cloudwatch merics like CPU, jvm mem pressure, ebs Throughput throttling

Hi,

During the peak hour, the CPU of OpenSearch cluster also high, ~85-90%.

Also, can you share detail about of limitation of CCR that affect to the performance.

And is there any way to tune the performance of OpenSearch that we can replace to use it?