CCR performance statistics for large clusters

Hi all,
With CCR we can achieve the replication with active-passive model , Do we have been any performance numbers for write heavy clusters?

Can we enable it for clusters with 200+ nodes with write throughput of 1000 or more docs per second?

Hi dinesh,

We have not run performance tests with 200 nodes clusters. All replication tasks are per node level and actually per shard level. So, if you’ve even distribution of limited number of shards per node, replication should run smooth.

1000 docs parallel ingestion is tested and as long as your CPU is within limits, replication guarantees 1 minute maximum lag. We’ve performance numbers on nodes with 128GB heap, if your CPU is around 60% replication runs smoothly within 1m lag.

If you’re enabling it to your production workload, please gradually verify the performance by starting with 1 index replication and then enabling in others.

Also can you explain a bit more about your requirement ?

  • 1000 docs is per node ? or cluster wide ?
  • What is the document size ?

Hi @amkhar ,
thanks for replying.

Having a perf test for fairly large dataset would be helpful to many customers .

My requirement is approx 1000 docs per node for cluster size of 100 to 200 nodes .
Document size would be around 1 KB.

RAM and CPU is not a concern , I can get high flavour machines for such clusters.

Let me know your thoughts.