# Benchmark Opensearch with ESRally **URL:** \ **Category:** OpenSearch\ **Created:** [November 22, 2021, 8:29am UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708 "2021-11-22T08:29:58Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![rickster](https://avatars.discourse-cdn.com/v4/letter/r/65b543/32.png) [@rickster](https://forum.opensearch.org/u/rickster)\ **Post date:** [November 22, 2021, 8:29am UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/1 "2021-11-22T08:29:58Z")
Hi, is it possible to benchmark my Opensearch instance with ESRally or what way would you do it? rickster ---
**Author:** ![reta](https://avatars.discourse-cdn.com/v4/letter/r/cdc98d/32.png) [@reta](https://forum.opensearch.org/u/reta)\ **Post date:** [November 22, 2021, 2:10pm UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/2 "2021-11-22T14:10:18Z")
Hi, Yes, it is possible, see please the [Upgrade from Elasticsearch OSS to OpenSearch - OpenSearch documentation](https://opensearch.org/docs/latest/upgrade-to/upgrade-to/#upgrade-to-opensearch) for compatibility notes, you will need to add the following setting to your `opensearch.yml`: ```auto compatibility.override_main_response_version: true ``` Beyond that, please note that some tracks (aka datasets) are using the mapping data types which are not supported by OpenSearch (fe [Text type family | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/text.html#match-only-text-field-type)), so you may not be able to run races across all tracks (but most of them just work). Hope it helps, thank you. ---
**Author:** ![rickster](https://avatars.discourse-cdn.com/v4/letter/r/65b543/32.png) [@rickster](https://forum.opensearch.org/u/rickster)\ **Post date:** [November 22, 2021, 2:35pm UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/3 "2021-11-22T14:35:07Z")
Thank you for your help. I will have a look on that. ---
**Author:** ![rickster](https://avatars.discourse-cdn.com/v4/letter/r/65b543/32.png) [@rickster](https://forum.opensearch.org/u/rickster)\ **Post date:** [November 25, 2021, 8:22am UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/4 "2021-11-25T08:22:55Z")
Okay after a bit of research I’m still confused. Is it right that ESRally is more for benchmarking troughput and latency? I’m asking because, my cluster has high CPU Usage during night, probably because of running some delete by queries. Currently I store document in indices by resource. Thats the reason why I have to delete documents which have reached retention with a delete by query. So I want to try to switch to a time based index straegy with an index per day or week, where I can then delete the whole index instead. Therefore I want to messure the cpu and memory between these two setups. Whats the best way to do this? ---
**Author:** ![reta](https://avatars.discourse-cdn.com/v4/letter/r/cdc98d/32.png) [@reta](https://forum.opensearch.org/u/reta)\ **Post date:** [November 25, 2021, 2:56pm UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/5 "2021-11-25T14:56:58Z")
That is correct (afaik): esrally intends to benchmark indexing throughput and search/aggregation latencies. It seems like in your case, you may need to outline your own goals: as far as I understand, your are focusing on server-side resource utilisation only. Not sure if benchmarking is what you need in this case (judging on the context you have provided). Logically, deleting whole index is significantly more lightweight operation that running delete by query. If you could spin off a cluster with this new indexing strategy side by side, you could compare the server-side resource consumption between both deployments. Not sure if it is very helpful. ---
**Author:** ![rickster](https://avatars.discourse-cdn.com/v4/letter/r/65b543/32.png) [@rickster](https://forum.opensearch.org/u/rickster)\ **Post date:** [November 25, 2021, 3:20pm UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/6 "2021-11-25T15:20:07Z")
Thanks for your quick response. You juged right on the context 😉 I will try to restructure my data to a time based index strategy on a separate cluster and compare the ressource usage of them using prometheus and grafana. ---
**Author:** ![fatcloud](https://avatars.discourse-cdn.com/v4/letter/f/ecd19e/32.png) [@fatcloud](https://forum.opensearch.org/u/fatcloud)\ **Post date:** [December 13, 2022, 9:42am UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/7 "2022-12-13T09:42:34Z")
Does this still work? I got: ```auto elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not Elasticsearch and we do not support this unknown product ``` error when try benchmarking OpenSearch 2.4, I already set ```auto compatibility.override_main_response_version: true ``` ---
**Author:** ![ralph](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/ralph/32/1194_2.png) [@ralph](https://forum.opensearch.org/u/ralph)\ **Post date:** [December 13, 2022, 2:20pm UTC](https://forum.opensearch.org/t/benchmark-opensearch-with-esrally/7708/8 "2022-12-13T14:20:48Z")
i think that there’s now a fork of ESRally available (but i haven’t used either one yet): > **[GitHub - opensearch-project/opensearch-benchmark: OpenSearch Benchmark - a...](https://github.com/opensearch-project/opensearch-benchmark)** > > OpenSearch Benchmark - a community driven, open source project to run performance tests for OpenSearch - GitHub - opensearch-project/opensearch-benchmark: OpenSearch Benchmark - a community driven,...