# High increase in scrolls after change to HTTPS

**URL:** <https://forum.opensearch.org/t/high-increase-in-scrolls-after-change-to-https/17252>\
**Category:** OpenSearch\
**Tags:** discuss\
**Created:** [December 27, 2023, 12:11pm UTC](https://forum.opensearch.org/t/high-increase-in-scrolls-after-change-to-https/17252 "2023-12-27T12:11:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aurelian](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@aurelian](https://forum.opensearch.org/u/aurelian)\
**Post date:** [December 27, 2023, 12:11pm UTC](https://forum.opensearch.org/t/high-increase-in-scrolls-after-change-to-https/17252/1 "2023-12-27T12:11:52Z")

</div>

**Versions** :  
OpenSearch 2.9.0 cluster

**Describe the issue** :  
We have a service running in spark cluster that reads a lot of documents in a batch from OpenSearch cluster. Due to recent change of cluster to use _HTTPS_, we have observed high increase in scrolls which eventually breaches limit of 500 scrolls, stopping the service.  
Error:

> _Caused by: org.opensearch.hadoop.rest.OpenSearchHadoopInvalidRequest: org.opensearch.hadoop.rest.OpenSearchHadoopRemoteException: rejected\_execution\_exception: Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max\_open\_scroll\_context] setting._

This is not the case when cluster uses _HTTP_. It feels strange that _HTTPS_ can have impact on scroll number.

Method used for fetching data: `sparkContext.openSearchJsonRDD(resource, query, cfg)`

Query used: `{ "query": {"query_string": {"query": "someTime:[\"2023-01-01T01:27:00.334Z\" TO \"2023-01-01T01:30:00.334Z\"]"}}}`

One solution to this would be to increase scroll limit, but that impacts cluster performance.  
Does anyone know why is there increase in scrolls after HTTPS update? API issue?

Please let me know if you need more info to clarify the problem.

---

<div class="post-metadata">

**Author:** ![aurelian](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@aurelian](https://forum.opensearch.org/u/aurelian)\
**Post date:** [January 11, 2024, 1:23pm UTC](https://forum.opensearch.org/t/high-increase-in-scrolls-after-change-to-https/17252/2 "2024-01-11T13:23:19Z")

</div>

After some investigation, https was not the issue.  
_read_ role does not include scroll clearing permission, so the users were creating a lot of scrolls without clearing them.  
Now that permission has been added, no scroll issues have been observed.