Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Rest Client 1.3.7
Java Client 1.0.0
opensearch-java 1.0.0 javadoc (org.opensearch.client)
Describe the issue:
I’m having trouble understanding the documentation for implementing a scroll request since I will be executing queries that exceed the maximum 10000 responses.
Currently, my code uses SearchResponse as a stopgap measure like so:
List query = new ArrayList<>();
query.add(new MatchQuery.Builder().field(“customerName”).query(FieldValue.of(name)).build()
._toQuery());
BoolQuery boolQuery = new BoolQuery.Builder().must(query).build();
SearchResponse response = client.search(s → s.index(“customer-index”).size(10000).query(boolQuery._toQuery()),
Customer.class);
client is an OpensearchClient
This query is easily expandable by adding more MatchQuerys to the Query List.
This is simply enough, but I am utterly loss on how I might accomplish the same thing with ScrollRequest and response beyond the following:
ScrollRequest request = new ScrollRequest.Builder().build();
ScrollResponse scrollResponse = client.scroll(request, Customer.class);
This has no query and no size. I know there is the scrollId() method.
Is the following the proper implementation?
String scrollId = response.scrollId();
ScrollRequest request = new ScrollRequest.Builder().scrollId(scrollId).build();
ScrollResponse scrollResponse = client.scroll(request, Customer.class);
List<Hit> scrollHits = scrollResponse.hits().hits();
while(scrollHits != null && scrollHits.size() > 0) {
for (Hit hit : scrollHits) {
Customer customer = hit.source();
customerList.add(customer);
}
request = new ScrollRequest.Builder().scrollId(scrollId).build();
scrollResponse = client.scroll(request, Customer.class);
scrollHits = scrollResponse.hits().hits();
}
ClearScrollRequest.Builder clearScrollRequestBuilder = new ClearScrollRequest.Builder();
clearScrollRequestBuilder.scrollId(scrollId);
client.clearScroll(clearScrollRequestBuilder.build());