Atomicity and error detection when calling IOpenSearchClient.BulkAsync()

[jyoung]

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Using version 1.4.0 of .NET OpenSearch.Client package.

Describe the issue:

I am using the BulkAsync method to index 1000 entries into OpenSearch index.

var response = await _openSearchClient.BulkAsync(bulkRequest, tokenSource.Token);

We are encountering an issue where the disk consumption exceeds 95%, which is causing indexing to fail. However, from what we see, the response value returned from BulkAsync doesn’t indicate any problem even when the disk consumption is above this threshold.

To detect this problem, we are planning to check whether the index is allowing writes by doing this:

var indexStats = _openSearchClient.Indices.Get(_index);
var isReadOnly = indexStats.Indices.First().Value.Settings.BlocksReadOnlyAllowDelete ?? false;

If we perform the above check before calling BulkAsync, and then only proceed if (!isReadOnly), what guarantees does OpenSearch offer for the BulkAsync call succeeding (assuming no errors in the input data that we provide via bulkRequest)?

What about an edge case where disk consumption is sitting at 94.9% and the BulkAsync call will push disk consumption above the 95% threshold? Will it successfully commit the records, and only after put the index into read-only mode? Or is it possible that the data will be partially committed?

We need a reliable way to ensure that the OpenSearch index is always updated. False negatives are okay (e.g. if OpenSearch does update the index, and we report that it failed) but false positives are not acceptable.

[xtansia]

Hi @jyoung,

Are you checking response.IsValid to confirm success of the bulk call?

That should return false if there’s items that failed, and you can get more detailed information about the specific failed items withresponse.ItemsWithErrors.

However, if these are indicating no failures, but the documents are not being indexed, this would need to be asked/reported in the server/cluster specific forums/GitHub as in that instance it’s regarding server behaviour rather than the client itself.

[jyoung]

Hi @xtansia, thanks for the reply.

Yes, I am checking response.IsValid and it’s returning true even in the scenario where it failed to index all of the provided documents. So there does appear to be a problem on the server side of OpenSearch.

I will follow up in another forum as you recommended.