# Atomicity and error detection when calling IOpenSearchClient.BulkAsync()

**URL:** https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462
**Category:** OpenSearch Client Libraries
**Tags:** opensearch-net
**Created:** [October 27, 2023, 1:06am UTC](https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462 "2023-10-27T01:06:44Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![jyoung](https://avatars.discourse-cdn.com/v4/letter/j/e19b73/32.png) [@jyoung](https://forum.opensearch.org/u/jyoung)
#### Post date: [October 27, 2023, 1:06am UTC](https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462/1 "2023-10-27T01:06:44Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

Using version 1.4.0 of .NET `OpenSearch.Client` package.

**Describe the issue** :

I am using the `BulkAsync` method to index 1000 entries into OpenSearch index.

`var response = await _openSearchClient.BulkAsync(bulkRequest, tokenSource.Token);`

We are encountering an issue where the disk consumption exceeds 95%, which is causing indexing to fail. However, from what we see, the `response` value returned from `BulkAsync` doesn’t indicate any problem even when the disk consumption is above this threshold.

To detect this problem, we are planning to check whether the index is allowing writes by doing this:

```auto
var indexStats = _openSearchClient.Indices.Get(_index);
var isReadOnly = indexStats.Indices.First().Value.Settings.BlocksReadOnlyAllowDelete ?? false;

```

If we perform the above check before calling `BulkAsync`, and then only proceed `if (!isReadOnly)`, what guarantees does OpenSearch offer for the `BulkAsync` call succeeding (assuming no errors in the input data that we provide via `bulkRequest`)?

What about an edge case where disk consumption is sitting at 94.9% and the `BulkAsync` call will push disk consumption above the 95% threshold? Will it successfully commit the records, and only after put the index into read-only mode? Or is it possible that the data will be partially committed?

We need a reliable way to ensure that the OpenSearch index is always updated. False negatives are okay (e.g. if OpenSearch _does_ update the index, and we report that it failed) but false positives are not acceptable.

---

<div class="post-metadata">

### Author: ![xtansia](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/xtansia/32/4961_2.png) [@xtansia](https://forum.opensearch.org/u/xtansia)
#### Post date: [November 24, 2023, 1:51am UTC](https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462/2 "2023-11-24T01:51:30Z")

</div>

Hi @jyoung,

Are you checking `response.IsValid` to confirm success of the bulk call?

That should return false if there’s items that failed, and you can get more detailed information about the specific failed items with`response.ItemsWithErrors`.

However, if these are indicating no failures, but the documents are not being indexed, this would need to be asked/reported in the server/cluster specific forums/GitHub as in that instance it’s regarding server behaviour rather than the client itself.

---

<div class="post-metadata">

### Author: ![jyoung](https://avatars.discourse-cdn.com/v4/letter/j/e19b73/32.png) [@jyoung](https://forum.opensearch.org/u/jyoung)
#### Post date: [December 4, 2023, 11:30pm UTC](https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462/3 "2023-12-04T23:30:32Z")

</div>

Hi @xtansia, thanks for the reply.

Yes, I am checking `response.IsValid` and it’s returning `true` even in the scenario where it failed to index all of the provided documents. So there does appear to be a problem on the server side of OpenSearch.

I will follow up in another forum as you recommended.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)
#### Post date: [February 2, 2024, 11:30pm UTC](https://forum.opensearch.org/t/atomicity-and-error-detection-when-calling-iopensearchclient-bulkasync/16462/4 "2024-02-02T23:30:44Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
