# Losing top documents when query reaches the \`terminate\_after\` limit

**URL:** <https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432>\
**Category:** OpenSearch\
**Tags:** troubleshoot\
**Created:** [October 25, 2023, 1:54pm UTC](https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432 "2023-10-25T13:54:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![eagle8046725](https://avatars.discourse-cdn.com/v4/letter/e/43a26b/32.png) [@eagle8046725](https://forum.opensearch.org/u/eagle8046725)\
**Post date:** [October 25, 2023, 1:54pm UTC](https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432/1 "2023-10-25T13:54:57Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):  
OpenSearch/Dashboard = 2.9.0  
OS = Windows 11

**Describe the issue** :  
I have indexed some documents containing nested documents and I perform queries with the .NET `OpenSearch.Client` to retrieve the results sorted by a document field.

What I notice is that, when the results are many, I don’t get back the most recent results (the ones that according to my sort should be at the top).  
From what I can see, this is related to the `terminate_after` setting.  
It appears to happen whenever I hit this limit, either after I set the limit in my query, or if I don’t set it and I hit some default internal limit of 10000 documents.

I have tried various things, like :

- Trying to change my sort field/order of my index, and make it align with my query sort, which was not possible since my documents contained nested documents.
- Trying to use the `.TrackTotalHits()` instead of the `.TerminateAfter()` method, but it didn’t help.
- Trying to set a document score dependent on the sort field, using the `.FieldValueFactor()` method. But that created other complications, that required me to restructure my query.

Does anyone have any idea why this happens and how to fix it?  
I am totally clueless.  
I don’t know if the last option (with the `FieldValueFactor`) could work, but I was hoping there is a better option, because what I have seen is that my filtering stopped working when I used it.

BTW, this is my code:

```auto

            var response = openSearchClient.Search<Document>(s => s
                .Index(documentsIndexName)
                .Query(q => q
                    //.FunctionScore(fs => fs
                    // .Functions(fu => fu
                    // .FieldValueFactor(fvq => fvq
                    // .Field(f => f.IdDocument)
                    // .Factor(4)
                    // .Modifier(FieldValueFactorModifier.None)
                    // .Filter(ff => ff.Bool(bq => bq
                    // //////// <------ When I move the contents of the Bool query below in here, filtering stops working.
                    // ))
                    // )
                    // )
                    //)
                    .Bool(bq => bq
                        .Must(mq =>
                        {
                            var query = GetUserAppliedFilters(mq );
                            return query;
                        })
                        .Filter(mq => {
                            var query = GetGroupAccessFilters(mq ) && GetUserAccessFilters(mq );
                            return query;
                        })
                        )
                    )
                    .Sort(sort => {
                        // Some sort logic here....
                    })
                    .TerminateAfter(maxNumberOfResultsToReturn)
                    //.TrackTotalHits(new TrackTotalHits(maxNumberOfResultsToReturn))
                    .TrackScores(false)
                    .From(request.CurrentPage * request.PageSize)
                    .Size(request.PageSize)
                    //.Explain(true)
                    .Source(false)
                );

```

**Configuration** :

**Relevant Logs or Screenshots** :

---

<div class="post-metadata">

**Author:** ![radu.gheorghe](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/radu.gheorghe/32/3516_2.png) [@radu.gheorghe](https://forum.opensearch.org/u/radu.gheorghe)\
**Post date:** [October 26, 2023, 2:28pm UTC](https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432/2 "2023-10-26T14:28:12Z")

</div>

I thought about index sorting, too - maybe there’s a way you can do without nested documents?

---

<div class="post-metadata">

**Author:** ![eagle8046725](https://avatars.discourse-cdn.com/v4/letter/e/43a26b/32.png) [@eagle8046725](https://forum.opensearch.org/u/eagle8046725)\
**Post date:** [October 26, 2023, 5:22pm UTC](https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432/3 "2023-10-26T17:22:35Z")

</div>

Unfortunately, no.  
I really need to use nested documents, and I need to query their fields using AND operators.

---

<div class="post-metadata">

**Author:** ![eagle8046725](https://avatars.discourse-cdn.com/v4/letter/e/43a26b/32.png) [@eagle8046725](https://forum.opensearch.org/u/eagle8046725)\
**Post date:** [November 3, 2023, 3:18pm UTC](https://forum.opensearch.org/t/losing-top-documents-when-query-reaches-the-terminate-after-limit/16432/4 "2023-11-03T15:18:04Z")

</div>

It turns out that using this works:

```auto
  //.TerminateAfter(maxNumberOfResultsToReturn) // Remove this!
  .TrackTotalHits(new TrackTotalHits(maxNumberOfResultsToReturn))

```

I had tried this in the past and didn’t seem to work, but it does now.  
Some weird build issue, most likely…

**Update:**

Actually, no. This was still missing results. This did help, but it is a bit horrible:

```auto
  .TrackTotalHits(new TrackTotalHits(1000000000/*a huge value here*/))

```