# \[Q\] Filter too complex / Shards failed

**URL:** <https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072>\
**Category:** General Feedback\
**Created:** [July 12, 2023, 12:45pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072 "2023-07-12T12:45:22Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sebastian](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@Sebastian](https://forum.opensearch.org/u/Sebastian)\
**Post date:** [July 12, 2023, 12:45pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/1 "2023-07-12T12:45:22Z")

</div>

I’m new to this forum (& Open Search in general), so please go easy on me 🙂

**[Problem]**  
I’m trying to sort for some IPs but the Query DSL filter i normally use doesn’t work with this quantity of IPs (ca. 16000)

```auto
{
  "query": {
    "terms": {
      "ip.keyword": [
          "1.1.1.1",
          "1.1.1.2"
          etc...
      ]
    }
  }
}

```

**[Question]**  
Are there any workarounds to this problem?  
maybe a way to divide the filter into smaller bits?

**[Error messages]**

- too\_complex\_to\_determinize\_exception at `shard` `0`
- Type: too\_complex\_to\_determinize\_exception
- Reason: Determinizing automaton with 212586 states and 212585 transitions would require more than 10000 effort.
- Shards Failed: 1
- Shards Skipped: 14
- Shards Total: 15

any help will be appreciated, thanks 🙂

---

<div class="post-metadata">

**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\
**Post date:** [July 12, 2023, 10:15pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/2 "2023-07-12T22:15:37Z")

</div>

Hey @Sebastian

Using this documentation [here](https://opensearch.org/docs/latest/query-dsl/)  
Example:

```auto
GET winlogbeat-2023.07.12/_search
{
    "query": {
      "term": {
        "host.ip": "192.168.1.100,192.168.1.200"
     }
    }
  }

```

---

<div class="post-metadata">

**Author:** ![radu.gheorghe](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/radu.gheorghe/32/3516_2.png) [@radu.gheorghe](https://forum.opensearch.org/u/radu.gheorghe)\
**Post date:** [July 13, 2023, 2:14pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/3 "2023-07-13T14:14:03Z")

</div>

I’m confused, I don’t think the `terms` query uses automatons, this is used more for wildcard, fuzzy or regex queries. For `terms` there’s a `index.max_terms_count` that defaults to 65K which should be enough for what you need.

Are you sure that the error message that you’re describing (can you paste it here?) corresponds to this query and not a different one?

---

<div class="post-metadata">

**Author:** ![Sebastian](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@Sebastian](https://forum.opensearch.org/u/Sebastian)\
**Post date:** [July 17, 2023, 2:23pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/4 "2023-07-17T14:23:39Z")

</div>

Hi Radu, here is an image of the “show details” window

 ![Screenshot 2023-07-17 at 15.43.09](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/3be3f6f42c0a1b6671295ca787c4f9271ef3f294.png)

---

<div class="post-metadata">

**Author:** ![Sebastian](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@Sebastian](https://forum.opensearch.org/u/Sebastian)\
**Post date:** [July 22, 2023, 7:11pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/5 "2023-07-22T19:11:55Z")

</div>

> [@Sebastian](#):
>
> Hi Radu, here is an image of the “show details” window
> 
> ![Screenshot 2023-07-17 at 15.43.09](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/3be3f6f42c0a1b6671295ca787c4f9271ef3f294.png)

does this help?  
Thanks.

---

<div class="post-metadata">

**Author:** ![radu.gheorghe](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/radu.gheorghe/32/3516_2.png) [@radu.gheorghe](https://forum.opensearch.org/u/radu.gheorghe)\
**Post date:** [July 26, 2023, 12:53pm UTC](https://forum.opensearch.org/t/q-filter-too-complex-shards-failed/15072/6 "2023-07-26T12:53:56Z")

</div>

Can you somehow get the JSON request and response as well? I assume those two would be in the two other tabs?