Query filters are not applied

OpenSearch
1

Versions: OS 3.2.0

Describe the issue: I want to use a python script to send multiple queries to OS. However, I get always the most recent log entry. The filter of the query is not applied.

My query is like this:

GET /syslog-*/_search
{
  "query": {
    "query_string": {
      "query": "syslog_program:suricata and fw_name: sws01.*********** and suricata.alert.signature:* and suricata.src_ip:* and suricata.dest_ip:10.10.* and not suricata.src_ip:10.10.*"
    }  
  },
  "size": 2,
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

I want to filter to syslog_program: suricata, but I get all syslog_program values. In case of the screenshot, it is “charon”. The usage of match is not possible, since I have several query strings I iterate. I do not get syntaxt errors, I guess it is more a logical error, but I do not get it.

Configuration: nothing relevant

Relevant Logs or Screenshots:

grafik
grafik1919×761 129 KB

2

@dennis_u can you provide the mappings of this index? Is the query being applied to a text field or keyword?

If you have the following mappings:

PUT syslog-demo
{
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },

      "syslog_program": { "type": "keyword" },
      "fw_name":        { "type": "keyword" },

      "suricata": {
        "properties": {
          "alert": {
            "properties": {
              "signature": { "type": "keyword" }
            }
          },
          "src_ip":  { "type": "ip" },
          "dest_ip": { "type": "ip" }
        }
      }
    }
  }
}

You can use this query:

GET syslog-demo/_search
{
  "size": 10,
  "sort": [{ "@timestamp": "desc" }],
  "query": {
    "query_string": {
      "query": "syslog_program:\"suricata\" AND fw_name:\"sws01.example\" AND _exists_:suricata.alert.signature AND _exists_:suricata.src_ip AND suricata.dest_ip:\"10.10.0.0/16\" AND NOT suricata.src_ip:\"10.10.0.0/16\""
    }
  }
}
3

Thx for your reply.

I thought the value assignment in logstash does this. But I read, this is independent.

I wanted to change the value in the index pattern, but is not possible:

grafik
grafik804×529 28.1 KB

Do I have to change it in the indexes? I create everyday a new one.

I am wondering that I can use OSD for searching, but not direct on OS level.

(P.S.: suricata creates several dest_ip fields. I am aware of it that I used suricata.alert.dest_ip in the opener)

Edit: Ah, the way to go is to create an ‘index template’ for (in my case) syslog-* and to put the mappings in there. Every new index has the new mappings, right?

4

@dennis_u yes, index template is the way to go if you want to ensure the mapping is consistent.

5

I guess space after fw_name: might cause a problem. /_validate/query?explain=true Validate query - OpenSearch Documentation might give clue what’s going on here.