Route based on tags set in grok processor

Data Prepper
1

When grok parser cannot match event message it sets tag like this
tags_on_match_failure: [‘grok_parse_error’]

How later I can use this tag to route this failed event to opensearch sink which will create index named: failed_grok_cisco

I tried with simple:

.....
  route:
    - grok_parse_error: "hasTags(\"grok_parse_error\")"

 sink:
    - opensearch:
        routes:
          - grok_parse_error
        hosts: [ "{{ os_nodes_datapreper }}" ]
        insecure: true
        username: "admin"
        password: "{{ admin_password }}"
        index: failed_grok_cisco-%{yyyy.MM.dd}
        dlq_file: "{{ data_prep_home }}/pipelines/dlq_cisco-grok-failed.err"

but even when malformed event is sent, the index is not created. Looks like the routing is not working.

What am I missing?

I guess a lot, as above is naive approach. How can I store the whole event (original, before grok) in the failed_grok_cisco index? I need a dedicated subpipeline for this and do something extra?

2

Stupid mistake, I put two sink: in pipeline, after fixing this the index was populated.