I am really wondering if it is possible to use custom log sources with detectors. I tried configure just the minimal setup but fail.
Here is a draft of what I do:
Have logs sent to OpenSeach.
Those logs end up in an index ‘app-2024-04-03’ and so forth with an index pattern ‘app-*’
I have (and need) an index template to define fields and set a default ingest pipeline for processing.
I created a log type ‘custom-app’ (without further details regarding the log; maybe this is wrong, are there requirements?)
I created one rule for that log type.
When I now try to create one detector using that log type, that rule and the required mapping of the field the rule works on, it fails with “Invalid field mapping” and the debug reveals that the detector logic tries to create an index template conflicting with mine. I see no way of working without my own index template since it defines fields and parses the logs through several ingest pipelines. So, the crucial detail is having an index template. Without this, there is no issue.
I filed a bug report with further details of this behaviour:
What I am now interested in is if anybody is able to succeed with a scenario like this? Just some proof-of-concept that it is possible would help.