# Something broke with index templates in 2.5.0

**URL:** <https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276>\
**Category:** OpenSearch\
**Tags:** troubleshoot\
**Created:** [January 25, 2023, 4:18pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276 "2023-01-25T16:18:13Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![opoplawski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/opoplawski/32/338_2.png) [@opoplawski](https://forum.opensearch.org/u/opoplawski)\
**Post date:** [January 25, 2023, 4:18pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/1 "2023-01-25T16:18:13Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):  
opensearch-2.5.0-1.x86\_64

**Describe the issue** :

After the update to 2.5.0, winlogbeat index templates are not getting applied to new indexes.

**Configuration** :

I have an index template:

```auto
{
  "winlogbeat-7.12.1" : {
    "order" : 1,
    "index_patterns" : [
      "winlogbeat-7.12.1-*"
    ],
    "settings" : {
      "index" : {
        "codec" : "best_compression",
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        },
        "refresh_interval" : "5s",
        "number_of_shards" : "1",
        "max_docvalue_fields_search" : "200",
        "query" : {
          "default_field" : [
            "message",
            "tags",
            "agent.ephemeral_id",
...

```

That seems to no longer be applied.

**Relevant Logs or Screenshots** :

New index settings:

```auto
# curl -X GET "https://XXX:9200/winlogbeat-7.12.1-2023.01.25/_settings?pretty" 
{
  "winlogbeat-7.12.1-2023.01.25" : {
    "settings" : {
      "index" : {
        "creation_date" : "1674604801637",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "sXqVcTjUTqWfiugG-4uBoA",
        "version" : {
          "created" : "136267827"
        },
        "provided_name" : "winlogbeat-7.12.1-2023.01.25"
      }
    }
  }
}

```

Previous index setting:

```auto
# curl -X GET "https://XXX:9200/winlogbeat-7.12.1-2023.01.24/_settings?pretty" 
{
  "winlogbeat-7.12.1-2023.01.24" : {
    "settings" : {
      "index" : {
        "codec" : "best_compression",
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        },
        "refresh_interval" : "5s",
        "number_of_shards" : "1",
        "provided_name" : "winlogbeat-7.12.1-2023.01.24",
        "max_docvalue_fields_search" : "200",
        "query" : {
          "default_field" : [
            "message",
            "tags",
            "agent.ephemeral_id",
...

```

The issue seems to be that I have overlapping index template patterns:

```auto
winlogbeat-7.12.1 [winlogbeat-7.12.1-*] 1
.opensearch-sap-alias-mappings-index-template-winlogbeat- [winlogbeat-*] 0 [.opensearch-sap-alias-mappings-component-winlogbeat-]

```

and now:

```auto
[2023-01-24T09:15:29,837][WARN][o.o.c.m.MetadataIndexTemplateService] [os-sea01.nwra.com] index template [.opensearch-sap-alias-mappings-index-template-winlogbeat-] has index patterns [winlogbeat-*] matching patterns from existing older templates [winlogbeat-7.12.1] with patterns (winlogbeat-7.12.1 => [winlogbeat-7.12.1-*]); this template [.opensearch-sap-alias-mappings-index-template-winlogbeat-] will take precedence during new index creation

```

which seems to be new behavior.

If I try to delete that template though I get a 404 error:

```auto
# curl -X DELETE "https://XXX:9200/_template/.opensearch-sap-alias-mappings-index-template-winlogbeat-"
{"error":{"root_cause":[{"type":"index_template_missing_exception","reason":"index_template [.opensearch-sap-alias-mappings-index-template-winlogbeat-] missing"}],"type":"index_template_missing_exception","reason":"index_template [.opensearch-sap-alias-mappings-index-template-winlogbeat-] missing"},"status":404}

```

---

<div class="post-metadata">

**Author:** ![opoplawski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/opoplawski/32/338_2.png) [@opoplawski](https://forum.opensearch.org/u/opoplawski)\
**Post date:** [January 25, 2023, 4:28pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/2 "2023-01-25T16:28:10Z")

</div>

Ah, I had the call wrong, this worked:

```auto
DELETE /_index_template/.opensearch-sap-alias-mappings-index-template-winlogbeat-

```

I have no idea where the .opensearch-sap… index template came from. I also have:

```auto
.opensearch-sap-detectors-queries-index-template [.opensearch-sap-s3-detectors-queries*, .opensearch-sap-others_compliance-detectors-queries*, .opensearch-sap-others_application-detectors-queries*, .opensearch-sap-dns-detectors-queries*, .opensearch-sap-others_cloud-detectors-queries*, .opensearch-sap-others_web-detectors-queries*, .opensearch-sap-windows-detectors-queries*, .opensearch-sap-cloudtrail-detectors-queries*, .opensearch-sap-others_macos-detectors-queries*, .opensearch-sap-ad_ldap-detectors-queries*, .opensearch-sap-test_windows-detectors-queries*, .opensearch-sap-network-detectors-queries*, .opensearch-sap-apache_access-detectors-queries*, .opensearch-sap-linux-detectors-queries*, .opensearch-sap-others_apt-detectors-queries*, .opensearch-sap-others_proxy-detectors-queries*] 0                

```

Ah, I know now - this came from my attempt to create a security analytics detector with an index pattern of winlogbeat-\*. That was an unexpected effect.

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [January 26, 2023, 11:32pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/3 "2023-01-26T23:32:22Z")

</div>

@opoplawski starting from 2.5 when detector is created with **index pattern** , **alias** or **datastream** , index template with alias mappings (as component template) will be created, to maintain user-selected field aliases in future rollovers/additions of new indices in pattern/alias/datastream.

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [January 27, 2023, 10:33pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/4 "2023-01-27T22:33:21Z")

</div>

@pdz can you elaborate on this please?

I am trying to find some documentation on this process because creating detectors is failing for me.  
Probably due to using datastreams and overly complex WinlogBeat 8.5 index template.

Alex

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [January 28, 2023, 12:20am UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/5 "2023-01-28T00:20:04Z")

</div>

can you paste errors here please?

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [January 28, 2023, 8:36pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/6 "2023-01-28T20:36:12Z")

</div>

This is from Opensearch-Dashboards (sorry my container is still called Kibana as it’s shorter 🙂 ):

```auto
kibana | Security Analytics - DetectorsService - getDetector: StatusCodeError: [security_analytics_exception] Failed applying mappings to index
kibana | at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
kibana | at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
kibana | at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
kibana | at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
kibana | at IncomingMessage.emit (events.js:412:35)
kibana | at IncomingMessage.emit (domain.js:475:12)
kibana | at endReadableNT (internal/streams/readable.js:1333:12)
kibana | at processTicksAndRejections (internal/process/task_queues.js:82:21) {
kibana | status: 500,
kibana | displayName: 'InternalServerError',
kibana | path: '/_plugins/_security_analytics/mappings',
kibana | query: {},
kibana | body: {
kibana | error: {
kibana | root_cause: [Array],
kibana | type: 'security_analytics_exception',
kibana | reason: 'Failed applying mappings to index',
kibana | caused_by: [Object]
kibana | },
kibana | status: 500
kibana | },
kibana | statusCode: 500,
kibana | response: '{"error":{"root_cause":[{"type":"security_analytics_exception","reason":"Failed applying mappings to index"}],"type":"security_analytics_exception","reason":"Failed applying mappings to index","caused_by":{"type":"
illegal_argument_exception","reason":"Limit of total fields [2000] has been exceeded"}},"status":500}',
kibana | toString: [Function (anonymous)],
kibana | toJSON: [Function (anonymous)]
kibana | }
kibana | {"type":"response","@timestamp":"2023-01-28T20:20:20Z","tags":[],"pid":1,"method":"post","statusCode":200,"req":{"url":"/_plugins/_security_analytics/mappings","method":"post","headers":{"host":"logging","user-agen
t":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0","accept":"*/*","accept-language":"de,en-US;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://logging/app/opensearch
_security_analytics_dashboards","content-type":"application/json","osd-version":"2.5.0","content-length":"243","origin":"https://logging","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-origin","te":"
trailers","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-for":"172.16.1.199","connection":"close"},"remoteAddress":"172.18.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/10
9.0","referer":"https://logging/app/opensearch_security_analytics_dashboards"},"res":{"statusCode":200,"responseTime":830,"contentLength":9},"message":"POST /_plugins/_security_analytics/mappings 200 830ms - 9.0B"}

```

This is from Opensearch:

```auto
[2023-01-28T20:20:20,799][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000018/Bwk7woWaRgSmBaiETS4M4g]
[2023-01-28T20:20:20,840][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000002/2dS3N0UJRs2xu2WMIdBAVg]
[2023-01-28T20:20:20,873][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000015/P3McMsnzTLerXNSMrwGVlw]
[2023-01-28T20:20:20,912][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000006/d1-jtlPJS8yjZIR-6YfHug]
[2023-01-28T20:20:20,943][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000001/xEHdG6DYR9ukD5kqZSMxmg]
[2023-01-28T20:20:20,976][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000010/ncZT3zNpROWpnIy81ouV3Q]
[2023-01-28T20:20:21,010][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000011/3LVjz60KSKyqzWTZDLhdCw]
[2023-01-28T20:20:21,046][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000005/wNIp48dyTOCeiYIeQORemA]
[2023-01-28T20:20:21,076][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000007/gZH7-DsIRveiCdOBaZwT9w]
[2023-01-28T20:20:21,108][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000009/vWPMsT55Svi2VQtwLdrXrA]
[2023-01-28T20:20:21,139][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000013/Z5uOmashQ7W20fCxvooRlg]
[2023-01-28T20:20:21,171][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000019/Kj3d1VFWRTK7E1fP6pxh-A]
[2023-01-28T20:20:21,206][INFO][o.o.c.m.MetadataMappingService] [opensearch] [.ds-winlogbeat-000019/Kj3d1VFWRTK7E1fP6pxh-A] update_mapping [_doc]
[2023-01-28T20:20:21,208][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000003/B96TiycDR3aWyPFx2RMsqw]
[2023-01-28T20:20:21,248][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000008/5FdbCqgBSL2mCQUO8-a83A]
[2023-01-28T20:20:21,285][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000012/i4U1ehmnT6qLQCPX_Yv6Ew]
[2023-01-28T20:20:21,320][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000014/K3SAMkqmRNKTtLlv1zkRnw]
[2023-01-28T20:20:21,356][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000017/hXlM3qUTR--JZYXisWvFvg]
[2023-01-28T20:20:21,391][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000004/Hc3xJXweQza5nVSL0UX1uQ]
[2023-01-28T20:20:21,422][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.ds-winlogbeat-000016/pzyx_bi4QtC5kZieqxJDMQ]
[2023-01-28T20:20:21,503][INFO][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch] Detected cluster change event for destination migration
[2023-01-28T20:20:21,503][INFO][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch] Reset destination migration process.
[2023-01-28T20:20:21,504][WARN][r.suppressed] [opensearch] path: /_plugins/_security_analytics/mappings, params: {}
org.opensearch.securityanalytics.util.SecurityAnalyticsException: Failed applying mappings to index
        at org.opensearch.securityanalytics.mapper.MapperService$2.onFailure(MapperService.java:128) [opensearch-security-analytics-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:78) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.securityanalytics.mapper.MapperService$3.onResponse(MapperService.java:215) [opensearch-security-analytics-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.securityanalytics.mapper.MapperService$3.onResponse(MapperService.java:207) [opensearch-security-analytics-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:113) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:107) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.action.support.RetryableAction$RetryingListener.onResponse(RetryableAction.java:181) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.action.ActionListener$2.onResponse(ActionListener.java:106) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.action.admin.indices.mapping.put.TransportPutMappingAction$1.onResponse(TransportPutMappingAction.java:180) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.action.admin.indices.mapping.put.TransportPutMappingAction$1.onResponse(TransportPutMappingAction.java:176) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.metadata.MetadataMappingService$1.onAllNodesAcked(MetadataMappingService.java:375) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService$SafeAckedClusterStateTaskListener.onAllNodesAcked(MasterService.java:717) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService$AckCountDownListener.finish(MasterService.java:852) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService$AckCountDownListener.onNodeAck(MasterService.java:843) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService$DelegatingAckListener.onNodeAck(MasterService.java:765) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.coordination.Coordinator$CoordinatorPublication$4$1.onSuccess(Coordinator.java:1709) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.ClusterApplierService$SafeClusterApplyListener.onSuccess(ClusterApplierService.java:657) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:494) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282) [opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245) [opensearch-2.5.0.jar:2.5.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
Caused by: java.lang.IllegalArgumentException: Limit of total fields [2000] has been exceeded
        at org.opensearch.index.mapper.MappingLookup.checkFieldLimit(MappingLookup.java:192) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.index.mapper.MappingLookup.checkLimits(MappingLookup.java:184) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.index.mapper.DocumentMapper.validate(DocumentMapper.java:329) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:491) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:447) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.index.mapper.MapperService.merge(MapperService.java:419) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.metadata.MetadataMappingService$PutMappingExecutor.applyRequest(MetadataMappingService.java:300) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.metadata.MetadataMappingService$PutMappingExecutor.execute(MetadataMappingService.java:244) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService.executeTasks(MasterService.java:867) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:424) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:295) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:206) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:190) ~[opensearch-2.5.0.jar:2.5.0]
        at org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:228) ~[opensearch-2.5.0.jar:2.5.0]
        ... 6 more

```

So I guess by using the current WinlogBeat ECS template, I get close to the 1000 fields limit of the index. And some change by the Security Analytics module is putting that mark even a lot higher - I already adjusted the index template to be 2000.

What’s bothersome about this is that Security Analytics is now setup in some way but I don’t yet have a detector I can query via API. Some jobs are already at work behind the scenes though:

```auto
[2023-01-28T20:26:29,571][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.opensearch-sap-ad_ldap-alerts/vV5Ch9CKR1eis7Spm72EwQ]
[2023-01-28T20:26:29,577][INFO][o.o.a.a.AlertIndices] [opensearch] Index mapping of .opensearch-sap-ad_ldap-alerts is updated
[2023-01-28T20:26:29,578][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.opensearch-sap-ad_ldap-alerts-history-2023.01.27-1/-9_iBWstRway72uKM4HjtQ]
[2023-01-28T20:26:29,584][INFO][o.o.a.a.AlertIndices] [opensearch] Index mapping of .opensearch-sap-ad_ldap-alerts-history-2023.01.27-1 is updated
[2023-01-28T20:26:29,594][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.opensearch-sap-ad_ldap-findings-2023.01.27-1/UJuKxD7DQgWt5fj4SbRqrA]
[2023-01-28T20:26:29,596][INFO][o.o.a.a.AlertIndices] [opensearch] Index mapping of .opensearch-sap-ad_ldap-findings-2023.01.27-1 is updated
[2023-01-28T20:26:29,610][INFO][o.o.p.PluginsService] [opensearch] PluginService:onIndexModule index:[.opensearch-sap-ad_ldap-detectors-queries-000001/pmYVYK5PTpSGKBic4sLTtA]
[2023-01-28T20:26:29,615][ERROR][o.o.a.u.AlertingException] [opensearch] Alerting error: java.lang.IllegalArgumentException: analyzer [rule_analyzer] has not been configured in mappings
[2023-01-28T20:26:29,616][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch] uncaught exception in thread [DefaultDispatcher-worker-2]
org.opensearch.alerting.util.AlertingException: analyzer [rule_analyzer] has not been configured in mappings
        at org.opensearch.alerting.util.AlertingException$Companion.wrap(AlertingException.kt:70) ~[opensearch-alerting-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.alerting.util.DocLevelMonitorQueries.updateQueryIndexMappings(DocLevelMonitorQueries.kt:359) ~[opensearch-alerting-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.alerting.util.DocLevelMonitorQueries.access$updateQueryIndexMappings(DocLevelMonitorQueries.kt:41) ~[opensearch-alerting-2.5.0.0.jar:2.5.0.0]
        at org.opensearch.alerting.util.DocLevelMonitorQueries$updateQueryIndexMappings$1.invokeSuspend(DocLevelMonitorQueries.kt) ~[opensearch-alerting-2.5.0.0.jar:2.5.0.0]
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
        at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285) [kotlinx-coroutines-core-1.1.1.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]
Caused by: java.lang.Exception: java.lang.IllegalArgumentException: analyzer [rule_analyzer] has not been configured in mappings
        ... 9 more
uncaught exception in thread [DefaultDispatcher-worker-2]
AlertingException[analyzer [rule_analyzer] has not been configured in mappings]; nested: Exception[java.lang.IllegalArgumentException: analyzer [rule_analyzer] has not been configured in mappings];
        at org.opensearch.alerting.util.AlertingException$Companion.wrap(AlertingException.kt:70)
        at org.opensearch.alerting.util.DocLevelMonitorQueries.updateQueryIndexMappings(DocLevelMonitorQueries.kt:359)
        at org.opensearch.alerting.util.DocLevelMonitorQueries.access$updateQueryIndexMappings(DocLevelMonitorQueries.kt:41)
        at org.opensearch.alerting.util.DocLevelMonitorQueries$updateQueryIndexMappings$1.invokeSuspend(DocLevelMonitorQueries.kt)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
Caused by: java.lang.Exception: java.lang.IllegalArgumentException: analyzer [rule_analyzer] has not been configured in mappings
        ... 9 more

```

When trying this first, the **.opensearch-sap-ad\_ldap-detectors-queries** index (and others) was rolling over every minute, which pretty soon brought my system to the 1000 shards limit ☹

This was the error I received before: [Security Analytics error when using Datastreams](https://forum.opensearch.org/t/security-analytics-error-when-using-datastreams/12304/1)

I’d be glad to provide any further information - this really rocks and I would love to start using it.

Alex

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [January 30, 2023, 1:55pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/7 "2023-01-30T13:55:01Z")

</div>

Unfortunately, there is a bug when source\_index has more mappings then default max number of mappings(1000). Until this is fixed in next version, you can resolve it by adding following index template:

```auto
PUT _index_template/opensearch-sap-query-index-temp
{
  "index_patterns": [
    ".opensearch-sap-s3-detectors-queries*",
    ".opensearch-sap-others_compliance-detectors-queries*",
    ".opensearch-sap-others_application-detectors-queries*",
    ".opensearch-sap-dns-detectors-queries*",
    ".opensearch-sap-others_cloud-detectors-queries*",
    ".opensearch-sap-others_web-detectors-queries*",
    ".opensearch-sap-windows-detectors-queries*",
    ".opensearch-sap-cloudtrail-detectors-queries*",
    ".opensearch-sap-others_macos-detectors-queries*",
    ".opensearch-sap-ad_ldap-detectors-queries*",
    ".opensearch-sap-network-detectors-queries*",
    ".opensearch-sap-apache_access-detectors-queries*",
    ".opensearch-sap-linux-detectors-queries*",
    ".opensearch-sap-others_apt-detectors-queries*",
    ".opensearch-sap-others_proxy-detectors-queries*"
  ],
  "template": {
    "settings": {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        },
        "analysis": {
          "analyzer": {
            "rule_analyzer": {
              "char_filter": [
                "rule_ws_filter"
              ],
              "tokenizer": "keyword"
            }
          },
          "char_filter": {
            "rule_ws_filter": {
              "pattern": "(_ws_)",
              "type": "pattern_replace",
              "replacement": " "
            }
          }
        },
        "hidden": "true"
      }
    },
    "mappings": {},
    "aliases": {}
  },
  "priority": 500
}

```

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [January 30, 2023, 4:44pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/8 "2023-01-30T16:44:44Z")

</div>

Thank you (also for providing the answer to my Datastream question)!

Now I get:

```auto
kibana | status: 400,
kibana | displayName: 'BadRequest',
kibana | path: '/_plugins/_security_analytics/mappings',
kibana | query: {},
kibana | body: {
kibana | error: {
kibana | root_cause: [Array],
kibana | type: 'illegal_argument_exception',
kibana | reason: 'composable template [winlogbeat] template after composition with component templates [.opensearch-sap-alias-mappings-component-winlogbeat] is invalid',
kibana | caused_by: [Object]
kibana | },

```

There is no further error in Opensearch ☹  
When I google this error I only get a reference to some change in the timestamp field for datastreams in ElasticSearch 8: [Component templates are considered invalid without timestamp\_field · Issue #58956 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/58956)

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [January 30, 2023, 5:48pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/9 "2023-01-30T17:48:53Z")

</div>

> [@adn77](#):
>
> `.opensearch-sap-alias-mappings-component-winlogbeat`

How is your datastream index template setup? Can please provide steps you used.

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [January 30, 2023, 9:47pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/10 "2023-01-30T21:47:45Z")

</div>

So much to learn - I didn’t realize that _\_template_ and _\_index\_template_ are two different things 🙂  
I checked and there are no more legacy templates (although I might have used them before).

I export the winlogbeat index template like this:

```auto
./winlogbeat-8.5.3.exe export template | \
 sed -E 's/winlogbeat-[^"]*/winlogbeat/g;s/flattened/keyword/g;s/constant_keyword/keyword/g;s/match_only_text/text/g;s/wildcard/keyword/g' | \
 jq 'del( .template.settings.index.lifecycle ) | .template.settings.index.default_pipeline |= "winlogbeat-routing"' \
 > winlogbeat_index-template.json

curl -X POST http://localhost:9200/_index_template/winlogbeat -H 'content-type: application/json' -d @winlogbeat_index-template.json

```

POSTing this to _\_index\_template/winlogbeat_ I get something like  
**GET \_index\_template/winlogbeat**

```auto
{
  "index_templates": [
    {
      "name": "winlogbeat",
      "index_template": {
        "index_patterns": [
          "winlogbeat"
        ],
        "template": {
          "settings": {
            "index": {
              "mapping": {
                "total_fields": {
                  "limit": "10000"
                }
              },
              "refresh_interval": "5s",
              "number_of_shards": "1",
              "max_docvalue_fields_search": "200",
              "query": {...},
              "default_pipeline": "winlogbeat-routing",
              "number_of_replicas": "0"
            }
          },
          "mappings": {...}
        },
        "composed_of": [],
        "priority": 150,
        "data_stream": {
          "timestamp_field": {
            "name": "@timestamp"
          }
        }
      }
    }
  ]
}

```

Index pattern and ingestion pipelines are exported from winlogbeat.exe as well.  
There is a manual ISM policy rolling over any datastream index daily.

Even if I add the composed index reference like this, the error remains the same:

```auto
  "composed_of": [".opensearch-sap-alias-mappings-component-winlogbeat"]

```

Thanks again for all your help!

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [January 30, 2023, 10:03pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/11 "2023-01-30T22:03:04Z")

</div>

When creating a detector, SAP should find existing template for your **datastream** and insert that alias mappings component template into **composed\_of** array.

Can you please do:

```auto
GET _component_template/.opensearch-sap-alias-mappings-component-winlogbeat

```

and paste results here?

If you don’t have any important data on SA plugin, I would recommend following:

1. delete all “.opensearch-sap\*” indices
2. Apply this template:

```auto
PUT _index_template/opensearch-sap-query-index-temp
{
  "index_patterns": [
    ".opensearch-sap-s3-detectors-queries*",
    ".opensearch-sap-others_compliance-detectors-queries*",
    ".opensearch-sap-others_application-detectors-queries*",
    ".opensearch-sap-dns-detectors-queries*",
    ".opensearch-sap-others_cloud-detectors-queries*",
    ".opensearch-sap-others_web-detectors-queries*",
    ".opensearch-sap-windows-detectors-queries*",
    ".opensearch-sap-cloudtrail-detectors-queries*",
    ".opensearch-sap-others_macos-detectors-queries*",
    ".opensearch-sap-ad_ldap-detectors-queries*",
    ".opensearch-sap-network-detectors-queries*",
    ".opensearch-sap-apache_access-detectors-queries*",
    ".opensearch-sap-linux-detectors-queries*",
    ".opensearch-sap-others_apt-detectors-queries*",
    ".opensearch-sap-others_proxy-detectors-queries*"
  ],
  "template": {
    "settings": {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        },
        "analysis": {
          "analyzer": {
            "rule_analyzer": {
              "char_filter": [
                "rule_ws_filter"
              ],
              "tokenizer": "keyword"
            }
          },
          "char_filter": {
            "rule_ws_filter": {
              "pattern": "(_ws_)",
              "type": "pattern_replace",
              "replacement": " "
            }
          }
        },
        "hidden": "true"
      }
    },
    "mappings": {},
    "aliases": {}
  },
  "priority": 500
}

```

1. try creating detector

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [January 30, 2023, 11:13pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/12 "2023-01-30T23:13:21Z")

</div>

> [@pdz](#):
>
> `GET _component_template/.opensearch-sap-alias-mappings-component-winlogbeat`

```auto
{
  "component_templates": [
    {
      "name": ".opensearch-sap-alias-mappings-component-winlogbeat",
      "component_template": {
        "template": {
          "mappings": {
            "properties": {
              "winlog.event_data.AuthenticationPackageName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-SubjectLogonId": {
                "path": "winlog.event_data.SubjectLogonId",
                "type": "alias"
              },
              "winlog.event_data.LogonId": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "server.user.hash": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-Description": {
                "path": "winlog.event_data.Description",
                "type": "alias"
              },
              "winlog-event_data-param1": {
                "path": "winlog.event_data.param1",
                "type": "alias"
              },
              "winlog.event_data.IpAddress": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-param2": {
                "path": "winlog.event_data.param2",
                "type": "alias"
              },
              "winlog-user-type": {
                "path": "winlog.user.type",
                "type": "alias"
              },
              "winlog.provider_name": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.Signed": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "host.hostname": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.computer_name": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-TargetUserName": {
                "path": "winlog.event_data.TargetUserName",
                "type": "alias"
              },
              "winlog.event_data.KeyLength": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "windows-message": {
                "path": "message",
                "type": "alias"
              },
              "winlog-event_data-Signed": {
                "path": "winlog.event_data.Signed",
                "type": "alias"
              },
              "winlog-computerObject-name": {
                "path": "winlog.computerObject.name",
                "type": "alias"
              },
              "winlog.keywords": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.DeviceName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-ScriptBlockText": {
                "path": "winlog.event_data.ScriptBlockText",
                "type": "alias"
              },
              "winlog-event_data-DeviceName": {
                "path": "winlog.event_data.DeviceName",
                "type": "alias"
              },
              "winlog-event_data-ProcessId": {
                "path": "winlog.event_data.ProcessId",
                "type": "alias"
              },
              "winlog.event_data.OriginalFileName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.Detail": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-LogonId": {
                "path": "winlog.event_data.LogonId",
                "type": "alias"
              },
              "winlog-event_data-AuthenticationPackageName": {
                "path": "winlog.event_data.AuthenticationPackageName",
                "type": "alias"
              },
              "winlog.event_data.Company": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_id": {
                "path": "winlog.event_id",
                "type": "alias"
              },
              "winlog.event_data.TargetUserName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-OriginalFileName": {
                "path": "winlog.event_data.OriginalFileName",
                "type": "alias"
              },
              "winlog.event_data.TargetServerName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.TargetUserSid": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-Product": {
                "path": "winlog.event_data.Product",
                "type": "alias"
              },
              "winlog.event_data.param1": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.param2": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-FileVersion": {
                "path": "winlog.event_data.FileVersion",
                "type": "alias"
              },
              "winlog-event_data-ServiceName": {
                "path": "winlog.event_data.ServiceName",
                "type": "alias"
              },
              "winlog-event_data-LogonProcessName": {
                "path": "winlog.event_data.LogonProcessName",
                "type": "alias"
              },
              "winlog-event_data-PrivilegeList": {
                "path": "winlog.event_data.PrivilegeList",
                "type": "alias"
              },
              "winlog.event_data.FileVersion": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-Status": {
                "path": "winlog.event_data.Status",
                "type": "alias"
              },
              "winlog-event_data-Detail": {
                "path": "winlog.event_data.Detail",
                "type": "alias"
              },
              "winlog-channel": {
                "path": "winlog.channel",
                "type": "alias"
              },
              "winlog.event_data.State": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.ServiceName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-State": {
                "path": "winlog.event_data.State",
                "type": "alias"
              },
              "winlog.event_data.SubjectUserSid": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.Description": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.computerObject.name": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.PrivilegeList": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-IpAddress": {
                "path": "winlog.event_data.IpAddress",
                "type": "alias"
              },
              "winlog-event_data-TargetLogonId": {
                "path": "winlog.event_data.TargetLogonId",
                "type": "alias"
              },
              "winlog-provider_name": {
                "path": "winlog.provider_name",
                "type": "alias"
              },
              "server-user-hash": {
                "path": "server.user.hash",
                "type": "alias"
              },
              "winlog-computer_name": {
                "path": "winlog.computer_name",
                "type": "alias"
              },
              "winlog-event_data-KeyLength": {
                "path": "winlog.event_data.KeyLength",
                "type": "alias"
              },
              "winlog-keywords": {
                "path": "winlog.keywords",
                "type": "alias"
              },
              "winlog.user.type": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.ScriptBlockText": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.Status": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.SubjectLogonId": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.TargetLogonId": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "host-hostname": {
                "path": "host.hostname",
                "type": "alias"
              },
              "winlog-task": {
                "path": "winlog.task",
                "type": "alias"
              },
              "winlog-event_data-TargetUserSid": {
                "path": "winlog.event_data.TargetUserSid",
                "type": "alias"
              },
              "winlog.event_data.ProcessId": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "timestamp": {
                "path": "@timestamp",
                "type": "alias"
              },
              "winlog-event_data-ProcessPath": {
                "path": "winlog.event_data.ProcessPath",
                "type": "alias"
              },
              "winlog.task": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-user-name": {
                "path": "winlog.user.name",
                "type": "alias"
              },
              "winlog.event_data.ProcessPath": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-SubjectUserName": {
                "path": "winlog.event_data.SubjectUserName",
                "type": "alias"
              },
              "winlog.event_data.SubjectUserName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.IntegrityLevel": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-LogonType": {
                "path": "winlog.event_data.LogonType",
                "type": "alias"
              },
              "winlog-event_data-SubjectUserSid": {
                "path": "winlog.event_data.SubjectUserSid",
                "type": "alias"
              },
              "winlog.user.name": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_id": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.LogonProcessName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "message": {
                "type": "text"
              },
              "winlog.event_data.Workstation": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "@timestamp": {
                "type": "date"
              },
              "winlog-event_data-IntegrityLevel": {
                "path": "winlog.event_data.IntegrityLevel",
                "type": "alias"
              },
              "winlog.event_data.LogonType": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.channel": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-SubjectDomainName": {
                "path": "winlog.event_data.SubjectDomainName",
                "type": "alias"
              },
              "winlog.event_data.Path": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-Path": {
                "path": "winlog.event_data.Path",
                "type": "alias"
              },
              "winlog.event_data.SubjectDomainName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-TargetServerName": {
                "path": "winlog.event_data.TargetServerName",
                "type": "alias"
              },
              "winlog-event_data-ProcessName": {
                "path": "winlog.event_data.ProcessName",
                "type": "alias"
              },
              "winlog.event_data.Product": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog.event_data.ProcessName": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              "winlog-event_data-Company": {
                "path": "winlog.event_data.Company",
                "type": "alias"
              },
              "winlog-event_data-Workstation": {
                "path": "winlog.event_data.Workstation",
                "type": "alias"
              }
            }
          }
        },
        "version": 0
      }
    }
  ]
}

```

I already performed steps 1.-3. before - just tried again now with the same error as before.  
While the component\_template with the alias mapping is created, it does not get added to the winlogbeat index template. As I posted above, it does not help either if I added it manually to the composed\_of array ☹

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [February 2, 2023, 11:41am UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/14 "2023-02-02T11:41:13Z")

</div>

Yea there is an issue with handling that template field inside index\_template.

Can you use concrete indices, until this is fixed? With concrete indices use index pattern: winlog\*

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [February 2, 2023, 3:55pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/15 "2023-02-02T15:55:15Z")

</div>

😃 I will have to…

Thank you for all the work! I am looking forward to having this work with data\_streams.

If there is anything I can do to find a fix for this, please let me know!

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [February 3, 2023, 10:22am UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/16 "2023-02-03T10:22:16Z")

</div>

I just tried that on a production workload.

- Started from scratch (fresh cluster)
- created **opensearch-sap-query-index-temp**
- modified winlogbeat index-template (not containing data\_stream references) - this worked on my test setup
- switched to index-ingestion in FluentD

I now have a winlogbeat-YYYY-MM-DD index and I am using **winlogbeat** \* as pattern for the detector.

Unfortunately the error still turns up:

```auto
  path: '/_plugins/_security_analytics/mappings',
  query: {},
  body: {
    error: {
      root_cause: [Array],
      type: 'illegal_argument_exception',
      reason: 'composable template [winlogbeat] template after composition with component templates [.opensearch-sap-alias-mappings-component-winlogbeat] is invalid',
      caused_by: [Object]
    },
    status: 400
  },

```

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [February 19, 2023, 11:01pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/17 "2023-02-19T23:01:32Z")

</div>

Oops, sorry I missed this message.

You have to create index template with component template (put it in composed\_of array). There is issue with template field in index templates in 2.5

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [February 20, 2023, 10:35pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/18 "2023-02-20T22:35:41Z")

</div>

My winlogbeat index template looks something like this now:

```auto
PUT _index_template/winlogbeat
{
        "index_patterns": [
          "winlogbeat*"
        ],
        "template": {
          "settings": {
            "index": {
              "mapping": {
                "total_fields": {
                  "limit": "10000"
                }
              },
              "refresh_interval": "5s",
              "number_of_shards": "1",
              "max_docvalue_fields_search": "200",
              "query": {
                "default_field": [...]
              },
              "default_pipeline": "winlogbeat-routing",
              "analysis": {
                "analyzer": {
                  "winlogbeat_powershell_script_analyzer": {
                    "pattern": """[\W&&[^-]]+""",
                    "type": "pattern"
                  }
                }
              },
              "number_of_replicas": "0"
            }
          },
          "mappings": { ... }
        },
        "composed_of": [".opensearch-sap-alias-mappings-component-winlogbeat"],
        "priority": 150
}

```

But the error remains the same ☹

I really appreciate all your work - if you tell me to shut up and wait for 2.6, I’ll do so 🙂  
Otherwise if there is something I can do please let me know!

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [February 20, 2023, 11:27pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/19 "2023-02-20T23:27:35Z")

</div>

You’re using “template” field in index template. What you can do instead is create Component Template, then add it to composed\_of array of index template. Take a look at this example: [link](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html)

---

<div class="post-metadata">

**Author:** ![adn77](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@adn77](https://forum.opensearch.org/u/adn77)\
**Post date:** [February 28, 2023, 4:21pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/20 "2023-02-28T16:21:23Z")

</div>

I moved the winlogbeat index\_template to a component\_template, switched back to data\_streams and tried again. I’ve been able to create a detector now, but I don’t know if it’s working until there’s an alert 😃

```auto
GET _index_template/winlogbeat
{
  "index_templates": [
    {
      "name": "winlogbeat",
      "index_template": {
        "index_patterns": [
          "winlogbeat*"
        ],
        "composed_of": [
          ".opensearch-sap-alias-mappings-component-winlogbeat",
          "winlogbeat"
        ],
        "priority": 150,
        "data_stream": {
          "timestamp_field": {
            "name": "@timestamp"
          }
        }
      }
    }
  ]
}

```

I am starting to realize now the issue lies with the mapping.  
What happens if the first component\_template defines

```auto
              "winlog.event_data.LogonId": {
                "ignore_above": 1024,
                "type": "keyword"
              }

```

and the second

```auto
              "winlog-event_data-LogonId": {
                "path": "winlog.event_data.LogonId",
                "type": "alias"
              }

```

How is the detector mapping generated anyways?

---

<div class="post-metadata">

**Author:** ![pdz](https://avatars.discourse-cdn.com/v4/letter/p/51bf81/32.png) [@pdz](https://forum.opensearch.org/u/pdz)\
**Post date:** [February 28, 2023, 4:54pm UTC](https://forum.opensearch.org/t/something-broke-with-index-templates-in-2-5-0/12276/21 "2023-02-28T16:54:18Z")

</div>

> [@adn77](#):
>
> `winlog-event_data-LogonId`

Detector is making percolate queries from parsed sigma rules. Field names used in these queries could be original field names used in Sigma rule, or if defined in mapping yml file, could be converted/mapped to new value. In your example you would need **winlog-event\_data-LogonId** field in your log index for rule which uses this field to work. That can be either alias or “normal” field. If your log index doesn’t have field named like this, you will get a chance to create an alias field with rule field name which points to some other field in index.(during detector create/update flow)