Hello
I am having issue with mapper_parsing. Error states that, ““reason”=>“object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value”}”. Because of this, alot of the logs aren’t being pushed to OpenSearch.
ERROR:
“2022-10-04 03:04:53 +0000 [error]: #0 [out_os] Could not push log to Elasticsearch: {“took”=>4, “errors”=>true, “items”=>[{“index”=>{”_index"=>“”, “_type”=>“_doc”, “_id”=>“hGn0oIMB7e53OWfmE5-p”, “status”=>400, “error”=>{“type”=>“mapper_parsing_exception”, “reason”=>“object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value”}}}, {“index”=>{“_index”=>“”, “_type”=>“_doc”, “_id”=>“hWn0oIMB7e53OWfmE5-p”, “status”=>400, “error”=>{“type”=>“mapper_parsing_exception”, “reason”=>“object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value”}}}, {“index”=>{“_index”=>“”, “_type”=>“_doc”, “_id”=>“hmn0oIMB7e53OWfmE5-p”, “status”=>400, “error”=>{“type”=>“mapper_parsing_exception”, “reason”=>“object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value”}}}]}"
I have tried couple of things.
1.tried to rename the key name from fluentd to “rename_key” → did not work
<filter kubernetes.var.log.containers.**.log>
@type rename_key
enable_ruby true
rename_rule1 cdn-proxy-(.+) cdn-proxy
rename_rule2 (\s.+) input
</filter>
- tried to remove all the key names conflicting with the “kubernetes.labels.app” like “kubernetes.labels.app.kubernetes.io/instance,kubernetes.labels.app.kubernetes.io/managed-by,kubernetes.labels.app.kubernetes.io/name…”. but, then all the logs from the application got stopped
<filter pattern>
@type record_modifier
# remove key1 and key2 keys from record
remove_keys key1,key2
</filter>
- From Dev tools, tried creating templates like this:
PUT _index_template/celia-template
{
"index_patterns": [
"index-pattern*"
],
"template": {
"mappings": {
"properties": {
"kubernetes.labels.app": {
"type": "object"
}
}
}
},
"priority": 200,
"version": 3,
"_meta": {
"description": "using component templates"
}
}
4.Tried to ingest pipe line:
PUT _ingest/pipeline/my_rename_pipeline
{
"description" : "describe pipeline",
"processors" : [
{
"rename": {
"field": "fieldCamelcase",
"target_field": "fieldCamelCase"
}
}
]
}
None of the options helped me. Could anyone please let me know, if there is any option for blocking certain objects(key-value pairs) or overcoming this issue for logs from kubernetes to OpenSearch via fluentd and get remaining part of the logs.
Thank you in advance