Otel metrics handling: field naming (@ in field names)

Data Prepper
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch and OpenSearch Dashboards 2.7.0,
DataPrepper 2.2.1
and OTEL Collector contrib 0.75.0.

Describe the issue:
Data Prepper plugin otel-metrics-raw-processor is modifying incoming field names.
In file
\data-prepper-plugins\otel-metrics-raw-processor\src\main\java\org\opensearch\dataprepper\plugins\processor\otelmetrics\OTelMetricsProtoHelper.java

one can see that dots are replaced by ‘@’ or that field names are prefixed with resource.attributes for example.

For example: in OpenSearch I will find

"resource.attributes.service@name": "cartservice"

If I compare this to Loki (where I have sent logs from OTEL Collector) it looks like this

  "resources": {
...
    "service.name": "cartservice",
...
}

(In Loki it seems also not OTEL compliant, OTEL calls the field resource.)

I am little bit confused and would like to understand the intention behind using ‘@’ and the specific prefix handling.

Ideally I have expected something like this

  "resource": {
...
    "service.name": "cartservice",
...
}

Configuration:

  processor:
    - otel_metrics_raw_processor:
        calculate_histogram_buckets: true
        calculate_exponential_histogram_buckets: true
        exponential_histogram_max_allowed_scale: 10
        flatten_attributes: true

Relevant Logs or Screenshots:

2

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.