OpenTelemetry integration for all signals (log,metrics and traces)

Data Prepper
#1

Hello,

We are using the OpenSearch stack and we would adopt the OpenTelemetry standard in order to handle all Observability concerns: metrics, traces and logs.

At the moment, we have successfully configured the OpenTelemetry collector in order to export Traces to Dataprepper.

About Logs, I know that actually, I could use Fluentbit + Dataprepper, but I would stay as simple as possible and avoid using other collectors (like Fluentbit)

What about metrics and (direct) logs?
Will DataPrepper handle also these signals? If yes, what is the Roadmap?
Is it right that, in the future, OpenSearch will natively support OpenTelemetry standards without using DataPrepper?

Thank you

#2

Hello @ziomill,

Thank you for raising this question. First, a community member has submitted a contribution to support metrics ingestion. It is available in this PR: Support OpenTelemetry Metrics by kmssap · Pull Request #1154 · opensearch-project/data-prepper · GitHub. We expect to release it in Data Prepper 1.4.

Regarding logs, we currently do only support ingest over HTTP using the Fluentbit format. We do expect to support OTel logs in the future, but have no concrete plans right not. The OTel Logs model is still experimental unlike trace and metrics which are more stable. While we do not have current plans to supports logs, we are happy to take community contributions similar to the metrics PR.

#3

Hello @dlv
For now, I will take a look at the metrics contribution.
Thank you!

#4

Do we have any sample or document for enabling metrics?

I’ve tried the below snip part of my existing trace pipeline. I think it’s not enabled metrics source… the default 21891 port is not able to telnet. another side my otel agent unable to push the metrics

metrics-pipeline:
source:
otel_metrics_source:
processor:
- otel_metrics_raw_processor:
sink:
- opensearch:
hosts: [ “http://10.81.211.22:9200” ]
username: “”
password: “”

Thanks!

#5

Hi @arulselvanj,

Could you share your otel config as well as your environment setup (is otel or data prepper or both running in Docker, etc). I am able to ingest metric data with the following sample configs (with otel collector running in Docker and Data Prepper running locally not in Docker)

Otel config to receive metrics from some application with prometheus and sending to the otel_metrics_source running on port 21891

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: ["sample_application"]

exporters:
  otlp/2:
    endpoint: host.docker.internal:21891
    insecure: true
  logging:

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      exporters: [logging, otlp/2]

Sample Data Prepper pipeline configuration from otel_metrics_source to OpenSearch running locally

metricpipeline:
  source:
    otel_metrics_source:
      ssl: false
  processor:
    - otel_metrics_raw_processor:
  sink:
    - opensearch:
        hosts: [ "https://localhost:9200" ]
        index: metrics_index
        username: "******"
        password: "******"
#6

awesome thanks @graytaylor0 for your help, I missed the “index” property under the OpenSearch sink. After adding that I could able to receive metrics and push them to OpenSearch