# OpenTelemetry integration for all signals (log,metrics and traces)

**URL:** https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872
**Category:** Data Prepper
**Tags:** feature-request
**Created:** [March 10, 2022, 9:18am UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872 "2022-03-10T09:18:59Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![ziomill](https://avatars.discourse-cdn.com/v4/letter/z/ea666f/32.png) [@ziomill](https://forum.opensearch.org/u/ziomill)
#### Post date: [March 10, 2022, 9:18am UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/1 "2022-03-10T09:18:59Z")

</div>

Hello,

We are using the **OpenSearch** stack and we would adopt the **OpenTelemetry** standard in order to handle all Observability concerns: metrics, traces and logs.

At the moment, we have successfully configured the OpenTelemetry collector in order to export **Traces** to Dataprepper.

About **Logs** , I know that actually, I could use **Fluentbit + Dataprepper** , but I would stay as simple as possible and avoid using other collectors (like Fluentbit)

What about metrics and (direct) logs?  
Will DataPrepper handle also these signals? If yes, what is the Roadmap?  
Is it right that, in the future, OpenSearch will natively support OpenTelemetry standards without using DataPrepper?

Thank you

---

<div class="post-metadata">

### Author: ![dlv](https://avatars.discourse-cdn.com/v4/letter/d/5f9b8f/32.png) [@dlv](https://forum.opensearch.org/u/dlv)
#### Post date: [March 14, 2022, 3:40pm UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/2 "2022-03-14T15:40:04Z")

</div>

Hello @ziomill,

Thank you for raising this question. First, a community member has submitted a contribution to support metrics ingestion. It is available in this PR: [Support OpenTelemetry Metrics by kmssap · Pull Request #1154 · opensearch-project/data-prepper · GitHub](https://github.com/opensearch-project/data-prepper/pull/1154). We expect to release it in Data Prepper 1.4.

Regarding logs, we currently do only support ingest over HTTP using the Fluentbit format. We do expect to support OTel logs in the future, but have no concrete plans right not. The OTel Logs model is still experimental unlike trace and metrics which are more stable. While we do not have current plans to supports logs, we are happy to take community contributions similar to the metrics PR.

---

<div class="post-metadata">

### Author: ![ziomill](https://avatars.discourse-cdn.com/v4/letter/z/ea666f/32.png) [@ziomill](https://forum.opensearch.org/u/ziomill)
#### Post date: [March 14, 2022, 6:12pm UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/3 "2022-03-14T18:12:53Z")

</div>

Hello @dlv  
For now, I will take a look at the metrics contribution.  
Thank you!

---

<div class="post-metadata">

### Author: ![arulselvanj](https://avatars.discourse-cdn.com/v4/letter/a/b4bc9f/32.png) [@arulselvanj](https://forum.opensearch.org/u/arulselvanj)
#### Post date: [August 1, 2022, 5:35pm UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/4 "2022-08-01T17:35:47Z")

</div>

Do we have any sample or document for enabling metrics?

I’ve tried the below snip part of my existing trace pipeline. I think it’s not enabled metrics source… the default 21891 port is not able to telnet. another side my otel agent unable to push the metrics

metrics-pipeline:  
source:  
otel\_metrics\_source:  
processor:  
- otel\_metrics\_raw\_processor:  
sink:  
- opensearch:  
hosts: [“[http://10.81.211.22:9200](http://10.81.211.22:9200)” ]  
username: “”  
password: “”

Thanks!

---

<div class="post-metadata">

### Author: ![graytaylor0](https://avatars.discourse-cdn.com/v4/letter/g/ecc23a/32.png) [@graytaylor0](https://forum.opensearch.org/u/graytaylor0)
#### Post date: [August 1, 2022, 9:17pm UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/5 "2022-08-01T21:17:39Z")

</div>

Hi @arulselvanj,

Could you share your otel config as well as your environment setup (is otel or data prepper or both running in Docker, etc). I am able to ingest metric data with the following sample configs (with otel collector running in Docker and Data Prepper running locally not in Docker)

Otel config to receive metrics from some application with prometheus and sending to the `otel_metrics_source` running on port 21891

```auto
receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: ["sample_application"]

exporters:
  otlp/2:
    endpoint: host.docker.internal:21891
    insecure: true
  logging:

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      exporters: [logging, otlp/2]

```

Sample Data Prepper pipeline configuration from `otel_metrics_source` to OpenSearch running locally

```auto
metricpipeline:
  source:
    otel_metrics_source:
      ssl: false
  processor:
    - otel_metrics_raw_processor:
  sink:
    - opensearch:
        hosts: ["https://localhost:9200"]
        index: metrics_index
        username: " ******"
        password: " ******"

```

---

<div class="post-metadata">

### Author: ![arulselvanj](https://avatars.discourse-cdn.com/v4/letter/a/b4bc9f/32.png) [@arulselvanj](https://forum.opensearch.org/u/arulselvanj)
#### Post date: [August 2, 2022, 12:59pm UTC](https://forum.opensearch.org/t/opentelemetry-integration-for-all-signals-log-metrics-and-traces/8872/6 "2022-08-02T12:59:27Z")

</div>

awesome thanks @graytaylor0 for your help, I missed the “index” property under the OpenSearch sink. After adding that I could able to receive metrics and push them to OpenSearch
