I have established a stack using Open Telemetry, Data Prepper, Opensearch, and Opensearch Dashboard. The application has been automatically instrumented to send traces to the Open Telemetry collector, and the data is then ingested into DataPrepper. From DataPrepper, the trace data is sent to the Opensearch index. I have created pipelines for Raw Traces and Service Map in Data Prepper. Although the indexes are created successfully, no traces are displayed on the Opensearch dashboard, and I am receiving the error “No data matches the selected filter. Clear the filter and/or increase the time range to see more results.”
Data prepper Pipe Line configuration
entry-pipeline:
delay: “100”
source:
otel_trace_source:
port: 21890
ssl: false
buffer:
bounded_blocking:
buffer_size: 10240
batch_size: 160
sink:
- pipeline:
name: “raw-trace-pipeline”
- pipeline:
name: “service-map-pipeline”
raw-trace-pipeline:
source:
pipeline:
name: “entry-pipeline”
buffer:
bounded_blocking:
buffer_size: 10240
batch_size: 160
processor:
- otel_traces:
- otel_trace_group:
hosts: [ “https://:9200” ]
insecure: true
username: “admin”
password: “XXXXXX”
sink:
- opensearch:
hosts: [“https://:9200”]
insecure: true
username: “admin”
password: “XXXXXX”
index: “otel-v1-apm-span-%{yyyy.MM.dd}”
- file:
path: /tmp/otel_trace_data.log
service-map-pipeline:
delay: “100”
source:
pipeline:
name: “entry-pipeline”
buffer:
bounded_blocking:
buffer_size: 10240
batch_size: 160
processor:
- service_map:
sink:
- opensearch:
hosts: [“https://:9200”]
insecure: true
username: “admin”
password: “XXXXXX”
index: “otel-v1-apm-service-map-%{yyyy.MM.dd}”
- file:
path: /tmp/otel_map_data.log
I have upload the trace data that is written to a file.
Service Map data written to a file is below
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“profile”,“hashId”:“MN5V7aCeGRSmGYDW6mDnbA==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“login”,“hashId”:“0/0toqDtUgKPOqBEHSGqCA==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“index”,“hashId”:“d/jKCuu8DdAlr8qNesohYg==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“logout”,“hashId”:“xKlDNdgmej6q8JgBlod0sA==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“rregister”,“hashId”:“MuPW8vaDgZYN0qkcj7MzmA==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“register”,“hashId”:“7O5gfql75GR+uNr8+ZKeaw==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“mmenu”,“hashId”:“FnU2niTmnZejA6Pp7W5NTQ==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“ccreate”,“hashId”:“FCDKpI4YBUFZ41rFagpL5Q==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“restuarant”,“hashId”:“KKPR7BELgmG3zl389+dPCw==”}
{“serviceName”:“foodapp-nocode-instru”,“kind”:null,“destination”:null,“target”:null,“traceGroupName”:“rlogin”,“hashId”:“KaNpPe83IPpKCEAWJwzvpg==”}
