Tutorial how to load Google Cloud logs to OpenSearch

Data Prepper
1

I am looking for tutorial on how to import application logs from Google Cloud Run ( a container running rust application) into OpenSearch.

So far I have not been able to find an instruction or tutorial on how to do that.
Please help!

2

I’m not aware of any tutorials for your exact usecase. You could look into using Fluentbit to push your logs to Data Prepper and then to OpenSearch.

Here’s a guide on using Fluentbit with Data Prepper - data-prepper/log_analytics.md at main · opensearch-project/data-prepper · GitHub

I’m not too familiar with Cloud Run but I found this guide for setting up Fluentbit on Google Kubernetes - Customizing Fluent Bit for Google Kubernetes Engine logs  |  Google Cloud Platform Community.

I hope this helps!

3

Thank You travisbenedict!

Seems that setting up Fluentbit could achieve my goal.

Since my application is based on rust I am also considering opentelemetry-rust GitHub - open-telemetry/opentelemetry-rust: OpenTelemetry API and SDK for Rust as input to Data Prepper. That kind of combination could also do the trick but it is slightly different approach.