Hello everyone, I am quite new to logging and opensearch.I want the logs of the docker containers in my virtual machine and I want to use opensearch, but I couldn’t find much about this topic on the internet.can anyone help me with this topic? yml file, routing etc.
Solutions for this can vary - docker has logging drivers you can configure to ship the stdout logs from the docker socket to various things you’ll need something to ingest the logs then ship to opensearch in most cases Configure logging drivers | Docker Docs
There are a few solutions people have posted about where logs are written to a bind mount on the local fs and something like logstash reads and then ships the logs as well as things like logspout which has be stale for a while but still works.
Ultimately there isn’t a specific opensearch solution due to the fact that everyone runs docker in varying ways and applications may have some nuances specific to their setups.
Hope that helps.
First of all, thank you very much for your answer. You can find my yaml file below, but I still can’t access my logs, where am I going wrong.
version: ‘3’
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD} # Sets the demo admin user password when using demo configuration, required for OpenSearch 2.12 and higher
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- ‘5601’
environment:
OPENSEARCH_HOSTS: ‘[“https://opensearch-node1:9200”,“https://opensearch-node2:9200”]’
networks:
- opensearch-net
logstash:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin
container_name: logstash
restart: unless-stopped
ports:
- 5044:5044
volumes:
- /docker/app/logstash/pipeline/:/usr/share/logstash/pipeline/
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
Hey @bastus11,
how does your Logstash pipeline look and how do you send your Logs to your Logstash Container?
Another way instead of sending the Logs from your Docker Host to Logstash is to map the Docker Log Dir into your Logstash Container and let Logstash pick up the Logs Files.
Hi my docker-compose.yaml file is as follows
---
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD} # Sets the demo admin user password when using demo configuration, required for OpenSearch 2.12 and higher
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- '5601'
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
logstash:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin
container_name: logstash
restart: unless-stopped
ports:
- 5044:5044
volumes:
- ~/opens/pipeline:/usr/share/logstash/pipeline/
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
and my logstash configuration file is as follows
input {
beats {
port => 5044
}
}
filter {
}
output {
opensearch {
hosts => ["https://opensearch-node1:9200", "https://opensearch-node2:9200"]
index => "your_index_name"
}
}
But my logs are as follows
LOGSTASH
RUNNER
I don’t know what to do 
You are on a good way but there are a few things missing in your Logstash config to be functional.
First you of all you need to specify an user and a password.
You could do that with the Script securityadmin.sh but there is a easier way to create a Logstash User in the Dashboards under the Option “Security”.
You have specified that you want to use https but just to be sure add the Option ssl_enabled => true into your output definition.
Lastly if you haven’t imported the CA Certificate from OpenSearch you will need this Option as well: ssl_verification_mode => none
Hello I changed my compose and configuration file as follows
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node1 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_INSTALL_DEMO_CONFIG=true" # Prevents execution of bundled demo script which installs demo certificates and security configurations to OpenSearch
- "DISABLE_SECURITY_PLUGIN=true" # Disables Security plugin
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
ports:
- 9200:9200 # REST API
- 9600:9600 # Performance Analyzer
networks:
- opensearch-net # All of the containers will join the same Docker bridge network
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node2 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_INSTALL_DEMO_CONFIG=true" # Prevents execution of bundled demo script which installs demo certificates and security configurations to OpenSearch
- "DISABLE_SECURITY_PLUGIN=true" # Disables Security plugin
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data # Creates volume called opensearch-data2 and mounts it to the container
networks:
- opensearch-net # All of the containers will join the same Docker bridge network
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601 # Map host port 5601 to container port 5601
expose:
- "5601" # Expose port 5601 for web access to OpenSearch Dashboards
environment:
- 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200","http://opensearch-node2:9200"]'
- "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
networks:
- opensearch-net
logstash:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin
container_name: logstash
restart: unless-stopped
ports:
- 5044:5044
volumes:
- ~/opens/pipeline:/usr/share/logstash/pipeline/
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
input {
beats {
port => 5044
}
}
# Required - An input plugin to pass some data to OpenSearch - file or AP
filter {
# Can be empty
}
output {
# All of the following connection details
opensearch {
# Your node IP addresses from the Instaclustr Console
hosts => ["http://opensearch-node1:9200", "http://opensearch-node2:9200"]
# SSL enabled
ssl => false
ssl_certificate_verification => false
# Path to your cluster certificate .pem downloaded earlier
# The Logstash Username and Password created earlier
# The name of the index
index => "logstash"
}
}
And my logs are better now
But I still don’t see any logs on this screen. Am I looking in the wrong place?
That’s great that you don’t get any errors with http. We will get to the security settings after that.
You can check on Dev Tool with GET logstash/_search if there are any Documents in this Index.
If yes there is a Issue with displaying the Events in the Dashboard if no there is a Problem with your Logstash pipeline.
The output is as follows
{
“took”: 2,
“timed_out”: false,
“_shards”: {
“total”: 1,
“successful”: 1,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 0,
“relation”: “eq”
},
“max_score”: null,
“hits”:
}
}
Isn’t that what you call a pipeline?
input {
beats {
port => 5044
}
}
# Required - An input plugin to pass some data to OpenSearch - file or AP
filter {
# Can be empty
}
output {
# All of the following connection details
opensearch {
# Your node IP addresses from the Instaclustr Console
hosts => ["http://opensearch-node1:9200", "http://opensearch-node2:9200"]
# SSL enabled
ssl => false
ssl_certificate_verification => false
# Path to your cluster certificate .pem downloaded earlier
# The Logstash Username and Password created earlier
# The name of the index
index => "logstash"
}
}
So it seems that Logstash doesn’t receive any Data.
How do you send your Events from your Host to Logstash?
You have the Beats Input plugin configured so i guess you use filebeat, are you?
Could you please share that config as well?
I have given them above.I use 2 things 1. docker-compose.yml and the other is logstash conf.Is there anything else I need to add?
So your Config is configured for receiving Events through a Logshipper.
You need to configure Docker to send Events to your Logstash Host for example like this.
If you are planing to do that you need to adjust your Logstash config from beats input to syslog.
Another way is to mount the log Directory from Docker into your Logstash Container and configure the file input for Logstash.
I can’t figure out how to put this in my yaml file, can you help me please?
Hey @bastus11,
before we get to the configuration you needed, I wanted to ask what was too complicated for you to understand in the documentation? I would like to know so that we can improve it for the next users who encounter the same problems as you.
You need to configure Docker like this.
Paste the following in the file /etc/docker/daemon.json.
{
"log-driver": "syslog",
"log-opts": {
"syslog-address": "udp://127.0.0.1:42069"
}
}
Adjust your docker-compose.yml like this:
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node1 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_INSTALL_DEMO_CONFIG=true" # Prevents execution of bundled demo script which installs demo certificates and security configurations to OpenSearch
- "DISABLE_SECURITY_PLUGIN=true" # Disables Security plugin
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
ports:
- 9200:9200 # REST API
- 9600:9600 # Performance Analyzer
networks:
- opensearch-net # All of the containers will join the same Docker bridge network
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node2 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligibile to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_INSTALL_DEMO_CONFIG=true" # Prevents execution of bundled demo script which installs demo certificates and security configurations to OpenSearch
- "DISABLE_SECURITY_PLUGIN=true" # Disables Security plugin
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data # Creates volume called opensearch-data2 and mounts it to the container
networks:
- opensearch-net # All of the containers will join the same Docker bridge network
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601 # Map host port 5601 to container port 5601
expose:
- "5601" # Expose port 5601 for web access to OpenSearch Dashboards
environment:
- 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200","http://opensearch-node2:9200"]'
- "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
networks:
- opensearch-net
logstash:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin
container_name: logstash
restart: unless-stopped
ports:
- 42069:42069/udp
volumes:
- ./pipeline:/usr/share/logstash/pipeline/
- /var/lib/docker/containers:/data:ro
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
At lastly here is the Logstash config File:
input {
syslog {
port => 42069
}
}
# Required - An input plugin to pass some data to OpenSearch - file or AP
filter {
# Can be empty
}
output {
# All of the following connection details
opensearch {
# Your node IP addresses from the Instaclustr Console
hosts => ["http://opensearch-node1:9200", "http://opensearch-node2:9200"]
# SSL enabled
ssl => false
ssl_certificate_verification => false
# Path to your cluster certificate .pem downloaded earlier
# The Logstash Username and Password created earlier
# The name of the index
index => "logstash"
}
}
First of all, thank you very much for your help sir 
. I did the syslog configuration in the document, but I could not find how to change the logstash.conf file accordingly.
input {
syslog {
port => 42069
}
}