Hello everyone! I’m just getting started with OpenSearch and I’m having trouble with one of the steps. I’ve been trying to figure it out for days. I’d be really grateful if you could guide me, thank you.
I installed OpenSearch Ver. 3, OpenSearch Dashboard Ver. 3, and used Logstash 9.2.4, from Elastic as recommended and Winlogbeat-9.2.3 for Windows.
I deployed it using Docker Compose.
P.S: I removed the links to the repository to bypass the restrictions on links in the topic
Docker-Compose.yml
services:
opensearch-node1:
image: opensearchproject/opensearch:3
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- OPENSEARCH_JAVA_OPTS=-Xms4096m -Xmx4096m # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD} # Sets the demo admin user password when using demo configuration, required for OpenSearch 2.12 and higher
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes: - opensearch-data1:/usr/share/opensearch/data
ports: - 9200:9200
- 9600:9600 # required for Performance Analyzer
networks: - opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:3
container_name: opensearch-node2
environment: - cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- OPENSEARCH_JAVA_OPTS=-Xms4096m -Xmx4096m
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes: - opensearch-data2:/usr/share/opensearch/data
networks: - opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:3
container_name: opensearch-dashboards
ports: - 5601:5601
expose: - ‘5601’
environment:
OPENSEARCH_HOSTS: ‘[“opensearch-node1:9200”,“opensearch-node2:9200”]’
networks: - opensearch-net
logstash:
image: docker.elastic.co/logstash/logstash:9.2.4
container_name: logstash
ports: - “5044:5044”
volumes: - /home/user/OpenSearch/logstash/config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf
- /home/user/OpenSearch/root-ca.pem:/usr/share/logstash/config/root-ca.pem
command: >
/bin/sh -c ’
echo “Checking and installing plugins…”;
if ! /usr/share/logstash/bin/logstash-plugin list | grep -q logstash-output-opensearch; then
/usr/share/logstash/bin/logstash-plugin install logstash-output-opensearch;
fi;
if ! /usr/share/logstash/bin/logstash-plugin list | grep -q logstash-input-opensearch; then
/usr/share/logstash/bin/logstash-plugin install logstash-input-opensearch;
fi;
echo “Starting Logstash…”;
/usr/share/logstash/bin/logstash -f /usr/share/logstash/pipeline/logstash.conf
’
networks: - opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
For Logstash, I used the logstash-output-opensearch and logstash-input-opensearchh plugins.
logstash.conf
input {
beats {
port => 5044
}
}
filter {
}
output {
if [type] == “beats_input” {
opensearch {
hosts => ["opensearch-node1:9200", "opensearch-node2:9200"]
user => "admin"
password => "password"
index => "winlogs-%{+YYYY.MM.dd}"
ssl => true
ssl_certificate_verification => false
cacert => "/usr/share/logstash/config/root-ca.pem"
manage_template => false
}
}
#stdout { codec => rubydebug }
}
The problem is that all containers start, and logs from WinLogBeat are being sent to Logstash. I can see this if I enable debug mode, but no data appears in OpenSearch. I can’t create a Create Index Pattern.
