How to setup security on containerised opensearch?

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
server opensearchproject/opensearch:2.4.0

Describe the issue:
I’m trying to set up a single-node cluster using the above docker container (although running under podman + systemd on rockylinux). This will ingest from filebeat so I have a custom opensearch.yml mounted in to allow me to define compatibility.override_main_response_version: true. Using the demo security certs that all works fine, opensearch runs filebeat can connect to it.

However, if I stop and restart the container then it fails with:

Likely root cause: OpenSearchException[Unable to read /usr/share/opensearch/config/esnode.pem (/usr/share/opensearch/config/esnode.pem). Please make sure this files exists and is readable regarding to permissions. Property:]

which makes sense - the demo certs get created in the container’s config directory and don’t persist, but the paths to them have been automatically added to the opensearch.yml which does persist because it’s mounted in.

How do people deal with that?

I tried creating my own (self-signed) certs following the docs and mounting those in too. It looks like the container is waiting for something to run, which seems like it’ll be painful to “orchestrate” with podman/systemd if I have to exec into the container at the right point (I.e. after opensearch is running) to run that. Or can I just specify that as the command to run for the container, e.g.

/usr/bin/podman run ... opensearchproject/opensearch:2.4.0 plugins/opensearch-security/tools/

or something?
Thanks for any help, seems like this must be a common problem but I can’t find a solution in the docs.


### This section copies a running container's default config ### docker-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.

# # minimum_master_nodes need to be explicitly set when bound on a public IP
# # set to 1 to allow single node clusters
# discovery.zen.minimum_master_nodes: 1

# Setting to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# discovery.type: single-node

### Custom config section ###

# Fake version for filebeat:
compatibility.override_main_response_version: true

Relevant Logs or Screenshots:

@lost Do you use docker-compose? Try the following procedure.

No. This is a single-node cluster with a systemd unit running the container via (rootless) podman.