OpenSearch docker-compose configuration exiting in CI

OpenSearch
1

Versions: OpenSearch 2.19.2

Describe the issue:

I’m able to successfully use a very simple docker-compose OpenSearch config locally on my macbook, but the same config exits without any informative errors or logs in CI (GitHub action). What might explain the different behavior locally vs in CI? Is there a way to enable more logging to figure out what the underlying issue is?

Configuration:

I’ve configured OpenSearch in my docker-compose.yaml as follows:

  opensearch:
    image: opensearchproject/opensearch:2.19.2
    restart: unless-stopped
    environment:
      discovery.type: single-node
      bootstrap.memory_lock: true
      http.host: 0.0.0.0
      transport.host: 0.0.0.0
      cluster.routing.allocation.disk.threshold_enabled: false
      OPENSEARCH_JAVA_OPTS: "-Xms1g -Xmx1g"
      OPENSEARCH_INITIAL_ADMIN_PASSWORD: "[redacted]"
      _JAVA_OPTIONS: -XX:UseSVE=0
    volumes:
      - os_data:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600
    networks:
      - internal
    healthcheck:
      test:
        [
          "CMD",
          "curl",
          "-k",
          "-u",
          "admin:[redacted]",
          "--fail",
          "--insecure",
          "https://localhost:9200/_cluster/health",
        ]
      interval: 2s
      timeout: 5s
      retries: 20
      start_period: 30s
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536

No dashboards, no additional nodes.

Relevant Logs or Screenshots:

This works fine locally on my macbook:

Enabling OpenSearch Security Plugin

Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin 

OpenSearch 2.12.0 onwards, the OpenSearch Security Plugin a change that requires an initial password for 'admin' user. 

Please define an environment variable 'OPENSEARCH_INITIAL_ADMIN_PASSWORD' with a strong password string. 

If a password is not provided, the setup will quit. 

 For more details, please visit: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/⁠

### OpenSearch Security Demo Installer

### ** Warning: Do not use on production or public reachable systems **

OpenSearch install type: rpm/deb on Linux 6.10.14-linuxkit aarch64

OpenSearch config dir: /usr/share/opensearch/config/

OpenSearch config file: /usr/share/opensearch/config/opensearch.yml

OpenSearch bin dir: /usr/share/opensearch/bin/

OpenSearch plugins dir: /usr/share/opensearch/plugins/

OpenSearch lib dir: /usr/share/opensearch/lib/

Detected OpenSearch Version: 2.19.2

Detected OpenSearch Security Version: 2.19.2.0

Admin password set successfully.

### Success

[...]

When trying to spin up this container in a GitHub action though, it never starts successfully. It seems to exit before the Security Demo Installer gets a chance to run. The first time it exits, it has exit code 0, and subsequent exits (due to the restart: unless-stopped) have exit code 1. The only logging I see is:

opensearch-1  | Enabling OpenSearch Security Plugin
opensearch-1  | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin 
opensearch-1  | OpenSearch 2.12.0 onwards, the OpenSearch Security Plugin a change that requires an initial password for 'admin' user. 
opensearch-1  | Please define an environment variable 'OPENSEARCH_INITIAL_ADMIN_PASSWORD' with a strong password string. 
opensearch-1  | If a password is not provided, the setup will quit. 
opensearch-1  |  For more details, please visit: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/

opensearch-1 exited with code 1

I can’t seem to get any additional logging. I’ve been able to print out the environment variables and have verified that the OPENSEARCH_INITIAL_ADMIN_PASSWORD is correct. I’ve also tried bumping up to larger runners, but that didn’t help either. Both the array syntax and key:value syntax for environment variables in docker-compose are resulting in the same (what appears to me to be valid and accurate) environment both locally and in CI.

2

@esaron I’ve tested your docker-compose.yml on my Mac M3 and it worked with no issues. I’ve also tested it on my Ubuntu server that runs on the Intel chipset. It failed as yours.

Once I removed the below line it worked on Ubuntu too.

`-XX:UseSVE=0`

This line is designed for ARM processors. I think GitHub action spins a Docker container on an Intel based platform.

1 Like