I’m trying to build a docker opensearch stack with LDAP authentication, but I run into the following error every time i’m trying to use a config to replace the config.yml.
I’m deploying the stack using portainer (not sure if thats an influence or not).
my docker-compose:
version: '3.5'
services:
opensearch-node1:
image: opensearchproject/opensearch:1.3.2
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.type=single-node
- bootstrap.memory_lock=false # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
configs:
- source: ldap_test
target: /usr/share/opensearch/plugins/opensearch_security/securityconfig/config.yml
volumes:
- esdata:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:1.3.2
container_name: opensearch-dashboards
environment:
- OPENSEARCH_HOSTS=["https://opensearch-node1:9200"]
ports:
- 5601:5601
configs:
- source: opensearch_kibana
target: :/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
networks:
- opensearch-net
logstash:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.3
container_name: logstash
command: logstash -f /config-dir/logstash_http_json.conf
volumes:
- logstash:/config-dir
environment:
- OPENSEARCH_HOSTS='["https://opensearch-node1:9200"]'
ports:
- 5043:5043
networks:
- opensearch-net
volumes:
esdata:
logstash:
configs:
ldap_test:
external: true
opensearch_kibana:
external: true
networks:
opensearch-net:
I’ve created a config “ldap_test” which contains the following:
_meta:
type: "config"
config_version: 2
config:
dynamic:
http:
anonymous_auth_enabled: false
authc:
internal_auth:
order: 0
description: "HTTP basic authentication using the internal user database"
http_enabled: true
transport_enabled: true
http_authenticator:
type: basic
challenge: false
authentication_backend:
type: internal
ldap_auth:
order: 1
description: "Authenticate using LDAP"
http_enabled: true
transport_enabled: true
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: ldap
config:
enable_ssl: true
enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:
- xxxxxx:636
bind_dn: xxxxxx
password: "xxxxxxxx"
userbase: 'xxxxxxxxx'
usersearch: '(mail={0})'
username_attribute: null
authz:
ldap_roles:
description: "Authorize using LDAP"
http_enabled: true
transport_enabled: true
authorization_backend:
type: ldap
config:
enable_ssl: true
enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:
- xxxxx:636
bind_dn: xxxxxxxxx
password: "xxxxxxx"
userbase: 'xxxxxxxxxx'
usersearch: '(mail={0})'
username_attribute: null
skip_users:
- admin
- kibanaserver
rolebase: 'xxxxxxxxxxxxxxxx'
rolesearch: '(member={0})'
userroleattribute: null
userrolename: memberOf
rolename: cn
resolve_nested_roles: true
the exact and complete error for those interested:
OpenSearch Security Demo Installer
** Warning: Do not use on production or public reachable systems **
Basedir: /usr/share/opensearch
OpenSearch install type: rpm/deb on NAME="Amazon Linux"
OpenSearch config dir: /usr/share/opensearch/config
OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin
OpenSearch plugins dir: /usr/share/opensearch/plugins
OpenSearch lib dir: /usr/share/opensearch/lib
Detected OpenSearch Version: x-content-1.3.2
Detected OpenSearch Security Version: 1.3.2.0
### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../securityconfig and execute:
"/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv
### or run ./securityadmin_demo.sh
### To use the Security Plugin ConfigurationGUI
### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
Enabling OpenSearch Security Plugin
[2022-05-19T07:32:31,960][INFO ][o.o.n.Node ] [opensearch-node1] version[1.3.2], pid[102], build[tar/6febcf7b53ff189de767e460e905e9e5aeecc8cb/2022-05-04T03:58:48.328641Z], OS[Linux/4.18.0-305.el8.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.15/11.0.15+10]
[2022-05-19T07:32:31,961][INFO ][o.o.n.Node ] [opensearch-node1] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true]
[2022-05-19T07:32:31,962][INFO ][o.o.n.Node ] [opensearch-node1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-6124132963470950871, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
uncaught exception in thread [main]
[2022-05-19T07:32:32,034][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch-node1] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: Could not load plugin descriptor for plugin directory [opensearch_security]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:182) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-1.3.2.jar:1.3.2]
at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) ~[opensearch-1.3.2.jar:1.3.2]
Caused by: java.lang.IllegalStateException: Could not load plugin descriptor for plugin directory [opensearch_security]
at org.opensearch.plugins.PluginsService.readPluginBundle(PluginsService.java:452) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.findBundles(PluginsService.java:437) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.getPluginBundles(PluginsService.java:430) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:181) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.3.2.jar:1.3.2]
... 6 more
Caused by: java.nio.file.NoSuchFileException: /usr/share/opensearch/plugins/opensearch_security/plugin-descriptor.properties
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) ~[?:?]
at java.nio.file.Files.newByteChannel(Files.java:371) ~[?:?]
at java.nio.file.Files.newByteChannel(Files.java:422) ~[?:?]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~[?:?]
at java.nio.file.Files.newInputStream(Files.java:156) ~[?:?]
at org.opensearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:229) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.readPluginBundle(PluginsService.java:449) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.findBundles(PluginsService.java:437) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.getPluginBundles(PluginsService.java:430) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:181) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.3.2.jar:1.3.2]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.3.2.jar:1.3.2]
... 6 more
java.lang.IllegalStateException: Could not load plugin descriptor for plugin directory [opensearch_security]
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/opensearch/plugins/opensearch_security/plugin-descriptor.properties
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:156)
at org.opensearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:229)
at org.opensearch.plugins.PluginsService.readPluginBundle(PluginsService.java:449)
at org.opensearch.plugins.PluginsService.findBundles(PluginsService.java:437)
at org.opensearch.plugins.PluginsService.getPluginBundles(PluginsService.java:430)
at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:181)
at org.opensearch.node.Node.<init>(Node.java:396)
at org.opensearch.node.Node.<init>(Node.java:319)
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412)
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
at org.opensearch.cli.Command.main(Command.java:101)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
For complete error details, refer to the log at /usr/share/opensearch/logs/opensearch-cluster.log
Killing performance analyzer process 103
OpenSearch exited with code 1
Performance analyzer exited with code 143
thanks in advance for pointing me in the right direction!
