@keri The error in the OpenSearch Dashboards logs means that the security plugin is disabled in OpenSearch. Therefore it can’t find a handler for /_plugins/_security/tenantinfo
The disabled security plugin will also cause the error for /_plugins/_security/authinfo.
Could you share a full start-up log file of the OpenSearch when the security plugin is enabled?
Heres the full startup-log
ubuntu@ews-open-search-1:~/opensearch-2.3.0$ ./bin/opensearch
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/home/ubuntu/opensearch-2.3.0/lib/opensearch-2.3.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/home/ubuntu/opensearch-2.3.0/lib/opensearch-2.3.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2022-11-11T00:52:25,665][INFO ][o.o.n.Node ] [ews-open-search-1] version[2.3.0], pid[292525], build[tar/6f6e84ebc54af31a976f53af36a5c69d474a5140/2022-09-09T00:07:12.137133581Z], OS[Linux/5.4.0-104-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.4/17.0.4+8]
[2022-11-11T00:52:25,668][INFO ][o.o.n.Node ] [ews-open-search-1] JVM home [/home/ubuntu/opensearch-2.3.0/jdk], using bundled JDK [true]
[2022-11-11T00:52:25,669][INFO ][o.o.n.Node ] [ews-open-search-1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms8g, -Xmx8g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-14318078273275099065, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/home/ubuntu/opensearch-2.3.0/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -XX:MaxDirectMemorySize=4294967296, -Dopensearch.path.home=/home/ubuntu/opensearch-2.3.0, -Dopensearch.path.conf=/home/ubuntu/opensearch-2.3.0/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2022-11-11T00:52:26,670][WARN ][stderr ] [ews-open-search-1] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[2022-11-11T00:52:26,670][WARN ][stderr ] [ews-open-search-1] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2022-11-11T00:52:26,671][WARN ][stderr ] [ews-open-search-1] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2022-11-11T00:52:26,685][INFO ][o.o.s.s.t.SSLConfig ] [ews-open-search-1] SSL dual mode is disabled
[2022-11-11T00:52:26,686][INFO ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] OpenSearch Config path is /home/ubuntu/opensearch-2.3.0/config
[2022-11-11T00:52:26,976][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] JVM supports TLSv1.3
[2022-11-11T00:52:26,979][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] Config directory is /home/ubuntu/opensearch-2.3.0/config/, from there the key- and truststore files are resolved relatively
[2022-11-11T00:52:27,298][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] TLS Transport Client Provider : JDK
[2022-11-11T00:52:27,299][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] TLS Transport Server Provider : JDK
[2022-11-11T00:52:27,299][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] TLS HTTP Provider : JDK
[2022-11-11T00:52:27,300][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2]
[2022-11-11T00:52:27,300][INFO ][o.o.s.s.DefaultSecurityKeyStore] [ews-open-search-1] Enabled TLS protocols for HTTP layer : [TLSv1.3, TLSv1.2]
[2022-11-11T00:52:27,533][INFO ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] Clustername: ews-opensearch-prd
[2022-11-11T00:52:27,545][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] Directory /home/ubuntu/opensearch-2.3.0/config has insecure file permissions (should be 0700)
[2022-11-11T00:52:27,546][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/admin.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,546][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/node1.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,547][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] Directory /home/ubuntu/opensearch-2.3.0/config/certs has insecure file permissions (should be 0700)
[2022-11-11T00:52:27,547][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/admin.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,548][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/node1.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,548][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/node1-key.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,549][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/root-ca-key.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,549][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/root-ca.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,550][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/node.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,550][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/certs/admin-key.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,550][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/root-ca.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,551][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/node.pem has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,551][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/setting.sh has insecure file permissions (should be 0600)
[2022-11-11T00:52:27,552][WARN ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] File /home/ubuntu/opensearch-2.3.0/config/root-ca.srl has insecure file permissions (should be 0600)
[2022-11-11T00:52:28,159][INFO ][o.o.p.c.PluginSettings ] [ews-open-search-1] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
[2022-11-11T00:52:28,603][INFO ][o.o.i.r.ReindexPlugin ] [ews-open-search-1] ReindexPlugin reloadSPI called
[2022-11-11T00:52:28,605][INFO ][o.o.i.r.ReindexPlugin ] [ews-open-search-1] Unable to find any implementation for RemoteReindexExtension
[2022-11-11T00:52:28,643][INFO ][o.o.j.JobSchedulerPlugin ] [ews-open-search-1] Loaded scheduler extension: opendistro_anomaly_detector, index: .opendistro-anomaly-detector-jobs
[2022-11-11T00:52:28,672][INFO ][o.o.j.JobSchedulerPlugin ] [ews-open-search-1] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions
[2022-11-11T00:52:28,674][INFO ][o.o.j.JobSchedulerPlugin ] [ews-open-search-1] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config
[2022-11-11T00:52:28,680][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [aggs-matrix-stats]
[2022-11-11T00:52:28,681][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [analysis-common]
[2022-11-11T00:52:28,681][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [geo]
[2022-11-11T00:52:28,681][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [ingest-common]
[2022-11-11T00:52:28,682][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [ingest-geoip]
[2022-11-11T00:52:28,682][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [ingest-user-agent]
[2022-11-11T00:52:28,683][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [lang-expression]
[2022-11-11T00:52:28,683][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [lang-mustache]
[2022-11-11T00:52:28,683][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [lang-painless]
[2022-11-11T00:52:28,683][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [mapper-extras]
[2022-11-11T00:52:28,684][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [opensearch-dashboards]
[2022-11-11T00:52:28,684][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [parent-join]
[2022-11-11T00:52:28,684][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [percolator]
[2022-11-11T00:52:28,684][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [rank-eval]
[2022-11-11T00:52:28,685][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [reindex]
[2022-11-11T00:52:28,685][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [repository-url]
[2022-11-11T00:52:28,685][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [systemd]
[2022-11-11T00:52:28,685][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded module [transport-netty4]
[2022-11-11T00:52:28,686][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-alerting]
[2022-11-11T00:52:28,686][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-anomaly-detection]
[2022-11-11T00:52:28,687][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-asynchronous-search]
[2022-11-11T00:52:28,687][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-cross-cluster-replication]
[2022-11-11T00:52:28,687][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-geospatial]
[2022-11-11T00:52:28,687][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-index-management]
[2022-11-11T00:52:28,688][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-job-scheduler]
[2022-11-11T00:52:28,688][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-knn]
[2022-11-11T00:52:28,688][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-ml]
[2022-11-11T00:52:28,688][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-notifications]
[2022-11-11T00:52:28,688][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-notifications-core]
[2022-11-11T00:52:28,689][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-observability]
[2022-11-11T00:52:28,689][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-performance-analyzer]
[2022-11-11T00:52:28,689][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-reports-scheduler]
[2022-11-11T00:52:28,689][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-security]
[2022-11-11T00:52:28,690][INFO ][o.o.p.PluginsService ] [ews-open-search-1] loaded plugin [opensearch-sql]
[2022-11-11T00:52:28,713][INFO ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
[2022-11-11T00:52:28,738][INFO ][o.o.e.NodeEnvironment ] [ews-open-search-1] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [91gb], net total_space [96.7gb], types [ext4]
[2022-11-11T00:52:28,739][INFO ][o.o.e.NodeEnvironment ] [ews-open-search-1] heap size [8gb], compressed ordinary object pointers [true]
[2022-11-11T00:52:28,829][INFO ][o.o.n.Node ] [ews-open-search-1] node name [ews-open-search-1], node ID [JC5B0pUQR9miRBqWdfshuQ], cluster name [ews-opensearch-prd], roles [master]
[2022-11-11T00:52:32,281][WARN ][o.o.s.c.Salt ] [ews-open-search-1] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2022-11-11T00:52:32,335][INFO ][o.o.s.a.i.AuditLogImpl ] [ews-open-search-1] Message routing enabled: true
[2022-11-11T00:52:32,346][ERROR][o.o.s.c.AdminDNs ] [ews-open-search-1] Unable to parse admin dn /C=ID/ST=Jakarta/L=Jakarta/O=BRI/OU=DDB/CN=ADMIN
javax.naming.InvalidNameException: Invalid name: /C=ID/ST=Jakarta/L=Jakarta/O=BRI/OU=DDB/CN=ADMIN
at javax.naming.ldap.Rfc2253Parser.parseAttrType(Rfc2253Parser.java:155) ~[?:?]
at javax.naming.ldap.Rfc2253Parser.doParse(Rfc2253Parser.java:108) ~[?:?]
at javax.naming.ldap.Rfc2253Parser.parseDn(Rfc2253Parser.java:70) ~[?:?]
at javax.naming.ldap.LdapName.parse(LdapName.java:806) ~[?:?]
at javax.naming.ldap.LdapName.<init>(LdapName.java:125) ~[?:?]
at org.opensearch.security.configuration.AdminDNs.<init>(AdminDNs.java:69) [opensearch-security-2.3.0.0.jar:2.3.0.0]
at org.opensearch.security.OpenSearchSecurityPlugin.createComponents(OpenSearchSecurityPlugin.java:797) [opensearch-security-2.3.0.0.jar:2.3.0.0]
at org.opensearch.node.Node.lambda$new$15(Node.java:696) [opensearch-2.3.0.jar:2.3.0]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) [?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) [?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) [?:?]
at org.opensearch.node.Node.<init>(Node.java:710) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.node.Node.<init>(Node.java:347) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-2.3.0.jar:2.3.0]
at org.opensearch.cli.Command.main(Command.java:101) [opensearch-cli-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137) [opensearch-2.3.0.jar:2.3.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103) [opensearch-2.3.0.jar:2.3.0]
[2022-11-11T00:52:32,388][INFO ][o.o.s.f.SecurityFilter ] [ews-open-search-1] <NONE> indices are made immutable.
[2022-11-11T00:52:32,724][INFO ][o.o.a.b.ADCircuitBreakerService] [ews-open-search-1] Registered memory breaker.
[2022-11-11T00:52:33,075][INFO ][o.o.m.c.b.MLCircuitBreakerService] [ews-open-search-1] Registered ML memory breaker.
[2022-11-11T00:52:33,599][INFO ][o.o.t.NettyAllocator ] [ews-open-search-1] creating NettyAllocator with the following configs: [name=opensearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={opensearch.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2022-11-11T00:52:33,700][INFO ][o.o.d.DiscoveryModule ] [ews-open-search-1] using discovery type [zen] and seed hosts providers [settings]
[2022-11-11T00:52:34,183][WARN ][o.o.g.DanglingIndicesState] [ews-open-search-1] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-11-11T00:52:34,696][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [ews-open-search-1] PerformanceAnalyzer Enabled: false
[2022-11-11T00:52:34,730][INFO ][o.o.n.Node ] [ews-open-search-1] initialized
[2022-11-11T00:52:34,731][INFO ][o.o.n.Node ] [ews-open-search-1] starting ...
[2022-11-11T00:52:34,878][INFO ][o.o.t.TransportService ] [ews-open-search-1] publish_address {10.40.50.233:9300}, bound_addresses {10.40.50.233:9300}
[2022-11-11T00:52:35,087][INFO ][o.o.b.BootstrapChecks ] [ews-open-search-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-11-11T00:52:35,094][INFO ][o.o.c.c.Coordinator ] [ews-open-search-1] cluster UUID [j3aisPUXSJ-E3ZRQQkunCw]
[2022-11-11T00:52:35,213][INFO ][o.o.c.s.MasterService ] [ews-open-search-1] elected-as-cluster-manager ([1] nodes joined)[{ews-open-search-1}{JC5B0pUQR9miRBqWdfshuQ}{tRKM8MUMSm6oW7ZMnpRKiQ}{10.40.50.233}{10.40.50.233:9300}{m}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_CLUSTER_MANAGER_TASK_, _FINISH_ELECTION_], term: 20, version: 112, delta: cluster-manager node changed {previous [], current [{ews-open-search-1}{JC5B0pUQR9miRBqWdfshuQ}{tRKM8MUMSm6oW7ZMnpRKiQ}{10.40.50.233}{10.40.50.233:9300}{m}{shard_indexing_pressure_enabled=true}]}
[2022-11-11T00:52:35,280][INFO ][o.o.c.s.ClusterApplierService] [ews-open-search-1] cluster-manager node changed {previous [], current [{ews-open-search-1}{JC5B0pUQR9miRBqWdfshuQ}{tRKM8MUMSm6oW7ZMnpRKiQ}{10.40.50.233}{10.40.50.233:9300}{m}{shard_indexing_pressure_enabled=true}]}, term: 20, version: 112, reason: Publication{term=20, version=112}
[2022-11-11T00:52:35,292][INFO ][o.o.a.c.ADClusterEventListener] [ews-open-search-1] Cluster is not recovered yet.
[2022-11-11T00:52:35,298][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [ews-open-search-1] Detected cluster change event for destination migration
[2022-11-11T00:52:35,328][INFO ][o.o.i.i.ManagedIndexCoordinator] [ews-open-search-1] Cache cluster manager node onClusterManager time: 1668102755328
[2022-11-11T00:52:35,334][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [ews-open-search-1] Config override setting update called with empty string. Ignoring.
[2022-11-11T00:52:35,347][INFO ][o.o.h.AbstractHttpServerTransport] [ews-open-search-1] publish_address {10.40.50.233:9200}, bound_addresses {10.40.50.233:9200}
[2022-11-11T00:52:35,347][INFO ][o.o.n.Node ] [ews-open-search-1] started
[2022-11-11T00:52:35,348][INFO ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] Node started
[2022-11-11T00:52:35,348][INFO ][o.o.s.c.ConfigurationRepository] [ews-open-search-1] Will attempt to create index .opendistro_security and default configs if they are absent
[2022-11-11T00:52:35,349][INFO ][o.o.s.c.ConfigurationRepository] [ews-open-search-1] Background init thread started. Install default config?: true
[2022-11-11T00:52:35,350][INFO ][o.o.s.OpenSearchSecurityPlugin] [ews-open-search-1] 0 OpenSearch Security modules loaded so far: []
[2022-11-11T00:52:35,396][INFO ][o.o.c.s.ClusterSettings ] [ews-open-search-1] updating [plugins.index_state_management.template_migration.control] from [0] to [-1]
[2022-11-11T00:52:35,400][INFO ][o.o.a.c.HashRing ] [ews-open-search-1] Node added: [JC5B0pUQR9miRBqWdfshuQ]
[2022-11-11T00:52:35,403][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [ews-open-search-1] Detected cluster change event for destination migration
[2022-11-11T00:52:35,408][INFO ][o.o.a.c.HashRing ] [ews-open-search-1] All nodes with known AD version: {JC5B0pUQR9miRBqWdfshuQ=ADNodeInfo{version=2.3.0, isEligibleDataNode=false}}
[2022-11-11T00:52:35,410][INFO ][o.o.a.c.HashRing ] [ews-open-search-1] Rebuild AD hash ring for realtime AD with cooldown, nodeChangeEvents size 0
[2022-11-11T00:52:35,410][INFO ][o.o.a.c.HashRing ] [ews-open-search-1] Build AD version hash ring successfully
[2022-11-11T00:52:35,411][INFO ][o.o.a.c.ADClusterEventListener] [ews-open-search-1] Init AD version hash ring successfully
[2022-11-11T00:52:35,435][INFO ][o.o.g.GatewayService ] [ews-open-search-1] recovered [2] indices into cluster_state
[2022-11-11T00:52:35,438][INFO ][o.o.s.c.ConfigurationRepository] [ews-open-search-1] Index .opendistro_security already exists
[2022-11-11T00:52:35,439][INFO ][o.o.s.c.ConfigurationRepository] [ews-open-search-1] Node started, try to initialize it. Wait for at least yellow cluster state....
[2022-11-11T00:53:35,331][INFO ][o.o.i.i.ManagedIndexCoordinator] [ews-open-search-1] Performing move cluster state metadata.
[2022-11-11T00:53:35,332][INFO ][o.o.i.i.MetadataService ] [ews-open-search-1] ISM config index not exist, so we cancel the metadata migration job.
@keri You still have admin_dn incorrectly configured. Take a look at my prevoius posts.
[2022-11-11T00:52:32,346][ERROR][o.o.s.c.AdminDNs ] [ews-open-search-1] Unable to parse admin dn /C=ID/ST=Jakarta/L=Jakarta/O=BRI/OU=DDB/CN=ADMIN
javax.naming.InvalidNameException: Invalid name: /C=ID/ST=Jakarta/L=Jakarta/O=BRI/OU=DDB/CN=ADMIN
at javax.naming.ldap.Rfc2253Parser.parseAttrType(Rfc2253Parser.java:155) ~[?:?]
at javax.naming.ldap.Rfc2253Parser.doParse(Rfc2253Parser.java:108) ~[?:?]
at javax.naming.ldap.Rfc2253Parser
@pablo when i run the securityadmin.sh its not running, and heres the error :
ubuntu@ews-open-search-1:~/opensearch-2.3.0/plugins/opensearch-security/tools$ OPENSEARCH_JAVA_HOME=/home/ubuntu/opensearch-2.3.0/jdk ./securityadmin.sh
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
Security Admin v7
ERR: Parsing failed. Reason: Specify at least -ks or -cert
usage: securityadmin.sh [-arc] [-backup <folder>] [-cacert <file>] [-cd
<directory>] [-cert <file>] [-cn <clustername>] [-dci] [-dg] [-dra]
[-ec <cipers>] [-ep <protocols>] [-er <number of replicas>] [-era]
[-esa] [-f <file>] [-ff] [-h <host>] [-i <indexname>] [-icl] [-key
<file>] [-keypass <password>] [-ks <file>] [-ksalias <alias>]
[-kspass <password>] [-kst <type>] [-migrate <folder>] [-mo
<folder>] [-nhnv] [-p <port>] [-prompt] [-r] [-rev] [-rl] [-si]
[-sniff] [-t <file-type>] [-ts <file>] [-tspass <password>] [-tst
<type>] [-us <number of replicas>] [-vc <version>] [-w]
and this my latest configuration of opensearch.yml
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]
node.max_local_storage_nodes: 3
plugins.security.disabled: false
plugins.security.ssl.transport.pemcert_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/node.pem
plugins.security.ssl.transport.pemkey_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/node.pem
plugins.security.ssl.http.pemkey_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /home/ubuntu/opensearch-2.3.0/config/certs/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
- '/C=ID/ST=Jakarta/L=Jakarta/O=xxx/OU=xxxDDB/CN=ADMIN'
plugins.security.nodes_dn:
- '/C=ID/ST=Jakarta/L=Jakarta/O=xxx/OU=xxx/CN=NODE'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
@keri Please review my previous post and fix node_dn and admin_dn. Without fixing these, the security plugin in your OpenSearch nodes won’t work.
plugins.security.authcz.admin_dn:
- '/C=ID/ST=Jakarta/L=Jakarta/O=xxx/OU=xxxDDB/CN=ADMIN'
plugins.security.nodes_dn:
- '/C=ID/ST=Jakarta/L=Jakarta/O=xxx/OU=xxx/CN=NODE'
securityadmin.sh relies on admin_dn. As mentioned before it must be configured as per documentation.
Please review the below documentation which describes securityadmin.sh options.