Before opening an issue on GitHub, I wanted to see if there may be something I missed on my end, as I’m having trouble starting a simple OpenSearch on Docker.
Here’s my docker run command:
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.0.0
And here’s the result (infinite looping) log:
[2021-07-28T11:17:22,451][ERROR][o.o.s.s.h.n.SecuritySSLNettyHttpServerTransport] [350fac761c4a] Exception during establishing a SSL connection: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 474554202f20485454502f312e310d0a486f73743a206c6f63616c686f73743a393230300d0a557365722d4167656e743a20476f2d687474702d636c69656e742f312e310d0a4163636570743a206170706c69636174696f6e2f6a736f6e0d0a4163636570742d456e636f64696e673a20677a69700d0a0d0a
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 474554202f20485454502f312e310d0a486f73743a206c6f63616c686f73743a393230300d0a557365722d4167656e743a20476f2d687474702d636c69656e742f312e310d0a4163636570743a206170706c69636174696f6e2f6a736f6e0d0a4163636570742d456e636f64696e673a20677a69700d0a0d0a
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1261) ~[netty-handler-4.1.59.Final.jar:4.1.59.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) ~[netty-handler-4.1.59.Final.jar:4.1.59.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508) ~[netty-codec-4.1.59.Final.jar:4.1.59.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447) ~[netty-codec-4.1.59.Final.jar:4.1.59.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:620) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:583) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.59.Final.jar:4.1.59.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.59.Final.jar:4.1.59.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.59.Final.jar:4.1.59.Final]
at java.lang.Thread.run(Thread.java:832) [?:?]
OpenSearch seems to be running though, as I can curl it without problems, but I cannot start a development OpenSearch Dashboard though, it gives me this log:
server log [11:17:14.260] [error][data][opensearch] [ConnectionError]: socket hang up
I haven’t tried it recently, but afaik I had the same problem using the sample docker-compose.
I managed to start a dev OSD (I still have the constant exception on OpenSearch though), but I think we should update the documentation or the default config values of opensearch_dashboards.yml.
Considering that the defaults of the docker image are to set the admin:admin credentials and enable ssl, I think we should match these settings in the OSD defaults:
diff --git a/config/opensearch_dashboards.yml b/config/opensearch_dashboards.yml
index d281b987ee..65aa1df2e4 100644
--- a/config/opensearch_dashboards.yml
+++ b/config/opensearch_dashboards.yml
@@ -25,7 +25,7 @@
#server.name: "your-hostname"
# The URLs of the OpenSearch instances to use for all your queries.
-#opensearch.hosts: ["http://localhost:9200"]
+#opensearch.hosts: ["https://localhost:9200"]
# OpenSearch Dashboards uses an index in OpenSearch to store saved searches, visualizations and
# dashboards. OpenSearch Dashboards creates a new index if the index doesn't already exist.
@@ -48,8 +48,8 @@
# the username and password that the OpenSearch Dashboards server uses to perform maintenance on the OpenSearch Dashboards
# index at startup. Your OpenSearch Dashboards users still need to authenticate with OpenSearch, which
# is proxied through the OpenSearch Dashboards server.
-#opensearch.username: "opensearch_dashboards_system"
-#opensearch.password: "pass"
+#opensearch.username: "admin"
+#opensearch.password: "admin"
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the OpenSearch Dashboards server to the browser.
@@ -68,7 +68,7 @@
#opensearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
-#opensearch.ssl.verificationMode: full
+#opensearch.ssl.verificationMode: none
What do you think ? I will surely open an issue for this case.
Technically, the docker-compose works out-of-the-box, as I can launch it and get access to OSD without any trouble, but the exception logs still spawn.
For the “dev” OSD I speak about in my messages, I mean that I cloned the repo and yarn start-ed it. In this case it doesn’t work with a dockerized OS if I don’t apply the changes I posted above.
Are you running anything else on your box? Perhaps something that’s trying to connect over and over to OpenSearch?
With regards to OpenSearch Dashboards, that makes more sense. I believe that the docker/docker compose initialization process does a lot of the work you are describing - which makes sense for a quick start. However, it’s not baked into OpenSearch as those are not sane defaults for a production environment.
Your intention was to start developing on OpenSearch Dashboards? Or just running? [I ask because I want to know if we should make a change in documentation or developer guides]
Aah yes you’re right I guess. I have multiple services that may be trying to connect to it. I can’t debug it right now, but I don’t think that there should be this huge exception thrown in the logs anyway. Could be investigated maybe!
EDIT: I can confirm that it’s one of my services that triggers the exception, they were trying to contact a previously-running ElasticSearch instance.
I’m currently migrating a Kibana plugin to OSD, so I couldn’t just use the docker image for OSD. I may also be developing for OSD in the near future.
I agree with you that having “none” as a default for the certificate checks is a no-no, that’s why I thought about updating the documentation (or the dev guides) rather than the config. It may be worth adding a small appendix to it!
Thank you so much! This worked for me. Though I found it easier to instead add plugins.security.disabled=true to the docker-compose.yml. That way I won’t have to make a Dockerfile for a new image.
