Hi,
I would like to work with node and client certificates in Kubernetes
Let’s assume I have:
Let’s say I managed to generate the certificates and everything is working great
But now what will happen if I need to scale my OpenDistro cluster?
Is there any solution for that in terms of the certificates generation?
Thanks!
Hi @mfuxi
You can generate new SSL certificates using the root CA cert of existing ES nodes with openssl.
CN or SAN of the new SSL certificate has to match the FQDN of the new node.
Thanks, @pablo
I struggled a lot yesterday with using my own self-certificates and at the end it didn’t work for me
When I deployed OpenDistro with my self-signed certificates the first pod which was Kibana kept on CrashLooping with the following error:
FATAL Error: [config validation of [elasticsearch].ssl.admin]: definition for this key is missing
Any idea what could this mean?
I was trying to enable in any way I could the following flag:
elasticsearch.ssl.admin.enabled
But it just won’t work, can you share an example of how to deploy the chart with a self-signed certificates? (I added the certificates to the needed secrets and used them just as the chart documentation suggests)
What is your ODFE version? Could you share the elasticsearch.yml file from that node?
Is that error coming out from Elasticsearch logs or container/pod?
Can’t find definition of that parameter in ES official documentation and ODFE.
The elasticsearch.yml stays the same as it was in the example unfortunately, according to the documentation it should have changed according to the flags I provided.
Regarding where the error is coming from, it’s when I do:
kubectl logs opendistro-es-kibana-.....
And this is the full error:
{"type":"log","@timestamp":"2021-08-08T15:24:34Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2021-08-08T15:24:34Z","tags":["fatal","root"],"pid":1,"message":"{ Error: [config validation of [elasticsearch].ssl.admin]: definition for this key is missing\n at ObjectType.validate (/usr/share/kibana/node_modules/@kbn/config-schema/target/out/types/type.js:62:19)\n at ConfigService.validateAtPath (/usr/share/kibana/node_modules/@kbn/config/target/config_service.js:165:23)\n at MapSubscriber.getDistinctConfig.pipe.operators_1.map [as project] (/usr/share/kibana/node_modules/@kbn/config/target/config_service.js:172:83)\n at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:49:35)\n at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n at DistinctUntilChangedSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)\n at DistinctUntilChangedSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:55:26)\n at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n at ReplaySubject._subscribe (/usr/share/kibana/node_modules/rxjs/internal/ReplaySubject.js:76:28)\n at ReplaySubject.Observable._trySubscribe (/usr/share/kibana/node_modules/rxjs/internal/Observable.js:44:25)\n at ReplaySubject.Subject._trySubscribe (/usr/share/kibana/node_modules/rxjs/internal/Subject.js:102:51)\n at ReplaySubject.Observable.subscribe (/usr/share/kibana/node_modules/rxjs/internal/Observable.js:30:22)\n at MapSubscriber.shareReplayOperation (/usr/share/kibana/node_modules/rxjs/internal/operators/shareReplay.js:48:32)\n at Observable.subscribe (/usr/share/kibana/node_modules/rxjs/internal/Observable.js:25:31)\n at MapOperator.call (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:32:23)\n cause:\n { Error: definition for this key is missing\n at ObjectType.onError (/usr/share/kibana/node_modules/@kbn/config-schema/target/out/types/type.js:87:20)\n at type.Type.schema.error (/usr/share/kibana/node_modules/@kbn/config-schema/target/out/types/type.js:52:53)\n at finish (/usr/share/kibana/node_modules/joi/lib/types/any/index.js:529:50)\n at type._validate (/usr/share/kibana/node_modules/joi/lib/types/any/index.js:618:24)\n at type._base (/usr/share/kibana/node_modules/joi/lib/types/object/index.js:212:45)\n at type._validate (/usr/share/kibana/node_modules/joi/lib/types/any/index.js:614:37)\n at type._validateWithOptions (/usr/share/kibana/node_modules/joi/lib/types/any/index.js:674:29)\n at module.exports.internals.Any.root.validate (/usr/share/kibana/node_modules/joi/lib/index.js:146:23)\n at ObjectType.validate (/usr/share/kibana/node_modules/@kbn/config-schema/target/out/types/type.js:57:72)\n at ConfigService.validateAtPath (/usr/share/kibana/node_modules/@kbn/config/target/config_service.js:165:23)\n at MapSubscriber.getDistinctConfig.pipe.operators_1.map [as project] (/usr/share/kibana/node_modules/@kbn/config/target/config_service.js:172:83)\n at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:49:35)\n at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n at DistinctUntilChangedSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)\n at DistinctUntilChangedSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:55:26) cause: undefined, path: [ 'ssl', 'admin' ] } }"}
FATAL Error: [config validation of [elasticsearch].ssl.admin]: definition for this key is missing
In my opinion this error states that Kibana doesn’t know elasticsearch.ssl.admin option. I couldn’t find that in docs either. You should check all your ODFE and Kube config files and remove that option.
Also this doesn’t look like security plugin issue either. It’s your kibana.yml file validation fails.
Sounds right, but the error started when I enabled the self-signed certs and did all the changes that were needed for that, before that when I used the demo certs I didn’t see this error
SSL certificate can’t cause a validation error. Maybe your pod config was already changed and the restart of the pod applied an incorrect option.