Migrating from xpack to security

Paladox · January 8, 2023, 5:46pm

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.2.0

Describe the issue:
How would i migrate:

xpack.security.authc.anonymous.username: elastic
xpack.security.authc.anonymous.roles: superuser
xpack.security.authc.anonymous.authz_exception: true

to security? I’ve looked everywhere and haven’t really found a way.

I just want similar behaviour to how it’s done in ES. We have the firewall blocked so it’s safe for us.

Configuration:

opensearch.yml

plugins.security.ssl.http.enabled: "true"
plugins.security.ssl.http.pemkey_filepath: "/etc/opensearch/ssl/wildcard.miraheze.org-2020-2-key.pem"
plugins.security.ssl.http.pemcert_filepath: "/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.crt"
plugins.security.ssl.http.pemtrustedcas_filepath: "/etc/opensearch/ssl/Sectigo.pem"
plugins.security.ssl.transport.pemkey_filepath: "/etc/opensearch/ssl/wildcard.miraheze.org-2020-2-key.pem"
plugins.security.ssl.transport.pemcert_filepath: "/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.crt"
plugins.security.ssl.transport.pemtrustedcas_filepath: "/etc/opensearch/ssl/Sectigo.pem"
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.nodes_dn:
  - 'CN=*.miraheze.org'
plugins.security.ssl_cert_reload_enabled: "true"
plugins.security.allow_default_init_securityindex: "true"
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]

config.yml

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: true

pablo · January 17, 2023, 9:30am

@Paladox Just to confirm. Are you looking for anonymous authentication in Opensearch?

Paladox · January 21, 2023, 10:43pm

@pablo sorry for late reply, yes please!

Basically just replacing:

            'xpack.security.enabled'                         => true,
            'xpack.security.http.ssl.enabled'                => true,
            'xpack.security.http.ssl.key'                    => '/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.key',
            'xpack.security.http.ssl.certificate'            => '/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.crt',
            'xpack.security.transport.ssl.enabled'           => true,
            'xpack.security.transport.ssl.key'               => '/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.key',
            'xpack.security.transport.ssl.certificate'       => '/etc/opensearch/ssl/wildcard.miraheze.org-2020-2.crt',
            'xpack.security.transport.ssl.verification_mode' => 'certificate',
            # We use a firewall so this is safe
            'xpack.security.authc.anonymous.username'        => 'elastic',
            'xpack.security.authc.anonymous.roles'           => 'superuser',
            'xpack.security.authc.anonymous.authz_exception' => true,

with OS equivalent. We never used the -os version of es (either i forgot or didn’t know that existed).

Paladox · January 31, 2023, 11:36pm

Any update please?

pablo · February 2, 2023, 11:45pm

@Paladox I had to dig in my notes. In OpenSearch, anonymous authentication is not configured in opensearch.yml

Please follow the steps below.

Set “anonymous_auth_enabled” to true in config.yml

Add the below line to opensearch_dashboards.yml

opensearch_security.auth.anonymous_auth_enabled: true

In roles.yml set the following role. (The permissions are just an example.)

opendistro_security_anonymous:
  cluster_permissions:
  - "unlimited"
  index_permissions:
  - index_patterns:
    - "*"
    allowed_actions:
    - "unlimited"
  tenant_permissions:
  - tenant_patterns:
    - "global_tenant"
    allowed_actions:
    - "kibana_all_write"

In roles_mapping.yml configure the mapping of the anonymous backend role

opendistro_security_anonymous:
  backend_roles:
  - "opendistro_security_anonymous_backendrole"

Please remember to apply security plugin configuration changes with securityadmin.sh script.

Paladox · February 5, 2023, 7:56pm

Thank you so much! it works!

Although running /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh
I get:

Will connect to test131.miraheze.org:9200 ... done
Connected as "CN=*.miraheze.org"
ERR: "CN=*.miraheze.org" is not an admin user
Seems you use a node certificate. This is not permitted, you have to use a client certificate and register it as admin_dn in opensearch.yml

(I ofc deleted /var/lib/opensearch and restarted the service to recreate it but was seeing how’d I update it if it was in production because we wouldn’t be able to delete it all to recreate it when we change the config).

Paladox · February 5, 2023, 8:04pm

Adding:

plugins.security.authcz.admin_dn:
  - 'CN=*.miraheze.org'

allowed ,me to proceed but I get:

ERR: Seems you use a node certificate which is also an admin certificate
     That may have worked with older OpenSearch Security versions but it indicates
     a configuration error and is therefore forbidden now.
OpenSearch Version: 2.2.0

although it still continued after that.

WARNING: nor OPENSEARCH_JAVA_HOME nor JAVA_HOME is set, will use /usr/bin/java
Security Admin v7
Will connect to test131.miraheze.org:9200 ... done
Connected as "CN=*.miraheze.org"
ERR: Seems you use a node certificate which is also an admin certificate
     That may have worked with older OpenSearch Security versions but it indicates
     a configuration error and is therefore forbidden now.
OpenSearch Version: 2.2.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: miraheze-general
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!
Populate config from /etc/opensearch/opensearch-security/
Will update '/config' with /etc/opensearch/opensearch-security/config.yml (legacy mode)
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/opensearch/opensearch-security/roles.yml (legacy mode)
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/opensearch/opensearch-security/roles_mapping.yml (legacy mode)
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/opensearch/opensearch-security/internal_users.yml (legacy mode)
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/opensearch/opensearch-security/action_groups.yml (legacy mode)
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/nodesdn' with /etc/opensearch/opensearch-security/nodes_dn.yml (legacy mode)
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/opensearch/opensearch-security/whitelist.yml (legacy mode)
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/opensearch/opensearch-security/audit.yml (legacy mode)
   SUCC: Configuration for 'audit' created or updated
SUCC: Expected 7 config types for node {"updated_config_types":["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"],"updated_config_size":7,"message":null} is 7 (["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"]) due to: null
Done with success

Paladox · February 12, 2023, 2:06pm

@pablo do you have any idea ^?

pablo · February 13, 2023, 9:59am

@Paladox Does your admin_dn in opensearch.yml contian ‘CN=*.miraheze.org’ ?

Paladox · February 14, 2023, 4:06pm

@pablo Yes for “plugins.security.authcz.admin_dn”.

Paladox · February 19, 2023, 4:18pm

@pablo bump please

pablo · February 20, 2023, 12:12am

@Paladox You’re using a wildcard certificate in admin_dn and nodes_dn. When you run securityadmin.sh the certificate’s subject matches both node_dn and admin_dn. That’s why the security plugin reports it as an error.

Admin certificate should never be a node certificate. I strongly suggest using a specific CN for admin_dn entries instead of wildcards.

Paladox · March 4, 2023, 4:13pm

Seems that causes it to fail with:

Connected as "CN=*.miraheze.org"
ERR: "CN=*.miraheze.org" is not an admin user

Paladox · March 5, 2023, 4:47pm

So from what I understand you have to create a admin cert that you use with the script and use plugins.security.authcz.admin_dn that matches it. But in the docs it says something about a root ca/key.

How do I do it so we can use our paid certificate for the nodes and then just a self generated one for the admin cert?

pablo · March 5, 2023, 5:26pm

@Paladox If the admin cert and node cert has a different root CA then you can bind both CA’s in one pem certificate and use it with plugins.security.ssl.http.pemtrustedcas_filepath option in opensearch.yml.

The root PEM file should look like the one below.

-----BEGIN CERTIFICATE-----
<admin Root CA certificate> 
-----END CERTIFICATE----- 
-----BEGIN CERTIFICATE-----
<node Root CA certificate > 
-----END CERTIFICATE-----

Paladox · April 5, 2023, 10:03pm

I guess the same applies to private kets / certificates?

Paladox · April 6, 2023, 7:47pm

Hmm… I guess not.

I generated a self signed certificate for the admin cert. We’re using the wildcard cert for the rest http/transport (the one we paid for)

On the other hand I’m getting:

**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
WARNING: nor OPENSEARCH_JAVA_HOME nor JAVA_HOME is set, will use /usr/bin/java
Security Admin v7
WARN: It makes no sense to specify -cd as well as -r
Will connect to test131.miraheze.org:9200 ... done
ERR: An unexpected SSLHandshakeException occured: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
See https://opensearch.org/docs/latest/clients/java-rest-high-level/ for troubleshooting.
Trace:
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
See https://opensearch.org/docs/latest/clients/java-rest-high-level/ for troubleshooting.
	at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:947)
	at org.opensearch.client.RestClient.performRequest(RestClient.java:332)
	at org.opensearch.client.RestClient.performRequest(RestClient.java:320)
	at org.opensearch.security.tools.SecurityAdmin.execute(SecurityAdmin.java:462)
	at org.opensearch.security.tools.SecurityAdmin.main(SecurityAdmin.java:159)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:353)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:296)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:291)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1076)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1063)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1010)
	at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:285)
	at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:345)
	at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:523)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1335)
	... 19 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 25 more

any ideas how to fix that?

Topic		Replies	Views
Migration from Elastic Search 7.10 with xpack.security to Opensearch 2x OpenSearch	1	399	July 27, 2023
Migration es7.10.2 opensearch 2.4.0 Security upgrade	9	1269	December 19, 2023
Transport client authentication no longer supported error in new installed cluster and security plugin enabled Open Source Elasticsearch and Kibana discuss , troubleshoot , configure	1	414	December 22, 2023
Locked out of .opendistro_security Security configure	4	742	March 3, 2022
X-pack to aws security conversion for LDAP Security	1	965	April 15, 2021

Migrating from xpack to security

Related topics