S3-repository doesn't work with private pki?

OpenSearch
,
1

Hello,

I’m trying to configure s3-repository plugins with a self-hosted Ceph.

Ceph is configured with a certificate from my private PKI, and I have configured Opensearch to use a truststore containing CA and root CA certificates:

plugins.security.ssl.transport.truststore_filepath: /usr/share/opensearch/config/opensearch-security/certs/truststore_.jks
plugins.security.ssl.http.truststore_filepath: /usr/share/opensearch/config/opensearch-security/certs/truststore.jks

s3.client.default.endpoint: ceph-rgw.int
s3.client.default.max_retries: 3
s3.client.default.path_style_access: true
s3.client.default.protocol: https
s3.client.default.read_timeout: 50s
s3.client.default.use_throttle_retries: true

Unfortunately, register repository fail despite the configuration of the truststore and the plugins does not find CA certificates :

{
  "error": {
    "root_cause": [
      {
        "type": "repository_verification_exception",
        "reason": "[dev-opensearch] path [backup/dc01] is not accessible on cluster-manager node"
      }
    ],
    "type": "repository_verification_exception",
    "reason": "[dev-opensearch] path [backup/dc01] is not accessible on cluster-manager node",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Unable to upload object [backup/dc01/tests-AhCHCRc_ThuW_zjTeccvhQ/master.dat] using a single upload",
      "caused_by": {
        "type": "sdk_client_exception",
        "reason": "sdk_client_exception: Unable to execute HTTP request: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
        "caused_by": {
          "type": "i_o_exception",
          "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
          "caused_by": {
            "type": "validator_exception",
            "reason": "validator_exception: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
            "caused_by": {
              "type": "sun_cert_path_builder_exception",
              "reason": "sun_cert_path_builder_exception: unable to find valid certification path to requested target"
            }
          }
        }
      }
    }
  },
  "status": 500
}

It seems the plugin does not use the variables plugins.security.ssl.transport.truststore_filepath and plugins.security.ssl.http.truststore_filepath ?

How can I configure s3-repository to use the truststore ?

Thanks for your help
Regards

2

Anyone have an idea ?

3

Hey @sab ,

The plugins.security.ssl settings are applicable to security plugin [1] and have nothing to do with S3 repository plugin. The way you could provide your own keystore is through these system properties:

javax.net.ssl.keyStore
javax.net.ssl.keyStorePassword
javax.net.ssl.keyStoreType

Hope it helps (but it may impact other plugins if any)

[1] GitHub - opensearch-project/security: 🔐 Secure your cluster with TLS, numerous authentication backends, data masking, audit logging as well as role-based access control on indices, documents, and fields

4

Hello @reta

Thanks for your reply.
Your solution work fine but yes it impact other plugins.

I think, the better solution is finaly to put all my CA cert in ${JAVA_HOME}/jdk/lib/security/cacerts

And if you want to not maintain two keystore, you can use it in plugins.security.ssl.http.truststore_filepath and plugins.security.ssl.transport.truststore_filepath, but requires to allow opensearch to read the file:

#  cat jdk/conf/security/java.policy 
grant {
    permission java.io.FilePermission "/usr/share/opensearch/jdk/lib/security/cacerts", "read";
};

Regards