403 error for the user with certificate based authentication

We are trying to use Cert based authentication as client auth and followed this: Client certificate authentication - OpenSearch documentation. I created a new certificate and assigned that CN to the role “all_access” by calling below API. But, When I call any API with this cert, I am getting 403 error.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "no permissions for [cluster:monitor/health] and User [name=<username>, backend_roles=[], requestedTenant=null]"
      }
    ],
    "type" : "security_exception",
    "reason" : "no permissions for [cluster:monitor/health] and User [name=<username>, backend_roles=[], requestedTenant=null]"
  },
  "status" : 403
}
curl -k --cert /etc/elasticsearch/ssl/admin.pem --key /etc/elasticsearch/ssl/admin-key.pem --location --request PUT 'https://localhost:9200/_opendistro/_security/api/rolesmapping/readall' --header 'Content-Type: application/json' -d '{
  "backend_roles" : ["all_access"],
  "users" : [ "CN name in cert." ]
}'

Can you please help, what am I missing here. It seems the user is not mapped to any role. But, I am calling above API to map user to the role.

@ddramireddy Try adding --cacert ca.crt to your curl command.

Is the admin_dn in opensearch.yml the same as DN of the admin.pem?

Thank you @pablo for the reply.

This CURL command to create new user is working and I am able to create a new user with admin.pem. But, with the new created user, If I try to access cluster health API, I am getting the permissions error. These are the steps I did.

  1. Created a new user
curl -k --cert /etc/ssl/admin.pem --key /etc/ssl/admin-key.pem  --location --request PUT 'https://localhost:9200/_opendistro/_security/api/internalusers/esclient.de1-es \
> --header 'Content-Type: application/json' \
> -d '{
>   "password" : "testsampledasada",
>   "opendistro_security_roles": ["security_rest_api_access"],
>   "backend_roles": ["readall_and_monitor"]
> }'
{"status":"OK","message":"'esclient.de1-es' updated."}
  1. Assign that user to a back-end role for cert based auth.
curl -k --cert /etc/ssl/admin.pem --key /etc/ssl/admin-key.pem  --location --request PUT 'https://localhost:9200/_opendistro/_security/api/rolesmapping/readall' --header 'Content-Type: application/json' --header 'Authorization: Basic YWRtaW46YWRtaW4=' -d '{
  "backend_roles" : ["readall_and_monitor" ],
  "users" : [ "esclient.de1-es" ]
}'
{"status":"OK","message":"'readall' updated."}

  1. Now, calling the cluster health API with the new user is failing.
curl -k --cert /home/eng/esclient.de1-es.crt --key /home/eng/esclient.de1-es.key  -X GET "https://localhost:9200/_cluster/health?pretty"
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "no permissions for [cluster:monitor/health] and User [name=esclient.de1-zpa-es, backend_roles=[], requestedTenant=null]"
      }
    ],
    "type" : "security_exception",
    "reason" : "no permissions for [cluster:monitor/health] and User [name=esclient.de1-es, backend_roles=[], requestedTenant=null]"
  },
  "status" : 403
}

My understanding is, From the certificate, it picks up the CN, i.e. esclient-de1-es and with that user, it will match against the back-end role. In this case, in step-1 & 2, we mapped user to readall_and_monitor role. So, it should be able to query elastic search cluster. But this is failing with 403 error that user doesn’t have permission.

I added --cacert option as well. It didn’t help. As I am using -k option, it disables ssl check at client side. The error I am receiving is user don’t have permissions from elastic search. For some reason, elastic search seems not able to map this user to any role.

@ddramireddy Could you share your config.yml file?
What is the ODFE version?