401, AuthenticationException while connecting to OpenSearch using opensearch-py basic authentication

[rizwan_kn]

Hi,

I have an AWS Lambda function written in python which uses opensearch-py==2.2.0

I am able to successfully connect to my OpenSearch cluster, check if an index exists and create an index through the below code,

Code:

from opensearchpy import OpenSearch

es = OpenSearch(hosts = [{'host': host, 'port': 443}], http_auth = ('username', 'password'), use_ssl = True, verify_certs = True)

es_index = "test_index"

if not es.indices.exists(index=es_index):
    logger.info("ES Index does not exist, creating")
    es.indices.create(index=es_index, body={
        'settings' : {
                'index' : {
                    'number_of_shards':2
                }
        }
    })

I have a new requirement to create an index pattern as well, after doing some initial research I was able to find that it can be done through the below python code,

Code:

url = '/_dashboards/api/saved_objects/index-pattern'
payload = {"attributes":{"title":"test_index_pattern"}}
headers = {"Content-Type": "application/json", "osd-xsrf": "true", "security_tenant": "global" }

response = es.transport.perform_request(method='POST', url=url, headers=headers, body=payload)

But when I execute this code I am getting the below error,

Error:

"errorType": "AuthenticationException", "errorMessage": "AuthenticationException(401, 'Unauthorized', 'Unauthorized')

When I update my client to use a different authentication method as below I am able to successfully create the index pattern. With this approach the role used to run the code must be mapped to the backed role since my cluster is FGAC enabled.

Code:

from opensearchpy import OpenSearch, RequestsHttpConnection, AWSV4SignerAuth

service = 'es'
credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, region, service)

es = OpenSearch(hosts = [{'host': host, 'port': 443}], http_auth = auth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection)

To summarize, when I use basic authentication I am able to perform other api calls without any issues, but when I try to create the index pattern through the above approach it is giving me a 401 authentication error. Can someone let me know if why this is happening ?

My requirement is to create an index pattern programmatically using python. Since I have already initialized a client through basic authentication and doing other api calls using opensearchpy, I would like to use the same client to do the ‘perform_request’ api call and create the index pattern.