AttributeError: 'Opensearch' object has no attribute 'options' while inserting documents via bulk API

Hi,

I was trying to insert documents using bulk api, but it’s giving attribute error.

Ref link-

Code-

import pandas as pd
from elasticsearch import Elasticsearch
from elasticsearch.helpers import bulk
from opensearchpy import OpenSearch

# Datetime, String, Interger Example Dataframe 

listDate = ['2020-01-01 00:00:00','2020-01-01 00:01:00','2020-01-01 00:02:00', '2020-01-01 00:03:00']
listStrings = ['a','b','c','d']
listInterger = [1, 2, 3, 4 ]

df = pd.DataFrame([ x for x in zip(listDate,listStrings,listInterger)], columns=['date','string', 'interger'])
df['date'] = pd.to_datetime(df['date'])

host = 'localhost'
port = 9200
auth = ('admin', 'admin') # For testing only. Don't store credentials in code.
# ca_certs_path = '/full/path/to/root-ca.pem' # Provide a CA bundle if you use intermediate CAs with your root CA.

# Optional client certificates if you don't want to use HTTP basic authentication.
# client_cert_path = '/full/path/to/client.pem'
# client_key_path = '/full/path/to/client-key.pem'

# Create the client with SSL/TLS enabled, but hostname verification disabled.

client = OpenSearch(
            hosts = [{'host': host, 'port': port}],
            http_compress = True, # enables gzip compression for request bodies
            http_auth = auth,
        # client_cert = client_cert_path,
        # client_key = client_key_path,
            use_ssl = True,
            verify_certs = False,
            ssl_assert_hostname = False,
            ssl_show_warn = False,
            # ca_certs = ca_certs_path
             )

# Create an index with non-default settings.
index_name = 'bulk'
index_body = {
  'settings': {
    'index': {
      'number_of_shards': 4
    }
  }
}
response = client.indices.create(index_name, body=index_body)
df = pd.DataFrame(data = {'date' : df['date'],
                          'strings': df['string'],
                          'interger' : df['interger']})
 
documents = df.to_dict(orient='records')
print(documents)
bulk(client, documents, index='bulk',doc_type='foo', raise_on_error=True)

Error-

(devenv) E:\env>python bulk.py
[{'date': Timestamp('2020-01-01 00:00:00'), 'strings': 'a', 'interger': 1}, {'date': Timestamp('2020-01-01 00:01:00'), 'strings': 'b', 'interger': 2}, {'date': Timestamp('2020-01-01 00:02:00'), 'strings': 'c', 'interger': 3}, {'date': Timestamp('2020-01-01 00:03:00'), 'strings': 'd', 'interger': 4}]Traceback (most recent call last):
  File "E:\env\bulk.py", line 55, in <module>
    bulk(client, documents, index='bulk',doc_type='foo', raise_on_error=True)
  File "E:\env\devenv\lib\site-packages\elasticsearch\helpers\actions.py", line 524, in bulk
    for ok, item in streaming_bulk(
  File "E:\env\devenv\lib\site-packages\elasticsearch\helpers\actions.py", line 410, in streaming_bulk
    client = client.options()
AttributeError: 'OpenSearch' object has no attribute 'options'

Pls let me know how to use bulk api in opensearch.

Welcome to the community @divyank_1 - I wonder if the information in this topic would help with your error: Problem with mapping data (timestamp) types by using Bulk API - #3 by q2dg

@dtaivpp @pablo - any input is appreciated

Hey the first issue I see here is you are trying to use the Elasticsearch bulk method with the OpenSearch client.

This is what the code is seeing

from elasticsearch.helpers import bulk
from opensearchpy import OpenSearch

client = OpenSearch(x,y,z)

# Here is the first thing I see
# the bulk method is imported from the Elasticsearch
# python library but then passed an OpenSearch client
bulk(client, y, z)

You should probably be able to do something along these lines:

from opensearchpy import OpenSearch
client = OpenSearch(x,y,z)

body = {some body ndjson}

client.bulk(body, x, y, z)

Here are the docs for that: client — OpenSearch Python Client documentation

1 Like

Thanks @kris @dtaivpp for the replies.

As @dtaivpp mentioned I was using elastic helpers for bulk operation
from elasticsearch.helpers import bulk.

After changing to opensearch helpers, I am able to do bulk operations.
from opensearchpy import OpenSearch, RequestsHttpConnection, helpers.

2 Likes

Happy to help and glad its working well for you!

1 Like