Getting Error (429) while Inserting Data in Index

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

AWS OpenSearch 2.3

Describe the issue:

While Inserting the Bulk Data from multiple resource in multiple index I will get many time status code 429.

Kindly Help me out what should we do to resolve the same to insert multiple data with the help of bulk command

Configuration:

Relevant Logs or Screenshots:

2

A 429 typically happens when the thread pool queue for writes fills up. Then OpenSearch will start rejecting data.

You’d basically have to try to make sure OpenSearch writes keep up with the load. Here are a few things to try:

  • make sure the client (e.g. Logstash) doesn’t use too many threads or batches that are too tiny
  • make sure that your cluster is balanced (i.e. don’t have hotspots) and that indices aren’t too big
  • if your IO is slow, try making the tlog async
  • increase the refresh interval if you can afford it
  • remove data from the mapping that you don’t need (e.g. maybe you don’t need both keyword and text for all strings)
  • add more nodes :slight_smile: