OpenSearch PHP Client: 404 Error on Document Insertion in Production

[zabs]

Problem

I’m experiencing a 404 NotFoundHttpException when trying to insert documents into an OpenSearch products index using the opensearch-project/opensearch-php client in a Laravel application, but only in production. The same setup for a dev and staging OpenSearch instances is working fine.

What works:

• All operations (including insertion) work fine in dev and staging
• Query, search, delete, and update operations work fine in production
• Direct curl commands for document insertion work in production
• The products index exists and is accessible in all environments

What fails:

• Only programmatic document insertion via PHP client fails in production

Environment

• Framework: Laravel (Docker container)
• OpenSearch PHP Client: opensearch-project/opensearch-php
• Production OpenSearch: https://opensearch.prd.company.com
• Staging OpenSearch: https://opensearch.stg.company.com:443 (working)

Code

Here’s the method that fails:

public function set(string $product_id, array $product_data): array
{
    $request = [
        'index' => 'products',
        'id' => $product_id, // e.g 'product_123'
        'body' => $product_data, // e.g ['name' => 'Product Name', 'price' => 100]
    ];

    return $this->client->index($request); // 404 error occurs here
}

Reproduction

This curl command works successfully in production:

curl -X POST "https://opensearch.prd.company.com/products/_doc/HELLO-PC" \
  -H "Content-Type: application/json" \
  -u admin:*** \
  -d '{"title": "Hello PC", "price": 999.99, "category": "Gaming PCs"}'

But the PHP client method above throws a 404 error.

Error Stack Trace

OpenSearch\Exception\NotFoundHttpException: 404 NotFoundHttpException
#0 /vendor/opensearch-project/opensearch-php/src/OpenSearch/HttpTransport.php(56): 
   OpenSearch\Exception\HttpExceptionFactory::create(404, '')
#1 /vendor/opensearch-project/opensearch-php/src/OpenSearch/Client.php(2182): 
   OpenSearch\HttpTransport->sendRequest('POST', '/products/_doc/...', Array, Array, Array)
#2 /vendor/opensearch-project/opensearch-php/src/OpenSearch/Client.php(1114): 
   OpenSearch\Client->performRequest(Object(OpenSearch\Endpoints\Index))
#3 /app/Services/ProductStore/ProductStoreClient.php(65): 
   OpenSearch\Client->index(Array)

Investigation Results

1. Index exists and is accessible:

   curl -X GET "https://opensearch.prd.company.com/products"
   # Returns index mapping successfully
   

2. Other operations work programmatically in production:

   • Query operations: Working
   • Search operations: Working
   • Delete operations: Working
   • Update operations: Working

3. Environment comparison:

   • Dev: All operations work
   • Staging: All operations work
   • Production: Only document insertion fails

Question

Why would the OpenSearch PHP client throw a 404 error for document insertion when:

• The same operation works via curl
• The index exists and other operations work
• The same code works in other environments

What could cause this environment-specific issue with only the index() method in the PHP client?

[pablo]

@zabs Do you see any errors in OpenSearch logs? Did you use the same user for PHP script and curl command?