OpenAI Flow Agent not Workking

OpenSearch
1

Versions:

  • OpenSearch: 3.6.0

  • OpenSearch Dashboards: 3.6.0

I encountered an issue while configuring the OpenSearch Assistant with a custom OpenAI connector.

The connector was created successfully and is visible in the .plugins-ml-connector index. Below is the connector configuration that was used:

{
  "name": "OpenAI Chat Connector",
  "description": "Connector to OpenAI GPT-4o Mini",
  "version": "1",
  "protocol": "http",
  "parameters": {
    "endpoint": "api.openai.com",
    "max_tokens": "8000",
    "temperature": "0.7",
    "response_filter": "$.choices[0].message.content",
    "model": "gpt-4o-mini"
  },
  "actions": [
    {
      "action_type": "PREDICT",
      "method": "POST",
      "url": "https://${parameters.endpoint}/v1/chat/completions",
      "headers": {
        "Authorization": "Bearer ${credential.openAI_key}"
      },
      "pre_process_function": "connector.pre_process.openai.chat_completion",
      "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": ${parameters.messages} }"
    }
  ]
}

After creating the connector, I attempted to create an AI Agent using the OpenSearch Assistant UI. However, when testing the agent, the request failed and an error was returned.

Configuration:

Relevant Logs or Screenshots:

image
image1594Ă—921 68 KB

2

@parthpatel thank you for the question. Have you seen the previous forum case here. It seems to be related to the same issue.

Can you try and confirm if this works for you?

3

Further more, after testing locally with 3.6.0, I can confirm the following works as expected:

1. Cluster settings:


PUT /_cluster/settings
{
  "persistent": {
    "plugins.ml_commons.only_run_on_ml_node": false,
    "plugins.ml_commons.trusted_connector_endpoints_regex": ["^https://api\\.openai\\.com/.*$"]
  }
}

2. Create connector (response_filter in parameters, not in action):

POST /_plugins/_ml/connectors/_create
{
  "name": "OpenAI Chat Connector",
  "description": "Connector for OpenAI with conversation memory",
  "version": 1,
  "protocol": "http",
  "parameters": {
    "endpoint": "api.openai.com",
    "model": "gpt-4o-mini",
    "response_filter": "$.choices[0].message.content"
  },
  "credential": {
    "openAI_key": "sk-proj-..."
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://${parameters.endpoint}/v1/chat/completions",
      "headers": {
        "Authorization": "Bearer ${credential.openAI_key}",
        "Content-Type": "application/json"
      },
      "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": [{\"role\":\"system\",\"content\":\"You are a helpful assistant.\"},{\"role\":\"user\",\"content\":\"${parameters.prompt}\"}] }"
    }
  ]
}

Save the connector_id from the response.

3. Register model:

POST /_plugins/_ml/models/_register
{
  "name": "OpenAI gpt-4o-mini",
  "function_name": "remote",
  "connector_id": "<connector_id>"
}

Check task until COMPLETED:

GET /_plugins/_ml/tasks/<task_id>

Save the model_id.

4. Deploy model:

POST /_plugins/_ml/models/<model_id>/_deploy

Check task until COMPLETED:

GET /_plugins/_ml/tasks/<task_id>

5. Verify model works before touching agents:

POST /_plugins/_ml/models/<model_id>/_predict
{
  "parameters": {
    "prompt": "say hello"
  }
}

6. Create inner conversational_flow agent:

POST /_plugins/_ml/agents/_register
{
  "name": "OpenAI Chat Agent",
  "type": "conversational_flow",
  "description": "Conversational agent with memory",
  "app_type": "os_chat",
  "memory": {
    "type": "conversation_index"
  },
  "tools": [
    {
      "type": "MLModelTool",
      "name": "chat_model",
      "description": "General purpose chat model",
      "parameters": {
        "model_id": "<model_id>",
        "prompt": "${parameters.chat_history:-}Human: ${parameters.question}\n\nAssistant:"
      },
      "include_output_in_agent_response": true
    }
  ]
}

Save the agent_id (inner agent).

7. Create root flow agent:

POST /_plugins/_ml/agents/_register
{
  "name": "Dashboard Root Agent",
  "type": "flow",
  "description": "Root agent for Dashboard Assistant",
  "tools": [
    {
      "type": "AgentTool",
      "name": "LLMResponseGenerator",
      "parameters": {
        "agent_id": "<inner_agent_id>"
      },
      "include_output_in_agent_response": true
    }
  ],
  "memory": {
    "type": "conversation_index"
  }
}

Save the agent_id (root agent).

8. Register as os_chat_root_agent:

PUT /.plugins-ml-config/_doc/os_chat
{
  "type": "os_chat_root_agent",
  "configuration": {
    "agent_id": "<root_agent_id>"
  }
}

In order to register the last step 8, I had to add plugins.security.system_indices.enabled=false in opensearch.yml file.

Hope this helps

4

@Anthony The previous approach worked successfully for me, and I was able to create a conversational agent.

My next objective is to create an agent that can answer questions based on the data stored in my OpenSearch index. Additionally, I would like to configure this agent as the default agent within OpenSearch.

I have attempted to achieve this using the following configuration; however, it is not working as expected


PUT /_plugins/_ml/agents/7glVcp4BOqDJoG0wS3MK
{
  "name": "Agami AI Agent",
  "type": "flow",
  "description": "Flow agent that searches OpenSearch cluster",
  "tools": [
    {
      "type": "QueryPlanningTool",
      "include_output_in_agent_response": false,
      "parameters": {
        "model_id": "0Y7gbp4BOqDJoG0wA_sz",
        "generation_type": "llmGenerated",
        "prompt": "Generate an OpenSearch DSL query for the following question about index ${parameters.index_name}: ${parameters.question}"
      }
    },
    {
      "type": "SearchIndexTool",
      "include_output_in_agent_response": true,
      "parameters": {
        "index": "${parameters.index_name}",
        "query": "${parameters.query}"
      }
    },
    {
      "type": "ListIndexTool",
      "include_output_in_agent_response": true
    },
    {
      "type": "IndexMappingTool",
      "include_output_in_agent_response": true,
      "parameters": {
        "index": "${parameters.index_name}"
      }
    }
  ]
}


POST /_plugins/_ml/agents/7glVcp4BOqDJoG0wS3MK/_execute
{
  "parameters": {
    "index_name": "ss4o_logs-app-2026.05.21",
    "question": "How many documents are in this index?"
  }
}

After This i got error 
{
  "status": 500,
  "error": {
    "type": "JsonParseException",
    "reason": "System Error",
    "details": "Unrecognized token '$': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1]"
  }
}

Also, I want to set this agent as the default so that I can query from Query Assistant

5

@parthpatel You can connect the AI Assistant to a particular index using the following:

POST /_plugins/_ml/connectors/_create
{
  "name": "OpenAI gpt-4o-mini (search-aware)",
  "version": 1,
  "protocol": "http",
  "parameters": {
    "endpoint": "api.openai.com",
    "model": "gpt-4o-mini",
    "response_filter": "$.choices[0].message.content"
  },
  "credential": {
    "openAI_key": "sk-proj-..."
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://${parameters.endpoint}/v1/chat/completions",
      "headers": { "Authorization": "Bearer ${credential.openAI_key}" },
      "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": [{\"role\":\"system\",\"content\":\"${parameters.system_prompt}\\n\\nChat history:\\n${parameters.chat_history:-}\\n\\nSearch results from OpenSearch:\\n${parameters.SearchIndexTool.output:-}\"},{\"role\":\"user\",\"content\":\"${parameters.question}\"}] }"
    }
  ]
}

POST /_plugins/_ml/models/_register?deploy=true
{
  "name": "OpenAI gpt-4o-mini (search-aware)",
  "function_name": "remote",
  "connector_id": "<NEW_CONNECTOR_ID>"
}

POST /_plugins/_ml/agents/_register
{
  "name": "Index-Aware Chat Agent",
  "type": "conversational_flow",
  "app_type": "os_chat",
  "description": "Chat agent with natural language search over my_products",
  "memory": { "type": "conversation_index" },
  "tools": [
    {
      "type": "QueryPlanningTool",
      "parameters": {
        "model_id": "<NEW_MODEL_ID>",
        "index_name": "my_products",
        "question": "${parameters.question}"
      },
      "include_output_in_agent_response": false
    },
    {
      "type": "SearchIndexTool",
      "parameters": {
        "input": "{\"index\": \"my_products\", \"query\": ${parameters.QueryPlanningTool.output}}"
      },
      "include_output_in_agent_response": false
    },
    {
      "type": "MLModelTool",
      "description": "Synthesize search results into a natural language answer",
      "parameters": {
        "model_id": "<NEW_MODEL_ID>",
        "system_prompt": "You are a helpful product catalog assistant. Use the OpenSearch search results provided to answer the user's question about products. If results are empty, say so."
      }
    }
  ]
}

PUT /.plugins-ml-config/_doc/os_chat
{
  "type": "os_chat_root_agent",
  "configuration": {
    "agent_id": "<NEW_AGENT_ID>"
  }
}

You should be able to use AI Assistant to ask question based on the index name “my_products”. However, I was not able to get this working using multiple indices however.

Alternatively you can use the following to query indices via devtools:

POST /_plugins/_ml/connectors/_create
{
  "name": "OpenAI Chat Connector",
  "description": "Connector for OpenAI supporting query planning and search",
  "version": 1,
  "protocol": "http",
  "parameters": {
    "endpoint": "api.openai.com",
    "model": "gpt-4o-mini",
    "response_filter": "$.choices[0].message.content"
  },
  "credential": {
    "openAI_key": "sk-proj-..."
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://${parameters.endpoint}/v1/chat/completions",
      "headers": {
        "Authorization": "Bearer ${credential.openAI_key}",
        "Content-Type": "application/json"
      },
      "request_body": "{\"model\":\"${parameters.model}\",\"messages\":[{\"role\":\"system\",\"content\":\"${parameters.system_prompt}\"},{\"role\":\"user\",\"content\":\"${parameters.user_prompt}\"}]}"
    }
  ]
}

POST /_plugins/_ml/models/_register
{
  "name": "openai-gpt4o-mini",
  "function_name": "remote",
  "description": "GPT-4o-mini via OpenAI",
  "connector_id": "<CONNECTOR_ID>"
}

POST /_plugins/_ml/agents/_register
{
  "name": "NL Search Agent",
  "type": "flow",
  "description": "Natural language search - specify index_name and question",
  "tools": [
    {
      "type": "QueryPlanningTool",
      "parameters": {
        "model_id": "<MODEL_ID>",
        "index_name": "${parameters.index_name}",
        "question": "${parameters.question}"
      },
      "include_output_in_agent_response": true
    },
    {
      "type": "SearchIndexTool",
      "parameters": {
        "input": "{\"index\": \"${parameters.index_name}\", \"query\": ${parameters.QueryPlanningTool.output}}"
      }
    }
  ]
}

POST /_plugins/_ml/agents/<AGENT_ID>/_execute
{
  "parameters": {
    "question": "Show me all electronics",
    "index_name": "my_products"
  }
}