# External Hosted Model Error: Invalid payload

**URL:** <https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795>\
**Category:** Machine Learning\
**Created:** [June 17, 2025, 3:37am UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795 "2025-06-17T03:37:27Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![wijamw](https://avatars.discourse-cdn.com/v4/letter/w/50afbb/32.png) [@wijamw](https://forum.opensearch.org/u/wijamw)\
**Post date:** [June 17, 2025, 3:37am UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/1 "2025-06-17T03:37:27Z")

</div>

**Versions** (OpenSearch Dashboard v 3.0.0):

**Describe the issue** :  
Azure OpenAI external host connector is added. But the model return `IllegalArgumentException: Invalid payload: { \"messages\": ${parameters.messages}, \"temperature\": 0.7 }`  
Strangely, this issue appear when I tried to test the model using this: [RAG chatbot with a conversational flow agent - OpenSearch Documentation](https://docs.opensearch.org/docs/latest/tutorials/gen-ai/chatbots/rag-conversational-agent/#step-24-test-the-model)  
But when I tried use the test described in the bluprint, it is working fine.  
I tried to register an agent with below config but it also throws an error.

**Configuration** :  
I follow the connector blueprint via [ml-commons/docs/remote\_inference\_blueprints/azure\_openai\_connector\_chat\_blueprint.md at main · opensearch-project/ml-commons · GitHub](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/azure_openai_connector_chat_blueprint.md)

**Relevant Logs or Screenshots** :

1. Agent Configuration

```auto
POST /_plugins/_ml/agents/_register
{
  "name": "Plan execute and reflect agent",
  "type": "plan_execute_and_reflect",
  "description": "this is a test agent",
  "llm": {
    "model_id": "dMEk-JYBuC4BoDwKx4IS"
  },
  "memory": {
    "type": "conversation_index"
  },
  "parameters": {
    "_llm_interface": "openai/v1/chat/completions"
  },
  "tools": [
    {
      "type": "ListIndexTool"
    },
    {
      "type": "SearchIndexTool"
    },
    {
      "type": "IndexMappingTool"
    }
  ],
  "app_type": "os_chat"
}

```

1. Connector Config  
 ![connector](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/d/dae9a9d6a731a7d59bfcad1c26a924924bb7922e.png)

---

<div class="post-metadata">

**Author:** ![pybot](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pybot](https://forum.opensearch.org/u/pybot)\
**Post date:** [June 19, 2025, 1:18am UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/2 "2025-06-19T01:18:25Z")

</div>

Hi @wijamw

Simple predict and agent use require the request\_bodies to be configured differently.

In order to use the agent with your open AI connector, you need to ensure that the connector is configured appropriately. To be specific, you need to ensure that the below fields are present in your request\_body according to how Azure Open AI accepts these fields:

```auto
${parameters.system_prompt} -- where the system prompt must be passed in the body
${parameters._chat_history:-} -- how the chat history is passed
${parameters.prompt} -- where the actual prompt must be provided
${parameters._interactions:-} -- where interactions between tool uses are present
${parameters.tool_configs:-} -- how the tools are provided via function calling to the model

```

Refer to this document for how you can configure openai-gpt-4o with this agent:

> **[Plan-execute-reflect agents](https://docs.opensearch.org/docs/latest/ml-commons-plugin/agents-tools/agents/plan-execute-reflect/#openai-gpt-4o)**
>
> Plan-execute-reflect agents Introduced 3.0

It should be similar for Azure Open AI as well. Let me know if you need any additional help.

---

<div class="post-metadata">

**Author:** ![wijamw](https://avatars.discourse-cdn.com/v4/letter/w/50afbb/32.png) [@wijamw](https://forum.opensearch.org/u/wijamw)\
**Post date:** [June 26, 2025, 2:59pm UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/3 "2025-06-26T14:59:09Z")

</div>

Thank you. I did what you recommend and use system\_prompt parameter on the agent creation but I got this error when I test the agent.

 ![image (14)](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/4/48afa2fefda2a3b3898fc1b7c2e55b1e5f312aa8.png)

I tried to look around but could not find any articles about this issue.  
Do you have any insight of what could possibly be the problem here? My configuration or the AI Models endpoint?

---

<div class="post-metadata">

**Author:** ![pybot](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pybot](https://forum.opensearch.org/u/pybot)\
**Post date:** [June 27, 2025, 12:17am UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/4 "2025-06-27T00:17:34Z")

</div>

Can you please share the latest connector/model creation and agent creation scripts?

The error is implying that the host is unreachable which probably means there is an issue with the connector setup.

---

<div class="post-metadata">

**Author:** ![wijamw](https://avatars.discourse-cdn.com/v4/letter/w/50afbb/32.png) [@wijamw](https://forum.opensearch.org/u/wijamw)\
**Post date:** [July 2, 2025, 2:00am UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/5 "2025-07-02T02:00:17Z")

</div>

I apologise for not including it before.  
Agent creation script:

```auto
POST /_plugins/_ml/agents/_register
{
  "name": "My Plan Execute Reflect Agent",
  "type": "plan_execute_and_reflect",
  "description": "Agent for dynamic task planning and reasoning",
  "llm": {
    "model_id": "input_your_model_id",
    "parameters": {
      "prompt": "${parameters.question}"
    }
  },
  "memory": {
    "type": "conversation_index"
  },
  "parameters": {
    "_llm_interface": "openai/v1/chat/completions",
    "system_prompt": "You are part of an OpenSearch cluster. When you deliver your final result, include a comprehensive report. This report MUST:\n1. List every analysis or step you performed.\n2. Summarize the inputs, methods, tools, and data used at each step.\n3. Include key findings from all intermediate steps — do NOT omit them.\n4. Clearly explain how the steps led to your final conclusion.\n5. Return the full analysis and conclusion in the 'result' field, even if some of this was mentioned earlier.\n\nThe final response should be fully self-contained and detailed, allowing a user to understand the full investigation without needing to reference prior messages. Always respond in JSON format."
  },
  "tools": [
    { "type": "ListIndexTool" },
    { "type": "SearchIndexTool" },
    { "type": "IndexMappingTool" }
  ],
  "app_type": "os_chat"
}

```

Connector Settings:

```auto
{
  "name": "azure-openai-agent-connector",
  "version": "1",
  "description": "Connector for Azure OpenAI using GPT-4.1",
  "protocol": "http",
  "parameters": {
    "api-version": "2024-12-01-preview",
    "endpoint": "resource_endpoint.openai.azure.com",
    "temperature": "0.7",
    "model": "gpt-4.1",
    "deploy-name": "deployment_name"
  },
  "actions": [
    {
      "action_type": "PREDICT",
      "method": "POST",
      "url": "https://${parameters.endpoint}/openai/deployments/${parameters.deployment-name}/chat/completions?api-version=${parameters.api-version}",
      "headers": {
        "api-key": "${credential.api_key}"
      },
      "request_body": """{ "messages": [{"role":"system","content":"${parameters.system_prompt}"},${parameters._chat_history:-}{"role":"user","content":"${parameters.user_prompt}"}], "temperature": ${parameters.temperature}, "max_tokens": 4096 }"""
    }
  ],
  "created_time": 1750826267478,
  "last_updated_time": 1750826267478
}

```

---

<div class="post-metadata">

**Author:** ![pybot](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pybot](https://forum.opensearch.org/u/pybot)\
**Post date:** [July 3, 2025, 9:09pm UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/6 "2025-07-03T21:09:33Z")

</div>

Your request body should look like this:

```auto
"request_body": "{ \"model\": \"${parameters.model}\", \"messages\": [{\"role\":\"developer\",\"content\":\"${parameters.system_prompt}\"},${parameters._chat_history:-}{\"role\":\"user\",\"content\":\"${parameters.prompt}\"}${parameters._interactions:-}]${parameters.tool_configs:-} }"

```

In the request body please also ensure:  
`prompt`, `interactions`, `tool_configs` are also part of it.

Secondly, your ${parameters.deployment-name} has no corresponding parameter to it. Because of which the url that is being hit is probably malformed or not found. Please make sure url is accurate. Internally, the parameter values are replaced in the url. If you wish, you can put in the entire url itself without using the parameters.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [September 1, 2025, 9:10pm UTC](https://forum.opensearch.org/t/external-hosted-model-error-invalid-payload/24795/7 "2025-09-01T21:10:07Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.