The DSL query is being generated in a different format while executing the agent

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 3.3.0

Describe the issue: Following the OpenSearch documentation, I successfully registered and deployed a model using Amazon Bedrock Claude 3.7 Sonnet, and then registered an agent with the corresponding model ID.

I executed the agent using the following payload:

POST /_plugins/_ml/agents/xxxx/_execute
{
  "parameters": {
    "question": "Find customer details having name 'Allison Hill'",
    "index_name": "customer_data"
  }
}

Result

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "result": """{"metrics":{"latencyMs":1975.0},"output":{"message":{"content":[{"text":"{\n  \"query\": {\n    \"match_phrase\": {\n      \"name\": \"Allison Hill\"\n    }\n  }\n}"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"cacheReadInputTokenCount":0.0,"cacheReadInputTokens":0.0,"cacheWriteInputTokenCount":0.0,"cacheWriteInputTokens":0.0,"inputTokens":4972.0,"outputTokens":37.0,"serverToolUsage":{},"totalTokens":5009.0}}"""
        }
      ]
    }
  ]
}

However, as per the documentation, the following result should be returned while executing the agent.

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "result": "{'query':{'match_phrase':{'name':'Allison Hill'}}}"
        }
      ]
    }
  ]
}

Please suggest if any configuration is missing. Additionally, how to log the agent trace showing reasoning, tool call, tool output, observations internally by the Query Planning tool?

1 Like

@arnabmca2006 Could you share exact link to the documentation that you’ve followed?

Below link I have followed
https://docs.opensearch.org/latest/ml-commons-plugin/agents-tools/tools/query-planning-tool/

1 Like

@arnabmca2006 I think it might be what you’re looking for.

OpenAI: "response_filter": "$.choices[0].message.content"
Anthropic Claude (Amazon Bedrock Converse API): "response_filter": "$.output.message.content[0].text"

I used that with my agent

POST /_plugins/_ml/agents/_register
{
  "name": "Agentic Search with Claude 4",
  "type": "flow",
  "description": "A test agent for query planning.",
  "tools": [
    {
      "type": "QueryPlanningTool",
      "description": "A general tool to answer any question",
      "parameters": {
        "response_filter": "$.output.message.content[0].text",
        "model_id": "kIHzCJoBAjTizxdxU0T5",
        "generation_type": "user_templates",
        "search_templates": [
          {
            "template_id": "flower_species_search_template",
            "template_description": "This template searches for flowers that match the given species using a match query."
          },
          {
            "template_id": "flower_petal_length_range_template",
            "template_description": "This template searches for flowers within a specific petal length range using a range query."
          }
        ]
      }
    }
  ]
}

Running query with agent.

POST /_plugins/_ml/agents/3IEHCZoBAjTizxdxykSA/_execute
{
  "parameters": {
    "question": "How many iris flowers of type setosa are there?",
    "index_name": "iris-index"
  }
}

Response:

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "result": """{"size":0.0,"query":{"term":{"species.keyword":"setosa"}},"aggs":{"setosa_count":{"value_count":{"field":"species.keyword"}}}}"""
        }
      ]
    }
  ]
}

Thanks for your quick support and help.

Any idea, how to log the agent trace showing reasoning, tool call, tool output, observations internally by the Query Planning tool?

@arnabmca2006 for agent tracing we have a processor Agentic context - OpenSearch Documentation need to attach it to the same pipeline for agent tracing. It will let you know the tool called and dsl query generated

I have reviewed the link you provided, but it does not clearly explain how to enable agent tracing and metrics while executing the Query Planning Tool, i.e., POST /_plugins/_ml/agents/agent_id/_execute.

Could you please provide an example demonstrating how to enable these features?

Agent tracing would be available if you use Query Planner Tool with Agentic Search. A complete example is provided Using conversational agents - OpenSearch Documentation and Agentic search - OpenSearch Documentation .
You need to setup an agent with Query Planner Tool.

POST /_plugins/_ml/agents/_register
{
  "name": "E-commerce Search Agent",
  "type": "conversational",
  "description": "Intelligent e-commerce search with product discovery",
  "llm": {
    "model_id": "your-model-id",
    "parameters": {
      "max_iteration": 20
    }
  },
  "memory": {
    "type": "conversation_index"
  },
  "parameters": {
    "_llm_interface": "openai/v1/chat/completions"
  },
  "tools": [
    {
      "type": "ListIndexTool",
      "name": "ListIndexTool"
    },
    {
      "type": "IndexMappingTool",
      "name": "IndexMappingTool"
    },
    {
      "type": "WebSearchTool",
      "parameters": {
        "engine": "duckduckgo"
      }
    },
    {
      "type": "QueryPlanningTool"
    }
  ],
  "app_type": "os_chat"
}

Then create a search pipeline with agentic_context processor

PUT _search/pipeline/agentic-pipeline
{
  "request_processors": [
    {
      "agentic_query_translator": {
        "agent_id": "your-ecommerce-agent-id"
      }
    }
  ],
  "response_processors": [
    {
      "agentic_context": {
        "agent_steps_summary": true,
        "dsl_query": true
      }
    }
  ]
}

and finally make a search request with the same pipeline

GET /_search?search_pipeline=agentic-pipeline
{
  "query": {
    "agentic": {
      "query_text": "Find me white shoes under 150 dollars"
    }
  }
}

Thanks for sharing the example. While aligning it with my POC, I am getting the following output.

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Agentic search failed - Agent execution error - Agent ID: [EFWCJJoBtkxVx3l6_kS6], Error: [No valid 'dsl_query' found in conversational agent response]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Agentic search failed - Agent execution error - Agent ID: [EFWCJJoBtkxVx3l6_kS6], Error: [No valid 'dsl_query' found in conversational agent response]",
    "caused_by": {
      "type": "illegal_state_exception",
      "reason": "No valid 'dsl_query' found in conversational agent response"
    }
  },
  "status": 400
}

From my understanding, the provided sample is related to Conversational Agents for Agentic Search However, earlier, I was able to generate proper DSL queries using Flow Agents for Agentic Search Using flow agents - OpenSearch Documentation.

Could you please confirm whether the same approach can be used with Agentic Search as well? If so, please provide an example demonstrating how to enable these features?

Yes the provided one is for conversational agent. Can you provide few details like

  1. Are you using Using flow agents - OpenSearch Documentation for agentic search?
  2. What type of natural language query are you sending?
  3. Did you create search pipeline with agentic_query_translator processor and attach it to search request?

For the conversational one I follow the below steps where I got error while running the Agentic Search

Create a customer_data index

PUT customer_data
{
  "mappings": {
    "properties": {
      "customer_id": { "type": "text", "fielddata": true },
      "name": { "type": "text", "fielddata": true },
      "address1": { "type": "keyword" },
      "address2": { "type": "keyword" },
      "state": { "type": "text", "fielddata": true },
      "city": { "type": "text", "fielddata": true },
      "zip": { "type": "integer" },
      "phone": { "type": "text", "fielddata": true },
      "email": { "type": "text", "fielddata": true }
    }
  }
}

Ingest sample data

POST _bulk
{ "index": { "_index": customer_data", "_id": "CUST_00001" } }
{ "customer_id": "CUST_00001", "name": "Allison Hill", "address1": "610 Kathleen Mount", "address2": "", "state": "OH", "city": "Chicago", "zip": 32555, "phone": "(220)747-7266x17864", "email": "tracyhorn@example.org" }
{ "index": { "_index": customer_data", "_id": "CUST_00002" } }
{ "customer_id": "CUST_00002", "name": "Noah Rhodes", "address1": "4320 Hicks Loaf", "address2": "", "state": "FL", "city": "Phoenix", "zip": 85000, "phone": "+1-905-777-1575", "email": "aaronrios@example.org" }
{ "index": { "_index": customer_data", "_id": "CUST_00003" } }
{ "customer_id": "CUST_00003", "name": "Angie Henderson", "address1": "45971 Cox Springs Suite 932", "address2": "", "state": "GA", "city": "Chicago", "zip": 87785, "phone": "811-587-4060x193", "email": "burnsmichael@example.net" }
{ "index": { "_index": customer_data", "_id": "CUST_00004" } }
{ "customer_id": "CUST_00004", "name": "Daniel Wagner", "address1": "0115 Byrd Court Suite 732", "address2": "", "state": "IL", "city": "San Jose", "zip": 77330, "phone": "822-835-2551x38726", "email": "castrobryan@example.com" }
{ "index": { "_index": customer_data", "_id": "CUST_00005" } }
{ "customer_id": "CUST_00005", "name": "Cristian Santos", "address1": "96856 Webster Bypass Suite 973", "address2": "Apt. 296", "state": "OH", "city": "San Antonio", "zip": 20261, "phone": "557-835-8453x0191", "email": "james98@example.com" }
{ "index": { "_index": customer_data", "_id": "CUST_00006" } }
{ "customer_id": "CUST_00006", "name": "Connie Lawrence", "address1": "679 Calderon Square Apt. 644", "address2": "", "state": "MI", "city": "New York", "zip": 24461, "phone": "231.967.9477", "email": "zwilliams@example.org" }

Create a model

POST /_plugins/_ml/models/_register?deploy=true
{
  "name": "Claude 4 sonnet Query Planner tool Model",
  "function_name": "remote",
  "description": "Claude 4 sonnet for Query Planning",
  "connector": {
    "name": "Bedrock Claude 4 Sonnet Connector",
    "description": "Amazon Bedrock connector for Claude 4 Sonnet",
    "version": 1,
    "protocol": "aws_sigv4",
    "parameters": {
      "region": "<<region>>",
      "service_name": "bedrock",
      "model": "us.anthropic.claude-sonnet-4-20250514-v1:0"
    },
    "credential": {
      "access_key": "<<access_key>>",
      "secret_key": "<<secret_key>>",
      "session_token": "<<session_token>"
    },
    "actions": [
      {
        "action_type": "predict",
        "method": "POST",
        "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/converse",
        "headers": {
          "content-type": "application/json"
        },
        "request_body": "{ \"system\": [{\"text\": \"${parameters.system_prompt}\"}], \"messages\": [${parameters._chat_history:-}{\"role\":\"user\",\"content\":[{\"text\":\"${parameters.user_prompt}\"}]}${parameters._interactions:-}]${parameters.tool_configs:-} }"
      }
    ]
  }
}

Register an agent

{
  "name": "Customer Search Agent",
  "type": "conversational",
  "description": "Intelligent customer search",
  "llm": {
    "model_id": "<<model_id>>",
    "parameters": {
      "max_iteration": 20
    }
  },
  "memory": {
    "type": "conversation_index"
  },
  "tools": [
    {
      "type": "ListIndexTool",
      "name": "ListIndexTool"
    },
    {
      "type": "IndexMappingTool",
      "name": "IndexMappingTool"
    },
    {
      "type": "WebSearchTool",
      "parameters": {
        "engine": "duckduckgo"
      }
    },
    {
      "type": "QueryPlanningTool"
    }
  ],
  "app_type": "os_chat"
}

Configure a search pipeline

PUT _search/pipeline/agentic-pipeline
{
  "request_processors": [
    {
      "agentic_query_translator": {
        "agent_id": "<<agent_id>>"
      }
    }
  ],
  "response_processors": [
    {
      "agentic_context": {
        "agent_steps_summary": true,
        "dsl_query": true
      }
    }
  ]
}

Run an agentic search

GET /_search?search_pipeline=agentic-pipeline
{
  "query": {
    "agentic": {
      "query_text": "Fetch all customers who live in Chicago, OH"
    }
  }
}

Response

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Agentic search failed - Agent execution error - Agent ID: [V1VTKpoBtkxVx3l630dc], Error: [No valid 'dsl_query' found in conversational agent response]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Agentic search failed - Agent execution error - Agent ID: [V1VTKpoBtkxVx3l630dc], Error: [No valid 'dsl_query' found in conversational agent response]",
    "caused_by": {
      "type": "illegal_state_exception",
      "reason": "No valid 'dsl_query' found in conversational agent response"
    }
  },
  "status": 400
}

@arnabmca2006 thanks for the detailed answer. Since you are using claude model we have released a new https://opensearch.org/downloads/ 3.3.2 release with better query generation for claude model. Can you try it?

As suggested, I have upgraded OpenSearch to version 3.3.2. I ran the same commands mentioned earlier, but I’m now encountering the following error. Could you please advise if I’m missing any configuration in the previously mentioned commands?

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Agentic search failed - Agent execution error - Agent ID: [wg1UOZoBpBqhAj5Dy79s], Error: [No valid 'dsl_query' found in conversational agent response. The agent must return a JSON object with 'dsl_query' field. Please check the agent configuration and prompts.]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Agentic search failed - Agent execution error - Agent ID: [wg1UOZoBpBqhAj5Dy79s], Error: [No valid 'dsl_query' found in conversational agent response. The agent must return a JSON object with 'dsl_query' field. Please check the agent configuration and prompts.]",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "No valid 'dsl_query' found in conversational agent response. The agent must return a JSON object with 'dsl_query' field. Please check the agent configuration and prompts."
    }
  },
  "status": 400
}

Earlier, I had registered the Flow Agents and used Query Planning Tool to generate the DSL query, which correctly returned the DSL query. However, my requirement is while generating the DSL query, how to log the agent trace showing reasoning, tool call, tool output, observations internally by the Query Planning tool?

Register an agent under a model

POST /_plugins/_ml/agents/_register
{
  "name": "Agentic Search with Claude 4",
  "type": "flow",
  "description": "A test agent for query planning.",
  "tools": [
    {
      "type": "QueryPlanningTool",
      "parameters": {
        "model_id": "<<model_id>>",
        "response_filter": "$.output.message.content[0].text"
      }
    }
  ]
}

Execute the agent to generate the DSL

POST /_plugins/_ml/agents/<<agent_id>>/_execute
{
  "parameters": {
    "inputs": "",
    "question": "Fetch all customers who live in Chicago, OH",
    "index_name": "customer_data"
  }
}

Result

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "result": """{"query":{"bool":{"must":[{"match":{"city":"Chicago"}},{"match":{"state":"OH"}}]}}}"""
        }
      ]
    }
  ]
}

If you just want to see agent tracing for flow agent while agent execution you can pass

POST /_plugins/_ml/agents/<<agent_id>>/_execute
{
  "parameters": {
    "inputs": "",
    "question": "Fetch all customers who live in Chicago, OH",
    "index_name": "customer_data",
    "verbose": true
  }
}

For flow agent agent tracing in agentic search can’t be supported because all the tools works sequentially present in the agent. You can still get the generated dsl query by using Agentic context - OpenSearch Documentation in your search pipeline.

You can also try conversation agent for agentic search with claude now and with agent tracing here. Using conversational agents - OpenSearch Documentation

Also, can you please provide us more details on your use case for agentic search?

I am developing a POC where users can enter queries in natural language, and OpenSearch retrieves relevant data using its Agentic Search capabilities.

@arnabmca2006 sounds good. Let us know if you have more questions. Feel free to connect with us on opensource slack. https://opensearch.org/slack/

I have sorted out the issue I faced earlier. Thanks.

1 Like