# Conversational Search w/ Bedrock Claude Sonnet 3 **URL:** \ **Category:** OpenSearch\ **Tags:** discuss, troubleshoot, configure\ **Created:** [November 25, 2024, 9:58pm UTC](https://forum.opensearch.org/t/conversational-search-w-bedrock-claude-sonnet-3/22567 "2024-11-25T21:58:55Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![champstank](https://avatars.discourse-cdn.com/v4/letter/c/4bbf92/32.png) [@champstank](https://forum.opensearch.org/u/champstank)\ **Post date:** [November 25, 2024, 9:58pm UTC](https://forum.opensearch.org/t/conversational-search-w-bedrock-claude-sonnet-3/22567/1 "2024-11-25T21:58:55Z")
**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser): OpenSearch v2.18.0 **Describe the issue** : Working to implement the Conversational Search feature with AWS Bedrock Claude Sonnet 3 LLM. The backend in the connector code is building an invalid payload and the connector code examples seem to be either old for Claude v2 or only using the inference setup on it’s own when referencing the Connector Blueprint examples. I’ve tried a handful of changes in the passing of the parameters.messages but it’s not aligning properly yet. I’m running the **public.ecr.aws/opensearchproject/opensearch:2.18.0** docker container locally. How can I set up this connector so it can work properly using Conversational Search with RAG and Bedrock Claude? I followed this tutorial documentation to set it up [https://opensearch.org/docs/latest/search-plugins/conversational-search/](https://opensearch.org/docs/latest/search-plugins/conversational-search/) and have been referencing this documentation for the connector code [https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote\_inference\_blueprints/bedrock\_connector\_anthropic\_claude3\_blueprint.md](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_anthropic_claude3_blueprint.md) **Configuration** : Here is my create\_connector for AWS Bedrock Claude Sonnet 3 def create\_connector(client: opensearchpy.client, model\_name: str, aws\_access\_key: str, aws\_secret\_key: str, aws\_session\_token: str, aws\_region: str = “us-east-1”) → str: “”"Creates an OpenSearch connector to the Amazon Bedrock Claude 3.5 model using AWS SigV4 authentication for conversational search. ``` Args: client (opensearchpy.client): OpenSearch client model_name (str): Name of the Bedrock model aws_access_key (str): AWS access key aws_secret_key (str): AWS secret key aws_session_token (str): AWS session token aws_region (str): AWS region (default: "us-east-1") Returns: str: Connector ID """ # Create the connector response = client.transport.perform_request( "POST", "/_plugins/_ml/connectors/_create", body={ "name": "Amazon Bedrock Claude 3.5 Connector", "description": "Connector for Amazon Bedrock Claude 3.5 model using AWS SigV4 authentication", "version": 1, "protocol": "aws_sigv4", "credential": { "access_key": aws_access_key, "secret_key": aws_secret_key, "session_token": aws_session_token }, "parameters": { "region": aws_region, "service_name": "bedrock", "model": model_name, "response_filter": "$.content[0].text" }, "actions": [ { "action_type": "predict", "method": "POST", "headers": { "Content-Type": "application/json", "Accept": "application/json", }, "url": f"https://bedrock-runtime.{aws_region}.amazonaws.com/model/{model_name}/invoke", "request_body": """{"messages":${parameters.messages},"anthropic_version":"bedrock-2023-05-31","max_tokens":300,"temperature":0.5})""" } ] } ) connector_id = response.get("connector_id") if connector_id: logger.info(f"Connector created with ID: {connector_id}") else: logger.error("Failed to create connector.") return connector_id ``` **Relevant Logs or Screenshots** : Error during search: Traceback (most recent call last): File “/var/folders/kh/4drf579d25s0nq2tsbfm7h940000gn/T/ipykernel\_72760/2737091731.py”, line 229, in query\_rag\_pipeline\_with\_generative\_qa response = client.search(index=index\_name, body={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/client/utils.py”, line 191, in \_wrapped return func(\*args, params=params, headers=headers, \*\*kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/client/ **init**.py”, line 2367, in search return self.transport.perform\_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/transport.py”, line 469, in perform\_request raise e File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/transport.py”, line 425, in perform\_request status, headers\_response, data = connection.perform\_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/connection/http\_urllib3.py”, line 332, in perform\_request self.\_raise\_error( File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/connection/base.py”, line 315, in \_raise\_error raise HTTP\_EXCEPTIONS.get(status\_code, TransportError)( opensearchpy.exceptions.RequestError: RequestError(400, ‘illegal\_argument\_exception’, ‘Invalid payload: {“messages”:[{“role”:“user”,“content”:[{“type”:“text”,“text”:“Generate a concise and informative answer in less than 100 words for the given question, taking into context: - An enumerated list of search results- A rephrase of the question that was used to generate the search results- The conversation historyCite search results using [${number}] notation.Do not repeat yourself, and NEVER repeat anything in the chat history.If there are any necessary steps or procedures in your answer, enumerate them.”},{“type”:“text”,“text”:“SEARCH RESULT 1: The metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.”},{“type”:“text”,“text”:“SEARCH RESULT 2: The metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.”},{“type”:“text”,“text”:“QUESTION: What's the population of NYC metro area in 2023?\n”},{“type”:“text”,“text”:“ANSWER:”}]}],“anthropic\_version”:“bedrock-2023-05-31”,“max\_tokens”:300,“temperature”:0.5})’)