Conversational Search w/ Bedrock Claude Sonnet 3

OpenSearch
, ,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch v2.18.0

Describe the issue:
Working to implement the Conversational Search feature with AWS Bedrock Claude Sonnet 3 LLM. The backend in the connector code is building an invalid payload and the connector code examples seem to be either old for Claude v2 or only using the inference setup on it’s own when referencing the Connector Blueprint examples. I’ve tried a handful of changes in the passing of the parameters.messages but it’s not aligning properly yet. I’m running the public.ecr.aws/opensearchproject/opensearch:2.18.0 docker container locally. How can I set up this connector so it can work properly using Conversational Search with RAG and Bedrock Claude? I followed this tutorial documentation to set it up https://opensearch.org/docs/latest/search-plugins/conversational-search/
and have been referencing this documentation for the connector code https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_anthropic_claude3_blueprint.md

Configuration:

Here is my create_connector for AWS Bedrock Claude Sonnet 3

def create_connector(client: opensearchpy.client, model_name: str, aws_access_key: str, aws_secret_key: str, aws_session_token: str, aws_region: str = “us-east-1”) → str:
“”"Creates an OpenSearch connector to the Amazon Bedrock Claude 3.5 model using AWS SigV4 authentication for conversational search.

Args:
    client (opensearchpy.client): OpenSearch client
    model_name (str): Name of the Bedrock model
    aws_access_key (str): AWS access key
    aws_secret_key (str): AWS secret key
    aws_session_token (str): AWS session token
    aws_region (str): AWS region (default: "us-east-1")
Returns:
    str: Connector ID
"""

# Create the connector
response = client.transport.perform_request(
    "POST",
    "/_plugins/_ml/connectors/_create",
    body={
        "name": "Amazon Bedrock Claude 3.5 Connector",
        "description": "Connector for Amazon Bedrock Claude 3.5 model using AWS SigV4 authentication",
        "version": 1,
        "protocol": "aws_sigv4",
        "credential": {
            "access_key": aws_access_key,
            "secret_key": aws_secret_key,
            "session_token": aws_session_token
        },
        "parameters": {
            "region": aws_region,
            "service_name": "bedrock",
            "model": model_name,
            "response_filter": "$.content[0].text"
        },
        "actions": [
            {
                "action_type": "predict",
                "method": "POST",
                "headers": {
                    "Content-Type": "application/json",
                    "Accept": "application/json",
                },
                "url": f"https://bedrock-runtime.{aws_region}.amazonaws.com/model/{model_name}/invoke",
                "request_body": """{"messages":${parameters.messages},"anthropic_version":"bedrock-2023-05-31","max_tokens":300,"temperature":0.5})"""
            }
        ]
    }
)

connector_id = response.get("connector_id")
if connector_id:
    logger.info(f"Connector created with ID: {connector_id}")
else:
    logger.error("Failed to create connector.")
return connector_id

Relevant Logs or Screenshots:

Error during search: Traceback (most recent call last):
File “/var/folders/kh/4drf579d25s0nq2tsbfm7h940000gn/T/ipykernel_72760/2737091731.py”, line 229, in query_rag_pipeline_with_generative_qa
response = client.search(index=index_name, body={
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/client/utils.py”, line 191, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/client/init.py”, line 2367, in search
return self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/transport.py”, line 469, in perform_request
raise e
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/transport.py”, line 425, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/connection/http_urllib3.py”, line 332, in perform_request
self._raise_error(
File “/Users/champstank/.pyenv/versions/opensearch/lib/python3.12/site-packages/opensearchpy/connection/base.py”, line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.RequestError: RequestError(400, ‘illegal_argument_exception’, ‘Invalid payload: {“messages”:[{“role”:“user”,“content”:[{“type”:“text”,“text”:“Generate a concise and informative answer in less than 100 words for the given question, taking into context: - An enumerated list of search results- A rephrase of the question that was used to generate the search results- The conversation historyCite search results using [${number}] notation.Do not repeat yourself, and NEVER repeat anything in the chat history.If there are any necessary steps or procedures in your answer, enumerate them.”},{“type”:“text”,“text”:“SEARCH RESULT 1: The metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.”},{“type”:“text”,“text”:“SEARCH RESULT 2: The metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.”},{“type”:“text”,“text”:“QUESTION: What's the population of NYC metro area in 2023?\n”},{“type”:“text”,“text”:“ANSWER:”}]}],“anthropic_version”:“bedrock-2023-05-31”,“max_tokens”:300,“temperature”:0.5})’)