Not able to run the predict APIs with external ml models like Hugging Face

[ramda]

Not able to run the predict APIs with external ml models like Hugging Face. Here are the steps i have followed
Version 2.10

1. Created a external Connector Model with Hugging Face

{
  "name": "Hugging Face GPT Connector",
  "description": "The connector to Hugging Face GPT Language Model",
  "version": 1,
  "protocol": "http",
  "parameters": {
    "endpoint": "https://api-inference.huggingface.co",
    "model": "gpt2",
    "temperature": 1.0
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "${parameters.endpoint}/models/gpt2",
      "request_body": "{ \"inputs\": \"${parameters.inputs}\", \"options\": {\"temperature\": ${parameters.temperature}} }"
    }
  ]
}

2. Created a Model Group
3. Registered the Model that uses the Connector

POST _plugins/_ml/models/_register

{
    "name": "Hugging Face Chat GPT2",
    "function_name": "remote",
    "model_group_id": "P_POHIsB9MGS2ILh14V7",
    "description": "LLM Chat GPT2 Model",
    "connector_id": "PvPDHIsB9MGS2ILh7IXX"
}

4. Deploy the Model

_plugins/_ml/models/QfPQHIsB9MGS2ILhF4Xf/_deploy

5. Verify the Model

_plugins/_ml/tasks/QvPSHIsB9MGS2ILhyoUd

Got the below response

{
    "model_id": "QfPQHIsB9MGS2ILhF4Xf",
    "task_type": "DEPLOY_MODEL",
    "function_name": "REMOTE",
    "state": "COMPLETED",
    "worker_node": [
        "bbFS6Sl3TjGZlDe1XCL-bg"
    ],
    "create_time": 1696995658269,
    "last_update_time": 1696995658343,
    "is_async": true
}

6. Predict the Model, here the step is failing.

POST _plugins/_ml/models/QfPQHIsB9MGS2ILhF4Xf/_predict

Body

{
  "parameters": {
    "messages": [
      {
        "role": "admin",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }
}

I am getting below error. Not getting any clue as what this error is all about

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Some parameter placeholder not filled in payload: inputs"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Some parameter placeholder not filled in payload: inputs"
    },
    "status": 400
}

[HenryLindeman]

I think this error is coming from the connector expecting a parameter called “inputs” and not finding one. Instead, there’s messages. perhaps use body

{
  "parameters": {
    "inputs": {
      "messages": [...]
    }
  }
}