Not able Predict API for OpenAI, Please Help me

Plugins Machine Learning
1

Hello, Opensearch Community
I cannot understand “model_state”: “PARTIALLY_DEPLOYED”
My opensearch version is 2.10.0 / tar install at Linux .
And My opensearch is standalone, because it is test opensearch.
I connect with OpenAI, make model_groups and models and deploy it.
There is 2 Questions.
What is PARTIALLY_DEPLOYED in “model_state” ?
“”"
<…>
“_source”: {
“last_deployed_time”: 1700641444886,
“model_version”: “2”,
“created_time”: 1700641139365,
“deploy_to_all_nodes”: true,
“description”: “test OpenAI gpt-3.5-turbo-16k model 20231122”,
“model_state”: “PARTIALLY_DEPLOYED”,
<…>
“”"
2. When I POST /plugins/ml/models/<model_id>/_predict {params, messages… etc.} command,
I response this… How can I solve it? Thx in advance.
“”"
{
“error”: {
“root_cause”: [
{
“type”: “illegal_argument_exception”,
“reason”: “Invalid JSON in payload”
}
],
“type”: “illegal_argument_exception”,
“reason”: “Invalid JSON in payload”
},
“status”: 400
}
“”"

Help me, plz!

2

  1. PARTIALLY_DEPLOYED means model is successfully deployed, but not in every nodes. It deployed in few nodes in the cluster.

  2. May be following this detailed blog could you help you to identify the issue.

1 Like
3

Thanks to reply, dhrubo!
I gonna check it right now!

4

You can also check the blueprint https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/open_ai_connector_chat_blueprint.md

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.