Unable to deploy the custom model in Opensearch

OpenSearch
1

I am trying to upload the custom model. It is registered successfully, but during deployment, I am getting issue as below
{
“model_id”: “jcyhWpUBiLYOqMKJU5V0”,
“task_type”: “DEPLOY_MODEL”,
“function_name”: “TEXT_EMBEDDING”,
“state”: “FAILED”,
“worker_node”: [
“kddpRYg9R9-Hj0GkixiUfw”,
“Gtk83tuLSx-DrYN2I_wx4w”,
“tEhaPrkWTGy_N210nfJBDA”
],
“create_time”: 1740983965882,
“last_update_time”: 1740983968556,
“error”: “”“{“kddpRYg9R9-Hj0GkixiUfw”:“all-MiniLM-L6-v2.pt file not found in: /home/elastic/data/opensearch/os_on_9200/data/ml_cache/models_cache/models/jcyhWpUBiLYOqMKJU5V0/1/all-MiniLM-L6-v2”,“Gtk83tuLSx-DrYN2I_wx4w”:“all-MiniLM-L6-v2.pt file not found in: /home/elastic/data/opensearch/os_on_9200/data/ml_cache/models_cache/models/jcyhWpUBiLYOqMKJU5V0/1/all-MiniLM-L6-v2”,“tEhaPrkWTGy_N210nfJBDA”:“all-MiniLM-L6-v2.pt file not found in: /home/elastic/data/opensearch/os_on_9200/data/ml_cache/models_cache/models/jcyhWpUBiLYOqMKJU5V0/1/all-MiniLM-L6-v2”}”“”,
“is_async”: true
}

I used zip file to upload the model.

Initialize ML Commons client

ml_client = MLCommonClient(client)

Define local model paths

model_path = “all-MiniLM-L6-v2_torchscript_sentence-transformer.zip” # Path to your model file
model_config_path = “all-MiniLM-L6-v2_torchscript.json” # Path to your config file

Register model

model_id_file_system = ml_client.register_model(model_path, model_config_path, isVerbose=True)

2

Mention the same name at all 3 places – your model, and in config.json for model name param and and even zip folder name – and try to deploy again via script.

3

Though I give same name for all 3 files, deployment failed

4

Can you please help with what names exactly you gave at all 3 places? please describe with details.

5

Hi, I can register the model, while deployment getting the issue as

“error”: “”“{“kddpRYg9R9-Hj0GkixiUfw”:”\nUnknown builtin op: aten::scaled_dot_product_attention.\nHere are some suggestions: \n\taten::_scaled_dot_product_attention\n\nThe original call is:\n File "code/torch/transformers/models/bert/modeling_bert.py", line 217\n x1 = torch.view(_52, [_54, int(_55), 16, 64])\n value_layer = torch.permute(x1, [0, 2, 1, 3])\n attn_output = torch.scaled_dot_product_attention(query_layer, key_layer, value_layer, attention_mask)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <— HERE\n attn_output0 = torch.transpose(attn_output, 1, 2)\n input = torch.reshape(attn_output0, [_42, _43, 1024])\n",“Gtk83tuLSx-DrYN2I_wx4w”:“\nUnknown builtin op: aten::scaled_dot_product_attention.\nHere are some suggestions: \n\taten::_scaled_dot_product_attention\n\nThe original call is:\n File "code/torch/transformers/models/bert/modeling_bert.py", line 217\n x1 = torch.view(_52, [_54, int(_55), 16, 64])\n value_layer = torch.permute(x1, [0, 2, 1, 3])\n attn_output = torch.scaled_dot_product_attention(query_layer, key_layer, value_layer, attention_mask)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <— HERE\n attn_output0 = torch.transpose(attn_output, 1, 2)\n input = torch.reshape(attn_output0, [_42, _43, 1024])\n”}“”",

I am using torch script format