Can't register a pre-trained model

OpenSearch
1

On an AWS managed service, version 2.17

Describe the issue:

trying to use one of the pretrained models listed on the docs:

the model is huggingface/sentence-transformers/multi-qa-MiniLM-L6-cos-v1
version 1.0.1
trying TORCH_FORMAT and ONNX

getting this error on the task:
“This model is not in the pre-trained model list, please check your parameters.”

what could i be doing wrong here?
thanks!

2

Hey, I just started getting this same issue yesterday and noticed the smallest change in the docs. The version for my model and your model is now 1.0.2 instead of 1.0.1.

I’m also on AWS managed OpenSearch. When I switched to model 1.0.2 I could register and deploy a pre-trained model again.

Don’t know why they’re actively blocking the old model versions… put on your tinfoil hats for that. But 1.0.2 should work.

3

Thanks for the reply!

You’re right but check this out, when using the TORCH_SCRIPT format on v 1.0.2 the registration succeeds but deploying throws this error:

“error”: “{"IDv1tSDKQvW1wgPGETaMWw":"\nUnknown builtin op: aten::scaled_dot_product_attention.\nHere are some suggestions: \n\taten::_scaled_dot_product_attention\n\nThe original call is:\n File \"code/torch/transformers/models/bert/modeling_bert.py\", line 165\n x1 = torch.view(_36, [_38, int(_39), 12, 32])\n value_layer = torch.permute(x1, [0, 2, 1, 3])\n attn_output = torch.scaled_dot_product_attention(query_layer, key_layer, value_layer, attention_mask)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <— HERE\n attn_output0 = torch.transpose(attn_output, 1, 2)\n input = torch.reshape(attn_output0, [_26, _27, 384])\n"}”,

ONNX version works for registering and deploying, seems like something is up indeed, tinfoil hat firmly in place.

4

We are also experiencing this issue. We use the Docker image [opensearchproject/opensearch:2.19.2] as part of our CI/CD pipeline to run integration tests, and we’ve encountered the problem described above. We are able to register the model if we update the version to 1.0.2, but the model deployment then fails.

Just to also confirm, when I change the model format to ONNX I can register and deploy the model.

POST /_plugins/_ml/models/_register
{
  "name": "huggingface/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
  "version": "1.0.2",
  "model_format": "ONNX"
}

I’m assuming they are interchangeable but would guess the embedding values (floating points) could be different? :man_shrugging:

5

As far as I know both formats represent the same weights so I assume the outputted values would also be the same… I could be wrong tho