# How to register local custom model? **URL:** \ **Category:** Machine Learning\ **Created:** [October 31, 2024, 2:12pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260 "2024-10-31T14:12:01Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![miroshuSan](https://avatars.discourse-cdn.com/v4/letter/m/b5a626/32.png) [@miroshuSan](https://forum.opensearch.org/u/miroshuSan)\ **Post date:** [October 31, 2024, 2:12pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260/1 "2024-10-31T14:12:01Z")
**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser): OpenSearch 2.17 **Describe the issue** : Im trying to register a custom model from hugging face but im having some issues with the URL. I created a zip file containing all the files from onnx folder from [intfloat/multilingual-e5-large at main](https://huggingface.co/intfloat/multilingual-e5-large/tree/main/onnx). Then i created a volume to add this zip file to the opensearch docker container. Then as URL for the model i simply wanted to use this local file but this didnt work. I also tried uploading this file to my google drive and using the link from google drive as URL but then im getting: “error”: “zip END header not found” Im guessing that i understood this URL and the file that is expected to be loaded from there completely wrong. Can someone please help me out here? Here is the request to register the model: { “name”: “multilingual-e5-large”, “version”: “1.0.0”, “description”: “Multilingual E5-Large”, “model\_format”: “ONNX”, “model\_group\_id”: “\_Gjj3pIBJrX4nqy0y2ST”, “model\_content\_hash\_value”: “9aaa222c5a6529ae4a69ab6925c863af595d106edf565434c8280777b3126d64”, “model\_config”: { “model\_type”: “xlm-roberta”, “embedding\_dimension”: 1024, “framework\_type”: “sentence\_transformers”, “all\_config”: “{"\_name\_or\_path":"tmp/","architectures":["XLMRobertaModel"],"attention\_probs\_dropout\_prob":0.1,"bos\_token\_id":0,"classifier\_dropout":null,"eos\_token\_id":2,"hidden\_act":"gelu","hidden\_dropout\_prob":0.1,"hidden\_size":1024,"initializer\_range":0.02,"intermediate\_size":4096,"layer\_norm\_eps":0.00001,"max\_position\_embeddings":514,"model\_type":"xlm-roberta","num\_attention\_heads":16,"num\_hidden\_layers":24,"output\_past":true,"pad\_token\_id":1,"position\_embedding\_type":"absolute","torch\_dtype":"float32","transformers\_version":"4.29.0","type\_vocab\_size":1,"use\_cache":true,"vocab\_size":250002}” }, “url”: “/usr/share/opensearch/models/multilingual-e5-large.zip” } ---
**Author:** ![dhrubo](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@dhrubo](https://forum.opensearch.org/u/dhrubo)\ **Post date:** [October 31, 2024, 2:45pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260/2 "2024-10-31T14:45:11Z")
You can try to use [Opensearch-py-ml](https://opensearch.org/docs/latest/clients/opensearch-py-ml/) Please follow this notebook: [Demo Notebook to trace Sentence Transformers model — Opensearch-py-ml 1.1.0 documentation](https://opensearch-project.github.io/opensearch-py-ml/examples/demo_tracing_model_torchscript_onnx.html) ---
**Author:** ![miroshuSan](https://avatars.discourse-cdn.com/v4/letter/m/b5a626/32.png) [@miroshuSan](https://forum.opensearch.org/u/miroshuSan)\ **Post date:** [November 4, 2024, 5:56pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260/3 "2024-11-04T17:56:05Z")
Hey! Thanks for the answer. I had a couple of problems and most of them i could fix but now im stuck again at the model deployment step. I basically used the same code from the notebook from your link. I only changed the model itself. In the last step, when im trying to deploy the model im getting: ```auto .venv/lib/python3.10/site-packages/opensearch_py_ml/ml_commons/ml_commons_client.py:357, in MLCommonClient.deploy_model(self, model_id, wait_until_deployed) 355 print("Model deployed only partially") 356 else: --> 357 raise Exception("Model deployment failed") 359 return self._get_task_info(task_id) ``` Then i tried to register the model only, without deployment (setting deploy\_model param to False) and this worked. But then when i tried to deploy the model manually by calling: ```auto https://localhost:9200/_plugins/_ml/models/aC9O-JIBJ8Vsn8aNmVZ-/_deploy ``` im getting: ```auto { "model_id": "A51B-JIBVoJr-PiFnVNy", "task_type": "DEPLOY_MODEL", "function_name": "TEXT_EMBEDDING", "state": "FAILED", "worker_node": [ "lNpYAlQ6T0Wu5ymYW4OQ4g" ], "create_time": 1730741918213, "last_update_time": 1730741921643, "error": "{\"lNpYAlQ6T0Wu5ymYW4OQ4g\":\"Error code - ORT_FAIL - message: Deserialize tensor onnx::MatMul_3915 failed.GetFileLength for /usr/share/opensearch/data/ml_cache/models_cache/models/A51B-JIBVoJr-PiFnVNy/9/intfloat/multilingual-e5-large/onnx__MatMul_3915 failed:Invalid fd was supplied: -1\"}", "is_async": true } ``` I tried to use TORCH\_SCRIPT instead of ONNX but im getting the same error from opensearch-py-ml and if i try to deploy the model manually: ```auto { "model_id": "aC9O-JIBJ8Vsn8aNmVZ-", "task_type": "DEPLOY_MODEL", "function_name": "TEXT_EMBEDDING", "state": "FAILED", "worker_node": [ "lNpYAlQ6T0Wu5ymYW4OQ4g" ], "create_time": 1730742866161, "last_update_time": 1730742933372, "error": "{\"lNpYAlQ6T0Wu5ymYW4OQ4g\":\"\\nUnknown builtin op: aten::scaled_dot_product_attention.\\nHere are some suggestions: \\n\\taten::_scaled_dot_product_attention\\n\\nThe original call is:\\n File \\\"code/ __torch__ /transformers/models/xlm_roberta/modeling_xlm_roberta/___torch_mangle_885.py\\\", line 38\\n x1 = torch.view(_10, [_12, int(_13), 16, 64])\\n value_layer = torch.permute(x1, [0, 2, 1, 3])\\n attn_output = torch.scaled_dot_product_attention(query_layer, key_layer, value_layer, attention_mask)\\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE\\n attn_output0 = torch.transpose(attn_output, 1, 2)\\n input = torch.reshape(attn_output0, [_0, _1, 1024])\\n\"}", "is_async": true } ``` ---
**Author:** ![miroshuSan](https://avatars.discourse-cdn.com/v4/letter/m/b5a626/32.png) [@miroshuSan](https://forum.opensearch.org/u/miroshuSan)\ **Post date:** [November 5, 2024, 2:06pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260/4 "2024-11-05T14:06:11Z")
Update: I found a workaround by using optimum-cli to export the model to ONNX and repeating the above mentioned steps. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\ **Post date:** [January 4, 2025, 2:06pm UTC](https://forum.opensearch.org/t/how-to-register-local-custom-model/22260/5 "2025-01-04T14:06:22Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.