How to upload an ML model in offline mode on OpenSearch 2.17?

Plugins Machine Learning
1

Hi everyone,

I’m using OpenSearch 2.17 and I would like to know:
What is the correct procedure to upload a machine learning model in offline mode (without internet access)?

I have already downloaded the model manually (in ONNX format).
I tested the _upload API using a file path, but I’m getting no response from OpenSearch, and the model doesn’t appear when I list models.

Thanks in advance for your help!

2

@MHR What message did you receive? You need to enable model registration via local file in the OpenSearch cluster.

PUT _cluster/settings
{
  "persistent": {
    "plugins.ml_commons.allow_registering_model_via_local_file": "true",
  }
}
3

Hi Pablo, thanks for you answer.

I already have this configuration in the cluster:
{
“persistent”: {
“plugins.ml_commons.allow_registering_model_via_url”: “true”,
“plugins.ml_commons.only_run_on_ml_node”: “true”,
“plugins.ml_commons.model_access_control_enabled”: “true”,
“plugins.ml_commons.native_memory_threshold”: “99”
}
}
Whats the next step please ?

4

@MHR You can use Python module opensearch_py_ml to import the model.
Check this thread, it has a working Python script example.

5

I tried this solution : Offline deployment pretrained of models - Plugins / Machine Learning - OpenSearch
I have the models registred in opensearch, but when i deploy i m having this error
{
“model_id”: “PtfADpgBFuMKdhl6fGIn_9”,
“task_type”: “DEPLOY_MODEL”,
“function_name”: “TEXT_EMBEDDING”,
“state”: “FAILED”,
“worker_node”: [
“LL4EfJJZTNWRUKK0Vv5ohg”,
“QUVT0ivpSVecE4Nrvgdptw”,
“YYu50U1AR0Glj3YkTRUlEQ”
],
“create_time”: 1752595176699,
“last_update_time”: 1752595177056,
“error”: “”“{“LL4EfJJZTNWRUKK0Vv5ohg”:“Failed to deploy model”,“QUVT0ivpSVecE4Nrvgdptw”:“Failed to deploy model”,“YYu50U1AR0Glj3YkTRUlEQ”:“Failed to deploy model”}”“”,
“is_async”: true
}

6

@MHR Did you test that with 2.17 only?

7

Yes

8

@MHR Did you test this script with pretrained model?
If so, could you share which one did you use?

Could you also check the OpenSearch logs? There will be more information than in that deployment task.

9

I didnt test the script yet.

Here are the opensearch logs when i deploy a model:

[ERROR]No controller is deployed because the model PtfADpgBFuMKdhl6fGIn is expected not having an enabled model controller. Please use the create model controller api to create one if this is unexpected.
[ERROR] Failed to deploy model PtfADpgBFuMKdhl6fGIn
ai.djl.engine.EngineException: Failed to save pytorch index file
[ERROR] Failed to retrieve model PtfADpgBFuMKdhl6fGIn
Caused by: Failed to save pytorch index file
Caused by: java.io.IOException: Offline model is enabled.

10

@MHR I’ve missed that offline mode. I’ve just tested and I’m getting the same error. I even installed PyTorch with the OpenSearch image and set DJL_OFFLINE, but no joy.

This issue has already been reported on GitHub.

Did you see that?

11

Thanks Pablo, yeah saw that, but no solution.
Is it possible to deploy the model on a cluster with internet access, then export it and import it on another cluster without internet access ?

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.