How can we deploy ML model (.zip) to nodes locally, not via SSL or the firewall

OpenSearch
, , ,
4

@Gsmitt Sorry for the confusion. Though I registered ML Model using private CDN url, deployment is the real problem.

OpenSearch uses djl library to load models externally, and each pretrained model would be downloaded by downloadPyTorch(code) and openUrl(code) method.

Do you have any idea for it?
(ref: Model deployment failure with ml-commons plugin in internet disabled environment - #3 by amank)

While trying to deploy a model, connecting to PyTorch fails (I think this is because the phase for depoyment doesn’t use private CDN.)

test-opensearch-cluster-ml-0 opensearch [2024-07-24T05:23:58,955][INFO ][o.o.m.m.MLModelManager   ] [test-opensearch-cluster-ml-0] Initializing the rate limiter with setting 4.0 per MINUTES (TPS limit 0.06666666666666667), evenly distributed on 1 nodes
test-opensearch-cluster-ml-0 opensearch [2024-07-24T05:23:58,955][INFO ][o.o.m.m.MLModelManager   ] [test-opensearch-cluster-ml-0] Initializing the rate limiter with setting 4.0 per MINUTES (TPS limit 0.06666666666666667), evenly distributed on 1 nodes
test-opensearch-cluster-ml-0 opensearch [2024-07-24T05:23:58,955][INFO ][o.o.m.m.MLModelManager   ] [test-opensearch-cluster-ml-0] Successfully redeployed model controller for model Hd-83pABm0qUjmUv3l96
test-opensearch-cluster-ml-0 opensearch [2024-07-24T05:24:03,470][WARN ][a.d.p.j.LibUtils         ] [test-opensearch-cluster-ml-0] Override PyTorch version: 1.13.1.
test-opensearch-cluster-ml-0 opensearch [2024-07-24T05:24:03,553][ERROR][o.o.m.e.a.DLModel        ] [test-opensearch-cluster-ml-0] Failed to deploy model Hd-83pABm0qUjmUv3l96
test-opensearch-cluster-ml-0 opensearch ai.djl.engine.EngineException: Failed to save pytorch index file
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.jni.LibUtils.downloadPyTorch(LibUtils.java:429) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.jni.LibUtils.findNativeLibrary(LibUtils.java:314) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.jni.LibUtils.getLibTorch(LibUtils.java:93) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:81) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:53) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:41) ~[pytorch-engine-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at ai.djl.engine.Engine.getEngine(Engine.java:190) ~[api-0.28.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.engine.algorithms.DLModel.doLoadModel(DLModel.java:188) ~[opensearch-ml-algorithms-2.15.0.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.engine.algorithms.DLModel.lambda$loadModel$1(DLModel.java:286) [opensearch-ml-algorithms-2.15.0.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at java.base/java.security.AccessController.doPrivileged(AccessController.java:571) [?:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.engine.algorithms.DLModel.loadModel(DLModel.java:252) [opensearch-ml-algorithms-2.15.0.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.engine.algorithms.DLModel.initModel(DLModel.java:142) [opensearch-ml-algorithms-2.15.0.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.engine.MLEngine.deploy(MLEngine.java:125) [opensearch-ml-algorithms-2.15.0.0.jar:?]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.model.MLModelManager.lambda$deployModel$52(MLModelManager.java:1067) [opensearch-ml-2.15.0.0.jar:2.15.0.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.core.action.ActionListener$1.onResponse(ActionListener.java:82) [opensearch-core-2.15.0.jar:2.15.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.ml.model.MLModelManager.lambda$retrieveModelChunks$73(MLModelManager.java:1680) [opensearch-ml-2.15.0.0.jar:2.15.0.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.core.action.ActionListener$1.onResponse(ActionListener.java:82) [opensearch-core-2.15.0.jar:2.15.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.action.support.ThreadedActionListener$1.doRun(ThreadedActionListener.java:78) [opensearch-2.15.0.jar:2.15.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:941) [opensearch-2.15.0.jar:2.15.0]
test-opensearch-cluster-ml-0 opensearch         at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.15.0.jar:2.15.0]
test-opensearch-cluster-ml-0 opensearch         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
test-opensearch-cluster-ml-0 opensearch         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
test-opensearch-cluster-ml-0 opensearch         at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
test-opensearch-cluster-ml-0 opensearch Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target