I am trying to get OpenSearch to work with deploying Machine Learning Models but am running into a connection issue from the JVM that runs opensearch, when it trys download any libraries from the djl site it gives an SSL error.
I can wget any files needed to the server and place them where they need to be but I can’t get the system to recognize the JniLibs file, libdjl_torch.so.
I was able to get the PyTorch libs by downloading the python package and moving those library files into the directory opensearch/ml_cache/pytorch/1.13.1-cpu-precxx11-linux-x86_64 and it sees them and does not try and download them again.
When I put the above JNI file in that directory it still trys to download it giving an SSL error, I also tried renaming it to 0.31.1-libdjl_torch.so as it seems that is how it gets named when put in the directory and still the file is trying to be downloaded.
This might be of some help - Connecting to externally hosted models - OpenSearch Documentation - it sounds like you’re trying to pull an externally hosted model. There’s some configuration options that have to be set first, in particular a set of trusted endpoints.
Let me know if that gets you moving in the right direction and we’ll go from there.
Awesome, @yeonghyeonKo! Would you mind creating a PR in ml-commons with a step-by-step development guide that includes detailed instructions? The community would benefit immensely from this.
Sure! @dhrubo I hope it will help others who are not familiar with the same issue.
Have you read my comment that detours Firewall issues by pre-downloading pytorch and tokenizers folders and copying them to the opensearch/data/ml_cache path in a ML Node?
(and then defining a docker image for new pods or using docker copy for existing pods)
I think there might be better way instead of the above because it still needs proxy CDN or downloading files from public internet to your air-gapped environments. In my point of view, it’s quite complicated for developers who just want to follow a guide.