What is the directory that is storing the JniLib files?

I am trying to get OpenSearch to work with deploying Machine Learning Models but am running into a connection issue from the JVM that runs opensearch, when it trys download any libraries from the djl site it gives an SSL error.
I can wget any files needed to the server and place them where they need to be but I can’t get the system to recognize the JniLibs file, libdjl_torch.so.

I was able to get the PyTorch libs by downloading the python package and moving those library files into the directory opensearch/ml_cache/pytorch/1.13.1-cpu-precxx11-linux-x86_64 and it sees them and does not try and download them again.

When I put the above JNI file in that directory it still trys to download it giving an SSL error, I also tried renaming it to 0.31.1-libdjl_torch.so as it seems that is how it gets named when put in the directory and still the file is trying to be downloaded.

Any advice or ideas would be greatly appreciated.

Hi @shorrocka -

This might be of some help - Connecting to externally hosted models - OpenSearch Documentation - it sounds like you’re trying to pull an externally hosted model. There’s some configuration options that have to be set first, in particular a set of trusted endpoints.

Let me know if that gets you moving in the right direction and we’ll go from there.

Thanks!

I faced the similar issue because of Air-gapped enviromemt. See the above link

I followed those steps and they got us almost all the way to where we need to be! Thank you for leading the way!

1 Like

Awesome, @yeonghyeonKo! Would you mind creating a PR in ml-commons with a step-by-step development guide that includes detailed instructions? The community would benefit immensely from this.

2 Likes

Sure! @dhrubo I hope it will help others who are not familiar with the same issue.

Have you read my comment that detours Firewall issues by pre-downloading pytorch and tokenizers folders and copying them to the opensearch/data/ml_cache path in a ML Node?
(and then defining a docker image for new pods or using docker copy for existing pods)

I think there might be better way instead of the above because it still needs proxy CDN or downloading files from public internet to your air-gapped environments. In my point of view, it’s quite complicated for developers who just want to follow a guide.

@dhrubo If you still think the above way is still valuable, I will create a PR as much detail as in ml-commons/docs/tutorials at main · opensearch-project/ml-commons · GitHub