Uploading a sentence transformer model of medical domain to OpenSearch

Plugins Machine Learning
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 2.7

Describe the issue: I want to upload the model pritamdeka/S-PubMedBert-MS-MARCO to OpenSearch using the following API:

POST /_plugins/_ml/models/_upload
{
  "name": "huggingface/pritamdeka/S-PubMedBert-MS-MARCO",
  "version": "1.0.1",
  "model_format": "TORCH_SCRIPT"
}

I get an error: Model is not in the list of pre-trained models.
My use case is related to semantic search on medical terminologies. Though huggingface/sentence-transformers/all-MiniLM-L12-v2 works to some extent, I wanted to upload a domain specific sentence transformer model for vector index search.

Can anybody guide me how to upload such a model to Opensearch, which is not in the list of presently supported pre-trained models(section 5 of Pretrained models - OpenSearch documentation)

Configuration:

Relevant Logs or Screenshots:

2

You can take a look at the following urls:

  1. What in the ML is going on around here? · OpenSearch
  2. Demo Notebook for Sentence Transformer Model Training, Saving and Uploading to OpenSearch — Opensearch-py-ml 1.1.0 documentation
  3. Demo Notebook for deploying CLIPTextModel to OpenSearch — Opensearch-py-ml 1.1.0 documentation

Please let me know if you have any other question. Thanks

3

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.