How to deploy Model2Vec Embedding Models

Plugins Machine Learning
1

Hi all,

I was wondering how to deploy embedding models generated using MinishLab/model2vec: Model2Vec: Distill a Small Fast Model from any Sentence Transformer (github.com) . They are supported by SentenceTransformers Release v3.2.0 - ONNX and OpenVINO backends offering 2-3x speedup; Static Embeddings offering 50x-500x speedups at ~10-20% performance cost · UKPLab/sentence-transformers (github.com) and they are very efficient in terms of speed and with small loss in accuracy.

2

We don’t have any pre-trained traced model support for this model. You can use follow this notebook to trace your model and register in opensearch. If you want any pre-trained traced model support from opensearch, please cut an issue to either ml-commons or opensearch-py-ml repos