We are releasing neural-search as an experimental feature with OpenSearch 2.4. The feature is created to enable neural embedding based search in OpenSearch. It provides the capability for indexing documents and doing neural search on the indexed documents. More details about the plugin can be found in this RFC: [RFC] OpenSearch neural-search plugin · Issue #11 · opensearch-project/neural-search · GitHub.
To use the plugin, you can download any pre-trained language model (e.g. from huggingface.co) and upload to OpenSearch. Meanwhile we’ll also publish a GPT model, along with a toolkit script, so you can fine tune your language model with your own corpus of data.
As mentioned, this feature is experimental as of now. We’d love to gather some feedback from you on this feature, to help us understand your need around it, and plan our next steps accordingly. Any comments/suggestions will be highly appreciated!