Good to know that OpenSearch will be more extensible and integrated with external systems. This will really help a variety of usecases. Regarding the RAG, is this going to be built-in in ml-commons or will it be supported through external APIs?
Yes, the ML extensibility capabilities are part of ml-commons. The external APIs will be mapped to the ml-commons APIs. The models managed by ml-commons will be an abstraction that could be associated with local and external models. The RAG workflow will be achieved through a workflow configuration that allows you to select the source of your embedding and generative models. You could use a self-managed generative model that you trained on your own or you could leverage a foundation model service.
Having an issue where i uploaded a model and tested the neural search plugin and it worked great. Then i deleted the model using the APIās but i keep seeing these errors in the log
ML model not in cache. Remove all of its cache files.
Cant tell what is looking for the deleted model ? I even deleted all the indexes and pipelines i had created. Any insights on what could be happening or what to look for ?
@srikant This log means there are some local cache folder not deleted yet for the model, the cron job trying to delete the file cache but seems these cache files canāt be deleted.
Which version are you using? You can check your <OpenSearch_home_folder>/data folder, you can manually delete the ml_cache folder, or djl folder if you are using version before 2.7.
Great, thanks that worked.
Running into an issue where Iām trying to store a vector size of 1536, and Iām getting an error āfailed to parseā. The parser gets to a certain part of the vector and just stops. How can I solve this?
hi, @rid, can you share more details like whatās the index setting? You can also test if the model can return correct embedding by calling predict API , refer to Model-serving framework - OpenSearch documentation
Are we expecting GA for this feature in the upcoming release 2.9?
1 Like
@asfoorial, yes, the ML framework and the neural search plugin will be GA. Apologies, on the slippage. We need some extra time to wrap things up with the model access controls.
As part of the ML framework, we will also release the first phase of an extensibility framework that will enable developers to create nativeAI services and ML platform integrations simply by defining JSON docs that describe the blueprint for the integration. Weāre releasing a connector for Amazon SageMaker and Amazon Bedrock for the first release. Weāre working with partners to develop 3rd-party options, and naturally, we would love for our community to contribute.