Given the fast pace of changes in the ML world right now, I suspect we will see frequent updates to the pre-trained models being offered which improve on the results.
Given that the inference process is the slowest part of indexing content, we’re planning on using the Predict API to cache the embedding vectors outside of OpenSeach that way we only have to run the inference process on content that actually changed when indexing our content.
How should we plan to manage updates to the models we are using?
If we’re using just an updated version of a model (e.g. move from opensearch-neural-sparse-encoding-doc-v1 v1.0.0 to v1.0.1), is it safe to use the embeddings created with the older models? Or will this create issues?
My thinking is that I probably needed to track a model version with our embeddings, that way when I prepare to switch to a newer model I can first go through all our documents, create embedding using the new model and once that has finished, update my indexes with the new embeddings and delete the old ones.
However, if the old embeddings should basically be expected to work, it certainly would be more straightforward to just update the in-place embeddings and then trigger a refresh once the embedding caches have been updated.
I just want to make sure I plan accordingly, but I also want to make sure I’m not over engineering a solution.