Scalability of Ak-NN with FAISS

Plugins k-NN
1

Hi all,

I’m looking to use OpenSearch as a distributed serving platform for FAISS at scale (order of multiple terabytes of dense, numerical records), scaling horizontally across multiple large AWS nodes.

Would folks generally expect this to be possible, or are there scalability limits to how FAISS interacts with the plug-in or how the plug-in interacts with OpenSearch (or just OpenSearch itself)

Thanks for reading and to all maintainers,

Rich