All, I’ve received requests from some partners and users for the ability to re-rank search results using an external service. An external re-ranker might be used to:
-
Improve lexical search results using a LLM model to re-rank results based on semantic similarity. This blog by Cohere describes the concept.
-
Re-rank search results for intelligent personalization. For instance, you might have a product catalog indexed in OpenSearch, which allows user to search for products by keywords. If you had context about the user running the search, a personalization model could trained to re-rank search results based on the user’s product propensities.
-
Re-rank search results based on sentiment. This could be useful for analytical purposes. Let’s say you had tweets indexed into OpenSearch, and you wanted to find all tweets that mentioned one of your product brands and rank the results by negative or positive sentiment.
The proposed solution will users to provision a re-rank connector similar to what we released in OpenSearch 2.9 for Cohere’s Embed API, but the connector will be built for Cohere’s Rerank API. We will then provide users the ability to configure a search pipeline so that search results could be sent through the connector for re-ranking.
Would love to get feedback, and identify users and partners who want this capability so that we can prioritize accordingly. Thanks!