Hi all,
I have been exploring OpenSearch for a project.
Just wanted to if unidirectional synonym is supported by OpenSearch.
ex: when I query “you” I am referring “you” as “Bob” but when I query as “Bob” it should not referred as “you”.
you → bob
bob != you
So without the full context this is a really hard problem. Seems like if you is definitely = bob then bob should also be = to you. Maybe a bit more context of the document types would help here in understanding the problem.
Generally speaking it seems like what you are looking for is a hyperonym. A good example of this would be all dogs are animals but not all animals are dogs.
Here are a few things that might not exactly solve the problem but might help guide you to a method for handling these.
Vector Scoring with Term Embeddings
Patterns for Elasticsearch Synonyms: Taxonomies and Managed Vocabularies
Improving Elastic Search Query Result with Query Expansion using Topic Modeling
@dtaivpp yes I am talking about the same, is it supported in the OpenSearch new version as I didn’t see documentation?
Thanks a lot, the article helps a lot.
So it seems like no one really has an implementation that solves this. I asked in a Search Relevancy Slack (which I strongly recommend you join) and here was some of the feedback they offered:
Alessandro: More than an Hypernym this is a task in natural language processing called “coreference resolution” and it has to do with discourse analysis (it may affect sequence of sentences). There is no out of the box integration for OpenSearch as far as I know (or Solr or Elasticsearch for what matters), but you can pre-process your text using open source libraries like spaCy or CoreNLP/coreference
Querqy, which is currently being ported to OpenSearch https://github.com/querqy/querqy-opensearch (I don’t know exactly how ready it is but it looks pretty solid) can handle unidirectional synonyms at query time - basically it’s a ‘query rewriter’. Check out the docs www.querqy.org - this is a solution used at scale by some very big companies (lots of our clients use it).
great I will surely go through it.
@CharlieOSC is it possible to integrate in AWS OpenSearch service , I am evaluating it for the first time have limited knowledge on that.
Sorry for the late reply, I’ve been on holiday. Querqy isn’t currently supported by the OpenSearch Service as far as I can see Plugins by engine version - Amazon OpenSearch Service …but plans are afoot so it may well be in the future! However I understand if this doesn’t solve your immediate problem.