Highlighting Issue: Mismatch Between Analyzer Output and Highlighting Results

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.11.0

Describe the issue:
I’ve noticed that the highlighted terms sometimes do not align with the terms produced by the analyzer. For example, when searching for “Developer,” the text gets highlighted incorrectly as “eloper, S” instead of the expected “Developer.”

  1. Analyzer Output:
    Here is the result of the analyzer when applied to the term “Developer”:
{
  "tokens": [
    {
      "token": "developers/sl",
      "start_offset": 0,
      "end_offset": 9,
      "type": "SYNONYM",
      "position": 0,
      "positionLength": 2
    },
    {
      "token": "developer/sl",
      "start_offset": 0,
      "end_offset": 9,
      "type": "SL",
      "position": 0,
      "positionLength": 2
    }
  ]
}
  1. Highlight Query:
    Here is the search and highlight query I am using
GET /my_index/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "terms": {
            "id_field": [
              "matched_value"
            ],
            "boost": 1
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "highlight": {
    "fields": {
      "highlighting_field": {
        "highlight_query": {
          "match_phrase": {
            "highlighted_field": "Developer"
          }
        }
      }
    }
  }
}
  1. Highlight Result:
    The text gets highlighted as follows:
"highlight": {
  "highlighting_field": [
    "Full Stack Dev<em>eloper, S</em>oftware Project Manager"
  ]
}

Issue: The highlighted term “eloper, S” is incorrect. Based on the analyzer output, I expect the term “Developer” to be highlighted instead.

Questions:

  1. Why is there a discrepancy between the analyzer output and the highlighting result?
  2. How can I ensure that the highlighting matches the analyzer output accurately?
  3. Are there specific configurations or settings that I might be missing?

Here’s the mapping

{
 ...
   "highlighted_field": {
          "type": "text",
          "analyzer": "custom_analyzer",
          "search_analyzer": "custom_search_analyzer"
        },
...
}

Any insights or suggestions would be greatly appreciated!

Thank you!