After upgrading to opensearch 1.3 from elasticsearch 7.10 `index` field expecting index id

OpenSearch Dashboards
1

We recently upgraded to Opensearch 1.3 from Elasticsearch 7.10 in AWS. After upgrading we are observing that the below query starts failing

https://<url>/_dashboards/app/discover#/?_a=(columns:!(_source),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,key:legId.traceId,negate:!f,params:(query:'95c81169d7485467'),type:phrase),query:(match_phrase:(legId.traceId:'95c81169d7485467')))),index:air-leg,interval:auto,query:(language:kuery,query:''),sort:!())&_g=(refreshInterval:(pause:!t))

The same works fine in elasticsearch 7.10.

In Opensearch if we replace index:air-leg with the index id like index:f4bd2db0-c73b-11ed-89c4-cf7874960abe then it works fine.

Is this change expected? If there is any github issue related to this can some please point me to that?

2

@tuk that id referrs to the index pattern ID. I dont know about elasticsearch 7.10, but opensearch does not query the index directly and always queries through the index pattern. There are two scenarios that come to mind here for me that could explain what you are seeing.

  1. Elasticsearch allows for that field to support index id’s directly and opensearch does not
  2. When you upgraded from elastic search to OpenSearch, the index pattern saved objects id’s did not stay the same or changed due to some underlying reason

I can tell you this, referring to an index directly is not supported in OpenSearch Dashboard’s right now. But if you update that o use an index pattern id, you should be good to go!

3

Thanks @ashwinpc for replying.

I think I was not clear on my question.

We do not want to query against index. We are using index pattern only (but name instead of id). In index:air-leg that I have mentioned in the question air-leg is an index pattern name.

4

I see. This might be a bug that was introduced in OpenSearch. Can you open an issue for this in the OpenSearch Dashboards GitHub repo?

5

I have created [BUG] Not able to query using index pattern name in opensearch dashboard 1.3 · Issue #3648 · opensearch-project/OpenSearch-Dashboards · GitHub

1 Like
6

The above issue is still not resolved. So for now I am getting the index-pattern id by querying opensearch dashboard and then using it in my query.

To get the index-pattern id I am using a query like below

debrajmanna@debrajmanna-DX6QR261G3 java % curl -X GET "https://staging.com/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "index-pattern.title": "air-leg"
    }
  },
  "_source": false
}'

After migration I am observing the above query returning two hits

...
"max_score" : 3.370884,
    "hits" : [
      {
        "_index" : ".kibana_3",
        "_type" : "_doc",
        "_id" : "index-pattern:air-leg",
        "_score" : 3.370884
      },
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:air-leg",
        "_score" : 2.8771467
      }
    ]

If I create the index-pattern after migation then it returns a single hit something like below

"max_score" : 2.170884,
    "hits" : [
      {
        "_index" : ".kibana_2",
        "_type" : "_doc",
        "_id" : "index-pattern:f4bd2db0-c73b-11ed-89c4-cf7874960abe",
        "_score" : 2.170884
      }
    ]

Can someone let me know what is the logic behind creation of .kibana_2 and .kibana_3 indices?

11

Different .kibana_<id> indices are created as the structure of the system index changes over time. You would always want to query the latest system index and to do so, you would want to run /.kibana/_search; .kibana is an alias that will always point to the correct system index.