Geoip mapping is not working

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

opensearch:2.5.0
opensearch-dashboards:2.5.0
logstash-oss-with-opensearch-output-plugin:8.4.0

Describe the issue:

I used to have an attribute called geoip.location and it’s data type is geo_point. I am using this geoip.location to map the geopoint in the Coordinate Map. It was working perfectly.

Suddenly somehow I found the geoip.location is gone. Instaed of geoip.location, I got geoip.geo.location.lat and geoip.geo.location.lon but I can’t use the Coordinate Map anymore.

Can someone help me? How can I get the geoip.location back or how can I get the Coordinate Map working?

Relevant Logs or Screenshots:

Screenshot 2023-03-14 150501
Screenshot 2023-03-14 150501332×544 14.2 KB

2

@junqiu Do you have any insight here?

3

The new format for your location data consists of two separate fields: geoip.geo.location.lat and geoip.geo.location.lon . This is different from the previous format you were using for geo_point, which had the entire location data stored in a single field called geoip.location.

Because we cannot update the mapping for an existing index, you can reindex your data with correct mapping.

Steps:

  1. Create mapping for new destination index
PUT destination-index-name
{
  "mappings": {
      "properties": {
        "geoip": {
          "properties": {
            "geo": {
              "properties": {
                "location": {
                  "type": "geo_point"
                }
              }
            }
          }
        }
      }
  }
}
  1. Reindex to copy all documents from existing index to destination index
POST _reindex
{
   "source":{
      "index":"existing-index-name"
   },
   "dest":{
      "index":"destination-index-name"
   }
}

Then you’ll have the geo_point mapping for geoip.geo.location in the new index.

4

@junqiu

Thank you for your help.

my destination index name has , like logstash-data-. The * stands for the date.

While I try to create a mapping for it, it shows " reason": “Invalid index name [logstash-data-*], must not contain the following characters [ , ", *, \, <, |, , >, /, ?]”,"

Is there any way I can create a mapping for such index?

Thanks you
Beilei

5

Hi @beilei

you can try to use an index template. An index template is a way to define settings and mappings for future indices that match a specified pattern. When you create an index that matches the template pattern, OpenSearch applies the template to the new index and uses the settings and mappings specified in the template.

like:

PUT _index_template/logstash-data-template
{
  "index_patterns": [
    "logstash-data-*"
  ],
  "mappings": {
    "properties": {
      "geoip": {
        "properties": {
          "geo": {
            "properties": {
              "location": {
                "type": "geo_point"
              }
            }
          }
        }
      }
    }
  }
}