Opensearch index patterns - Searchable Field (change "Index" property to true)

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch 7.10

Describe the issue:
Hi everyone.
I’m currently running a Telegraf instance with the [inputs.ping] plugin activated, pointing to a specific IP.
I’m receiving the ping back from the server with no issues, and I’m sending the data to a OpenSearch instance via the [outputs.elasticsearch] plugin in telegraf.conf (creating a Index named “webserver-%Y.%m.%d.”).
I then create the Index Pattern in the OpensearchDashboard based on the name “webserver-*” to catch all the data received with that pattern.
At this point I have the correct fields structure with all the “cpu.” fields and their values.
Here comes the problem: all the fields that OpenSearch automatically maps from the [inputs.ping] plugin via Telegraf, are mapped with the “searchable” property set to false, therefore making all these fields NOT queriable, so a bit useless when I need to filter certain values and show them in all the charts in the final dashboard.

I did search online for possible solutions, but it appears like I can’t actually update the index structure to change the “Searchable” property, without OpenSearch complaining about it. I’ve seen some cases using the “reindex API”, but in that case I can’t remap all my “webserver-*” indexes, but only specific ones like “webserver-2023-19-01”.
I can’t figure out if it’s some configuration to be done in the Telegraf.conf file, or if it’s an OpenSearch issue.

Any help will be appreciated, I’m really struggling and start to feel the frustration about this bit of code.

Configuration:
The configuration from the Telegraf.conf file.

[[outputs.elasticsearch]]
  ## The full HTTP endpoint URL for your Elasticsearch instance
  ## Multiple urls can be specified as part of the same cluster,
  ## this means that only ONE of the urls will be written to each interval
  urls = [ "http://localhost:9200" ] # required.
  ## Elasticsearch client timeout, defaults to "5s" if not set.
  timeout = "5s"
  ## Set to true to ask Elasticsearch a list of all cluster nodes,
  ## thus it is not necessary to list all nodes in the urls config option
  enable_sniffer = false
  ## Set to true to enable gzip compression
  enable_gzip = false
  ## Set the interval to check if the Elasticsearch nodes are available
  ## Setting to "0s" will disable the health check (not recommended in production)
  health_check_interval = "10s"
  ## Set the timeout for periodic health checks.
  # health_check_timeout = "1s"
  ## HTTP basic authentication details.
  ## HTTP basic authentication details
  username = "admin"
  password = "admin"
  ## HTTP bearer token authentication details
  # auth_bearer_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"

  ## Index Config
  ## The target index for metrics (Elasticsearch will create if it not exists).
  ## You can use the date specifiers below to create indexes per time frame.
  ## The metric timestamp will be used to decide the destination index name
  # %Y - year (2016)
  # %y - last two digits of year (00..99)
  # %m - month (01..12)
  # %d - day of month (e.g., 01)
  # %H - hour (00..23)
  # %V - week of the year (ISO week) (01..53)
  ## Additionally, you can specify a tag name using the notation {{tag_name}}
  ## which will be used as part of the index name. If the tag does not exist,
  ## the default tag value will be used.
  # index_name = "telegraf-{{host}}-%Y.%m.%d"
  # default_tag_value = "none"
  index_name = "webserver-%Y.%m.%d" # required.

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  ## Template Config
  ## Set to true if you want telegraf to manage its index template.
  ## If enabled it will create a recommended index template for telegraf indexes
  manage_template = false
  ## The template name used for telegraf indexes
  template_name = "telegraf"
  ## Set to true if you want telegraf to overwrite an existing template
  overwrite_template = false
  ## If set to true a unique ID hash will be sent as sha256(concat(timestamp,measurement,series-hash)) string
  ## it will enable data resend and update metric points avoiding duplicated metrics with diferent id's
  force_document_id = false

  ## Specifies the handling of NaN and Inf values.
  ## This option can have the following values:
  ##    none    -- do not modify field-values (default); will produce an error if NaNs or infs are encountered
  ##    drop    -- drop fields containing NaNs or infs
  ##    replace -- replace with the value in "float_replacement_value" (default: 0.0)
  ##               NaNs and inf will be replaced with the given number, -inf with the negative of that number
  # float_handling = "none"
  # float_replacement_value = 0.0

  ## Pipeline Config
  ## To use a ingest pipeline, set this to the name of the pipeline you want to use.
  # use_pipeline = "my_pipeline"
  ## Additionally, you can specify a tag name using the notation {{tag_name}}
  ## which will be used as part of the pipeline name. If the tag does not exist,
  ## the default pipeline will be used as the pipeline. If no default pipeline is set,
  ## no pipeline is used for the metric.
  # use_pipeline = "{{es_pipeline}}"
  # default_pipeline = "my_pipeline"
[[inputs.ping]]
  ## Hosts to send ping packets to.
  urls = ["172.xx.xx.xx", "172.xx.xx.xx", "172.xx.xx.xx"]

  ## Method used for sending pings, can be either "exec" or "native".  When set
  ## to "exec" the systems ping command will be executed.  When set to "native"
  ## the plugin will send pings directly.
  ##
  ## While the default is "exec" for backwards compatibility, new deployments
  ## are encouraged to use the "native" method for improved compatibility and
  ## performance.
  # method = "exec"

  ## Number of ping packets to send per interval.  Corresponds to the "-c"
  ## option of the ping command.
  # count = 1

  ## Time to wait between sending ping packets in seconds.  Operates like the
  ## "-i" option of the ping command.
  # ping_interval = 1.0

  ## If set, the time to wait for a ping response in seconds.  Operates like
  ## the "-W" option of the ping command.
  # timeout = 1.0

  ## If set, the total ping deadline, in seconds.  Operates like the -w option
  ## of the ping command.
  # deadline = 10

  ## Interface or source address to send ping from.  Operates like the -I or -S
  ## option of the ping command.
  # interface = ""

  ## Percentiles to calculate. This only works with the native method.
  # percentiles = [50, 95, 99]

  ## Specify the ping executable binary.
  # binary = "ping"

  ## Arguments for ping command. When arguments is not empty, the command from
  ## the binary option will be used and other options (ping_interval, timeout,
  ## etc) will be ignored.
  # arguments = ["-c", "3"]

  ## Use only IPv6 addresses when resolving a hostname.
  # ipv6 = false

  ## Number of data bytes to be sent. Corresponds to the "-s"
  ## option of the ping command. This only works with the native method.
  # size = 56

Relevant Logs or Screenshots:

As you can see, the fields received from [ping] are not searchable. I need them to be, in order to check their values in the final dashboard charts.

Thanks a lot for the help! If other informations are needed, don’t hesitate to ask. And sorry if this issue is considered not related to this forum, but I’m really out of ideas and I’m searching for help both in OpenSearch and Telegraf forums.

Gabriele

So what you will need to do is create an index template that will apply those settings on ingest. You will create a pattern similar to the one webserver-* so that everytime a new index is created it will apply the template with searchable=true.

2 Likes

I imagined something similar would have been needed. Thanks a lot.

FYI there are cases where the inputs received from Telegraf get flagged as not searchable, others where they are searchable! I’m not sure yet about the reason of this. After a bit of fiddling with docker and telegraf and opensearch instances, I ended up having those properties flagged as searchable!

I’ll keep your solution for future issues, thanks again!

On top of what @dtaivpp recommended you can also check mappings of individual indices.
See Mapping - OpenSearch documentation

You should compare mappings of indices which are searchable and which are not. If you find cases where mappings differ then the reason can be that there might be some index templates that match only some indices but are missing the other.

Just a practical tip…

1 Like