Filtering large responses (e.g. cluster settings including defaults)

OpenDistro
1

According to the OpenSearch configuration docs I can get a full list of the cluster settings by sending a

GET _cluster/settings?include_defaults=true

request through OpenSearch Dashboards or a curl equivalent.

The response object is rather large and I want to narrow it down a bit.
Here is the first 30 lines of it:

{
  "persistent" : {
    "plugins" : {
      "index_state_management" : {
        "metadata_migration" : {
          "status" : "1"
        },
        "template_migration" : {
          "control" : "-1"
        }
      }
    }
  },
  "transient" : { },
  "defaults" : {
    "cluster" : {
      "max_voting_config_exclusions" : "10",
      "metadata" : {
        "perf_analyzer" : {
          "config" : {
            "overrides" : ""
          },
          "pa_node_stats_setting" : "1",
          "state" : "0"
        }
      },
      "auto_shrink_voting_configuration" : "true",
      "election" : {
        "duration" : "500ms",
        "initial_timeout" : "100ms"

How could I get the response to only return the value of some particular setting (e.g. persistent.plugins.index_state_management.template_migration.control) or a some sub-object of properties/settings (e.g. defaults.cluster.metadata)?

The search API has such a nice query DSL for searching through indexed documents, but in this case, I just want to narrow down the response, be it the settings object as described above or more generally, any object I could possibly get in the response.

I couldn’t find the answer anywhere (these forums and the web as a whole), so I sincerely hope this is not a trivial hiccup described at the very top of some documentation I’ve missed.

PS: I know the response can be traversed and filtered programmatically. But using Dashboards’ Dev Tools for play testing is so practical, it would be really handy to narrow the response down to just the bits you’re interested in the same step.
Though I could accept the next best alternative, whichever you might recommend.

2

Hi @lukaromih -

What I would do is to submit feature requests to the github repository asking for new endpoints /_cluster/settings to give you subsets of the data. However, what I usually end up doing is piping the json result through a command line json parser like jq (jq). It has a syntax of its own for taking in json and displaying whatever you like.

Another solution that is a worthwhile exercise is to write a few lines of code in your favorite programming language to parse the json programatically. This is likely reinventing the wheel if you become intimate with jq.

Nate

1 Like
3

Thank you, @nateynate .

I think jq is exactly what I was looking for. And, like you said, writing my own solution would just be reinventing the wheel and possibly not as good one.

Sure, adding another feature would probably save a click here and a copy paste there, but it would just add to the total weight of the core search engine, which needs to stay as lean and performant as possible for production use.

jq on the other hand, being an outside tool, doesn’t add to that weight and adds very little extra effort, considering you can copy and paste the request from Dashboards and it’s automatically converted to curl format.

If anything, it’s probably a feature, which should best be added to the Dasboards’ dev tools experience.

However, for now, I won’t be opening a feature request on github, because I don’t think I’m experienced enough with the whole OpenSearch (+Dashboards) workflow, to warrant it. It might turn out, my perceived problem is not a problem at all or just and edge case at best/worst.

If there’s somebody else though, that would also benefit from such a feature and finds current tooling lacking, sure, open a feature request and write here you’ve done so, so I or somebody else doesn’t open a duplicate request sometime in the future.

2 Likes
4

Don’t be afraid to file a feature request. You never know what other people might find useful, and we can only build what the community wants with feedback from people like you!

Thanks for sharing your issue here for other people to learn from.

Nate

5

Through pure luck more than anything, I’ve found that such a feature already exists and is actually pretty well documented in Elasticsearch docs, so I’m very surprised Google didn’t point me in right direction sooner.

I’ve grown very acquainted with OpenSearh docs and I like their style and structure, both on menu level, as well as within each individual document. I’ve accepted, however, that it’ll take some time, for the project to mature, until it’s docs have full coverage of all options, settings, APIs, params, … Hopefully, it’ll happen, but until then, I’ll probably have to peek into Elastic’s docs from time to time, to cover those gaps.

1 Like