OpenSearch experimental setting true by default (datetime_formatter_caching)

OpenSearch
1

Hi all,

Just wanted to bring up a change that was added into OpenSearch 2.12 which was enabled by default and broke some functionality in our applications.

The setting is described here: Date - OpenSearch Documentation

Firstly, I don’t think this change is mentioned clearly in the 2.12 release notes. Additionally, this appears to be an experimental setting, as the path to disable it is at: opensearch.experimental.optimization.datetime_formatter_caching.enabled

The release notes state that: “Experimental features are disabled by default. For instructions on how to enable them, refer to the documentation for the feature.”

Why was this change enabled by default then?

We have set the above flag to get ‘false’ to set the datetime format to be the same as previous versions of OpenSearch, but want to know if this flag is here to stay or if it will get removed in a future release, and the change made in 2.12 will become the required OpenSearch default.

We are cautious to make the change in our applications to support the new default date format, as due to it being an experimental feature we feel it should not be relied upon and could be removed at any time. Therefore, we may have to revert again to the old way if this default is removed.

Just wanted to raise this here to gather general thoughts on why this was set to be ‘enabled’ by default, despite being an ‘experimental’ feature. And is this the new default date format something we should move to in our code?

Cheers,
Stewart

1 Like
2

Hi Stewart, thanks for bringing this up. You’re definitely right that experimental features should be disabled by default and this looks incorrect. I have created a GitHub issue to address this, please feel free to share your thoughts:

1 Like
3

Hey @Stewart could you provide some more details on what specifically broke for you as a part of this change? Feel free to share here or on the GitHub issue itself. Thanks!

4

Responded on the ticket :slight_smile: Thanks for raising that issue!

1 Like
5

@Stewart Thanks for reporting this.

I agree that this is a miss for default.

I believe the indexing would have still gone through for these use cases.

  1. What is the use case to update the mappings to use explicit format instead of continuing to rely on default format (which is backward compatible with the older one in terms of handling documents)?
  2. Would a mitigation to update the mappings to use the new default format strict_date_time_no_millis||strict_date_optional_time||epoch_millis helped here (instead of the older default strict_date_optional_time||epoch_millis)?