Unrecognized Audit Settings

jecanne · December 16, 2021, 2:30am

Hi all,

I recently set up an OpenSearch cluster and configured security. When trying to configure auditing, I ran into some issues. Here are the options I applied in my opensearch.yml file.

plugins.security.audit.type: internal_opensearch
plugins.security.audit.enable_rest: true
plugins.security.audit.enable_transport: true
plugins.security.audit.ignore_users:
  - admin
  - kibanaserver
plugins.security.audit.config.index: "'os-sirt-qa-auditlog-'YYYYMM"

When starting up the cluster, it seems like three of these above are not being recognized as valid settings; those three are:

plugins.security.audit.enable_rest
plugins.security.audit.enable_transport
plugins.security.audit.ignore_users

Interestingly these settings are mentioned in the official docs so they should be good I would think.

Below is the full stacktrace:

[2021-12-14T19:40:24,375][ERROR][o.o.b.Bootstrap          ] [os_sirt1] Exception
java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.ignore_users] did you mean [plugins.security.audit.config.username]?
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) [opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) [opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) [opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) [opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-1.2.0.jar:1.2.0]
	at org.opensearch.cli.Command.main(Command.java:101) [opensearch-cli-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) [opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) [opensearch-1.2.0.jar:1.2.0]
	Suppressed: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.enable_transport] did you mean [opendistro_security.audit.enable_transport]?
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.main(Command.java:101) [opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) [opensearch-1.2.0.jar:1.2.0]
	Suppressed: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.enable_rest] did you mean any of [opendistro_security.audit.enable_rest, plugins.security.audit.config.enable_ssl, plugins.security.audit.type]?
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.main(Command.java:101) [opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) [opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) [opensearch-1.2.0.jar:1.2.0]
[2021-12-14T19:40:24,383][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [os_sirt1] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.ignore_users] did you mean [plugins.security.audit.config.username]?
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:182) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-1.2.0.jar:1.2.0]
	at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) ~[opensearch-1.2.0.jar:1.2.0]
Caused by: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.ignore_users] did you mean [plugins.security.audit.config.username]?
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.0.jar:1.2.0]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.0.jar:1.2.0]
	... 6 more
	Suppressed: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.enable_transport] did you mean [opendistro_security.audit.enable_transport]?
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) ~[opensearch-1.2.0.jar:1.2.0]
	Suppressed: java.lang.IllegalArgumentException: unknown setting [plugins.security.audit.enable_rest] did you mean any of [opendistro_security.audit.enable_rest, plugins.security.audit.config.enable_ssl, plugins.security.audit.type]?
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:463) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) ~[opensearch-1.2.0.jar:1.2.0]
		at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) ~[opensearch-1.2.0.jar:1.2.0]

pablo · December 16, 2021, 3:05am

@jecanne What is the OpenSearch version?

jecanne · December 16, 2021, 3:18am

@pablo it’s 1.2.0 (see stacktrack referencing opensearch-1.2.0.jar)

pablo · December 16, 2021, 9:43am

@jecanne Try the below lines instead.

opendistro_security.audit.enable_rest: true
opendistro_security.audit.enable_transport: true
opendistro_security.audit.ignore_users:
  - admin
  - kibanaserver

jecanne · December 21, 2021, 3:17pm

@pablo sorry for the late response. This fixed the problem.

As for the settings, is there a place I can report this so it gets tracked and the values can be updated to their proper name? I might even try to fix this myself time permitting, but would love to at least track it.

pablo · December 22, 2021, 3:54pm

@jecanne According to OpenSearch logs these options are deprecated and will be removed in v2.0.0.

[2021-12-22T14:22:36,578][WARN ][o.o.s.c.ConfigurationRepository] [opensearch-node1] Following keys [opendistro_security.audit.enable_transport, opendistro_security.audit.enable_rest, opendistro_security.audit.ignore_users] are deprecated in opensearch settings. They will be removed in plugin v2.0.0.0

Instead, you can use the Audit logs configuration in Kibana UI which was implemented in OpenDistro 1.10.1

github.com

opendistro-for-elasticsearch/opendistro-build/blob/main/release-notes/opendistro-for-elasticsearch-release-notes-1.10.1.md#security-kibana-plugin

# Open Distro for Elasticsearch 1.10.1 Release Notes

Open Distro for Elasticsearch 1.10.1 is now available for [download](https://opendistro.github.io/for-elasticsearch/downloads.html).

The release consists of Apache 2 licensed Elasticsearch version 7.9.1, and Kibana version 7.9.1. Plugins in the distribution include Alerting, Index Management, Performance Analyzer (with Root Cause Analysis Engine), Security, SQL, Machine Learning with k-NN, and Anomaly Detection. Also, SQL JDBC/ODBC driver, SQL CLI Client, and PerfTop (a client for Performance Analyzer) are available for download now.


## Release Highlights

* Anomaly Detection supports a [command line interface](https://github.com/opendistro-for-elasticsearch/anomaly-detection/tree/master/cli) that allows users to create, start, stop and delete detectors, and work with multiple clusters using named profiles.
* Anomaly Detection supports three different types of [sample detectors and corresponding indices](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin/pull/272) that allow users to detect sample anomalies using logs related to HTTP response codes, eCommerce orders, and CPU and memory of a host.
* The Alerting feature now supports [email destinations](https://github.com/opendistro-for-elasticsearch/alerting/pull/244) to send notifications without using a web hook.
* K-NN supports [warmup API](https://github.com/opendistro-for-elasticsearch/k-NN#warmup-api) that allows users to explicitly load indices’ graphs used for approximate k-NN search into memory before performing their search workload. With this API, users no longer need to run random queries to prevent initial latency penalties for loading graphs into the cache.
* The updated Kibana plugin for Security streamlines security workflows, improves usability and adds audit and compliance logging configuration.

## Release Details

Open Distro for Elasticsearch 1.10.1 includes the following features, enhancements, bug fixes, infrastructure, documentation, maintenance, and refactoring updates.

## BREAKING CHANGES

This file has been truncated. show original

Topic		Replies	Views
Audit log setting parameters not working Security	4	387	April 18, 2023
Opensearch Security Plugin config ignoring openId setting Security configure	3	753	February 10, 2022
Resolved - Error configuring opensearch-dashboards in opensearch-security configuration Security troubleshoot , configure	4	765	September 22, 2022
SSL issue when server is restarted OpenSearch troubleshoot , configure	1	869	November 30, 2022
Support URLs instead of IPs on external_opensearch option for audit logs Security	3	213	September 15, 2023

Unrecognized Audit Settings

Related Topics