Unable to send logs from fluentd to opensearch

Security
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Opensearch: 2.11.0
Opensearch-Dashboards: 2.11.0

Fluentd:

gem 'fluentd' version '1.16.2'
gem 'fluent-plugin-elasticsearch' version '5.3.0'
gem 'fluent-plugin-filter_typecast' version '0.0.3'
gem 'fluent-plugin-opensearch' version '1.1.4'
gem 'fluent-plugin-redis' version '0.3.5'
gem 'fluent-plugin-s3' version '1.7.2'

Describe the issue:
Trying to configure fluentd to send logs to opensearch. Installed opensearch plugin and tried with the most basic configuration I can think of. The index fails with the log line in the “relevant logs or screenshots”. I don’t understand what is missing. (and I’d like to understand how to parse - myself - the error so that i can fix the permissions).

Thanks for helping.

Configuration:

Fluentd:

<source>
  @type dummy
  tag dummy
  dummy {"hello":"world"}
</source>

<match dummy>
    @type opensearch
    host 10.112.96.33
    port 9200
    ca_file /tutto/root-ca.pem
    scheme https
    # Verify actually false due to not using the correct cname. not the actual issue.
    ssl_verify false
    user fluentd
    password <XXX>
    logstash_format true
</match>

Opensearch:
I configured a new user named “fluentd” using password and backend role: fluentd.
I configured a backend role named “fluentd” with the following permissions:

Cluster permissions:

  • cluster_monitor
  • cluster:monitor/main
  • cluster:monitor/health
  • cluster_composite_ops_ro

Index permissions:

  • Indices: [applications-, logstash-, infra_*]
  • Permissions: [crud, create_index]

I did it trough dashboards → Security.

Relevant Logs or Screenshots:

[2023-10-20T11:11:05,896][INFO ][o.o.s.p.PrivilegesEvaluator] [opensearch-cluster-master-2] No cluster-level perm match for User [name=fluentd, backend_roles=[fluentd], requestedTenant=null] Resolved [aliases=[*], allIndices=[*], types=[*], originalRequested=[*], remoteIndices=[]] [Action [cluster:monitor/main]] [RolesChecked [own_index]]. No permissions for [cluster:monitor/main]
2

Solved:
You have to map user to roles… Setting backend_role on users is not enough, you have to go into roles and then map the user to the role.