Removing opensearch.requestHeadersWhitelist

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.7.0

Describe the issue:
In my organization we are currently working with opensearch 2.4.0, and all is great but
we are researching the option to upgrade to 2.7.0 (and above),

when I’ve tested with version 2.7.0, my client stopped working,
getting the following screen:

image
image874×412 14.9 KB

when I’ve removed the key
opensearch.requestHeadersWhitelist

from my opensearch_dashboards.yml, it suddenly starts working.

I know its deprecated and the new key is allowlist, but it is also not working.

I do not see anything relevant in the logs even after raising debug.
please tell me
tell me which information required and I will provide.

Configuration:

Relevant Logs or Screenshots:

1 Like
2

Hi @nap608

Could you please share your opensearch_dashboards.yml and opensearch.yml files? Please feel free to remove or change all sensitive data.

3

opensearch.yml

plugins.security.ssl.transport.pemcert_filepath: certificates/elk-transport-crt.pem
plugins.security.ssl.transport.pemkey_filepath: certificates/elk-transport-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/elastic_ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false 
plugins.security.ssl.http.enabled: true 
plugins.security.ssl.http.pemcert_filepath: certificates/elk-node-crt.pem
plugins.security.ssl.http.pemkey_filepath: certificates/elk-node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/elastic_ca.pem
plugins.security.allow_unsafe_democertificates: true 
plugins.security.allow_default_init_securityindex: true 
plugins.security.authcz.admin_dn: 
  - 'CN=CONTROL-M_EM_ES_admin,O=ChangeMe,L=ChangeMeL,ST=ChangeMeST,C=CM'

 
#plugins.security.audit.type: internal_opensearch 
plugins.security.enable_snapshot_restore_privilege: true 
plugins.security.check_snapshot_restore_write_privileges: true 
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"] 
plugins.security.system_indices.enabled: true 
plugins.security.system_indices.indices: [".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"] 
node.max_local_storage_nodes: 3 
######## End OpenSearch Security Demo Configuration ######## 
path.logs: /home/pxem1/ctm_em/log/services/workflow_insights
cluster.name: workflow_insights_cluster
network.host: 0
node.name: px-859b65ab
node.roles: ["initial_master","data","ingest","master","remote_cluster_client"]
action.auto_create_index: false
bootstrap.memory_lock: true
http.port: 19200
transport.port: 19300
cluster.initial_master_nodes: ['px-859b65ab']
discovery.seed_hosts: ['px-859b65ab']
plugins.security.nodes_dn: 
  - 'CN=CONTROL-M_EM_ES_transport,O=ChangeMe,L=ChangeMeL,ST=ChangeMeST,C=CM'

opensearch_dashboards.yml

---
server.port: 15061
server.host: '0.0.0.0'
opensearch.hosts: ['https://localhost:19200']
opensearch.ssl.verificationMode: certificate

opensearch.requestHeadersWhitelist: [authorization, securitytenant] # This is the line causing issues which I want to remove.

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Global"]
opensearch_security.multitenancy.tenants.enable_private: false
opensearch_security.readonly_mode.roles: ["kibana_read_only","kibana_read"]
opensearch_security.cookie.secure: false


# Use this setting if you are running opensearch-dashboards without https
#server.rewriteBasePath: true
server.basePath: "/services-proxy/kibana"
csp.warnLegacyBrowsers: false
csp.rules:
 - "script-src 'unsafe-eval' 'self' 'unsafe-inline'"
# Options for the logging rotate feature
logging.rotate:
   enabled: true
   everyBytes: 10000000
   keepFiles: 5
   usePolling: true
   pollingInterval: 10000

#logging.quiet : true
logging.verbose: true
opensearch.ssl.certificateAuthorities: ['/home/pxem1/ctm_em/ini/ssl/elastic_ca.pem']
logging.dest: /home/pxem1/ctm_em/log/services/workflow_insights/opensearch-dashboards.log
server.ssl.enabled: true
server.ssl.key: /home/pxem1/ctm_em/opensearch-dashboards/config/certificates/kibana-node-key.pem
server.ssl.certificate: /home/pxem1/ctm_em/opensearch-dashboards/config/certificates/kibana-node-crt.pem
4

@kris may I get some attention here please.

5

Hi @nap608,

Have you tested OpenSearch Dashboards with your OpenSearch cluster that has the same version? How did you deploy the OpenSearch cluster and the OpenSearch Dashboards?

1 Like
6

I’ve used same version (2.7.0) for both os and os-dashboards.

by deploy you mean installation?
I’ve downloaded the tar file for both os and os-dashboards
extracted and edited the yaml file (previous comments), and edited the roles files.

roles.yml


_meta:
  type: "roles"
  config_version: 2

kibana_only_read:
  reserved: false
  cluster_permissions:
    - cluster_composite_ops_ro
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'read'
  tenant_permissions:
    - tenant_patterns:
        - 'global_tenant'
      allowed_actions:
        - 'kibana_all_write'
# Restrict users so they can only view visualization and dashboard on OpenSearchDashboards
kibana_read_only:
  reserved: true

# The security REST API access role is used to assign specific users access to change the security settings through the REST API.
security_rest_api_access:
  reserved: true
 
# Allows users to view monitors, destinations and alerts
alerting_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/alerting/alerts/get'
    - 'cluster:admin/opendistro/alerting/destination/get'
    - 'cluster:admin/opendistro/alerting/monitor/get'
    - 'cluster:admin/opendistro/alerting/monitor/search'
    - 'cluster:admin/opensearch/alerting/findings/get'

# Allows users to view and acknowledge alerts
alerting_ack_alerts:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/alerting/alerts/*'

# Allows users to use all alerting functionality
alerting_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster_monitor'
    - 'cluster:admin/opendistro/alerting/*'
    - 'cluster:admin/opensearch/alerting/*'
    - 'cluster:admin/opensearch/notifications/feature/publish'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices_monitor'
        - 'indices:admin/aliases/get'
        - 'indices:admin/mappings/get'

# Allow users to read Anomaly Detection detectors and results
anomaly_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/ad/detector/info'
    - 'cluster:admin/opendistro/ad/detector/search'
    - 'cluster:admin/opendistro/ad/detectors/get'
    - 'cluster:admin/opendistro/ad/result/search'
    - 'cluster:admin/opendistro/ad/tasks/search'
    - 'cluster:admin/opendistro/ad/detector/validate'
    - 'cluster:admin/opendistro/ad/result/topAnomalies'

# Allows users to use all Anomaly Detection functionality
anomaly_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster_monitor'
    - 'cluster:admin/opendistro/ad/*'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices_monitor'
        - 'indices:admin/aliases/get'
        - 'indices:admin/mappings/get'

# Allow users to execute read only k-NN actions
knn_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/knn_search_model_action'
    - 'cluster:admin/knn_get_model_action'
    - 'cluster:admin/knn_stats_action'

# Allow users to use all k-NN functionality
knn_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/knn_training_model_action'
    - 'cluster:admin/knn_training_job_router_action'
    - 'cluster:admin/knn_training_job_route_decision_info_action'
    - 'cluster:admin/knn_warmup_action'
    - 'cluster:admin/knn_delete_model_action'
    - 'cluster:admin/knn_remove_model_from_cache_action'
    - 'cluster:admin/knn_update_model_graveyard_action'
    - 'cluster:admin/knn_search_model_action'
    - 'cluster:admin/knn_get_model_action'
    - 'cluster:admin/knn_stats_action'

# Allows users to read Notebooks
notebooks_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/notebooks/list'
    - 'cluster:admin/opendistro/notebooks/get'

# Allows users to all Notebooks functionality
notebooks_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/notebooks/create'
    - 'cluster:admin/opendistro/notebooks/update'
    - 'cluster:admin/opendistro/notebooks/delete'
    - 'cluster:admin/opendistro/notebooks/get'
    - 'cluster:admin/opendistro/notebooks/list'

# Allows users to read observability objects
observability_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/observability/get'

# Allows users to all Observability functionality
observability_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/observability/create'
    - 'cluster:admin/opensearch/observability/update'
    - 'cluster:admin/opensearch/observability/delete'
    - 'cluster:admin/opensearch/observability/get'

# Allows users to read and download Reports
reports_instances_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/reports/instance/list'
    - 'cluster:admin/opendistro/reports/instance/get'
    - 'cluster:admin/opendistro/reports/menu/download'

# Allows users to read and download Reports and Report-definitions
reports_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/reports/definition/get'
    - 'cluster:admin/opendistro/reports/definition/list'
    - 'cluster:admin/opendistro/reports/instance/list'
    - 'cluster:admin/opendistro/reports/instance/get'
    - 'cluster:admin/opendistro/reports/menu/download'

# Allows users to all Reports functionality
reports_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/reports/definition/create'
    - 'cluster:admin/opendistro/reports/definition/update'
    - 'cluster:admin/opendistro/reports/definition/on_demand'
    - 'cluster:admin/opendistro/reports/definition/delete'
    - 'cluster:admin/opendistro/reports/definition/get'
    - 'cluster:admin/opendistro/reports/definition/list'
    - 'cluster:admin/opendistro/reports/instance/list'
    - 'cluster:admin/opendistro/reports/instance/get'
    - 'cluster:admin/opendistro/reports/menu/download'

# Allows users to use all asynchronous-search functionality
asynchronous_search_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/asynchronous_search/*'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices:data/read/search*'

# Allows users to read stored asynchronous-search results
asynchronous_search_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/asynchronous_search/get'

# Allows user to use all index_management actions - ism policies, rollups, transforms
index_management_full_access:
  reserved: true
  cluster_permissions:
    - "cluster:admin/opendistro/ism/*"
    - "cluster:admin/opendistro/rollup/*"
    - "cluster:admin/opendistro/transform/*"
    - "cluster:admin/opensearch/notifications/feature/publish"
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices:admin/opensearch/ism/*'

# Allows users to use all cross cluster replication functionality at leader cluster
cross_cluster_replication_leader_full_access:
  reserved: true
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - "indices:admin/plugins/replication/index/setup/validate"
        - "indices:data/read/plugins/replication/changes"
        - "indices:data/read/plugins/replication/file_chunk"

# Allows users to use all cross cluster replication functionality at follower cluster
cross_cluster_replication_follower_full_access:
  reserved: true
  cluster_permissions:
    - "cluster:admin/plugins/replication/autofollow/update"
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - "indices:admin/plugins/replication/index/setup/validate"
        - "indices:data/write/plugins/replication/changes"
        - "indices:admin/plugins/replication/index/start"
        - "indices:admin/plugins/replication/index/pause"
        - "indices:admin/plugins/replication/index/resume"
        - "indices:admin/plugins/replication/index/stop"
        - "indices:admin/plugins/replication/index/update"
        - "indices:admin/plugins/replication/index/status_check"

# Allow users to read ML stats/models/tasks
ml_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/ml/stats/nodes'
    - 'cluster:admin/opensearch/ml/models/get'
    - 'cluster:admin/opensearch/ml/models/search'
    - 'cluster:admin/opensearch/ml/tasks/get'
    - 'cluster:admin/opensearch/ml/tasks/search'

# Allows users to use all ML functionality
ml_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster_monitor'
    - 'cluster:admin/opensearch/ml/*'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices_monitor'

# Allows users to use all Notifications functionality
notifications_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/notifications/*'

# Allows users to read Notifications config/channels
notifications_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/notifications/configs/get'
    - 'cluster:admin/opensearch/notifications/features'
    - 'cluster:admin/opensearch/notifications/channels/get'

# Allows users to use all snapshot management functionality
snapshot_management_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/snapshot_management/*'
    - 'cluster:admin/opensearch/notifications/feature/publish'
    - 'cluster:admin/repository/*'
    - 'cluster:admin/snapshot/*'

# Allows users to see snapshots, repositories, and snapshot management policies
snapshot_management_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/snapshot_management/policy/get'
    - 'cluster:admin/opensearch/snapshot_management/policy/search'
    - 'cluster:admin/opensearch/snapshot_management/policy/explain'
    - 'cluster:admin/repository/get'
    - 'cluster:admin/snapshot/get'

# Allows user to use point in time functionality
point_in_time_full_access:
  reserved: true
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'manage_point_in_time'

# Allows users to see security analytics detectors and others
security_analytics_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/securityanalytics/alerts/get'
    - 'cluster:admin/opensearch/securityanalytics/detector/get'
    - 'cluster:admin/opensearch/securityanalytics/detector/search'
    - 'cluster:admin/opensearch/securityanalytics/findings/get'
    - 'cluster:admin/opensearch/securityanalytics/mapping/get'
    - 'cluster:admin/opensearch/securityanalytics/mapping/view/get'
    - 'cluster:admin/opensearch/securityanalytics/rule/get'
    - 'cluster:admin/opensearch/securityanalytics/rule/search'

# Allows users to use all security analytics functionality
security_analytics_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/securityanalytics/alerts/*'
    - 'cluster:admin/opensearch/securityanalytics/detector/*'
    - 'cluster:admin/opensearch/securityanalytics/findings/*'
    - 'cluster:admin/opensearch/securityanalytics/mapping/*'
    - 'cluster:admin/opensearch/securityanalytics/rule/*'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices:admin/mapping/put'
        - 'indices:admin/mappings/get'

# Allows users to view and acknowledge alerts
security_analytics_ack_alerts:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opensearch/securityanalytics/alerts/*'
7

@nap608

As per the documentation, you need to use opensearch.requestHeadersAllowlist parameter.

Please remove opensearch.requestHeadersWhitelist and try to use the following configuration:

opensearch.requestHeadersAllowlist: ["securitytenant","Authorization"]

Have you tried to change network.host: 0 to network.host: 0.0.0.0 ?

8

Hi @Eugene7 ,
what are the ramifications of not using that field?

As I said, I’ve tried both:
opensearch.requestHeadersAllowlist: [“securitytenant”,“Authorization”]
and
opensearch. requestHeadersWhitelist: [“securitytenant”,“Authorization”]

and it didn’t work, only when removed it work.

Is there a way to fully understand which headers I should add?

" Have you tried to change network.host: 0 to network.host: 0.0.0.0 ?"
No, I didn’t, could you explain what would it do? I will try.

9

Hi @nap608 ,

  • _local_ or 0.0.0.0 : Binds to all available network interfaces on the local machine.
  • _site_ : Binds to all site-local (private) IPv4 addresses.
  • _global_ : Binds to all globally-scoped (public) IPv4 addresses.
  • _[networkInterface]_: Binds to the IP address of the specified network interface.
10

Hi @Eugene7 ,
How is it related to the headers whitelist ?

11

It looked unusual to me. Do you use a proxy? Can you try to open OSD in a different browser? Do you use a direct connection between OS and OSD?

12

I’m using proxy, yes.
I will try different browser.
I’m using direct connection between os and osd, yes.

But again it working without issues if i’m using version between 2.4.0 and 2.6.0,
issues start with 2.7.0.

This is blocking me from upgrading OS in my firm.