Just checking if there is a supported rolling upgrade path from opendistro-security plugin 0.X to 1.X ?
Some similar queries posted on the forums here, but I haven’t seen a definitive answer here or in the documentation -
For my specific query, we’re not running the full Opendistro suite of plugins, but are using the Opendistro Security plugin to complement elasticsearch-oss and provide LDAP auth functionality .
Current version: elasticsearch-oss v.6.8.1, opendistro-security 0.10.0.0-1
Target version: elasticsearch-oss v 7.10.2 , opendistro-security 1.13.1.0-1
Running a 5 node cluster - attempted a rolling upgrade, starting with a single node first . After upgrade (elasicsearch & security plugin) , the upgraded node rejoins the cluster OK, and Opendistro security is initialized on the node OK (using the existing .opendistro_security index)
LDAP auth seems to continue to work OK (tested by making requests to the upgraded node directly) , but native authentication using the internal users database does not work on the upgraded node (it still works fine on the non upgraded nodes). Local auth fails (order 1) and falls back to LDAP (order 2), but the internal users don’t exist in the LDAP directory of course.
Reluctant to proceed further as applications interacting with the cluster use the internal users DB . Disabling the plugin temporarily is not an option either, as it’s required for TLS communication between nodes.
I expect a full cluster shutdown followed by an upgrade on each node, and recreation of the .opendistro_security index will result in a successful upgrade - but I’d like to avoid a shutdown if possible.
Hi @lechuck
I managed to migrate however it was rather a messy process, I will describe below, hoping it helps.
Install ES6 OSS + odfe security plugin on the nodes (probably current state)
Disable shard allocation.
PUT _cluster/settings
{
“persistent”: {
“cluster.routing.allocation.enable”: “primaries”
}
}
perform synced flush POST _flush/synced
OPTIONAL: Temporarily stop the tasks associated with active machine learning jobs and datafeeds.
POST _ml/set_upgrade_mode?enabled=true
Download securityadmin.sh tool from new version of odfe plugin (1.4) and backup config.
./tools/securityadmin.sh -backup “./sgadmin7” -icl -key “…/config/kirk-key.pem” -cert “…/config/kirk.pem” -cacert “…/config/root-ca.pem” -nhnv
Verify the exported files:
./tools/securityadmin.sh -vc 6 -cd “./sgadmin7” -icl -key “…/config/kirk-key.pem” -cert “…/config/kirk.pem” -cacert “…/config/root-ca.pem” -nhnv
(fix any errors- if any)
Perform offline migration to new versions of odfe config using below command:
audit.yml (I had to update audit.yml file as below to avoid future issues)
_meta:
type: "audit"
config_version: 2
config:
# enable/disable audit logging
enabled: true
audit:
# Enable/disable REST API auditing
enable_rest: true
# Categories to exclude from REST API auditing
disabled_rest_categories:
- AUTHENTICATED
- GRANTED_PRIVILEGES
# Enable/disable Transport API auditing
enable_transport: true
# Categories to exclude from Transport API auditing
disabled_transport_categories:
- AUTHENTICATED
- GRANTED_PRIVILEGES
# Users to be excluded from auditing. Wildcard patterns are supported. Eg:
# ignore_users: ["test-user", "employee-*"]
ignore_users:
- kibanaserver
# Requests to be excluded from auditing. Wildcard patterns are supported. Eg:
# ignore_requests: ["indices:data/read/*", "SearchRequest"]
ignore_requests: []
# Log individual operations in a bulk request
resolve_bulk_requests: false
# Include the body of the request (if available) for both REST and the transport layer
log_request_body: true
# Logs all indices affected by a request. Resolves aliases and wildcards/date patterns
resolve_indices: true
# Exclude sensitive headers from being included in the logs. Eg: Authorization
exclude_sensitive_headers: true
compliance:
# enable/disable compliance
enabled: true
# Log updates to internal security changes
internal_config: true
# Log external config files for the node
external_config: false
# Log only metadata of the document for read events
read_metadata_only: true
# Map of indexes and fields to monitor for read events. Wildcard patterns are supported for both index names and fields. Eg:
# read_watched_fields: {
# "twitter": ["message"]
# "logs-*": ["id", "attr*"]
# }
read_watched_fields: {}
# List of users to ignore for read events. Wildcard patterns are supported. Eg:
# read_ignore_users: ["test-user", "employee-*"]
read_ignore_users:
- kibanaserver
# Log only metadata of the document for write events
write_metadata_only: true
# Log only diffs for document updates
write_log_diffs: false
# List of indices to watch for write events. Wildcard patterns are supported
# write_watched_indices: ["twitter", "logs-*"]
write_watched_indices: []
# List of users to ignore for write events. Wildcard patterns are supported. Eg:
# write_ignore_users: ["test-user", "employee-*"]
write_ignore_users:
- kibanaserver
roles.yml file contains system roles which can not be present as they are now built it. remove them, these are:
manage_snapshots
logstash
kibana_user
own_index
all_access
kibana_server
kibanaserver
readall
if you didn’t have any custom roles defined previously you should only have:
---
_meta:
type: "roles"
config_version: 2
stop the old ES and start the new ES with new plugin installed one by one, make sure to point the path.data in elasticsearch.yml to the old directory to ensure data persistence, if not using shared volumes.
once all nodes are upgrades and connected to cluster, delete the old index .opendistro_security with below command: ./securityadmin.sh -icl -key ".../config/kirk-key.pem" -cert ".../config/kirk.pem" -cacert ".../config/root-ca.pem" -nhnv -h <host> -dci
Enable shard allocation:
PUT _cluster/settings
{
“persistent”: {
“cluster.routing.allocation.enable”: null
}
}
Upload the new config with below command: ./securityadmin.sh -cd "./securityadmin_config" -icl -key ".../config/kirk-key.pem" -cert ".../config/kirk.pem" -cacert ".../config/root-ca.pem" -nhnv -h <host>