Versions 2.6.0 (tar distribution)
sudo ./securityadmin_demo.sh
** This tool will be deprecated in the next major release of OpenSearch **
** [DEPRECATION] Security Plugin Tools will be replaced · Issue #1755 · opensearch-project/security · GitHub **
Security Admin v7
Will connect to localhost:9200 … done
Connected as “***********************” <----omitted the actual line
OpenSearch Version: 2.6.0
Contacting opensearch cluster ‘opensearch’ …
Clustername: opensearch-alpha-st1
Clusterstate: RED
Number of nodes: 1
Number of data nodes: 0
.opendistro_security index already exists, so we do not need to create one.
ERR: .opendistro_security index state is RED.
Populate config from /opt/opensearch-2.6.0-alpha/config/opensearch-security/
Will update ‘/config’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/config.yml
FAIL: Configuration for ‘config’ failed because of java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-6 [ACTIVE]
Will update ‘/roles’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/roles.yml
FAIL: Configuration for ‘roles’ failed because of java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-7 [ACTIVE]
Will update ‘/rolesmapping’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/roles_mapping.yml
FAIL: Configuration for ‘rolesmapping’ failed because of java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-8 [ACTIVE]
Will update ‘/internalusers’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/internal_users.yml
FAIL: Configuration for ‘internalusers’ failed because of java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-9 [ACTIVE]
Will update ‘/actiongroups’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/action_groups.yml
FAIL: Configuration for ‘actiongroups’ failed because of java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-10 [ACTIVE]
Will update ‘/tenants’ with /opt/opensearch-2.6.0-alpha/config/opensearch-security/tenants.yml
I am able to run the API calls
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/_cat/nodes
10.5.xxx.34 21 94 3 0.19 0.15 0.18 m cluster_manager * nodem02.example.pvt-alpha
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/_cat/indices
red open .opensearch-observability pPrzHgYNQqGKYyMbL5Z0ig 1 0
red open .opendistro_security lUljvrkZSPa6vffbIOjp1w 1 1
I can view the index mapping
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/.opendistro_security
{“.opendistro_security”:{“aliases”:{},“mappings”:{},“settings”:{“index”:{“number_of_shards”:“1”,“auto_expand_replicas”:“0-all”,“provided_name”:“.opendistro_security”,“creation_date”:“1684907739056”,“number_of_replicas”:“1”,“uuid”:“VCWc_44PSoizk0ghd_UTbg”,“version”:{“created”:“136277827”}}}}}
But when I try to query the index I get the error due to the red state
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/.opendistro_security/_search
{“error”:{“root_cause”:,“type”:“search_phase_execution_exception”,“reason”:“all shards failed”,“phase”:“query”,“grouped”:true,“failed_shards”:},“status”:503}
Can I force assign the shards?
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/_cat/shards
.opendistro_security 0 p UNASSIGNED
.opendistro_security 0 r UNASSIGNED
.opensearch-observability 0 p UNASSIGNED
Ran allocation command
sudo curl -k -GET --key config/opensearch_admin_key.pem --cert config/opensearch_admin_cert.pem https://localhost:9200/_cluster/allocation/explain?pretty
{
“index” : “.opendistro_security”,
“shard” : 0,
“primary” : true,
“current_state” : “unassigned”,
“unassigned_info” : {
“reason” : “CLUSTER_RECOVERED”,
“at” : “2023-05-24T06:24:05.629Z”,
“last_allocation_status” : “no”
},
“can_allocate” : “no”,
“allocate_explanation” : “cannot allocate because allocation is not permitted to any of the nodes”
}
Ran command to disable the auto expand, getting this:
ERR: Unable to reload config due to /{“_nodes”:{“total”:1,“successful”:0,“failed”:1,“failures”:[{“type”:“failed_node_exception”,“reason”:“Failed node [gQLGEJt2QkKlBOLjl-pKtg]”,“node_id”:“gQLGEJt2QkKlBOLjl-pKtg”,“caused_by”:{“type”:“exception”,“reason”:“java.util.concurrent.TimeoutException: Timeout after 10SECONDS while retrieving configuration for INTERNALUSERS, WHITELIST, ACTIONGROUPS, AUDIT, ROLES, TENANTS, CONFIG, ALLOWLIST, ROLESMAPPING, NODESDN”,“caused_by”:{“type”:“timeout_exception”,“reason”:“Timeout after 10SECONDS while retrieving configuration for INTERNALUSERS, WHITELIST, ACTIONGROUPS, AUDIT, ROLES, TENANTS, CONFIG, ALLOWLIST, ROLESMAPPING, NODESDN”}}}]},“cluster_name”:“opensearch-alpha-st1”,“configupdate_response”:{“nodes”:{},“node_size”:0,“has_failures”:true,“failures_size”:1}}
Reload config on all nodes
Auto-expand replicas disabled
Then I tried to set the replica, .opendistro-security index still shows red
OpenSearch Version: 2.6.0
ERR: Unable to reload config due to /{“_nodes”:{“total”:1,“successful”:0,“failed”:1,“failures”:[{“type”:“failed_node_exception”,“reason”:“Failed node [gQLGEJt2QkKlBOLjl-pKtg]”,“node_id”:“gQLGEJt2QkKlBOLjl-pKtg”,“caused_by”:{“type”:“exception”,“reason”:“java.util.concurrent.TimeoutException: Timeout after 10SECONDS while retrieving configuration for INTERNALUSERS, ACTIONGROUPS, ROLES, AUDIT, CONFIG, ROLESMAPPING, NODESDN”,“caused_by”:{“type”:“timeout_exception”,“reason”:“Timeout after 10SECONDS while retrieving configuration for INTERNALUSERS, ACTIONGROUPS, ROLES, AUDIT, CONFIG, ROLESMAPPING, NODESDN”}}}]},“cluster_name”:“opensearch-alpha-st1”,“configupdate_response”:{“nodes”:{},“node_size”:0,“has_failures”:true,“failures_size”:1}}
Reload config on all nodes
Update number of replicas to 1 with result: true
