I patched the OS of our ES cluster today and suddenly users are seeing a blank white screen when trying to access the “Discover” app in Kibana.
Looking at the logs, its because the user can no longer access the indices in question
No index-level perm match for User [name=user.name, backend_roles=[Incident Response Users], requestedTenant=Incident Response] Resolved [aliases=[.kibana_-1461739569_incidentresponse], indices=[], allIndices=[.kibana_-1461739569_incidentresponse_2], types=[*], originalRequested=[.kibana_-1461739569_incidentresponse, .kibana_-1461739569_incidentresponse_2], remoteIndices=[]] [Action [indices:data/read/mget[shard]]] [RolesChecked [Incident Response Users, own_index]]
I am able to create a new tenant and grant access to that and it all works. If I delete the users .kibana indices they get created and that works too…
As an administrator I have access without issue. If I gave access specifically to these indices it works, but that is not how it’s supposed to be.
Any idea what went wrong so I can keep this from happening again?