I have finally managed to setup SAML. I created new topic that can be used as a reffer for setup.
My roles mapping also works properly.
I assume my biggest issue was not having same ES config on all nodes.
However, you can try to look into it here, I hope it will help you!