Going through https://github.com/opensearch-project/security-dashboards-plugin/blob/main/server/index.ts, I figured out that you can disable the plugin in the opensearch_dashboard yaml.
opensearch_security:
enabled: false
Also make sure to disable the security in the server/cluster.