Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Versions:
Operator: 2.6.1
Opensearch: 2.14.0
K8s: v1.24.17+rke2r1
OS: Ubuntu
Setup is based on Opensearch Operator
Describe the issue:
When there is any activity which Opensearch Operator does, it is not logging into the pod logs. Logging into pod what operator does in the background would help us troubleshoot better when there is an issue.
For example: I manually deleted the secret which was created by Operator i.e. opensearch-transport-http. However i didnt see any logs in the pods which shows this activity. All we could see is the below logs from the time we create operator which is only related to kube-rbac-proxy.
It would be great if we are capturing what operator does in the logs for better troubleshooting experience
Configuration:
k -n opensearch-operator logs opensearch-operator-controller-manager-6f4894fcfd-srs9x
Defaulted container "kube-rbac-proxy" out of: kube-rbac-proxy, operator-controller-manager
Flag --logtostderr has been deprecated, will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components
I1119 07:55:22.024167 1 flags.go:64] FLAG: --add-dir-header="false"
I1119 07:55:22.024377 1 flags.go:64] FLAG: --allow-paths="[]"
I1119 07:55:22.024399 1 flags.go:64] FLAG: --alsologtostderr="false"
I1119 07:55:22.024406 1 flags.go:64] FLAG: --auth-header-fields-enabled="false"
I1119 07:55:22.024416 1 flags.go:64] FLAG: --auth-header-groups-field-name="x-remote-groups"
I1119 07:55:22.024426 1 flags.go:64] FLAG: --auth-header-groups-field-separator="|"
I1119 07:55:22.024436 1 flags.go:64] FLAG: --auth-header-user-field-name="x-remote-user"
I1119 07:55:22.024443 1 flags.go:64] FLAG: --auth-token-audiences="[]"
I1119 07:55:22.024453 1 flags.go:64] FLAG: --client-ca-file=""
I1119 07:55:22.024460 1 flags.go:64] FLAG: --config-file=""
I1119 07:55:22.024467 1 flags.go:64] FLAG: --help="false"
I1119 07:55:22.024473 1 flags.go:64] FLAG: --http2-disable="false"
I1119 07:55:22.024480 1 flags.go:64] FLAG: --http2-max-concurrent-streams="100"
I1119 07:55:22.024490 1 flags.go:64] FLAG: --http2-max-size="262144"
I1119 07:55:22.024498 1 flags.go:64] FLAG: --ignore-paths="[]"
I1119 07:55:22.024510 1 flags.go:64] FLAG: --insecure-listen-address=""
I1119 07:55:22.024516 1 flags.go:64] FLAG: --kubeconfig=""
I1119 07:55:22.024522 1 flags.go:64] FLAG: --log-backtrace-at=":0"
I1119 07:55:22.024538 1 flags.go:64] FLAG: --log-dir=""
I1119 07:55:22.024546 1 flags.go:64] FLAG: --log-file=""
I1119 07:55:22.024552 1 flags.go:64] FLAG: --log-file-max-size="1800"
I1119 07:55:22.024559 1 flags.go:64] FLAG: --log-flush-frequency="5s"
I1119 07:55:22.024566 1 flags.go:64] FLAG: --logtostderr="true"
I1119 07:55:22.024573 1 flags.go:64] FLAG: --oidc-ca-file=""
I1119 07:55:22.024579 1 flags.go:64] FLAG: --oidc-clientID=""
I1119 07:55:22.024585 1 flags.go:64] FLAG: --oidc-groups-claim="groups"
I1119 07:55:22.024593 1 flags.go:64] FLAG: --oidc-groups-prefix=""
I1119 07:55:22.024599 1 flags.go:64] FLAG: --oidc-issuer=""
I1119 07:55:22.024604 1 flags.go:64] FLAG: --oidc-sign-alg="[RS256]"
I1119 07:55:22.024617 1 flags.go:64] FLAG: --oidc-username-claim="email"
I1119 07:55:22.024622 1 flags.go:64] FLAG: --one-output="false"
I1119 07:55:22.024628 1 flags.go:64] FLAG: --proxy-endpoints-port="10443"
I1119 07:55:22.024636 1 flags.go:64] FLAG: --secure-listen-address="0.0.0.0:8443"
I1119 07:55:22.024644 1 flags.go:64] FLAG: --skip-headers="false"
I1119 07:55:22.024650 1 flags.go:64] FLAG: --skip-log-headers="false"
I1119 07:55:22.024656 1 flags.go:64] FLAG: --stderrthreshold="2"
I1119 07:55:22.024662 1 flags.go:64] FLAG: --tls-cert-file=""
I1119 07:55:22.024667 1 flags.go:64] FLAG: --tls-cipher-suites="[]"
I1119 07:55:22.024678 1 flags.go:64] FLAG: --tls-min-version="VersionTLS12"
I1119 07:55:22.024684 1 flags.go:64] FLAG: --tls-private-key-file=""
I1119 07:55:22.024690 1 flags.go:64] FLAG: --tls-reload-interval="1m0s"
I1119 07:55:22.024714 1 flags.go:64] FLAG: --upstream="http://127.0.0.1:8080/"
I1119 07:55:22.024720 1 flags.go:64] FLAG: --upstream-ca-file=""
I1119 07:55:22.024726 1 flags.go:64] FLAG: --upstream-client-cert-file=""
I1119 07:55:22.024732 1 flags.go:64] FLAG: --upstream-client-key-file=""
I1119 07:55:22.024738 1 flags.go:64] FLAG: --upstream-force-h2c="false"
I1119 07:55:22.024745 1 flags.go:64] FLAG: --v="10"
I1119 07:55:22.024751 1 flags.go:64] FLAG: --version="false"
I1119 07:55:22.024762 1 flags.go:64] FLAG: --vmodule=""
W1119 07:55:22.123906 1 kube-rbac-proxy.go:155]
==== Deprecation Warning ======================
Insecure listen address will be removed.
Using --insecure-listen-address won't be possible!
The ability to run kube-rbac-proxy without TLS certificates will be removed.
Not using --tls-cert-file and --tls-private-key-file won't be possible!
For more information, please go to https://github.com/brancz/kube-rbac-proxy/issues/187
===============================================
I1119 07:55:22.123998 1 kube-rbac-proxy.go:284] Valid token audiences:
I1119 07:55:22.124122 1 kube-rbac-proxy.go:378] Generating self signed cert as no cert is provided
I1119 07:55:31.825089 1 kube-rbac-proxy.go:442] Starting TCP socket on 0.0.0.0:8443
I1119 07:55:31.825435 1 kube-rbac-proxy.go:490] Starting TCP socket on 0.0.0.0:10443
I1119 07:55:31.924141 1 kube-rbac-proxy.go:497] Listening securely on 0.0.0.0:10443 for proxy endpoints
I1119 07:55:31.924449 1 kube-rbac-proxy.go:449] Listening securely on 0.0.0.0:8443
Relevant Logs or Screenshots: