I use the Prometheus , Fluent bit , OpenSearch and OpenSearch dashboard. I want the Prometheus indexes and data are get in OpenSearch dashboard use with fluent bit.
this is my fluent bit value.yaml file code :
# Default values for fluent-bit.
kind: DaemonSet
replicaCount: 1
image:
repository: cr.fluentbit.io/fluent/fluent-bit
tag:
digest:
pullPolicy: IfNotPresent
testFramework:
enabled: true
namespace:
image:
repository: busybox
pullPolicy: Always
tag: latest
digest:
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
annotations: {}
name:
rbac:
create: true
nodeAccess: false
eventsAccess: false
podSecurityPolicy:
create: false
annotations: {}
openShift:
enabled: false
securityContextConstraints:
create: true
name: ""
annotations: {}
existingName: ""
podSecurityContext: {}
hostNetwork: false
dnsPolicy: ClusterFirst
dnsConfig: {}
hostAliases: []
securityContext: {}
service:
type: ClusterIP
port: 2020
internalTrafficPolicy:
loadBalancerClass:
loadBalancerSourceRanges: []
labels: {}
annotations: {}
externalIPs: []
serviceMonitor:
enabled: false
additionalEndpoints: []
prometheusRule:
enabled: false
dashboards:
enabled: false
labelKey: grafana_dashboard
labelValue: 1
annotations: {}
namespace: ""
lifecycle: {}
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /api/v1/health
port: http
resources: {}
ingress:
enabled: false
ingressClassName: ""
annotations: {}
hosts: []
extraHosts: []
tls: []
autoscaling:
vpa:
enabled: false
annotations: {}
controlledResources: []
maxAllowed: {}
minAllowed: {}
updatePolicy:
updateMode: Auto
enabled: false
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: 75
customRules: []
behavior: {}
podDisruptionBudget:
enabled: false
annotations: {}
maxUnavailable: "30%"
nodeSelector: {}
tolerations: []
affinity: {}
labels: {}
annotations: {}
podAnnotations: {}
podLabels: {}
minReadySeconds:
terminationGracePeriodSeconds:
priorityClassName: ""
env: []
envWithTpl: []
envFrom: []
extraContainers: []
initContainers: []
logLevel: info
hotReload:
enabled: false
image:
repository: ghcr.io/jimmidyson/configmap-reload
tag: v0.11.1
pullPolicy: IfNotPresent
resources: {}
config:
service: |
[SERVICE]
Daemon Off
Flush 1
Log_Level info
Parsers_File /fluent-bit/etc/parsers.conf
Parsers_File /fluent-bit/etc/conf/custom_parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port {{ .Values.metricsPort }}
Health_Check On
inputs: |
# [INPUT]
# Name tail
# Refresh_Interval 2
# Path /var/log/containers/*.log
# Tag kube.custommvc.*
# Mem_Buf_Limit 50MB
# Read_from_Head true
[INPUT]
Name systemd
Tag host.*
Systemd_Filter _SYSTEMD_UNIT=kubelet.service
Read_From_Tail On
[INPUT]
Name tail
Host prometheus-server.default.svc.cluster.local
Port 80
Tag pr
Path /api/v1/targets
Refresh_Interval 2
Parser access_log_ltsv
filters: |
[FILTER]
Name kubernetes
Match kube.custom.*
Kube_URL https://kubernetes.default:443
tls.verify Off
Merge_Log On
Keep_Log Off
K8S-Logging.Parser On
K8S-Logging.Exclude On
[Filter]
Name throttle
Match kube.custom.*
Rate 5
Window 30
Interval 60
outputs: |
[OUTPUT]
Name opensearch
Match pr
Host opensearch-cluster-master-headless.default.svc.cluster.local
Port 9200
Buffer_Size 15MB
HTTP_User admin
HTTP_Passwd TadhakDev01
Replace_Dots On
Retry_Limit false
Index prome
Suppress_Type_Name on
Include_Tag_Key on
tls on
tls.verify off
Generate_ID on
Type _doc
# [OUTPUT]
# Name es
# Match prometheus
# Host opensearch-cluster-master-headless.default.svc.cluster.local
# Port 9200
# Index Prome
# Logstash_Format On
# Logstash_Prefix Prome
# Logstash_Dateformat %Y.%m.%d
# Retry_Limit False
customParsers: |
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S %z
volumeMounts:
- name: config
mountPath: /fluent-bit/etc/conf
daemonSetVolumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: etcmachineid
hostPath:
path: /etc/machine-id
type: File
daemonSetVolumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: etcmachineid
mountPath: /etc/machine-id
readOnly: true
command:
- /fluent-bit/bin/fluent-bit
args:
- --workdir=/fluent-bit/etc
- --config=/fluent-bit/etc/conf/fluent-bit.conf
extraFiles: {}
i want with this fluent bit value.yaml code get automatically index in the opensearch dashboard.