Why does Packetbeat not populate all fields in the Opensearch packetbeat index?


Packetbeat only populates a small set of fields in the OS packetbeat index.
Data is coming in and a smaller set of fields populate - but lots of fields are missing and never populated although the config. looks ok.

As an example the network.protocol field is missing from the index but the network.transport and network.packets fields gets populated.

My config is: packetbeat —> logstash (without any filters) —> Opensearch 2.2
(The only packetbeat type that gets populated are of type flow.)

Do anyone see something odd/strange in my config. below that needs to be fixed?

setup.dashboards.enabled: false
setup.template.enabled: true
index.number_of_shards: 1
packetbeat.interfaces.device: any
packetbeat.interfaces.buffer_size_mb: 100

  • type: dns
    ports: [53]
    include_authorities: true
    include_additionals: true
  • type: http
    ports: [80, 8000, 8080, 9200]
  • type: mysql
    ports: [3306]
  • type: redis
    ports: [6379]
  • type: pgsql
    ports: [5432]
  • type: tls
    ports: [443, 993, 995, 5223, 8443, 8883, 9243]
    timeout: 30s
    period: 10s
    • add_cloud_metadata:
    • add_kubernetes_metadata:
      host: ${HOSTNAME}
      • ip_port:
      • field_format:
        format: ‘%{[ip]}:%{[port]}’
    • add_tags:
      tags: [“testing_packetbeat”]
      setup.ilm.overwrite: true
      hosts: [‘xxxxx.xxxxx.xx:8081’]
      ssl.certificate_authorities: [“/etc/packetbeat/beat.pem”]
      ssl.certificate: “/etc/packetbeat/beat.pem”
      ssl.key: “/etc/packetbeat/beat.key”

Would really appreciate if someone knows how it is possible to resolve this

Best Regards