Why does Packetbeat not populate all fields in the Opensearch packetbeat index?

General Feedback
1

Hi,

Packetbeat only populates a small set of fields in the OS packetbeat index.
Data is coming in and a smaller set of fields populate - but lots of fields are missing and never populated although the config. looks ok.

As an example the network.protocol field is missing from the index but the network.transport and network.packets fields gets populated.

My config is: packetbeat —> logstash (without any filters) —> Opensearch 2.2
(The only packetbeat type that gets populated are of type flow.)

Do anyone see something odd/strange in my config. below that needs to be fixed?

setup.dashboards.enabled: false
setup.template.enabled: true
setup.template.settings:
index.number_of_shards: 1
packetbeat.interfaces.device: any
packetbeat.interfaces.buffer_size_mb: 100
packetbeat.protocols:

  • type: dns
    ports: [53]
    include_authorities: true
    include_additionals: true
  • type: http
    ports: [80, 8000, 8080, 9200]
  • type: mysql
    ports: [3306]
  • type: redis
    ports: [6379]
  • type: pgsql
    ports: [5432]
  • type: tls
    ports: [443, 993, 995, 5223, 8443, 8883, 9243]
    packetbeat.flows:
    timeout: 30s
    period: 10s
    processors:
    • add_cloud_metadata:
    • add_kubernetes_metadata:
      host: ${HOSTNAME}
      indexers:
      • ip_port:
        matchers:
      • field_format:
        format: ‘%{[ip]}:%{[port]}’
    • add_tags:
      tags: [“testing_packetbeat”]
      setup.ilm.overwrite: true
      output.logstash:
      hosts: [‘xxxxx.xxxxx.xx:8081’]
      ssl.certificate_authorities: [“/etc/packetbeat/beat.pem”]
      ssl.certificate: “/etc/packetbeat/beat.pem”
      ssl.key: “/etc/packetbeat/beat.key”

Would really appreciate if someone knows how it is possible to resolve this

Best Regards
//Dennis