Data Prepper Aggregation & Peer Forwarding RFCs

Some Data Prepper users are looking forward to stageful aggregation of events. They want to perform aggregations such as combining multiple events into one final document. Or use aggregations to remove duplicate events.

I have created an RFC for stateful aggregation. It describes the planned approach and how teams running Data Prepper can use it. I also have a related RFC to support peer-forwarding as a core feature rather than a processor. This is part of the work needed for stateful aggregation. I made it a different RFC since it can support other plugins as well.

Are you interested in either of these features? Please read and comment on them. I expect that work may start in January. So there is time to discuss the approach and design.