Hi everyone,
I would need some information that I could not find on the official documentation.
Basically I would like to know how Dataprepper solves/mitigates the problems of Event recovery and Backpressure (default buffer?).
Thank you,
Regards
Giacomo
Could you please clarify what you mean by event recovery? Do you mean if Data Prepper went down?
Hi,
yes, for example a crash, or a system restart or a Dataprepper restart.
Thank you.
Since Data Prepper uses in-memory buffer, Data Prepper doesn’t survive crashes/system restart/Data Prepper restart. The data in the buffer is lost.
Data Prepper uses an in memory buffer. In the case of a sink taking a long time to process the request the buffer will fill up. Once the buffer is full, Data Prepper will throw exception to client to exert back pressure.
Does that answer your question? Please let me know! Thanks!
–David