Upserting into data stream?

Open Source Elasticsearch and Kibana
1

Recently I realised that one of our Data Stream’d sources had some changes occur after they were ingested.

I was wondering if it is possible to use the upsert method with data streams?

I got stuck immediately using the OpenSearchPy bulk method since doc_as_upsert seems to require using the “update” _op_type - however data streams only accept the “create” _op_type

I know Data Streams are intended for event logs which wouldn’t change / update - but I like the convenience of using data streams to handle index rollovers so it would be great to use them.

If upserting into data streams is not possible then could anyone direct me to a better method that includes both Index rollover AND upserting? would an Index Alias support both upserting and rollovers ?

Thanks!

2

I wonder if instead of upserting you could do a delete and re-create the doc with the updated information. Data streams as I understand are generally for immutable data such as logs that don’t really change.

1 Like
3

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.