Is there a way to stream or receive docs that have been indexed in Opensearch?

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Describe the issue:
We are looking into a way of streaming the indexed documents from OpenSearch that won’t involve continuously pinging the API with updated queries.

I was curious if this github discussion was relevant to this ask. [META] Streaming Indexing API · Issue #9065 · opensearch-project/OpenSearch (github.com)

Configuration:

Relevant Logs or Screenshots:

2

Hi @tbunkley ,

The feature is in development right now (if you are interested, please check [META] Streaming Indexing API · Issue #9065 · opensearch-project/OpenSearch · GitHub out). Some parts are already coming in 2.12.0 but it is not yet exposed. Thank you.

1 Like
3

Thanks @reta for your response. A couple of the references refer to streaming documents into opensearch. I wanted to clarify that my use case was for streaming indexed docs from opensearch?

4

Oh my apologies, initially this would apply to streaming ingestion (both for request and response), for example you would stream docs in and get the stream of ingested docs out. But if you have something like CDC or have a fan out stream in mind, out of OpenSearch, I am not aware of such development or feature requests. Thank you.

1 Like