# Generate tokens as a stream when calling predict

**URL:** <https://forum.opensearch.org/t/generate-tokens-as-a-stream-when-calling-predict/24616>\
**Category:** Machine Learning\
**Created:** [June 2, 2025, 2:02pm UTC](https://forum.opensearch.org/t/generate-tokens-as-a-stream-when-calling-predict/24616 "2025-06-02T14:02:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asfoorial](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@asfoorial](https://forum.opensearch.org/u/asfoorial)\
**Post date:** [June 2, 2025, 2:02pm UTC](https://forum.opensearch.org/t/generate-tokens-as-a-stream-when-calling-predict/24616/1 "2025-06-02T14:02:54Z")

</div>

OS 3.0.0/RHEL 8

Hi all, I was wondering how to call an LLM from within OpenSearch and get the result as a stream of tokens. I appreciate your help.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [August 1, 2025, 2:03pm UTC](https://forum.opensearch.org/t/generate-tokens-as-a-stream-when-calling-predict/24616/2 "2025-08-01T14:03:13Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.