Generate tokens as a stream when calling predict

Plugins Machine Learning
1

OS 3.0.0/RHEL 8

Hi all, I was wondering how to call an LLM from within OpenSearch and get the result as a stream of tokens. I appreciate your help.

Thanks

2

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.