Hi @phipiship1, can you describe the type of anomaly detection models that you like to create and the use cases that you like to support? Details about the data sources, algorithms and other characteristics like offline versus streaming would be helpful.
In terms of the roadmap, we do have plans to add forecasting to the anomaly detection plugin. We also play to continue to build our ML framework to support more models beyond text embedding models. In the future, we would like for you to be able to bring your own anomaly detection model, integrate it through the framework and use it through our various query interfaces (eg. DSL, SQL, PPL).
Furthermore, you also have the option to implement outlier detection and anomaly detection type support using our existing VectorDB capabilities. If you have the ML expertise, you could create an embedding model to, for instance, represent user behavior activity logs, build a k-NN index and perform similarity vector queries. You can detect outliers and anomalies by identifying vectors that are significantly dissimilar.
@phipiship1, I don’t have a tutorial to implement a solution like this, but we are working on building application samples and templates. Your use case is something we should keep on the radar (@wbeckler). I’ll also take note that a tutorial and/or blog post on this use case would be useful.
On that note, we have customers and partners like Graylog who have implement your use case using our existing anomaly detection capabilities. Here’s a presentation by Graylog on your use case (UEBA):