Hi all,
is there any detail on how OpenSearch will evolve in relation to the connectors for languages beside Java and in particular .NET ?
Considering the compatibiity matrix of Nest (cfr. https://github.com/elastic/elasticsearch-net/blob/master/readme.md#compatibility-matrix ) I can assume the that the current version of Nest could be compatible also for OpenSearch being it based on 7.10.x… but what about future?
As new features will be added to ES (and others possibily in OpenSearch) would Nest become unusable ?
Is there any plan (or need) to get the ES connectors under the OpenSearch project umbrella ?
2 Likes
Voicing my support and need here:
One of the reasons Elasticsearch got so popular is due to the broad support on many platforms.
Continuing with the current NEST implementation should work for a while, however, it will most likely stop working after some time has passed.
A definitive answer will make us, and many other, much more trusting in contributing to the endeavor
@jkowall, @searchymcsearchface what d’you think about this kind of issue?
I know the API compatibility will be guaranted for some time -I guess for the whole openseach 1.0.x branch- but what about the future? As soon as opensearch and elasticsearch will start to diverge consistently it would more and more probable to the existing connectors to become useless due to lack of support for specific features: the connectors provided by ES (i.e. Nest) could rely on new, specific ES features and at the same time it could lack support for new opensearch-specific features.
I know some focus had been set on the java-api and the python-api but it could be nice to have info about other languages connectors (i’m particularly interested in the .net connector).
The project is to remain as compatible as possible, the challenge is that Elastic likely wants nothing more than to fragment the community so this will likely not be possible. If Elastic breaks the library we will probably fork it and make sure it’s compatible. If not we’ll try to fix the compatibility as you see here: Restore support for Elasticsearch OSS distributions 7.0.x-7.10.2 by dblock · Pull Request #1018 · logstash-plugins/logstash-output-elasticsearch (github.com)
It depends on your use cases for the connector. I live in the Observability world where OpenSearch is used as a backend for searching telemetry. In this world the use of OpenTelemetry to do this logging, tracing, and metrics has vast benefits. We are working on the logging SDKs right now, but you can already use OpenSearch for trace data. If you have other use cases where you want to use OpenSearch as a database you’d have to check with Amazon maintainers who might keep the clients compatible.
OpenSearch uses semantic versioning, so it should be compatible through the 1.x.x, 2.x.x will bring any breaking changes that are needed, see the roadmap.
As far as the clients are concerned, the effort needed to make clients work in the future should be low. At the moment, the AWS team doesn’t have anyone working on the .net client space, but that doesn’t mean it will stay that way forever.
To be honest, my dream would be a single client that works for both ES and OpenSearch not controlled by a single company or project, a real OSS effort. That would be the biggest win for users.