There were some interesting questions and request in the Community Meeting today regarding client libraries, YAML for all the APIs, etc. Let’s continue the conversation here.
Paging @aparo and @lornajane - I’d love to know your thoughts further.
Technically speaking, isn’t it just a matter of time? At some point the ES APIs will start diverging and at that point client libs will need to be cloned, forked, and a version compatible with this fork will need to be maintained.
Otis
1 Like
This applies to both Java and Python packages.
1 Like
Looks like there’s an existing API description for an older API standard (OpenAPI specification | Elasticsearch Service Documentation | Elastic) which we could use as a starting point, I can’t immediately tell if this is the basis for the API that’s used, or if it’s generated from code but I’d be happy to dig into this more at some point.
Having an API description lets us generate reference docs from that, but also generate other useful things such as API clients, or Postman collections, or other integrations. Generated clients are as great as they sound - but they’re an excellent starting point that we can then add human-friendly wrappers to!
Will a distinct API evolve from ES and the forks over time? Sure. The speed at which this happens is a question. I suspect the core of this project will move slowly initially, especially when it comes to breaking changes at the API level. (Open source is rarely a rapid game)
Will clients need to be cloned/forked? That’s more complex. I guess the question to pose is how new clients need to come up. Is it a new client lib or is it a fork?
Yup. I believe the one linked is the Cloud service spec, so not quite the same. The API is at elasticsearch/tree/master/rest-api-spec (be careful, the one on the github right now is SSPL, but the to-be-released fork has Apache 2.0 version).
Now, the question brought up in the Community Meeting by @aparo is slightly more complex. The Open Distro plugins don’t have a single API spec file.
We are actually heavily using NEST (Elasticsearch client for c#) quite heavily at our company and from our experience, version compatibility is quite an issue there, which suggests it breaks quite easily.
If the NEST client would be deprecated we’ll actually have to stop upgrading TBH, not to mention documentation will get a bit wonky over time.
Fragile when you upgrading ES or the client library?
When upgrading ES, NEST usually requires an update as well or bugs begin to crop up
That makes sense. For the forks, a client library should probably just consider it to be 7.10.2 (for a while, at least).
My idea is to fork GitHub - elastic/elasticsearch-specification: Elasticsearch full specification that is Apache 2.0 for now.
Add the typescript api of the opendistro code. (Many of type declaration are already available in the OD repos).
Generate the code.
Because for many companies I developed ES code wrapper in Python/Java/Scala and I will also add the Query and Aggregation objects if they are no available.
1 Like
What would you need to for this to happen?
I create a place where put the code GitHub - aparo/opensearch-client-generator: OpenDistro Client code generator to be used with Elasticsearch
I’ll work on it in the spare time.
1 Like
That’s awesome! Let me know if you need anything else - I’m working on seeing if we can get full API specs.
We are too. The NEST clients are absolutely AWESOME but as you say, flaky on upgrade.
I understand a lot of ES installations run on Windows with NEST .net clients and I would love to see full commitment from OpenSearch on keeping these available. It will be a deal-breaker for A LOT of customers if not.
1 Like
Well, we’re using Linux if we’re being fully honest here.
NEST however, applies to any typed language (so any high-performance language) whatsoever