Long-term Direction

General Feedback
1

Hello,

The announcement is of particular interest to me, as I am the creator of ElasticHQ (http://www.elastichq.org/), likely the most downloaded/installed elasticsearch monitoring/management application. So…

Questions on long-term direction:

  1. Do you plan to maintain contributions under ASLv2?
  2. Do you plan to make all plugins interact with Kibana as its base?

Selfishly, I’d love to see how HQ fits in to this, but if Kibana is the core platform that negates the idea. In fact, I chose to roll my own management/monitoring option free of Kibana hooks, because I didn’t want the viability of my application to be at the mercy of Elastic, the company.

My own .02… Your FAQ seems to imply this is not a fork, yet by definition, it is in fact, a fork. Let’s call it what it is. :slight_smile:

1 Like
2

Hi Roy!

Thanks for stopping by. ElasticHQ looks really interesting.

1.) Yes it is our intent to maintain all contributions and projects that are part of Open Distro for Elasticsearch as Apache 2.0.
2.) It is our current thinking that we would provide management and monitoring functionality as extensions/plugins in Kibana, but that does not have to be exclusive. For example the PefTop feature is a CLI in the initial release. Totally get your concern about the Kibana dependency.

I know the fork question is going to generate a lot of discussion in the coming days. We are building off of the Apache-OSS binaries and then building plugins on-top to make Open Distro for Elasticsearch for the first release so I think saying it isn’t a fork is defensible. In fact, we intend to continue making contributions to the Apache 2.0-licensed Elasticsearch code, back to Elastic.

Happy to setup a call and we can discuss if there is any way we can work together - I will DM you.

Looking forward to it.

Carl

1 Like
3

Hi Carl,

I emailed you.

Fork discussions and other PR chatter has already started on HN: https://news.ycombinator.com/item?id=19359602

My only bit of advice to you folks is to consider the kibana plugin route. Elastic made a move around v2.x that undid the Java plugin API and standardized all tooling to be built on top of kibana. I chose to go my own route at that time and NOT build on top of kibana. I think removing the plugin API was a bad faith move against the plugin developer community, and so decided to roll my own stateful management/monitoring application. Building on top of their kibana base, exposes you long-term to be at the mercy of Elastic’s whims. My .02.

2 Likes
4

I agree that building on top of Kibana’s base is a big risk. Kibana constantly changes the architecture and it’s a pain to stay in sync with latest releases. The other concern is that Kibana keeps pushing new features behind X-Pack plugins. If this distro doesn’t provide the same features for a vanilla Kibana (one without X-Pack plugin code) then it will have a hard time getting adopters. People will have to buy X-Pack anyways to use some of those features which defeats the purpose of this endeavor. I hope that doesn’t happen and really look forward to this development.

Carl mentioned that they intend to continue making contributions to the Apache licensed Elasticsearch code. However, I would be surprised to see Elastic ever excepting code that competes with their X-Pack code. I guess time will tell.

5

These are absolutely fair points and areas we are going to have to monitor diligently. We are committed to keeping Kibana open as well as Elasticsearch and will be making the necessary investments. It is our goal that Open Distro for Elasticsearch will provide choice and a platform for innovation to customers who are committed to running a 100% open source platform.

6

Keep in mind that X-Pack is itself little more than plugin-ins for the OSS version of the Elastic Stack. In fact, it wasn’t that long ago that X-Pack was installed using elasticsearch-plugin install x-pack. Releasing plugins that “compete” with X-Pack is not anything different than what has long been available from SearchGuard and ReadonlyREST (security), ElastAlert (alerting), Sentinl (alerting & reporting), and more.

Elastic made the move to “pre-bundle” X-Pack when they made X-Pack open code. Similarly AWS also “pre-bundles” their plugins with the OSS foundation of Elasticsearch and Kibana. I would not consider this to be a fork. It is a distribution.

Of course there is some risk of Elastic changing their plugin APIs in the future to limit such competitive options. Such a move might force a fork down the road. Hopefully it won’t come to that.

X-Pack is a well integrated offering that includes additional features like Machine Learning (well… anomaly detection for time series data) and Canvas. Perhaps even more important to risk-averse enterprises, X-Pack customers have the ability to pick up the phone and dial for support directly from the source.

There is however another group of users, who can live with support from the community, and need little more than some basic security options to get all they need from the stack. For this group, Open Distro fits the bill. And there is some comfort knowing that it is backed by significant tech powerhouses like AWS and Netflix (not sure if Netflix is a backer or just a user).

Will this hurt Elastic a bit? Sure. Security is the #1 driver of X-Pack sales, forcing even small organizations to bite the bullet and buy a subscription. Afterall, security is like car insurance… none of us really want to spend money on it… but for production environments we have no choice. Open Distro has the potential to cost them a significant portion of this segment of customers, as well as some larger organizations who have enough in-house know-how to live without support.

I have no issue with the fact that Elastic sells value-added features for a premium. If they can’t make money, their stack wouldn’t be where it is today, and a lot of organizations/users would be worse off. However moves like making the Grok Debugger part of X-Pack (especially when the original is still available here) make it clear that there is more behind their X-Pack strategy than offering premium features.

I have to wonder whether Open Distro ever happens if Elastic would have given the OSS release basic login capability, and focused their commercial efforts on major value-adding options. However now that both exist, and arguably appeal to different user segments, I hope they are able to thrive together for the good of the community.

1 Like
7

Well said @robcowart. Here are my thoughts on the this news.

As someones who has been through X-Pack customer support, this Open Distro and what it is trying to provide is a welcome addition to the Elasticsearch community. You may get security from the X-Pack subscription but not great support. I agree that there have been many options out there already like ReadonlyREST people should look into for security.

Here’s my problem I have with X-Pack. Like you mentioned they questionably keep adding features to the X-Pack license, like Spaces and Autocompletion. I can see them start moving other UX improvements there as well after I saw those. I mean what’s to stop them from moving the new Kibana dark theme behind X-Pack?

I don’t think their support from buying an X-Pack subscription is worth it either. Support issues get fixed on Elastic’s schedule if you don’t have the know how to fix it yourself. This could take months for them to fix them. You can help speed the fix along if you know how and send out a pull request. You can apply your fix whenever you want and hopefully if it gets merged you won’t have to maintain the fix. However, when it does get merged and you install the newer version with the fix that version typically also comes with it’s own set of regression issues. Now this happens with almost any other software out there so why is this a problem? When those regression issues start happening behind X-Pack which has a license you can’t make modifications to, even if you have the know how, it’s a big problem. You’ll forever by stuck playing a game of which X-Pack issues can I live without for a couple months. It’s not a fun game to play.

I hope that we’ll see a truly open source platform that Open Distro is trying to provide.

1 Like