Guiding principles for the forks of Elasticsearch and Kibana

I’m a bit of an outsider on this, but it’s been a very interesting discussion to watch. Setting up principles and governance for a new open collaboration generally happens organically over time, but here since the software is well-established and the potential ecosystem is huge, the stakes are higher and clear answers are more urgent. The uniqueness of the situation encourages me to throw my 2 cents into the thoughts bucket.

I agree with @msw that writing down base principles, basically defining the common project that various stakeholders would join, is necessary before writing down governance. The details of the governance of the project evolves over time, while principles (the project itself, in a large sense) ideally should not.

That said, principles can include open governance principles, like for example the idea that every contributor should be on a level playing field from a decision-making perspective. The governance implementation of that (anyone can get elected to the top governance body, one contributor = one vote…) can be defined later. The governance can be bootstrapped by an initial named committee, or an initial “project leader”. It can take time before truly open governance is fully set up. But contributors can trust that it is the direction this is going, since it’s baked in the principles. Here it seems some of the tension is coming from the absence of a clearly stated open governance goal in the principles (“a level playing field” or “shared stewardship” might not be enough).

Now, the project can totally be successful without open governance. The principles that you just listed are great, and are what were cruelly missing from the original Elasticsearch project. The guarantees around building a common base, rather than free labor toward a specific company product, are what ultimately matters… and I think your list captures it very well.

Finally, open governance is not as necessary when there is trust. Building trust takes time. Here one of the issues here is that this is being discussed without the benefit of past experience and building on existing trust.

Note that this is orthogonal to who owns the trademark (the association between the chosen name and the “project” itself in a large sense). The nice thing here is that there is no name yet, so who owns it is not distracting from thinking about what really matters at this stage: defining what your common project is.

2 Likes

Thanks for sharing your thoughts, I agree this is a unique situation. Your last comment… Amazon owns the new name and the associated trademarks, it will be announced when the repos are public.

There is a major concern with trust, and building that will be a challenge especially without governance that prevents a single company from controlling the projects and the decisions which may be more difficult to make.

2 Likes

As @msw and @ttx mentioned, we can include governance as part of some written principles.

Here’s a shot at what I think is important, stated in the form of a principle:

  • Neutral governance and leadership. Project decisions are made by people who work at a diverse set of companies with no single company having a majority say in decisions that impact the project as a whole.

Based on what I’ve seen in this community so far, this is what quite a few of us are asking for. I also know that this may or may not be something that Amazon is willing or able to agree to do. I’ve worked for lots of companies where the company wanted to maintain control of the leadership, decision-making, and governance for the project, and there is nothing wrong with this approach. The important thing is for all of us to be honest about how this will work, and putting this decision off until later only creates frustration, hard feelings, and usually some negative PR (she says with experience having been in this situation).

@searchymcsearchface - My recommendation for Amazon is to be clear with us now about whether you plan to commit to neutral governance for this project or whether you will be keeping leadership, decision-making, and governance within Amazon. This will likely involve tough discussions internally with your execs, but doing this now will make things a lot easier for you and all of the rest of us in the long run.

2 Likes

thanks, @dawnfoster, this sounds exactly like a point which i’d like to see in the list of principles and which i’d like to see clarified sooner than later.

this might be splitting hairs, but i’m just wondering about the wording (note however that i’m not a native english speaker): would it make sense to explicitly state that nobody has a veto power? having a minority vote while still having a veto right still gives you more power (see e.g. United Nations Security Council veto power - Wikipedia - and no, i don’t want to start a political discussion about this here, it’s just the most prominent veto power and it shows how much controversy this can cause).

Hi @ralph, veto power is a bit tricky, and I wouldn’t recommend it in this situation, but it’s good to talk about it. I agree that if one person or company has veto power, then they have more power than the others, which gives them a majority say in the decision even if they aren’t a majority of whatever leadership body is set up, so I would prefer not to give a single company veto power.

It’s also worth mentioning what we did in the Knative community when one company wanted veto power over trademark decisions is that we gave every committee member an equal power to veto decisions.

1 Like

Exactly. @searchymcsearchface is there an ETA for this?

Thanks,
Otis

I agree that’s ideal to form a truly diverse community. But it can be hard to get to if the project is bootstrapped from employees at a single company.

Personally I don’t mind that much a single company ending up with a majority say if it is a reflection of their contribution to the project. The critical thing with open governance is that no company should be special-cased in governance rules (like have reserved seats, keep a veto right), and the governance should focus on individuals. If you regularly elect leadership based on one contributor = one vote, and the result is that the elected individuals happen to be mostly employed by a single company, well that’s still open governance in my book.

So I would settle for:

  • Company-neutral governance and leadership. The project is governed by a representation of the active contributors to the project. No company gets special rights on the project governance, and leaders should wear their project hat rather than their company hat when making decisions affecting the project.

I agree with this position. May I suggest that it should be easier to add a representative that makes the committee more diverse? Likewise, it should be harder, in my opinion, to add someone from an overrepresented group.

1 Like

Let me point to the original blog post:

We’re in this for the long haul, and will work in a way that fosters healthy and sustainable open source practices—including implementing shared project governance with a community of contributors.

I know people up and down the chain at Amazon are committed to this. The real key thing here is a “community of contributors” - contributors means folks, not companies (similar to what @ttx is saying). Underlying ‘shared project governance with a community of contributors’ also is that contribution comes before getting decision making ability.

Companies introduces a whole host of complexity (I have a list!) that is probably not something that can be adequately addressed in a 1-3 sentence principle. I understand how lopsided company influence can be problematic, evident by countless other OSS projects that are single-company, and I don’t think anyone wants this (see principle “A level playing field”).

However, let’s be clear here: when the repos are released, they will be 100% Amazon folks who have contributed past the fork point. Like @ttx mentions, this is bootstrapping. Once there are other folks that are making contributions, I think that’s when the contributors can start having the discussions about how decisions are made, given the problems that are faced by that group. Then I think more specific guidelines about company influence can be crystallized as problems present themselves.

4 Likes

I totally get this (I’ve been working in open source for 20+ years), but most of us folks do indeed work for companies who pay us to participate in open source projects, and as a company, we need to make hard decisions about which open source projects we pay our staff to participate in as part of their job. Sure, anyone can participate in these forks, but you’ll get more people participating who have more time to devote to the project if they are being paid to do it as part of their full time job.

Companies like to know upfront that our employees will be able to participate on a level playing field with no one company in control of the project. Or, if the project will be controlled by one company, we want to know this upfront so that we can make a decision about whether or not to devote the precious time of our employees to this project.

Let me ask another direct question. Do you have plans to donate these forks to a neutral foundation, and if so, when? Or do you plan to keep them owned by Amazon?

3 Likes

it’s a chicken and egg here: if you start as a 100% amazon (while not letting others in), then you want non-Amazon folks to contribute so that you “can start having the discussions about how decisions are made”, and without any governance showing how contributors get chosen, promoted, vetted etc., this will deter non-Amazon people and companies (echoing @dawnforster on companies’ considerations) from contributing. Especially after the community has been burnt with Elastic.

2 Likes

This topic has generated quite a few responses and I wanted to consolidate a change as the project moves to the next phase, as per today’s announcement.

A key clarification is needed to the the meaning of ‘we’ in the principles. As I mentioned earlier in the thread, the original wasn’t precise as to who ‘we’ referred to. The original intent was that ‘we’ would be all those contributing to the project. To rectify the preamble will be changed:

Original
“When we are successful, the to-be-named project will be:”

New
“When we (the contributors) are successful, the OpenSearch project will be:”

With this update, we have posted the principals to the opensearch.org site.

As the project grows, I hope the number of people and diversity of contributors multiplies.
Nothing is etched into stone with these principles. I suspect as folks start developing on and interacting as a community of contributors, these principles will need to evolve and be changed/augmented/adjusted to fit the real-life interactions that development brings. When you have anything additional to add, continue the conversation in this topic or start a new one for a specific principle.

2 Likes

Speaking personally, only for myself.

Yes, I think that open governance is a deeply discussed concern, especially when a community is working from a trust deficit. To me, one case study is Moby, where many in the community had concerns with the BDFL model used by Docker. I was an outsider there, too. But I think that introducing the TSC was a step too late because there was little trust left between some of the participants in Docker.

Yes, I fully agree with this. I think that it is an unforced error to not be extremely mindful about what expectations you are setting, and promises you are making, when inviting others to join in a new effort. If you set expectations that something will happen in the future (like, for example, that Istio assets would be transferred to a foundation, but then reneging on that promise), the damage to trust can be significant and require much more investment to rebuild trust after such an event than was ever needed at the start of a new venture.

These mistakes can spill over into adjacent efforts, like Knative and the trademark committee rules, largely due to the association with one entity and their past behavior.

As Open Source professionals we know that there is a great diversity in day-to-day interpersonal relationships and practices from community to community, and software package to software package. Sometimes corporate interests get in the way from Getting Stuff Done. I think we’ve all seen patches that are submitted to an upstream maintainer slow walked, or rejected outright with no explanation. But, to me, the truth is you can’t really know that employees will have a good experience collaborating and working in the open without trying it. And I think we have also experienced gatekeeping and frustrating decisions in projects that are part of a “neutral foundation” just the same as projects that have a primary corporate steward.

To me, there are no guarantees here. It is far more important to build resiliency as a community, and as stakeholders, that expects and allows for changes over time.

Speaking personally, I have a lot of concerns about the signaling involved in “donating” things to a “neutral foundation.” If I apply my own RFC governance tenets, I worry that sometimes industry consortia are formed because there is mutual distrust between competitors that are collaborating. To me, that is why there is such a focus on “neutral”. And also why the notably industry consortium–the Linux Foundation–has “Built on trust” in the headline of their website.

I think if you aren’t careful, these decisions–ones that are not easy to reverse (see my tenet #4)–may be a proxy for doing the hard work to earn trust among stakeholders (see my tenet #1).

I also think that using the word “donate” is part of the trust signaling, since it is a virtuous thing that good, trustworthy, selfless people do.

For any big decision that is not reversible, I think it is a good practice to have a principled debate on pros and cons, taking into account feedback from multiple stakeholders of varying types. What would the community get from moving assets to a different holding entity? What might the cost be to certain classes of stakeholders? Would we we want to raise a pool of funds through vendor membership levels? What would we do with the funds? What might that do to the community dynamics, speed in decision making, and ability to innovate? Are some practices within foundations becoming the new cathedral? Back when the GNOME Foundation was being established, its founders wrote some poignant words in their charter:

The GNOME foundation must not stifle the interest of outsiders. An ill-conceived foundation could discourage outsider participation directly, by establishing rules which limit the ability of potential contributors to make their mark, or indirectly, by engendering an alienating sense of elitism. The stained glass of the cathedral creates a colorful spectacle for those inside, but from the outside, the building is just a hulking grey edifice, intimidating and impenetrable.

No matter who is acting as a steward for important assets of an Open Source effort, you have to do the hard work to build an inclusive community (if it is a goal to have one—and I think that having one is an essential and valuable feature). I don’t see how committees with membership based on “substantial contribution” that vote on things, with veto powers, helps to build an inclusive community. I think it builds a set of rules so that one competitor feels like they are not going to be wronged by another competitor due to conflicting corporate interests.

Personally, I would rather see a diverse community of stakeholders around OpenSearch (users, developers, software vendors, cloud providers, upstream projects like Apache Lucene, etc.) that is laser focused on specific common interests, rather than a consortium of vendors trying to work out the rules that protect each in their own interests up front before deciding to commit to participate. I think that if Amazon acts as a steward for the trademark, or a non-profit is the steward for the trademark, is a distraction from the hard work that is needed in community building.

I believe that the things that make a community work well are uncovered over time when you work together in the open, working mindfully to address specific concerns and needs of all stakeholders.