Governance for the Forks

OpenDistro
1

As @carlmead suggested in the guiding principles thread, it makes sense to have a separate thread about governance. I’ve been working in open source for 20+ years, and lately, I’ve been spending a lot of time working on governance, so I thought I would start the thread and chime in with a few ideas.

With governance, you quickly get into a chicken and egg scenario with people wanting to wait until the project has more contributors before setting up any governance, but then who gets to make the decisions about the project in the meantime? Only Amazon? This creates barriers to getting those first few contributors because many of us won’t contribute until we have at least some neutral governance in place, even if it’s fairly minimal.

Several others mentioned putting these forks under a neutral foundation (Linux Foundation or something similar), which I think is absolutely the right step, but this can also take time, and you’ll want to have people from multiple companies involved as you transition into a foundation.

One way to do this is with a bootstrapped steering committee, similar to what Kubernetes had in the early days, with people from a variety of companies. This would allow the project to get started and help people understand how decisions are made, and it allows the SC to make decisions about things like the charter, governance model, and foundation transition. This bootstrapped steering committee can be seeded with a few smart people with experience in similar open source projects from several different companies who will be involved in the fork. It’s also a good idea to set this up with the promise to move to an elected steering committee within 1 year and with a limit on the number of people who can be elected from a single company (again, similar to Kubernetes).

In this type of structure, the SC would typically delegate technical decisions to existing maintainers, so this wouldn’t change anything about the day to day technical decisions involved in contributing to an open source project. With some form of neutral governance, it would make it more likely that the project will be able to get more contributors and begin to grow the maintainer base to include contributors outside of Amazon.

Related docs and resources about governance

8 Likes
2

I would be fine with a steering committee to start things off on the right foot. Having a 1 year term before governance is in place works for me. I am just concerned there is a lot of “closed door” and lack of multi-company involvement in whatever is happening right now with the fork. I agree with you that these technologies ultimately need a home with a foundation to help govern and control them (ASF or CNCF are good options).

I can tell you I would be happy to participate, and we have already been working on fixes and new features here at logz.io and would be able to have maintainers working on the forks.

4 Likes
3

@jkowall I’m also very concerned that the work is all happening behind closed doors within Amazon. I work at VMware, and we’re still deciding whether or not to embrace this fork. It’s my opinion (not speaking on behalf of VMware) that we shouldn’t get involved in this fork unless there is some type of neutral (multi-company) governance in place first, which is why I’m trying to push some governance decisions.

2 Likes
4

Thanks @dawnfoster I’ve heard the same feedback from other large companies who I don’t want to publicly call out. One of which is an open source organization who will likely adopt the fork due to Apache 2.0 requirements, but cannot participate until this is sorted out.

2 Likes
5

I get why Amazon is doing things behind closed doors for now. At least one reason is that doing it alone helps them move faster. So porting things and such is probably all easier and faster if Amazon’s people do it. I am grateful for any work they are putting into that. But I think it needs to be clear - inside and outside of Amazon - that as soon as the groundwork has been done the next step is to get the fork into one of the other orgs. These orgs have clear rules for number of contributors, diversity, community, etc. I think there is no need to overthink this. They have been doing that for 10-20 or more years. What I would watch out for here are any signs of prolonged thinking that the fork is not ready to donate. The “oh we just need to fix this one more thing” sort of stuff. This is why I think the principals in that other thread are not super important. The goal should not be establishing principals “here”, but getting the fork to an org that already has all these sorts of principals in place.

Otis

6 Likes
6

With my experience from the ASF and going through the incubator i think what AWS should do (and it seems it went this way atm):

  1. Bootstrap the technical stuff (Cleanup Codebase, get CI running, get ready to produce a release)
  2. Find a name then open up the source code, from now on everything is public (also decisions, discussions etc, no closed doors)
  3. Now let it flow and let the community grow (contributers, key people, frequent contributers etc)
  4. Establish a temporary SC with a few key people from the community (diverse from multiple companies/individuals). They will govern the project until:
  5. Let a few month pass and if the community is broad and strong enough start the transition
    to the ASF, Linux foundation, etc

Candidates should start with a community and have the potential to grow and renew this community by attracting new users and developers. Explain how the proposal fits this vision.

https://incubator.apache.org/guides/proposal.html
http://incubator.apache.org/cookbook/
https://cwiki.apache.org/confluence/display/INCUBATOR/New+Podling+Proposal

3 Likes
7

Hi Dr. Dawn - long time, no see.

So there are a number of competing constraints that we’re struggling with right now, and it’s causing some frustration.

Right now, 90% of activity is happening inside the AWS firewall, and none of us want it be that way. As I am sure you can imagine, the governance discussion has come up a number of times, and we’ve delayed it because we want that conversation to happen in the open, and there are a few prerequisites that are currently the focus of much of the effort. At the same time we are pushing every optional or future discussion out until the bits that have to be completed behind the firewall are done and this can become an open conversation.

So the priorities of the team have been two fold:

The first priority is getting a clean code base. This has involved ripping out lots of bits that weren’t open source or weren’t clearly licensed. Then of course it all still has to build. There’s also been a good chunk of disabling things like in-product advertising and telemetry bits. My suspicion is that the community as a whole will decide it doesn’t want those in the codebase at all, but again, we’re trying to focus on only making the decisions that aren’t optional (those are largely defined as the bits that require instruction or consultation with attorneys) Fortunately we have really good history in the form of commit logs, so anyone will be able to audit all of the actions that have been taken.

The second priority is a project name. An independent name is a requirement, and frankly most of those conversations are driven by cycles of reviews with trademark attorneys.

Those are the two things that are occupying everyone’s time. I know it’s frustrating because it seems like, aside from recurring status reports, that there is little happening and so many unanswered questions and undecided issues. Couple that with lots of energy and excitement; and well, I understand why there’s frustration. And lots of us want to make decisions about things like release cadence, and qualifications for committer status, and of course governance. The challenge is that virtually everyone on the AWS side is heads down on the two issues above, and that limits their ability to focus on important conversations like governance. I have a strong desire to make sure that those decisions get broad input from the community, including the devs from AWS who are almost solely focused on working the above two priorities.

If I were in your shoes, I’d also be recommending a wait and see attitude as well. I am hoping that we’ll soon have a code drop and new name - and that the wait will be measured in days or worst case, in weeks. I am hoping that folks will be patient with us just a bit longer.

In the interest of the open source ecosystem in general, I hope that people like you hold AWS accountable. If we start veering off, please call us on it. I, personally, want to have the governance conversation and discuss the relative merits of the LF, ASF, SPI, SFC, and $other_orgs_whose_initials_have_slipped_my_mind, and steps we might take in the interim. I explicitly want you (and lots of others) to participate in those conversations to help inform and drive them to decision. It is my firm (personal) belief that for this project to be successful it will need engagement and ownership from a broad community.

9 Likes
8

Hi David - it’s great to ‘see’ you again! I’m always amazed at what a small world we have in the open source community!

Based on the conversation so far, we have some agreement around:

  1. Working in the Open: moving this work outside of the Amazon firewall as soon as possible
  2. Neutral Decision-Making: having people working at multiple companies (not just Amazon) involved in decision-making
  3. Contributing to a Foundation: contributing these projects to a neutral foundation

1. Working in the Open

Forking a project is a huge effort. I know this is hard for all of the reasons @ke4qqq mentions and more, and I appreciate all of the work Amazon is doing on the fork. It sounds like the people within Amazon are as eager to get this out as the rest of us. I’m hoping they can get it into a “good enough” state to open up the GitHub repo soon while not waiting for everything to be perfect.

2. Neutral Decision-Making

I still think a bootstrapped steering committee is probably the easiest way to get overall project decision-making into a neutral group with representatives who work for a variety of different companies while we grow the contributor / maintainer base. Moving overall project decision-making outside of just Amazon will make it more likely to be accepted by a foundation and more likely to get contributions from some of the other big companies. As I mentioned before, I work at VMware, and we’re way more likely to contribute if it’s not controlled solely by Amazon.

3. Contributing to a Foundation

This one could get a bit tricky because there are a lot of foundations to choose from, and not all of them accept hostile forks, which is likely what they would consider this. For example, I’ve heard that the ASF has this policy, but I’m not sure. Also, I’m guessing that most of us have favorite foundations (I have a strong preference for the many Linux Foundation foundations). I’m guessing some of you have equally strong preferences for one of the other foundations, and in an environment where people have these types of strongly held views, community consensus can be difficult / impossible. One purpose of having a bootstrapped steering committee would be to weigh the pros and cons and make a decision about foundation selection.

What did I miss? What do the rest of you think?

6 Likes
9

It’s a small world indeed :slight_smile:

I’d like to chime in and agree loudly with all these points. Thank you so much to Amazon for all the effort already poured into this thing, it’s necessary but it’s not a small matter and I see and appreciate it!

I’m at Aiven and I’d also like to see diverse governance for this fork, and to support/contribute in whatever ways are needed as things evolve.

6 Likes
10

I would strongly encourage the community to consider having this be done under a foundation such as OASIS or the Linux Foundation from the start, as opposed to a bootstrapped committee. These bootstrapped committees can cause political issues (I can reference many public links here if necessary) & make transitioning to proper governance later on harder than it should be.

Organizations such as OASIS have out-of-the-box processes in place that make setting up a new project under proper governance extremely simple. Having experienced this first hand when we founded the Open Cybersecurity Alliance, I would be happy to give any guidance or feedback here, feel free to reach out.

1 Like
11

I agree that these bootstrap committees can be problematic (especially the ones that linger and don’t give up control early enough). I also have plenty of examples of those :slight_smile:

My first choice would be to have this under the Linux Foundation (or similar) on day 1, but I worry that this will further slow the progress of getting the code out so that the rest of us can see it, use it, and contribute.

What I really care about is having some kind of neutral governance from the start. I care less about exactly how we do it.

It would be great to hear more from the Amazon folks about which of these options work best for them given where they are in their process. Ultimately, since Amazon owns this right now, we need for them to agree on how to move this under an open governance model.

3 Likes
12

This has been a great conversation so far, and I want to thank you for all the feedback.

I am sure by now, you have seen the post announcing OpenSearch and the repos that are now open and ready for contributions. Our desire is to develop a community-led working model where those that contribute can influence decision making. The contributions may come in many forms like code, documentation, issues, and more.

Many of you have suggested transferring this project to a foundation. While this is eventually a possible destination, it’s too early to make that decision. The community is just beginning to form and we do not know who will invest and commit, and how. The long-term implications on the project of such a decision are unknown at this time.

As mentioned in the post, AWS will act as the steward of OpenSearch. We plan to use the principles we have posted to opensearch.org to define guidelines for how we make decisions about the project. If you have feedback on the principles, we will continue iterating and refining them. We want the principles to align with what we all can agree we want to see OpenSearch evolve into. In the meantime, let’s get to work on the code so that we can make sure OpenSearch is a great piece of software that innovates quickly.

It’s taken a lot of work to get here and we still have our work cut out for us. The good news is that Elastic commercial code and references have been removed, and we have a new name so that we can all move forward. We hope to earn your trust by listening to you, incorporating your ideas and contributions, and moving fast. We will remain open to all options as we progress from here, so please keep the feedback coming.

1 Like
13

This is a little orange flag for me. IMHO the community will form “tomorrow” if the door is open. And before the end of this summer you could have a a number of regular contributors.
Elasticsearch had contributors and a “community”, too. But it also had one vendor as the ultimate owner. AWS is in the exact same position now, ownership-wise. I think AWS has to express real desire NOT to keep this an AWS project. The above snippet gives me the impression AWS is not willing to let go. :frowning:

For example, you could say:

  • We are planning or hoping to build enough of a community to hand things off by September, 2021. We cannot guarantee it, of course, but this is out goal/plan/hope.
  • We think we can consider community to be big and active enough to hand off the project to a foundation if there are at least N non-AWS contributors who are active. An active contributor is a person who _______. Or you could say you’d be looking for at least N non-AWS contributors from at least M orgs.
  • We started to talk to foundations A, B, and C, to see if they are open to being the home for this project at all.
  • …

But AWS is not saying anything like this, which gives me a bad feeling… :frowning:

Otis

2 Likes
14

Jumping in here, I’m on the AWS Open Source Strategy & Marketing (OSSM) team, and I’ve been helping the OpenSearch team.

Skepticism is always fine, but I think it is really difficult to set quantitative goals like you’re asking before a community starts going. If I say 5 active contributors, someone will say “oh, they are limiting it to 5”. If I say 100, someone will say “they are setting the bar too high so that they never have to give it up”. I think the sky is the limit and OpenSearch wants anyone who wants to participate to join the community.

If you look at the way we named it, we didn’t call it Amazon OpenSearch or OpenSearch from AWS, we intentionally named it in a way that indicates independence. We drafted trademark guidelines to be extremely permissive (without a naked license) and to encourage broad community use. It has its own namespace on GitHub.

Having external contributors is absolutely a goal of OpenSearch. Having some of those contributors take on additional levels of responsibility in the project is a goal of OpenSearch. What I don’t know is who they will be, how many there will be, and when we will get there. I believe it will be sooner rather than later, but my psychic powers only work on days that don’t end in Y.

AWS’s role in OpenSearch is to be a leader in the community, but not to be the only one. We believe diversity is key to a healthy open source community. Obviously, we have a vested interest in OpenSearch, but we’re not considering it something where you have to be at Amazon to help drive.

This goes for governance too. I think this is a decision that needs to be made by a larger audience than just AWS (and revisited as needed).

As far as foundations go, that’s not off the table, but we need to get OpenSearch to a good place first. We’ve structured OpenSearch in the way that we have so that as the community grows, if the community decides that a foundation would be a better home, it can pursue that avenue. AWS is part of that community (and that decision), but we are not the only part.

We know that we have a lot of work to do (and a lot of things to learn) and we’re very glad for feedback like this! If you see us behaving in a way that you think is inappropriate or unfair, call us out.

At the end of the day, the proof is not in words, but in deeds. People who do good work will be offered greater responsibility, and if you don’t think that is happening in a timely or appropriate fashion, speak out. Every maintainer’s path starts with their first contribution. :slight_smile:

5 Likes
15

This sounds reassuring. But who is the community? Can anyone become a committer? How are decisions made? Can there be a community vote? Who in the community will have a binding vote? Have you formed some sort of steering group amoung AWS, Logz.io, RedHat etc? Without clarification of the formal decision making process of the project as it is right now, then there is no way of estimating how a proposal of moving to Foundation XYZ will be welcomed and decided upon further down the road.

2 Likes
16

the list of company names is an interesting thing - it’s obvious that AWS is in the driver seat right now because, well, they’re doing everything here. logz.io has been visible with their blog posts and some posts here in the forum. the news mentioned various companies being involved in some official manner (“kooperiert” = “cooperates”), e.g. this german news post) lists some companies as official partners, even though for some of them i haven’t seen this mentioned in any announcement:

Für den Fork kooperiert AWS unter anderem mit den Open-Source-Spezialisten von Red Hat, mit SAP, dem Finanzdienstleister Capital One sowie mit Logz.io […]

is there a list of companies which are currently officially involved in the fork (not just hanging out here in the forums and posting some questions/comments, as e.g. i’m doing)? i don’t know where the authors of this article got the list from.

17

It’s part of the official aws announcement Introducting OpenSearch.

We are not alone in our commitment to OpenSearch. Organizations as diverse as Red Hat, SAP, Capital One, and Logz.io have joined us in support.

I don’t see anything related on the new opensearch.org, but I there’s a list on the opendistro partners page, which I guess they will be sooner or later replicated/transferred to the OpenSearch site.

18

Sematext provides production support, consulting, and training for Elasticsearch, including Open Distro for ES. We will be doing the same for OpenSearch.

If OpenSearch really proves healthy in terms of both code/APIs/functionality and ownership/governance we will definitely consider using OpenSearch in Sematext Cloud, too.

But I think listing organizations as contributors or steering committee is kind of the wrong thing to do, isn’t it? Wouldn’t it be better to think of individuals? Sure, one can hope that individuals are from a number of different organizations, but shouldn’t we be thinking about individual contributors and not having companies steer the project?

Otis

3 Likes
19

+1 to this. I’m really hoping that we can get to the point where the people driving the project are employed by a diverse group of employers, not just Amazon. I know I talk about companies a lot, but I’m a big fan of governance models where leadership is by individuals, rather than organizations. I’ve seen too many projects where leadership seats are held by companies that rotate people in and out like musical chairs, which is super disruptive to actually getting any work done within those projects.

20

We at logz.io are 100% behind the project in terms of our engineers and I agree there will be individuals such as myself and others here contributing and we hope to be part of the team making decisions and helping run the maintainership of the codebase. I do not know when this will happen, but for now, I want a good stable distribution and one we can adopt.

From our business, we have made a commitment to switch to these distributions in the coming months for our services. I have been in touch with over 80 individuals who are going to be adopting and contributing. I’m sending them this reply today:

Thanks for getting in touch with us at logz.io when we announced that we’ll be working towards a new project to ensure that ElasticSearch and Kibana remain Apache 2.0 licensed open source projects. This initiative has been led by AWS, but many organizations have been working with them leading up to the launch of OpenSearch this week . If you have a look at the GitHub repos you will find the community picking up. Here at logz.io we have put forward several proposals for features and capabilities we will contribute to the new community. I hope the governance and organization for the maintainers come together in the next 3-6 months, for now, AWS is leading the work, but they are open to all collaborations and users. The team has been easy to work with and accepting of our contributions, so the signs are encouraging.

I hope to see you on the discussions on GitHub or you can participate in the community through the newly launched website for OpenSearch

Please contact me with any questions or opinions.

Thank you!

2 Likes