Sadly, unlike an email distro list, this forum prevents comments of less than 20 chars, which makes +1 / -1 comments impossible, so I’ve moved the vote to a GitHub issue.
Foe anyone interested in casting a vote, please head over to the GitHub issue
Im pretty sure you can create a poll right here: in the cog wheel there’s a Build Poll option. I think a new thread may be a good idea to get everyones attention though.
Usually polls don’t also allow comments, but I’m not sure about this forum. Honestly, I think the last time I used a forum for an OSS project was sometime around 2010, so I’m a bit rustly 
1 Like
I agree, github is probably the best place for it. I was just suggesting an alternative to the +1 -1 approach in this forum.
OK - so mulling over this I think we are placing the cart and horse in the wrong position. Especially considering the implicit affect on decision making in the project.
@nknize I appreciate the community orientation, but I don’t think a vote like this makes a ton of sense in that it the community might agree to something that ends up not being what happens or happens on an unpalatable timeline. That’s just… not great.
I’ve been having some conversations on how to enable this type of effort and get consensus with a larger group of stakeholders. Unfortunately, this is going to require coordination with a number of folks and that takes time which I don’t have in the near term.
I would ask that we hold for a little while longer on this before jumping right to a vote. I feel like that would be healthier for the community.
This is where not having any defined governance and not knowing how decisions are made is pretty frustrating for the community. Since @nknize is a maintainer and works at Amazon, it seemed like he was ok with making the decision to call for a vote. However, after your comments, I’m even more confused about how decisions are made in this project.
3 Likes
I admit I am also confused by this. After following/participating in this thread for weeks – I would have liked to think that the conclusions were valid.
3 Likes
So the response from the Amazon team to a request by the community to establish some semblance of structure, of understandable governance, of publicly disclosed and verifiable rules is a statement that says "we, the GodKing of this world, deem you unworthy to have such access, as only we, the GodKing, know when is the right time to allow you peasants to know what it is that you might be allowed to do.
Ok, that may be a bit over the top as far as characterizations of actions go, but then again, subtle suggestions don’t appear to make much impact here…
So, dear absolute leaders, please tell us what your vision is for the open governance of this AWSdistrocommunity. Because for the life of me, all I can see is a company that is categorically refusing to actually take the community seriously. And that’s… disappointing.
Oh, right, I remember the other threads on this topic, we aren’t ready to talk about that topic, either.
Now, the helpful yellow box tells me to be polite and respectful and constructive, and to criticize ideas, not people. So let me be more specific. The IDEA that the AWSdistrocommunity is in fact an open source community is a great one. Yet the visible ACTIONS of the AWS PEOPLE who show up here seem to actively show that the reality is a different one.
4 Likes
Who are the stakeholders you are referring to?
Thanks,
Otis
1 Like
So the word “decision” for me matters here. I come from the Apache community so I have a bias towards the meritocracy. IMHO, in OSS communities the majority voice is important and should be heard and listened to, ergo I believe my “decisions” can and should be overrule-able by the majority. I do have the ability to label issues, merge PRs, etc. (mostly because I have a long history with this and the Lucene codebase) so in that regard I’m a maintainer but I don’t feel I have any more “decision authority” in this community project than anyone else that has been actively participating in these discussions. This is why my reflex was to suggest taking a vote. This forum is a medium to voice that majority preference, and just as in any other free environment I don’t see a harm in taking a vote (maybe “poll” is the better word?) to capture the position of the majority. If the majority sides one direction and a single individual has the power to veto then I think that’s an issue with the community model that should be discussed and addressed. But I don’t think that’s happened yet? So I think this controversy is healthy and speaking up like everyone is doing will (and should) ultimately drive which direction the project goes. Call it out if that isn’t the case, and it should be addressed.
2 Likes
i do not necessarily agree 100% with the proposal of @dawnfoster (i’ll elaborate below), but i would definitely take it over the current situation where it’s basically Amazon calling the shots but also labeling it as a community-driven project. it was our understanding that this will be set up as a community-based project and would most likely end up with some established foundation (apache foundation, CNCF, etc.).
as pointed out by others it would be good to know who is currently calling the shots. @searchymcsearchface: are you the official public voice of Amazon for opensearch right now and thus only your statements are the rule? who takes decisions behind the scenes (somebody has to take decisions and you have written before that you need to check things internally)? when will this transition to a more public place (community meetings, this forum, github issues - whatever format, as long as it’s publicly visible)? and when will others be able to get involved on the same level so that this isn’t a pure amazon thing?
my opinion about the current proposal (i should have contributed my feedback here earlier, sorry):
i do not see a big differentiation between a community member and a contributor: if i post something in the forum, report a bug or request a new feature i’m also contributing. why am i only a contributor if i regularly submit PRs? what if i only submit one? i’ll be in the commit history from then on out, so i am a contributor to the project by all means.
what i’m missing in turn is the more managerial side of things (but maybe that’s also not the intention of this document and will be another - or will be discussed when the discussion comes back to whether opensearch should end up with some foundation): who decides on “big” things:
- decisions such as: should a new plugin be part of the opensearch branding or not
- managing the forums, CI/CD infrastructure, etc. (or deciding that some contributor (e.g. currently: Amazon) runs it for the project)
- administrative tasks like organising community meetings
- probably lots more which i am ignorant of but as a coder-at-heart try to ignore
i will probably abstain from voting either way on such a proposal since i don’t have a strong opinion about it and am more in the “meh, whatever - i just want to get my work done” category (sorry, had too much admin overhead and too little real software development happening lately in my live 
).
2 Likes
I am reading in this thread that there’s a strong desire from the contributors on this thread for us at Amazon to, not only say that this is a community driven project, but to actually have non-Amazonians drive the said project.
I’m right now with @ralph on the “I just want to get my work done”, and am personally focusing on backward compatibility. I do sit inside Amazon, but AFAIK there’s nothing stopping anyone outside of Amazon from being an active contributor, and eventually a maintainer as governance gets established in a thoughtful orderly manner. If there’s something in your way of contributing, please tell me and I will work on knocking that down.
I speculate the sensitivities stem from a thorny history. This same message was heard from Elastic but actions were very different. But you are right, OpenSearch is incubating and now is a good time to influence by actively getting involved with the code base and speaking up on the forums (both symbiotic and controversial) just as is occurring here. The more contributions and influence, the stronger the case to warrant elevated access. It is a bummer @dawnfoster closed the vote/poll thread. It’s okay if folks want to vote 0. Maybe open this as a github issue instead so we can keep it around and tweak it over a bit of time?
@dblock That is the whole idea of this thread. To establish the criteria for what you said.
This thread and the vote issue on github were not about whether or not one could contribute to the project. I am happy to be among those that have contributed, and that are also participating in these forums.
Yet, I don’t let the fact that I too “just want to get my work done” interfere with my awareness that the current state is not ideal and is lacking a clear public plan to evolve. If members of the community want to help with this why would AWS appose it? Handling this Contributor ladder issue is as much a contribution as adding code to the project is. It required time, effort, experience and communicating with members of the community to pull off.
edit: spelling
3 Likes
I agree with everything you said. I also didn’t see any opposition.
Personally, I had lots of problems with the proposal (I commented on the GitHub issue) and I don’t think it’s implementable (e.g. GitHub doesn’t support approvers that aren’t allowed to merge, AFAIK), so this ladder would remain good intentions, which is net worse than no ladder because it would just pretend like we are doing it. It was very premature to vote.
2 Likes
This is definitely implementable. It’s a stripped down version of what Kubernetes uses. They use a bot called Prow to manage with more granular permissions. A lot of the CNCF projects use something very similar.
You can learn more about GitHub permissions here. If we didn’t use something like Prow, we could still give Approvers the “Triage” role, which allows them to submit reviews that indicate approval, apply labels, assign people to issues, and other tasks that would be helpful during the review process before the code gets merged by a maintainer.
1 Like
The size & age of the codebase can make it easy to overlook that OpenSearch is a very young project. It’s valuable to have free-flowing conversation where we get to talk about what we’d all want to see, even if we can’t know in advance all the practical considerations of what’s implementable on what kind of timelines.
Re the proposal itself, I have some thoughts.
- Include users in the community. Users of OpenSearch, even anonymous ones who it takes extra work to hear from, are so important to the project, that we should be extra mindful to do the work to seek out and consider their perspective when making community decisions.
- Start simple, add complexity as needed. There are a lot of tiers here, and a lot of explicit and implicit administrative procedures being defined. Is there a core idea here that matters the most, where we could all focus our efforts?
- For approver/maintainer/admin roles, what is their responsibility to users? If a maintainer adds a feature that users depend on, what is their ongoing responsibility to support it, update it, and ensure a good transition to a new maintainer if they need to stop working on it?
1 Like
Keeping the ball moving because I think this is a healthy dialogue:
- “Include users in the community.” - 100%. I agree this falls under the “Community Member” label; in other words, a general responsibility for all contributors involved to contribute where they can.
- “Is there a core idea here that matters the most, where we could all focus our efforts?” - I’d like to focus efforts where I thought this was intended to focus efforts: how, and where, members of the community are granted elevated github privileges to assist in labeling, reviewing, and merging contributions so we can grow community ownership.
- “What is [a maintainer’s] ongoing responsibility to support [a feature], update it, and ensure a good transition to a new maintainer if they need to stop working on it?” - Usually in an OSS project contributed features are given to the collective group of maintainers to continue to build and grow. There is no expectation that a single maintainer carries the ongoing responsibility to maintain the contributed feature. If this were the case, Doug Cutting would be expected to continue to maintain Lucene. Community maintenance and enhancement is the mechanism to achieve the intent of having a successful community project outgrow the “founders”. This is also why the maintainers have to be mindful of new features that are not part of an experimental sandbox; accepting and promoting experimental features to “first class” brings a collective maintenance responsibility. Again, this is very much an ASF view, so I’m curious if others feel differently for the OpenSearch model.
1 Like
I would also love to have answers to these questions.
I think there is a fundamental disconnect within this community that is causing a lot of frustration and confusion. Quite a few people, including many of the Amazon engineers, have been advocating for something like the Apache Way, which would be great, but it is fundamentally incompatible with a project like this where decisions are being made by unknown “stakeholders” within Amazon. My attempt to move this contributor ladder discussion to an Apache-style vote by the community was shut down, again because of back room discussions with these Amazon “stakeholders”. Also, the existing maintainers were appointed within Amazon, again possibly by these same “stakeholders”, rather than earning this authority.
So to summarize what I have been seeing in this community by putting it into the Apache Way framework:
- Earned Authority: all individuals are given the opportunity to participate, but their influence is based on publicly earned merit – what they contribute to the community. Merit lies with the individual, does not expire, is not influenced by employment status or employer.
- No evidence of this yet, just Amazon appointed maintainers, which have clearly been influenced by employment status and employer.
- Community of Peers: individuals participate at the ASF, not organizations.
- So far, positions and decisions seem to be driven by Amazon management.
- Open Communications: The ASF requires all communications related to code and decision-making to be publicly accessible to ensure asynchronous collaboration… Private decisions on code, policies, or project direction are disallowed; off-list discourse and transactions must be brought on-list.
- There are clearly discussions happening with Amazon “stakeholders” that are not being fully disclosed.
- Consensus Decision Making: Apache Projects are overseen by a self-selected team of active volunteers who are contributing to their respective projects. Projects are auto-governing with a heavy slant towards driving consensus to maintain momentum and productivity. Whilst total consensus is not possible to establish at all times, holding a vote or other coordination may be required to help remove any blocks with binding decisions, such as when declaring a release.
- I have not been able to figure out how decision-making happens in this project, so I’d love to see the answers to @ralph’s questions that I quoted above.
- Responsible Oversight: The ASF governance model is based on trust and delegated oversight. Rather than detailed rules and hierarchical structures, ASF governance is principles-based, with self-governing projects providing reports directly to the Board. Apache Committers help each other by making peer-reviewed commits, employing mandatory security measures, ensuring license compliance, and protecting the Apache brand and community at-large from abuse.
- So far, I haven’t seen much trust or delegated oversight, but I’m not sure it this one applies here, since it’s more about making reports to the ASF and protecting the brand.
- Independence: the ASF is strictly vendor neutral. No organization is able to gain special privileges or control a project’s direction, irrespective of employing Committers to work on Apache projects or sponsorship status.
- Again, Amazon definitely has special privileges here.
- Community Over Code: the maxim “Community Over Code” is frequently reinforced throughout the Apache community, as the ASF asserts that a healthy community is a higher priority than good code. Strong communities can always rectify problems with their code, whereas an unhealthy community will likely struggle to maintain a codebase in a sustainable manner.
- So far, I’ve seen more evidence of struggles than evidence of a healthy community.
My point here is not that Amazon is bad or that the Apache Way is bad, but that the Apache Way is incompatible with the current way that Amazon is involved in this project, and you can’t claim to have the Apache Way while also managing this like an internal Amazon project.
2 Likes
I know that I’m being a complete pain in the ass here, but I also wanted to say that I have been on the other side of this exact position before. I’ve been the person working at $big_company$ where the company wanted to claim that the community was open while also controlling the future of the project from behind closed doors. I’ve been on the receiving end of people like me who were poking at governance and decision making, while getting direction from management that made it impossible to be truly open. I hated it. It was hard. It sucked. I eventually left this company and went to work somewhere else.
The lesson I learned from this is that it’s better to be clear about how the project is being run and who is responsible for making decisions, even if it means that the community is less open. By being honest with the rest of the community, you build trust, and it makes it easier for everyone to work together in a productive way that allows people to focus on developing the project and releasing it, instead of spending endless hours debating things like this.
2 Likes