I’ve mostly been watching the dialog and “liking” things that stand for a multi-company representation of contributors, maintainers, and community members. This has been really hard for Amazon and is counter to the culture at the organization. I have a lot of private meetings with Amazon folks and they want to do the right thing they are just unable to let go of control of the project to date. I really hope the behavior changes and things become open and multi-company. I know I am personally and my company is behind a community, and we have dedicated engineers (some posting on this thread) to work on OpenSearch.
I’m going to keep pushing, but this is moving very slowly and I’m not seeing the actions yet to back up the words I am hearing. Once we have a 1.0 of this project if that doesn’t change I believe that we’ll need to think about an alternate path for the project. I do not like to say this out loud, but I can’t sugar coat the potential outcome.
Has anyone seen any sign of progress from AWS’s side on choosing a governance model? I think it would benefit both AWS and the whole community if there was some sort of roadmap laid ouf with the end goal of incubating with a foundation is stated, along with a deadline, even if that deadline is as far away as 6 months.
I have to admit that all I have been able to read between the lines so far is that OpenSearch is going to be just another Vendor-owned open source project with good intentions but no real openness. I wonder how logz.io and the other co-founders feel about this? Will there be yet another fork?
This would be an anti-pattern, in my personal opinion. You’re suggesting that folks at AWS should choose a governance model. I think that governance is a living, evolving thing in cooperative efforts that last long periods of time. Governance should be tweaked by those who are governed (i.e., first the people doing the work, then all stakeholders of varying classes).
Governance should be discussed, debated, and then adopted and put into practice. I think that is happening here on the forum. It should not be discussed in private. Speaking for myself, I would rather it also not be debated on a video conference because it’s not very accessible to me due to other obligations. But, I am only one voice in this, and I’m not in a position of power where I am “deciding” anything. I wouldn’t want to assume any special privilege in being made any accommodation. I’m only here to ask questions and provide advice based on my experience with FOSS, and also my experience as a long-time Amazon employee (I know the culture and our practices quite well).
Using the services that a foundation can provide is mostly orthogonal to establishing governance, unless a foundation is strongly opinionated about how governance should be done (many are not). The decision about when (and if) incubating with a foundation is one that is made through the decision-making process that governance lays out. People sometimes go to the Apache Software Foundation incubator without knowing that graduation requires certain practiced elements of governance in accordance with ASF policy and philosophy. This doesn’t always work out for long term success.
I think that you’re reading things between the lines that are not actually there, and are based on words. What matters is actions. “Walking the walk” as we said at Red Hat.
I do not want to minimize your experiences collaborating with folks who work at Amazon so far. Your feedback is essential to our shared success. But I personally, think that you are generally mischaracterizing Amazon culture and how we operate, at least from my perspective. From the outside, there are glimpses of the foundations of Amazon culture visible, like our Leadership Principles and practices of using tenets for decision-making. An outsider might assume that the Amazon Leadership Principle of “Ownership” means tight control over a project like OpenSearch. That is not what a good owner does.
Ownership
Leaders are owners. They think long term and don’t sacrifice long-term value for short-term results. They act on behalf of the entire company, beyond just their own team. They never say “that’s not my job."
To me, Ownership is about stewardship. It’s caring for the whole, and not focusing on only the part. It’s about instilling an “ownership culture” where you are a part of what’s built. When building FOSS together, we all are Owners, and we all have to act on behalf of all stakeholders, beyond the needs of a single organization. I would tweak the Leadership Principle like this:
Ownership
Leaders are owners. They think long term and don’t sacrifice long-term value for short-term results. They act on behalf of the entire community, beyond just their own company. They never say “that’s not my job."
This is something that sets off alarm bells for me. When it comes to OpenSearch governance discussions (in particular), I believe we should be making them in the open, transparently. When it comes to partnership activities around AWS managed services (like the Amazon Elasticsearch Service), they should be done privately. I think this separation can be confusing, especially given a pending rename of the service to Amazon OpenSearch Service. But it is important to have that separation, as far as I’m concerned. Others may disagree.
In specific terms, what do you want to see changed regarding behavior?
Again, what actions do you want to see?
To me, the most important decisions happen in the day-to-day activities of the people doing the work. Writing code, submitting pull requests, getting PR reviews, planning features, determining release schedules, declaring an Alpha release, Beta release, 1.0. Are you concerned with how any of that is going?
I am not close to the day-to-day effort here, and anyone who is dedicating their time and attention has far more information than I do. What do dedicated developers have to say about how things are going with their collaborative work? Are they getting frustrated that PRs are going unanswered? That development related discussions are not happening via GitHub PRs, issues, and forum posts? I believe that direct, candid feedback provides the best information for addressing any immediate concerns.
Dirk, I agree that it is over the top. And, personally speaking, I am surprised that someone in your position (both personally and professionally) would characterize good-faith discussions about governance, which are subtle, nuanced, and generally drawn out, in this way.
What specific actions are you speaking of? I’m somewhat an outsider here, but I am personally invested in helping to make this overall effort successful. And, to me, “success” includes building an inclusive, invested, multi-stakeholder community around OpenSearch. That’s going to take time, discussion, debate, and ultimately trust.
Sadly trust seems to be in short supply in the world today.
As for what actions I am talking about? Reading through this thread (and several others) I ended up with the impression that trust is indeed in short supply. And that requests to create more transparency generally receive a “not yet, too early” response. I have been around open source projects for a while and believe that “when in doubt, be more open, more inclusive, more transparent” is generally a good rule. “How does someone become a maintainer” is not a surprising question. Nor is “how are decisions made”. Right now it seems the default answers remain “by working for AWS” and “by AWS employees”, respectively. And that’s what I am pointing out.
@dirkhh Same thing for me with being attacked a bit in the response @msw but I can deal with that and move on.
I believe this thread and our discussion GitHub shows proposed governance, but Amazon is not bought into the idea of how this should move forward. Amazon is the owner of this project, they must buy in to a model before we can progress.
Amazon historically hasn’t been a leader in Open Source. I do not need to rehash the truths and the history of this. It’s not constructive or indicative of the future. I know change is happening, or at least I hope this is the case. I want to see OpenSearch be an example of this change. Working with a broader community as a peer and not the dictator of what will and will not happen. Working in the open, embracing community best practices with regards to collaboration to earn the trust you mention.
It’s about including more folks to help with being maintainers and leaders of community discussions in an open manner. Not having everything controlled and managed by Amazon employees only. This includes codebases, community meetings, and the way the community functions.
No ad hominem dig was intended, and I apologize if the feedback caused any discomfort for you. Reading characterizations of actions of AWS PEOPLE (which I currently identify as due to my employment) was uncomfortable for me. But I believe that candid, yet empathetic, feedback is how we grow to be better, and I believe that you’re working in good faith to help establish and grow the OpenSearch community as a long-tenured open source person, not just acting as a VMware executive.
Indeed, you have. And you have lived through community schisms, vendor battles, and everything else. Same as me, and many others that currently work at AWS, but identify as FOSS advocates. We do know what openness looks like, feels like, and really is in practice.
I would like to chime in here as someone at the engineering level.
Collaboration on code and within discussions: this has been, so far, a positive experience I feel like there is a team effort and that being part of this effort is welcomed by the maintainers. Regarding discussions - They seem to be important to us all, and openness is evident in most threads.
Governance-related topics - this is the pain point at the moment. Something odd happens when discussions reach a call to action phase: The whole discussion is either marked as “premature”, or worse – as “invalid” by some members of the community.
I personally don’t believe there are discussions behind closed doors regarding the governance model. From my observations, this thread never required any stamp of approval from AWS. I appreciate the opinions that the vote may be premature - though, we would all appreciate a second argument stating when this thread’s conclusions would be relevant in their opinion.
The fact that these questions are not answered is the core issue here. If anyone thinks the vote is premature or invalid then it would be great to hear an agreeable alternative. If no alternative is given because the organization that controls the project currently needs to decide internally – that would be bad for the openness we are all working to achieve.
I feel like there is a team effort and that being part of this effort is welcomed by the maintainers. Regarding discussions - They seem to be important to us all, and openness is evident in most threads.