As we prepare to make the repos for the new forks of Elasticsearch and Kibana available, we are seeking feedback on the initial principles of the project. Please let us know what you think and if there is an additional principle you’d like to see.
When we are successful, the to-be-named project will be:
Great software. If it doesn’t solve your problems, everything else is moot. It’s going to be software you love to use.
Open source like we mean it. We are invested in this being a successful open source project for the long term. It’s all Apache 2.0. There’s no Contributor License Agreement. Easy.
A level playing field. We will not tweak the software so that it runs better for any vendor (including AWS) at the expense of others. If this happens, call it out and we will fix it as a community.
Used everywhere . Our goal is for as many people as possible to use it in their business, their software, and their projects. Use it however you want. Surprise us!
Made with your input. We will ask for public input on direction, requirements, and implementation for any feature we build.
Open to contributions. Great open source software is built together, with a diverse community of contributors. If you want to get involved at any level - big, small, or huge - we will find a way to make that happen. We don’t know what that looks like yet, and we look forward to figuring it out together.
Respectful, approachable, and friendly. This will be a community where you will be heard, accepted, and valued, whether you are a new or experienced user or contributor.
A place to invent. You will be able to innovate rapidly. This project will have a stable and predictable foundation that is modular, making it easy to extend.
We look forward to getting your thoughts and feedback.
Thank you for sharing this. A lot of that will come from the project eventually getting governed by one of the orgs that have been mentioned here previously, but it’s still good to see this here. Is donating the yet-to-be-named to one of those orgs still very much the plan for not too distant future, say later this year?
I wanted to echo Otis’ comment above. The community expects multiple companies to “steer” this project, meaning this is not Amazon making decisions and others following along. There should be a group of maintainers and a group of folks providing oversight of the project. No single company should control the future path of this project, including Amazon. This would give the community comfort knowing that the history would not repeat itself. Ideally, this is a requirement when being part of a foundation, but can also be in the values for the project which would be essential in giving other organizations comfort to being part of the community.
Those are not reassuring me that this will not be an Amazon-led project, which Amazon could shift as needed depending on the business goals. I understand point 3 helping to alleviate that, but I would be concerned that one or two companies could control the decision making which would be unfortunate.
Same thoughts here.
What sort of timeline have you discussed internally for donation to ASF, CNCF, or some other org? We can all understand you cannot commit to a specific date, but I imagine you can give us a sense. Is it 3 months from now? 6? 12? 2 years? Or maybe you’ve come up with a clear list of things that need to be in place before that can happen? I think seeing that would be a lot more valuable and reassuring than the “for now” guiding principles because I think we are all hoping that this “for now” is very short. Thank you for all your effort so far and for sharing information about progress, etc. I greatly appreciate that!
Totally hear you and understand governance is a top concern and the fear about “history repeating itself” is valid. We want to address these concerns or they could stymie the success and widespread adoption of the project. If we can separate these concerns from the principles, I think it would be useful. The principles stated above were not meant to answer the questions about how this project is governed long-term but to get aligned with the community about how we look at the project and what kind of community we all collectively want this to be. We would love your feedback on those principles and any suggestions.
Governance can take many forms. As we have stated, we intend to make this a community-led project. At scale any community-led project will need a certain amount of governance. Which form it takes should be driven by the community that emerges around the project and what will be optimal for the long-term sustainability and aligned with delivering great software. We are open to what that looks like and would love to hear your specific proposals and wants. However, before we make any decisions, we need to get the project launched and then see who all is contributing to the community and make sure we have the right set of constituents to make such important decisions.
I propose we start a separate thread on what folks want to see and accomplish (and avoid) with governance. We can discuss the steps, and pros and cons collectively and keep this thread focused on the principles above.
+10 for doing this right away and making everything very explicit and transparent. Some of us will, I am sure, repeat what we touched on in other messages, but yes, a thread focused on that makes sense.
@searchymcsearchface - during the community webinar yesterday, you mentioned that the principles discussion had been sidetracked by governance discussions. I think this is because some of those principles are hard to interpret in absence of governance, especially when they use the word “we” in the context of a principle that has implied decisions, since someone or some group needs to make these decisions.
For example:
In this principle, who is “we”? Amazon?
In the absence of neutral governance, this could be interpreted as “Amazon will ask for community input on features that Amazon builds”. This implies that Amazon will ask for input and feedback, but that ultimately Amazon gets to decide which features go into the forks. I suspect that you have good intentions and that this in not in the spirit of what you mean by this principle, but without neutral governance, there are no guarantees that the rest of us will have any real influence over which features go into the forks. Maybe I’m jaded after 20+ years of doing this, but I’ve seen this go badly in so many projects controlled by a single big company (OpenOffice, for example).
Along the lines of the “Open to Contributions” principle, I would love to see the community webinars pivoted into more collaborative community meetings. In the meetings for all of the other open source projects that I participate in, the participants can see and interact with each other, rather than having a single person control the meeting. It’s not a free for all; there is still someone facilitating the meeting and keeping us on track, but there are shared Google docs where we can all add agenda items and collectively take notes from the meetings. The meetings are also recorded so that those of us in awkward time zones or on vacation can watch them after the meeting, rather than having to work late / early. This model works in projects the size of Kubernetes and in much smaller projects as well, so I think it should work relatively easily for Open Distro as well. Let me know if I can help in any way.
100% agree with these comments. The “community” so far is AWS and people who decided to follow their lead. AWS must change the discussion in the community and have shared ownership. Launching this project without clarifying the governance is a problem.
I’ve been pushing for shared docs, open zoom meetings, recording the sessions, and other standard practices the OSS community uses. I have been pushing this and trying to use open docs with the AWS team for the last 2 months. This is not been adopted or accepted at all.
@searchymcsearchface how can we make progress on these issues in an open manner?
Thanks for surfacing this. I do see the principles and governance to be separate things - when I said sidetracked it wasn’t to lessen any discussion of the governance, just two things being perhaps parallel.
On “we”… You wouldn’t believe how much the team agonized on this particular word. The team tried constructing these without the word ‘we’ and it became hard to understand and more lengthy, ultimately ‘we’ was selected as the most clear, concise word although, admittedly, open to more interpretation that perhaps the team intended.
In this case “we” should be interpreted more broadly and applicable to the entire community of contributors (Amazon folks included). I think the most important (but easiest to ignore) line is “When we are successful, the to-be-named project will be” - defining ‘we’ to the project. That’s the intent at least; words are hard.
With regards to the community meetings - I hear you on this and personally like the open meeting format vs the webinar format. (Also, thanks for the email on the subject yesterday) We actually used this format successfully in the community meetings until the fork announcement. The possibility of the interaction is just deeper and organic questions come up that are healthy for the community. Open Zoom does leave us open to Zoom bombing, which was one of the considerations on switching formats, but the risk/reward here may land on the side of just taking the risk of bombing. I’ve resurfaced the issue.
I’m open to a contributory agenda - let me see what we can do for the next meeting. You make great points about recording - and they’re not unique. I’d like to see these points addressed in some way - all I can say is I’m working on it.
I’d also like a more collaborative meeting style. It isn’t possible to know who is there, and I’d recommend that at least the presenters enable their video by default. Yesterday I did wonder if I was the only attendee but I did see one other person post in the chat who wasn’t a presenter. The format of lecture-only, no interaction, doesn’t fit well with what I see expressed here on the boards - the enthusiasm and the sense of being part of something that’s really got some momentum.
I think this is where our opinions differ. I actually see the principles as being a part of the project governance, which is how many of the CNCF projects do this (K8s, for example).
As part of my work within the CNCF Governance WG, we’re planning to start suggesting that CNCF projects include a statement of principles within their governance.md files.
I think the principles and governance go hand in hand in a way that makes thinking about them separately very difficult.
@lornajane That’s good feedback - especially regarding what you see vs what we see. I wasn’t aware that you couldn’t see even the numbers of attendees - for someone running the meeting the UI is exactly the same as the regular zoom format.
@lornajane / @dawnfoster / @jkowall: All this being said, I think there is a lot of valid issues here regarding meeting format. Given this isn’t unique to the forks, I’d like to move the community meeting feedback to a thread in the general category for higher visibility
Hi. I’ll be direct here. It feels like a lot of dancing around the core issue. “We” is not about “We” or some other word, it is about the fact that “we” today really is “Amazon” and AFAICT there is nothing firm about opening up of the fork.
That “Thank you for surfacing this” makes it sound like a new thing has been brought up to the surface. But N people here have been trying to point this out for M weeks. I think they/we would all stop if “we” stopped being just Amazon or if the clear path to that was shared. But since that is not happening people keep expressing their concerns, fears, and hopes.
Please note I am not trying to be harsh or ungrateful. I’m just trying to point out as directly as possible what (I think) is needed. When that happens then words will stop being as hard because it will all be out in the open. I imagine that, despite the desire to give up 100% control of the fork there are N other things Amazon has to think through and go through before that can actually happen…
To me, it is difficult to talk about governance (more explicit rules and practices for separating work, making decisions, and resolving disputes) without having a shared understanding of common beliefs and principles. I believe that a common understanding of the vision is most important, and then being explicitly flexible in the details of how we work toward that vision together.
Or, to put it a different way, making decisions becomes much more easy when there is general agreement on the “rules of thumb” of a shared vision. The exact manner in which decisions are made matter less, since there is a good likelihood that anyone who understands the shared tenets would act similarly.
As an advocate for being intentional about writing down core principles, and using that as a tool in building a community of like-minded people to collaborate together in building (see my closing thoughts in this OSCON 2019 panel), I welcome this. Thank you for pushing for it. For me, these are the things to debate robustly, to the point where you reach a general consensus and satisfaction that they are going to be relatively durable through time, yet not chiseled in stone.
Meanwhile, I think the specific practices that can be used to organize a project, and make decisions, might necessarily need to change over time. During the early phases of a project, it may make sense to have a designated initial project leader. As a project grows, the work put on a single leader may exceed what any one person can do, and it then makes sense to find a way to divide up the work. (These phases may not directly map to the situation at hand, since this software project is already meaningfully large in size, with many people interested in joining in.)
If you are able to look at it that way, @dawnfoster, what decisions about governance (the specific norms and practices used to make decisions and resolve disputes) would you want to be able to generally answer with the assistance of written-down principles?
I do think that your feedback on the use of “we” in the draft (which, for background, I wasn’t involved in drafting or reviewing) is helpful, since it raises the question about who is “we” and who is “other.” I think that partially boils down to the question about roles, responsibilities, authority, role assignment, and so on. I think that it is very important for people of diverse backgrounds and affiliations to be able to assume roles that have defined responsibilities to be stakeholders in the effort. “We” should be each person involved in the overall effort.
So, let me propose my own set of tenets that might could be used when faced with decisions about governance. These are based on things that I’ve observed to work well in practice with other open source community-focused projects in many sizes. I am writing only as an individual here, and I have not passed these around internally for any discussion or feedback, which I think needs to happen here in the open.
Trust is fundamental. Lasting communities are built on trust, which builds slowly over time and is easy to lose. Trust is earned through establishing a shared vision, clearly setting expectations, being transparent, making promises to one another, and then keeping those promises. We mindfully reject proxies and shortcuts that may be substitutions for building trusting relationships between all stakeholders in this effort.
Shared stewardship. Each participant in this effort is a stakeholder, and partial owner. We mindfully establish policies and practices that re-enforce the continued growth of a diverse group of stakeholders in the project, such as adopting the Developer Certificate of Origin rather than a Contributor License Agreement that aggregates rights to a single party that are not available to everyone.
Permissionless Innovation. Through the use of open source licenses, well understood core principles, and shared norms, permissionless innovation conditions are supported by this effort. It is expected that some stakeholders that collaborate in the effort will also compete with one another. Permission is not required to innovate or to compete. All of the necessary permissions to do so are provided ahead of time through open source licenses and open, non-discriminatory policies.
Designed for speed and scale in making decisions. Federation of responsibility and decision-making authority scales better than coordination. Modularity and extensibility in the design of the software, and the community that produces it, reduces the need for coordinated decision making. Through the use of written principles and clear areas of responsibility, rapid high quality decisions are made locally by those who are most familiar with a situation. Only decisions that are difficult or impossible to reverse require more extensive deliberation.
Roles and responsibilities need not be equal. When federating responsibility, different layers of responsibility form with increasing scope. On points of the timeline of a project, some responsibilities may not be easily delegated (for example, trademark enforcement when a company is acting as the steward), or performed in the public (like handling reports of Code of Conduct violations, or addressing embargoed security vulnerabilities). Each stakeholder needs to act as a steward within their area of responsibility, making decisions that are in the best interest of all stakeholders. Disputes can often be resolved cooperatively by engaging with individuals that have a different role in the project.
Growing a community requires welcoming all. Investments made by some stakeholders will be variable through the long-term timeline of this effort. Mindfully creating leadership opportunities for newcomers that show interest in the effort means that historic contribution metrics should not be a primary component of a formula to allocate areas of ownership and responsibility, or to establish a stake as a stakeholder. Stakeholders come in all shapes and sizes, and every newcomer starts small.
We all wear multiple hats. Stakeholders in the project have different interests, and those interests are valuable to consider when making decisions, as it can provide multiple perspective. When participating in discussions, it is helpful to declare if you are trying to represent the interest of your employer or sponsor, stakeholders that have similar interests, or if you are providing an opinion as an individual. When an individual holds a role with responsibility to the community at large, they may find it helpfully to declare that they are wearing their “upstream hat” to avoid misunderstanding about who’s interest they are trying to represent. Some hats come with real or perceived authority, and wearing small hats is recommended.
Forks and Distributions are welcome. While a goal of this project is to produce high quality software that can be directly useful for a wide range of users, it is also a goal to produce one shared component of a diverse supply chain of software used to build products and services. Innovation and building value that is “downstream” of this effort is encouraged, as is “upstream first” collaboration to enable innovation that is generally useful. We adopt open, non-discriminatory policies that enable downstream consumption of the software produced by this effort in a “distribution” model to correctly identify its origin, and to also market products using a common name. We ask forks to rename their effort to avoid confusion, and we avoid introducing artificial impediments to do so. An exit should always be an option for someone that fundamentally disagrees with a decision.
Special hooks are not welcome. Maintaining significant special functionality that does not generally benefit multiple stakeholders in the project causes undue collective burden on the community as a whole. Stewards and stakeholders of the project should not ask for the community to take on that work.
Hopefully these provide some raw material for further discussion. I’m not particularly attached to these, but I think they’re relatively sound.
Thank you for all your effort so far and for sharing information about progress, etc. I greatly appreciate that!