Next OpenSearch Distribution Support

kris · December 6, 2022, 7:16pm

hello @dinesh - it is currently targeted for 2.5.0 and is being tracked here:

github.com/opensearch-project/opensearch-build

[Design] OpenSearch/Dashboards DEB Distribution Design Document

opened 12:15PM - 16 Aug 22 UTC

documentation distributions deb

Coming from #28 ------ # OpenSearch DEB Distribution Design Document H…ey everyone, I want to explain all details for the workflow of providing the *DEB packages* for the future releases on OpenSearch/Dashboards. In parts, this topic has already been dealt with in Issue [Design OpenSearch/Dashboards DEB/RPM Distribution Design Document · Issue #1452 · opensearch-project/opensearch-build · GitHub](https://github.com/opensearch-project/opensearch-build/issues/1452). Please feel free to comment and raising questions about it. To be transparent, I am currently working for the company Graylog, who would like to provide a *DEB packages* to their customers as well. ## Introduction As 2022/08, OpenSearch project published *Tarball*, *Docker Images*, *RPM packages* and *YUM packages* as part of the release artifacts. As a result, the community asked in multiple issues for *DEB packages* for OpenSearch. This design issue is based on the issue created in 2022/01 [Design OpenSearch/Dashboards DEB/RPM Distribution Design Document · Issue #1452 · opensearch-project/opensearch-build · GitHub](https://github.com/opensearch-project/opensearch-build/issues/1452). This design issue has already covered parts of the *DEB packages* build. But then mainly the building of the RPM package was described in the issue. ## Requirements ### [Milestone 1] OpenSearch build process for DEB package generation * The current workflow which already generates *Tarball* and *RPM packages* should be modified to also generate *DEB packages*. * The way the current packages are built should also be feasible for building the *DEB packages* by using the same behavior and arguments. * The build process which already exists should be used to create the contents of the *DEB packages*. * Create one *DEB package* for OpenSearch. * Create one *DEB package* for OpenSearch-Dashboards. * Post scripts should register the plugins. * Should include scripts for pre and post stuff. * Should include systemd and sysvinit configuration files. ### [Milestone 2] DEB Package Signing * The provided *DEB packages* should be signed with GPG to allow verify the source. * Should provide a GPG public keyring to verify the source of the packages. * Likewise, a checksum file with SHA512 should be provided. * Maybe `debsign` is an option to use (man page is [debsign(1) — devscripts — Debian bullseye — Debian Manpages](https://manpages.debian.org/bullseye/devscripts/debsign.1.en.html)) to sign the packages. ### [Milestone 3] DEB Package Design * The Debian package should consider the following the naming schema `opensearch_1.0.0_amd64.deb` and `opensearch-dashboards_1.0.0_amd64.deb`. * As for now, we should provide two *DEB packages*, one which contains OpenSearch and another for the Dashboards check [Opensearch 2.1.0 · OpenSearch](https://opensearch.org/downloads.html) * The package should be built for the architectures amd64 and arm64. * The packages should use the `prerm`, `postinst` features to store a script here which will be executed after the package has been unpacked. * With the help of these scripts, in the case when the packages have to be uninstalled, for example, the services should be stopped before removing files from the filesystem. * These scripts should also initialise plugins automatically. * We should be able to adopt the existing scripts which are already used for the RPM package. * Include a description, `LICENSE` and `README` file. * When assembling the *DEB packages*, dependencies should be added to the package definition. And also define to be incompatible with other versions of the same *DEB packages*. * The package should create user/group for the daemon. * The dashboards plugin package should require the opensearch package as an dependency. * The dashboards plugin package should automatically register the plugins in OpenSearch. ### [Milestone 4] DEB Package Publishing * Publish *DEB packages* files for direct downloading * Publish via *DEB repository* * Publish a *DEB repository* which can be created by for example [aptly - Debian repository management tool](https://www.aptly.info/) see also [DebianRepository - Debian Wiki](https://wiki.debian.org/DebianRepository). * The *DEB packages* and the *DEB repository* should be uploaded to the existing artifacts.opensearch.org with the same process which already exists. * Also the generated e.g. `Packages` files for the repository should be signed with PGP, `aptly` are able to sign the files. ### [Milestone 5] Installation The *DEB packages* should work on the current Debian (11.4 Bullseye) and Ubuntu (22.04.1 LTS) versions. * Users should be able to download the *DEB packages* directly from [Opensearch 2.1.0 · OpenSearch](https://opensearch.org/downloads.html) and installing them with: ```shell dpkg -i opensearch_2.0.0_amd64.deb opensearch-dashboards_2.0.0_amd64.deb ``` * Users should be able to add the APT repository and install the *DEB packages* with: ```shell apt-get install opensearch opensearch-dashboards ``` ### [Milestone 6] Validation The build system should apply tests to validate the quality of the delivered product before releasing to public. These tests should cover the latest Debian and Ubuntu distribution. * Validate *DEB packages* file downloading. * Validate *DEB packages* downloading over APT repository. * Validate *DEB packages* signing. * Validate *DEB packages* installing. * Validate *DEB packages* upgrading/downgrading. * Validate *DEB packages* uninstalling. ## Implementation The already existing workflow should be adapted and extended to build the *DEB packages* as well as to deploy them in the existing infrastructure to serve the files. To build the packages, the build system should use Debian Bullseye or Ubuntu 22.04.1 for this. ## Assemble the DEB distribution The existing scripts that already work for RPM packages should be adapted so that they also work for DEB packages. * *DEB packages* build should also based on the tar and plugin installation. * Scripts for pre and post hooks, systemd and sysvinit files should be included. # Usage ## Installation With `dpkg`: ```shell sudo dpkg -i opensearch_<version>_amd64.deb sudo dpkg -i opensearch-dashboards_<version>_amd64.deb ``` With the APT repository: ```shell # download the gpg key wget -qO - https://artifacts.opensearch.org/GPG-KEY-opensearch | sudo gpg --dearmor -o /usr/share/keyrings/opensearch-keyring.gpg # install dependencies sudo apt-get install apt-transport-https # add the opensearch repository echo "echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring.gpg] https://artifacts.opensearch.org/packages/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list" # install opensearch sudo apt-get update && sudo apt-get install opensearch opensearch-dashboards ``` ## Test Once the OpenSearch daemon is installed, the status can be queried from the cluster, checkout the API https://opensearch.org/docs/latest/opensearch/popular-api/ ## Uninstallation With `dpkg` or APT repository: ```shell sudo dpkg --remove opensearch_<version>_amd64.deb sudo dpkg --remove opensearch-dashboards_<version>_amd64.deb ``` ### Usage with systemd to manipulate the daemon state #### Start ```shell sudo systemctl enable --now opensearch sudo systemctl enable --now opensearch-dashboards ``` #### Stop ```shell sudo systemctl disable --now opensearch-dashboards sudo systemctl disable --now opensearch ``` #### Status ```shell sudo systemctl status opensearch sudo systemctl status opensearch-dashboards ``` # Reference links * https://manpages.debian.org/bullseye/devscripts/debsign.1.en.html * https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.en.pdf * [HOWTO: GPG sign and verify deb packages and APT repositories | Packagecloud Blog](https://blog.packagecloud.io/how-to-gpg-sign-and-verify-deb-packages-and-apt-repositories/) * [aptly - Debian repository management tool](https://www.aptly.info/)

Topic		Replies	Views
RPM Distributions delayed again? Build releases , discuss	60	3642	June 13, 2022
Debian Packages OpenSearch discuss	10	2179	January 26, 2023
Debian/Ubuntu installer please? OpenSearch	3	520	December 7, 2022
Support for Debian General Feedback	2	863	March 13, 2019
Query on availability of Opensearch and Opensearch dashboard RPM's OpenSearch discuss	3	724	March 30, 2022

Next OpenSearch Distribution Support

Related topics