Switch to self-hosted infrastructure for DEB package distribution. (#14290)

* Update DEB repository configuration to new infrastructure.

* Fix typo.

packaging/installer/methods/packages.md

@@ -39,7 +39,44 @@ For example, for stable release packages for RHEL 9 on 64-bit x86, the full URL
 https://repo.netdata.cloud/repos/stable/el/9/x86\_64/
 
 Our RPM packages and repository metadata are signed using a GPG key with a user name of ‘Netdatabot’. The
-current key fingerprint is `6588FDD7B14721FE7C3115E6F9177B5265F56346`
+current key fingerprint is `6588FDD7B14721FE7C3115E6F9177B5265F56346`. The associated public key can be fetched from
+`https://repo.netdata.cloud/netdatabot.gpg.key`.
+
+If you are explicitly configuring a system to use our repositories, the recommended setup is to download the
+appropriate repository configuration package from https://repo.netdata.cloud/repos/repoconfig and install it
+directly on the target system using the system package manager. This will ensure any packages needed to use the
+repository are also installed, and will help enable a seamless transition if we ever need to change our infrastructure.
+
+## Manual setup of DEB packages.
+
+Netdata’s official DEB repositories are hosted at https://repo.netdata.cloud/repos. We provide four groups of
+repositories at that top level:
+
+- `stable`: Contains packages for stable releases of the Netdata Agent.
+- `edge`: Contains packages for nightly builds of the Netdata Agent.
+- `repoconfig`: Provides packages that set up configuration files for using the other repositories.
+- `devel`: Is used for one-off development builds of the Netdata Agent, and can simply be ignored by users.
+
+Within each top level group of repositories, there are directories for each supported group of distributions:
+
+- `debian`: Is for Debian Linux and binary compatible distros.
+- `ubuntu`: Is for Ubuntu Linux and binary compatible distros.
+
+Under each of these directories is a directory for each supported release, corresponding to the release codename.
+
+These repositories are set up as what Debian calls ‘flat repositories’, and are available via both HTTP and HTTPS.
+
+As a result of this structure, the required APT sources entry for stable packages for Debian 11 (Bullseye) is:
+
+```
+deb http://repo.netdata.cloud/repos/stable/debian/ bullseye/
+```
+
+Note the `/` at the end of the codename, this is required for the repository to be processed correctly.
+
+Our DEB packages and repository metadata are signed using a GPG key with a user name of ‘Netdatabot’. The
+current key fingerprint is `6588FDD7B14721FE7C3115E6F9177B5265F56346`. The associated public key can be fetched from
+`https://repo.netdata.cloud/netdatabot.gpg.key`.
 
 If you are explicitly configuring a system to use our repositories, the recommended setup is to download the
 appropriate repository configuration package from https://repo.netdata.cloud/repos/repoconfig and install it

packaging/repoconfig/Makefile

@@ -4,23 +4,26 @@ all: $(FILES)
 
 netdata.list: netdata.list.in
 	cp netdata.list.in netdata.list
-	set -a && . /etc/os-release && sed -i -e "s/__DISTRO__/$${ID}/" -e "s/__SUITE__/$${VERSION_CODENAME}/" -e "s/__VARIANT__//" netdata.list
+	set -a && . /etc/os-release && sed -i -e "s/__DISTRO__/$${ID}/" -e "s/__SUITE__/$${VERSION_CODENAME}/" -e "s/__VARIANT__/stable/" netdata.list
 
 netdata-edge.list: netdata.list.in
 	cp netdata.list.in netdata-edge.list
-	set -a && . /etc/os-release && sed -i -e "s/__DISTRO__/$${ID}/" -e "s/__SUITE__/$${VERSION_CODENAME}/" -e "s/__VARIANT__/-edge/" netdata-edge.list
+	set -a && . /etc/os-release && sed -i -e "s/__DISTRO__/$${ID}/" -e "s/__SUITE__/$${VERSION_CODENAME}/" -e "s/__VARIANT__/edge/" netdata-edge.list
 
-netdata-archive-keyring.gpg:
+netdata.gpg.key:
-	curl -L https://packagecloud.io/netdata/netdata/gpgkey | gpg --dearmor > netdata-archive-keyring.gpg
+	curl -L https://repo.netdata.cloud/netdatabot.gpg.key > $@
 
-netdata-edge-archive-keyring.gpg:
+netdata-archive-keyring.gpg: netdata.gpg.key
-	curl -L https://packagecloud.io/netdata/netdata-edge/gpgkey | gpg --dearmor > netdata-edge-archive-keyring.gpg
+	gpg --dearmor > $@ < $<
 
-netdata-repoconfig-archive-keyring.gpg:
+netdata-edge-archive-keyring.gpg: netdata.gpg.key
-	curl -L https://packagecloud.io/netdata/netdata-repoconfig/gpgkey | gpg --dearmor > netdata-repoconfig-archive-keyring.gpg
+	gpg --dearmor > $@ < $<
+
+netdata-repoconfig-archive-keyring.gpg: netdata.gpg.key
+	gpg --dearmor > $@ < $<
 
 debian/tmp:
-	mkdir -p debian/tmp
+	mkdir -p $@
 
 install: $(FILES) debian/tmp
 	cp $(FILES) debian/tmp/
@@ -29,3 +32,4 @@ clean:
 	rm -f $(FILES)
 
 .PHONY: clean
+.INTERMEDIATE: netdatabot.gpg.key

packaging/repoconfig/debian/changelog

@@ -1,3 +1,10 @@
+netdata-repo (2-1) unstable; urgency=medium
+
+  * Switched to new package hosting infrastructure
+  * Removed apt-transport-https requirement
+
+ -- Netdata Builder <bot@netdata.cloud>  Wed, 18 Jan 2023 08:30:00 -0500
+
 netdata-repo (1-2) unstable; urgency=medium
 
   * Fixed package file naming for repo layout compliance

packaging/repoconfig/debian/control

@@ -8,12 +8,12 @@ Homepage: https://netdata.cloud
 
 Package: netdata-repo
 Architecture: all
-Depends: apt-transport-https, debian-archive-keyring, gnupg
+Depends: debian-archive-keyring, gnupg
 Conflicts: netdata-repo-edge
 Description: Configuration for the official Netdata Stable package repository.
 
 Package: netdata-repo-edge
 Architecture:all
-Depends: apt-transport-https, debian-archive-keyring, gnupg
+Depends: debian-archive-keyring, gnupg
 Conflicts: netdata-repo
 Description: Configuration for the official Netdata Edge package repository.

packaging/repoconfig/debian/copyright

@@ -4,7 +4,7 @@ Upstream-Contact: Costa Tsaousis <costa@netdata.cloud>
 Source: https://github.com/netdata/netdata
 
 Files: *
-Copyright: 2021 Netdata Inc.
+Copyright: 2021-2023 Netdata Inc.
 License: GPL-3+
  On Debian systems, the complete text of the GNU General Public
  License version 3 can be found in /usr/share/common-licenses/GPL-3.

packaging/repoconfig/netdata.list.in

@@ -1,2 +1,2 @@
-deb https://packagecloud.io/netdata/netdata__VARIANT__/__DISTRO__/ __SUITE__ main
+deb http://repo.netdata.cloud/repos/__VARIANT__/__DISTRO__/ __SUITE__/
-deb https://packagecloud.io/netdata/netdata-repoconfig/__DISTRO__/ __SUITE__ main
+deb http://repo.netdata.cloud/repos/repoconfig/__DISTRO__/ __SUITE__/