Skip to content

Automate PPA releases #137

@rtibbles

Description

@rtibbles

Overview

The current release process for the Debian package requires manual steps to copy packages across Ubuntu series via the Launchpad UI, and there is no automated path for promoting packages from the kolibri-proposed PPA to the kolibri PPA. This should be replaced with two reusable GitHub Actions workflows — following the same pattern as kolibri-installer-android — that can be consumed by the main Kolibri repository's release process:

  1. A build/pre-release workflow that builds the signed source package, uploads it to the kolibri-proposed PPA, copies it to all supported Ubuntu series, and monitors builds to completion
  2. A release workflow that promotes all packages from kolibri-proposed to the kolibri PPA and publishes a Debian Trixie-compatible PPA on GitHub Pages

Complexity: High

Context

The existing upload_signed_sources.yml workflow handles building, signing, and uploading the source package to the PPA via dput. This workflow will be refactored to include cross-series copying and build monitoring.

The kolibri-server repository provides the reference implementation. Its release.yml workflow and scripts/launchpad_copy.py demonstrate the full Launchpad API interaction lifecycle: checking if a source exists, uploading, waiting for builds, copying across series, promoting between PPAs, and publishing a GitHub Pages APT repository.

Key differences from kolibri-server:

  • Both workflows must be reusable (workflow_call) since they are consumed by the Kolibri repository — kolibri-server's workflow is triggered directly by release events
  • The release workflow includes GitHub Pages PPA publishing (Debian Trixie-compatible) — in kolibri-server this is part of the same monolithic release workflow behind an approval gate, but here it must be a separate reusable workflow since the approval gate lives in the Kolibri repository
  • The source package name is kolibri-source (binary: kolibri) rather than kolibri-server
  • The build process uses tarball input (Kolibri .tar.gz) rather than building from a git checkout with an installed Kolibri
  • The existing build_tools/generate_changelog.py is used for automated changelog generation at build time

This issue should leverage the automated changelog generation in this repository: build_tools/generate_changelog.py to allow builds to be regenerated predictably without committing the updated changelog to the repository.

The Change

Python script: scripts/launchpad_copy.py

A single consolidated Launchpad PPA management tool (following kolibri-server's pattern of a single script with subcommands) with:

  • check-source — Check if a source package version already exists in a PPA (returns 0 if found, 1 if missing). Used to skip redundant uploads.
  • copy-to-series — Copy packages from the source series to all other supported Ubuntu series within the kolibri-proposed PPA. Uses distro_info for dynamic series discovery (including ESM series). Handles "already copied" gracefully.
  • wait-for-published — Poll the Launchpad API until all builds across specified (or all discovered) series reach a terminal state (Successfully built, Failed to build, Chroot problem, Cancelled build, Build for superseded Source). Continues past individual build failures and reports all failures at the end.
  • promote — Copy all published packages for a specific version from kolibri-proposed to kolibri PPA using syncSources() with include_binaries=True. Handles obsolete series gracefully (skip and report, don't fail).

Constants: SOURCE_PACKAGE_NAME = "kolibri-source", PPA_OWNER = "learningequality", PROPOSED_PPA_NAME = "kolibri-proposed", RELEASE_PPA_NAME = "kolibri".

Helper script: scripts/create_lp_creds.py

A one-time credential generation helper (adapted from kolibri-server's scripts/create_lp_creds.py). Runs Launchpad.login_with() to open a browser for OAuth approval, then saves credentials to a file. The file content is stored as the LP_CREDENTIALS GitHub Actions secret.

Build/pre-release workflow (refactor upload_signed_sources.yml)

A reusable workflow (workflow_call + workflow_dispatch) that:

Inputs: tar-file-name or tar-url (Kolibri tarball), ref (checkout ref)
Secrets: GPG_SIGNING_KEY, GPG_PASSPHRASE, LP_CREDENTIALS
Outputs: deb-file-name (artifact name for the built .deb), version (full Debian version string)

Jobs:

  1. build_source_and_upload — Extracts version from tarball, writes LP credentials, checks if source already uploaded via launchpad_copy.py check-source, imports GPG key, builds and signs the source package, uploads to kolibri-proposed via dput. Outputs the Debian version string.
  2. build_deb — Calls the existing build_deb.yml reusable workflow to build a binary .deb (for later GitHub Pages use). Runs in parallel with the upload job.
  3. wait_for_source_published — Waits for builds to complete in the source series via launchpad_copy.py wait-for-published.
  4. copy_to_other_series — Copies to all supported Ubuntu series via launchpad_copy.py copy-to-series.
  5. wait_for_copies_published — Waits for all series builds to reach terminal states via launchpad_copy.py wait-for-published.

Release workflow (new release_ppa.yml)

A reusable workflow (workflow_call + workflow_dispatch) that:

Inputs: version (Debian package version), deb-file-name (artifact from pre-release), debian-repo-signing-key-id (optional, falls back to vars.DEBIAN_REPO_SIGNING_KEY_ID)
Secrets: LP_CREDENTIALS, DEBIAN_REPO_SIGNING_KEY, LE_BOT_APP_ID, LE_BOT_PRIVATE_KEY

Jobs:

  1. promote — Promotes packages from kolibri-proposed to kolibri PPA via launchpad_copy.py promote, then waits for publication via launchpad_copy.py wait-for-published --ppa kolibri.
  2. publish_github_pages_ppa — Downloads the .deb artifact, builds a Debian Trixie-compatible APT repository using reprepro, signs it with DEBIAN_REPO_SIGNING_KEY, and deploys to GitHub Pages using a GitHub App token for cross-repo authentication (since this workflow runs in the Kolibri repository's context). Runs in parallel with promotion.

Launchpad authentication:

  • Credentials are generated locally using scripts/create_lp_creds.py and stored as the LP_CREDENTIALS GitHub Actions secret
  • Workflows write credentials to a temporary file and set LP_CREDENTIALS_FILE for launchpadlib
  • Credentials are cleaned up in always() steps

Out of Scope

  • Triggering series-specific source builds for individual Ubuntu series (the workflow should be structured to allow this in the future, but the initial implementation copies binaries only)
  • Manual approval gating between pre-release and release — this is handled in the main Kolibri repository's release workflow, not in these reusable workflows
  • Changes to build_deb.yml or publish_debian_repo.yml (these existing reusable workflows are consumed as-is)

Acceptance Criteria

  • scripts/launchpad_copy.py implements check-source, copy-to-series, wait-for-published, and promote subcommands
  • Build/pre-release workflow is reusable via workflow_call and can be consumed by the Kolibri repository's release process
  • Release/promotion workflow is reusable via workflow_call and can be consumed by the Kolibri repository's release process
  • Release workflow includes GitHub Pages PPA publishing (Debian Trixie-compatible)
  • Cross-series copying uses the distro_info Python package for dynamic series discovery with no hardcoded series list
  • distro_info filters are configured to support as broad a range of Ubuntu versions as possible, including ESM series
  • Build monitoring polls the Launchpad API until all builds across all series reach a terminal state
  • Individual build failures are reported but do not prevent other series from completing
  • Promotion script copies all published packages from kolibri-proposed to kolibri PPA
  • Obsolete series are handled gracefully during promotion (skipped and reported, not failed)
  • check-source prevents redundant uploads when the source package already exists
  • A Launchpad credential helper script (scripts/create_lp_creds.py) is included
  • The existing upload_signed_sources.yml is refactored into the new build/pre-release workflow

Testing

  • Unit tests cover series discovery logic with distro_info filters (including ESM)
  • Unit tests cover build monitoring polling and terminal state detection
  • Unit tests cover error handling (individual build failures, obsolete series, already-copied packages)
  • Unit tests cover check-source, copy-to-series, wait-for-published, and promote subcommands
  • launchpadlib and distro_info calls are mocked in tests (scripts use conditional imports so tests run without launchpadlib installed)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions