Releasing

Current channel observation

The following is a dated 2026-09-06 observation: v0.2.0 is the latest published GitHub release. The trixie and resolute APT Release URLs return signed metadata for 0.2.0-1, main serves the same Trixie package, and legacy serves signed empty indexes. All three AUR entries serve 0.2.0-1. The production COPR serves 0.2.0-1 on Fedora 43, 44, and 45, from a build submitted by hand after the tagged release run failed to produce one. These are availability observations, not changes to the release policy below.

Versioning

Facelock uses Semantic Versioning:

• MAJOR (1.0.0): Breaking changes to config format, database schema, D-Bus interface, or CLI flags
• MINOR (0.2.0): New features, non-breaking config additions
• PATCH (0.1.1): Bug fixes, documentation, dependency updates

The project is pre-1.0. The public contract is:

Rust crate APIs are internal and not part of the versioning contract.

Prerelease identity conversions

Release input is strict SemVer: X.Y.Z or X.Y.Z-{alpha,beta,rc}.N. The same identity is converted explicitly for each package manager:

The first alpha Debian revisions are 0.2.0~alpha.1-1~deb13u1 (trixie) and 0.2.0~alpha.1-1~ubuntu26.04.1 (resolute).

Package rebuilds advance independently of the Cargo version. Debian and Arch increment their package revision for a rebuild of the same prerelease and reset it for the next semantic prerelease. RPM uses one monotonic prerelease counter across the whole series:

Debian: 0.1.4-1 < 0.2.0~alpha.1-1 < 0.2.0~alpha.1-2 < 0.2.0~alpha.2-1 < 0.2.0~beta.1-1 < 0.2.0~rc.1-1 < 0.2.0-1
RPM:    0.1.4-1 < 0.2.0-0.1.alpha.1 < 0.2.0-0.2.alpha.1 < 0.2.0-0.3.alpha.2 < 0.2.0-0.4.beta.1 < 0.2.0-0.5.rc.1 < 0.2.0-1
Arch:   0.1.4-1 < 0.2.0alpha1-1 < 0.2.0alpha1-2 < 0.2.0alpha2-1 < 0.2.0beta1-1 < 0.2.0rc1-1 < 0.2.0-1

scripts/release-versions.sh is the executable conversion contract. Repeating the same prerelease is a package rebuild; repeating a stable version is rejected because Debian/Arch would otherwise advance while RPM remains at release 1. Semantic version regressions are rejected before any file is edited. just test-release-matrix verifies the exact order with native dpkg --compare-versions, rpmdev-vercmp, and vercmp in disposable, digest-pinned containers.

How to Release

Automated (recommended)

just release 0.2.0
# or
just release 0.2.0-alpha.1

This will:

1. Convert and bump Cargo, Arch tag/pkgver/pkgrel, RPM Version/Release, and Debian upstream/revision metadata
2. Run cargo check --workspace to verify the version bump compiles
3. Preserve package rebuild ordering, including the monotonic RPM prerelease counter
4. Prompt you to update CHANGELOG.md (add entries under the new version heading)
5. Print the git commit / git tag / git push commands for you to run

Then push the tag to trigger the release workflow:

git push origin main --tags

What happens on tag push

The .github/workflows/release.yml workflow:

1. Validates the tag against the checked-in release identity and target matrix
2. Builds release binaries and uploads them as workflow artifacts
3. Prepares the pinned ONNX Runtime and lock-bound Cargo-vendor source components with their reviewed manifests and checksums
4. Builds two suite-specific TPM-enabled .deb packages for trixie and resolute
5. Builds the direct .rpm package in the pinned Fedora 44 container and validates contents
6. Validates the unlocked Nix flake evaluation (network inputs are resolved because dist/nix has no checked-in flake.lock)
7. For a stable tag, builds the signed codenamed APT repository plus the main and legacy compatibility suites until 0.3.0; missing APT signing secrets fail here, before the GitHub Release is public
8. Verifies the tag, assembles and validates every asset, writes MANIFEST.json, and publishes the release exactly once
9. After a stable GitHub Release is public, attempts to publish facelock, facelock-bin, and facelock-git to AUR; a missing AUR_SSH_KEY prints a skip notice and exits successfully, while an invalid configured key fails after publication
10. For a stable tag, triggers the GitHub Pages rebuild that deploys the updated APT repo

Validated prerelease tags set the GitHub Release prerelease output and upload direct artifacts, but skip stable APT and all AUR publication. The workflow guards use the validated release identity rather than substring matching.

COPR (Fedora) is not built by release.yml. It is handled by Packit, which reacts to the release the publish job makes public in step 8. A draft raises no release event, so nothing downstream fires until validation passes. The workflow does watch: a stable run's verify-copr job polls the public COPR API for the released EVR and fails the run if it never appears. It builds nothing and can undo nothing — the release is already public by then — it only makes a failed Packit submission loud on the day it happens. See the COPR section below.

Builders build, publish publishes

No builder writes to the release. Each one uploads a workflow artifact and a digest attestation naming what it produced, the image it produced it in, and the components it consumed. Until the publish job runs, the tag has no release at all: nothing is public and no downstream automation has seen anything.

That split is the point. Every builder compiles or packages code this project does not own, from every dependency's build.rs to rpmbuild and dpkg-buildpackage. A builder holding the publication credential is a builder that can publish whatever it likes. publish compiles nothing, so it is the only job that holds RELEASE_PAT and the only one with contents: write; every other job holds contents: read, and the workflow's own default is deny-all.

publish runs after every builder and validator, and it:

• verifies the tag exists, names the validated version, and points at the built commit; where the tag carries a signature it must verify. The job reads the tag and never creates, moves, or replaces one.
• stages exactly the canonical assets out of the builders' artifacts. The allowlist is derived from the validated version, Debian revision, and RPM counter, so an artifact built from another identity has no canonical name and a file a builder added beside the one it was asked to produce is never staged.
• holds every staged asset to the SHA-256 its builder attested. An asset that changed between its build and publication stops the release, as does one no builder attested or one two builders claim.
• holds each attestation to the provenance its slot may declare: the suite, the image dist/release-matrix.json pins, and the component names. A builder cannot report another image or an extra component into MANIFEST.json; a matrix the job cannot read stops the release instead of shortening the allowlist.
• trusts an attestation only once it hashes to the job output its builder recorded. Artifacts are shared, writable storage for every job in the run; a job output belongs to the job that wrote it. An attestation that was replaced after its job finished, or whose job recorded no output, is refused by name.
• creates the release as a draft carrying those assets, then writes MANIFEST.json over them, plus the source tarball digest, the pinned build-image digests, and the reviewed ONNX Runtime and Cargo-vendor component digests. It replaces the three-binary SHA256SUMS file, which covered a fraction of the release and was written before most of it existed.
• reads the draft back from the API, holds it to the allowlist a last time, holds each published asset's size, and digest where the API exposes one, to MANIFEST.json and the uploaded manifest to the file it wrote, and flips the draft to published once. A tag whose release is already published is refused before anything is written, so re-running the workflow after a failure is safe; re-running it after success goes red by design, at verify-creatable, with nothing written. One case needs a hand: if the Debian revision or RPM counter changed between runs, the draft still carries the asset built under the old name, and the readback refuses it. The failure names the file and the command that removes it, gh release delete-asset.

The workflow runs once at a time per tag (concurrency keyed by the ref, never cancelling the run in progress), so a re-run started while a run is inside publish queues behind it instead of racing it. Two drafts for one tag, which only a race could leave behind, are refused with the gh api --method DELETE command that removes the extra one.

A builder's extra output fails the release closed: a canonically named file in an artifact the allowlist does not expect it from is refused at staging, and the failure says so. Re-running only the failed publish job keeps that artifact, so the remedy is fixing the builder and re-running all jobs. A partial re-run of a single build-deb leg can similarly leave the other suite's attestation unbound (attestation deb-<suite> is not bound to a job output); the remedy is the same, re-run all jobs.

Two consequences for the maintainer:

• RELEASE_PAT is required. It is now the only credential that can write the release, so an unset secret fails the publish job. just release-preflight checks for it, and fails when it cannot check: without gh, or unauthenticated, the check is reported as unchecked and preflight does not pass.
• A signed tag must be verifiable on the runner. Importing the maintainer's public key is release infrastructure tracked by #235; until it lands, an unsigned tag is accepted and a signed one that the runner cannot verify stops the release.

The publication prerequisites include build-nix: its flake evaluation must pass, while its nix build step is advisory. No Nix lockfile is checked in, so evaluation and builds depend on the currently resolved flake inputs. An evaluation failure blocks the publish job before it creates a draft in that run; fix the flake and re-run the failed jobs, never tag again.

test/release-artifacts-contract.sh (just test-release-artifacts) proves this shape by fixture and by mutation. The workflow itself runs only on a tag, so the gate never tags anything to test it.

Debian package channels

Debian-family release support is exactly Debian 13 (Trixie) and Ubuntu 26.04 LTS (Resolute). Both codenamed suites ship one binary package named facelock with TPM support enabled. No rustup toolchain participates in Debian source builds. Bookworm and Noble artifacts may remain in historical releases, but those suites are unsupported and receive no new packages. Trixie package builds use the official Trixie Backports cargo and rustc; Resolute package builds use the native Ubuntu toolchain.

Both .deb packages are uploaded to the GitHub Release for direct download. Stable packages are published under the matching codename at https://tysmith.me/facelock/apt/.

Each Debian source package consists of the exact tagged main upstream tarball, the reviewed ORT component, the deterministic Cargo-vendor component, and the Debian quilt delta. Complete .dsc rebuilds run with network denied and empty Cargo/Rustup caches. Stable APT publication consumes exactly two suite manifests, one for Trixie and one for Resolute, before signing or writing the repository.

Each suite manifest contains exactly eight artifacts in this order: the main orig tarball, ORT orig component, Cargo-vendor orig component, Debian quilt delta, .dsc, .buildinfo, .deb, and .changes. The Cargo component carries a generated legal inventory covering every exact lock-bound crate; its specific DEP-5 stanza precedes the Facelock source catch-all. CI prepares that component with Rust 1.95.0 through the immutable dtolnay/rust-toolchain action commit 4360b52568e2003a75bf9bc1d59f33a8e3fc893c, matching the repository's pinned 1.95 toolchain channel.

Every built .deb passes .github/workflows/scripts/validate-deb.sh in the suite container before staging: package identity, forbidden transition fields, generated dependencies, the required file set, the hash-verified ORT bundle, and a lintian run that fails on error-severity tags. Deliberate deviations are suppressed in that script, each with a recorded reason; warnings are printed for review but do not gate.

Supported release matrix

dist/release-matrix.json is the checked-in authority. The release workflow, APT configuration, Packit targets, and this table are checked against it.

Production COPR requires Fedora 43, Fedora 44, and Fedora 45. Rawhide is the only optional allowed experimental production chroot, so it may be present or absent; missing any required chroot or enabling any unknown extra fails closed. Every Packit copr_build target must be an explicit member of the checked-in allowlist: fedora-43-x86_64, fedora-44-x86_64, or fedora-45-x86_64. Mutable aliases such as fedora-all, fedora-development, and their architecture-suffixed forms are rejected, as is any other undeclared target. Rawhide is not a release target and is not a Packit staging or production release target. Both fedora-rawhide and fedora-rawhide-x86_64 fail validation, and no alpha may publish to Rawhide.

Fedora 43 and Fedora 44 carry the full lifecycle. Fedora 45 carries required build/runtime smoke. Rawhide remains best-effort pinned Track D smoke only; a Rawhide-only failure is not alpha-blocking, and Rawhide cannot supply lifecycle, artifact, upgrade, rollback, served-version, or availability evidence. Promotion requires a separately reviewed amendment and full Fedora gates. Issue #236 owns the pre-tag and post-publication proof that optional Rawhide serves no alpha or candidate build.

Container identities are pinned by registry/index digest, with the linux/amd64 manifest digest retained where the registry exposes both. They were resolved from Docker Hub registry metadata and Fedora registry Docker-Content-Digest on 2026-08-18. The Arch repository identity is pinned separately to https://archive.archlinux.org/repos/2026/08/18/; every matrix-associated CI and AUR pacman invocation installs that exact mirror before refreshing package metadata.

Local distro validation

Before releasing, validate packages build and install correctly on each target:

# Automated (no camera needed)
just test-arch-pam       # Arch container PAM smoke tests
just test-rpm            # Fedora — validate file layout from manual install
just test-deb            # delegate to both exact supported-suite package gates
just test-deb-trixie-pkg    # Debian 13 — offline source rebuild, install, TPM, lifecycle
just test-deb-resolute-pkg  # Ubuntu 26.04 — the same complete package gate
just test-rpm-pkg        # Fedora — build real .rpm, install via dnf, validate
just test-rpm-lanes      # every declared Fedora target at its declared depth
just test-rpm-authselect # Fedora — retired-profile upgrade guard lifecycle
just test-packit-config  # Packit config schema — real `packit` in a pinned Fedora container
just test-copr           # COPR-equivalent build only — Packit SRPM + mock from-source rebuild (slow)
just test-copr-pkg 43    # the same rebuild, then install it and run the booted lifecycle
just test-copr-lanes     # every Packit/COPR target rebuilt from source at its declared depth

# Interactive (requires camera)
just test-deb-dev-shell      # Ubuntu .deb with host models — fast iteration
just test-rpm-dev-shell      # Fedora .rpm with host models — fast iteration
just test-deb-release-shell  # Ubuntu .deb clean room — real user experience
just test-rpm-release-shell  # Fedora .rpm clean room — real user experience

The test-rpm recipe validates file layout from manually installed binaries. test-deb delegates to both supported-suite *-pkg recipes. The *-pkg recipes build real packages using the same scripts as CI, install them with the actual package manager (dnf / dpkg), and validate the result — testing postinst scripts, dependency resolution, ORT bundling, tmpfiles triggers, and the full install path.

The *-dev-shell recipes mount host models for fast interactive camera testing. The *-release-shell recipes start from a clean package install with nothing from the host — run facelock setup to download models, then enroll and test.

Fedora lanes

Every Fedora recipe takes a release — just test-rpm-pkg 43, just test-copr 45 — and defaults to 44. just test-rpm-lanes runs each declared release target at the lifecycle depth dist/release-matrix.json gives it: full lifecycle for Fedora 43 and 44, build plus runtime smoke for branched Fedora 45. Rawhide is optional and experimental, has no lane, and can never stand in for a Fedora 43, 44, or 45 result.

Each Fedora target needs two lanes, not one. test-rpm-lanes proves the direct .rpm: host-built binaries, bundled ONNX Runtime. That is not the delivery path the matrix declares for Fedora, which is Packit publishing to COPR against Fedora's system ONNX Runtime. just test-copr-lanes proves that one at the same declared depths — test-copr-pkg 43, test-copr-pkg 44, test-copr-smoke 45. Each rebuilds the package from source in a mock chroot (the test-copr half), exports the RPM it built, installs it with dnf so the package's own Requires: onnxruntime resolves, and boots it for the same validation the direct lane runs. just test-packaging-matrix requires both, and test/packaging-evidence.py refuses a direct-RPM record offered as a COPR target's evidence.

The COPR lanes are slow even by this file's standards: each one compiles the whole workspace and runs the spec's %check inside the mock chroot, and mock needs a privileged container, so they run serially through a single staging path (target/copr-lane/facelock.rpm). Never run two at once.

test/fedora-lane-image.sh resolves each lane's digest-pinned base image from the matrix, so no Containerfile carries its own Fedora digest. It refuses a release the matrix does not declare and refuses one that has reached its EOL gate: Fedora 43 goes EOL on 2026-12-02, and from that date the Fedora 43 lane stops with a message instead of quietly testing an unmaintained release. Set RELEASE_MATRIX_TODAY to rehearse that date, the same override test/check-release-matrix.py reads. Retiring the lane means retiring its matrix rows, and moving the date is a deliberate matrix edit.

Fedora 43 is the only release carrying a gate today. The lookup is generic on fedora.<release>_eol_gate, so adding a 44_eol_gate or 45_eol_gate key gates those lanes immediately; until one exists, 44 and 45 run past their own end of life without complaint.

just test-rpm-lanes runs each release through the recipe its matrix lifecycle_depth names, and test/check-release-matrix.py requires that exact pairing, so a full lifecycle lane cannot be quietly downgraded to a smoke lane.

The full lifecycle lane also pins %config(noreplace): an unmodified /etc/facelock/config.toml is replaced in place on upgrade, a modified one survives byte for byte with the new file diverted to .rpmnew, and erase removes an unmodified copy outright while retaining a modified one as .rpmsave. docs/contracts.md carries the same contract.

The RPM embeds a read-only retired-profile upgrade guard in %pre. The model-free test-rpm-authselect gate boots Fedora with systemd and exercises real authselect and PAM password success/failure across fresh, unselected, selected-retired, custom-profile, malformed-state, and authselect-absent transactions. It never changes the host PAM stack. The exact retired facelock selection blocks with manual backup-and-reselection guidance; ordinary selections are preserved and the new RPM ships no authselect profile or dependency.

An already-installed v0.1.4 RPM cannot be retroactively guarded: direct uninstall runs only the scriptlets already installed from v0.1.4. Users must install a guarded release before a later uninstall so the guarded upgrade can retire the old authselect payload first.

Packaging gates in CI

.github/workflows/packaging.yml runs the lanes above in CI: both Debian suite gates, every declared Fedora lane, the Arch package built from the real dist/PKGBUILD, and the native version-ordering matrix. It downloads and checksum-verifies the ONNX models first, through .github/actions/fetch-models, so the daemon-start assertions execute instead of being counted as skipped.

Three schedules, because the full matrix takes about 1 h 45 min (measured 2026-09-02) and most pull requests touch no packaging:

The copr job never runs on a pull request regardless of the filter above -- mock needs a privileged container.

The pull-request filter is a changes job running .github/workflows/scripts/classify-changes.sh, which classifies the merge-base diff in plain bash and emits one output per lane. Each job gates on its own output -- if: needs.changes.outputs.deb == 'true' for the Debian suites, rpm for Fedora, arch for the Arch package, release_binaries for the Arch-container build the Fedora lanes stage from, release_matrix for the version-ordering matrix -- which reports a real "skipped" conclusion; GitHub's own paths: filter would leave a required check pending forever instead.

A path only one family's recipe or harness reads selects that family's lane. Everything else that reaches a package selects every lane; when in doubt, every lane.

Any package lane also selects release_matrix, since the versions it orders live in debian/changelog, the spec and the PKGBUILD; rpm also selects release_binaries, which it stages from. The three Rust files are listed because facelock pam remove --all runs from %preun, from Arch's pre_remove and from Debian's prerm, so a change to that command can abort a package removal without touching a packaging file. ci.yml and the other non-packaging workflows select nothing; a container digest bump inside packaging.yml itself still selects every lane, because a path cannot say which job's image moved. just test-classify-changes pins the table.

Residual risk. A Rust change outside those three files runs only the release_binaries lane on its own pull request: just build-release in the pinned Arch container, which proves the workspace still compiles the way the packages consume it, in minutes. It does not build or boot a package. A Rust change to daemon startup, a new runtime dependency, a file the spec does not ship: each of those leaves its own pull request green with the deb, rpm and Arch lifecycle jobs reported as skipped. Do not read that as packaging-verified. The nightly matrix catches it within a day, and the release gate below catches it before anything ships. When a change is packaging-relevant in a way the filter cannot see, run the lane by hand or add the path to classify-changes.sh.

The COPR jobs go further and skip pull requests entirely. Each one compiles the workspace inside a mock chroot, which needs a privileged container, and no pull request has been shown to get one on a rootless-podman runner; making that a required check before it is proven would block every packaging merge on an unproven capability. So a COPR-only break — something that shows up when the package is rebuilt from source or run against Fedora's system ONNX Runtime, and not otherwise — survives its own pull request even when the filter fires. The nightly and the pre-release workflow_dispatch are unfiltered and do run them; locally, just test-copr-lanes.

The Debian lanes skip the .dsc rebuild on pull requests. The full gate compiles the workspace twice per suite: once to assemble the candidate .deb, once more from the extracted .dsc in a clean image to prove the source package rebuilds standalone. Both are lto = true release builds and no cache applies, so the rebuild is about ten of the lane's twenty-six minutes on a runner that has reached the 90-minute cap (#337). packaging.yml sets FACELOCK_DEB_SKIP_DSC_REBUILD=1 on pull requests, which drops that second compile and nothing else: the .deb is still built from source with the network denied, its dependency closure still proved, the booted lifecycle still run. A lane that skipped the rebuild records depth=partial, which the release matrix requires of nothing, so just release-preflight refuses it -- it refuses pull-request runs regardless. The nightly, the dispatch, and a local just test-deb or just test-packaging-matrix keep the rebuild. What survives a pull request, then, is an incomplete source package: a file the .dsc does not carry, or a build that only works from the Git checkout. The nightly catches it within a day, the release gate before anything ships.

Release preflight (recommended)

Run this before creating/pushing a release tag:

just test-arch-loopback               # the same tiers on a synthetic camera; records the commit
just test-arch-camera-required        # or: camera + a person in frame; records the commit
gh workflow run packaging.yml --ref main   # the packaging matrix, at this commit
just release-preflight                # stable release checks
just release-preflight v0.2.0-rc.1   # prerelease checks; no stable secret access
just check
just test-arch-pam
just test-arch-camera-free

The end-to-end tiers come first because preflight cannot run them. It refuses to pass until one of two records names HEAD, so run whichever you choose after the last commit that will ship, not before:

• just test-arch-loopback runs test-arch-integration and test-arch-oneshot against a v4l2loopback node fed with a procedurally rendered face, with require_ir and require_frame_variance on, and writes the commit to .loopback-tier-verified. No camera, no person, a few minutes. It proves the capture, IR classification, liveness, enrollment, daemon, one-shot and PAM paths end to end; it cannot prove that a real sensor's frames match a real face. The loopback nodes it needs and the modprobe line are in Testing Safety.
• just test-arch-camera-required runs the same two tiers against /dev/video* with a person in frame and writes the commit to .hardware-tiers-verified. It is the only run that proves real-sensor recognition of a real face.

Either satisfies the gate (test/e2e-tier-evidence.sh). Those two tiers are the only automated evidence that face authentication works end to end: real D-Bus activation, the real PAM stack, real capture, and the one-shot path PAM falls back to. Nothing else ran them, and three of their assertions rotted undetected as a result (#139). A run done by hand at this exact commit is acknowledged by naming it: FACELOCK_LOOPBACK_TIER_ACK=<sha> or FACELOCK_HARDWARE_TIERS_ACK=<sha> on the just release-preflight command.

Preflight also refuses to pass without complete packaging evidence for HEAD. Every packaging lane writes a record of what it claimed and what it counted, and test/packaging-evidence.py accepts the set only when every lane the release matrix requires is present at this commit with zero skips and the ONNX models on hand (the contract is in docs/contracts.md, "Packaging matrix evidence"). That set includes a COPR lane per Packit release target beside the direct-RPM lane, so a green Fedora .rpm result alone leaves the evidence incomplete. Preflight reads it from the packaging-evidence-* artifacts a successful packaging.yml run at that exact commit uploaded, fetched with gh run download, or from .packaging-matrix-verified, which just test-packaging-matrix writes after running every lane locally. A run's green conclusion alone is not evidence: a path-filtered pull-request run skips every lane and still concludes "success". A pull-request run cannot satisfy it either: it builds the merge commit, not the commit being released. A FACELOCK_ALLOW_MISSING_MODELS=1 run is a diagnostic: it writes its partial lane records, and the marker is withheld. The one-line commit marker from before 0.2.0 is refused with a message naming the new format. A nightly run does not satisfy it either: nightly builds whatever main was at 07:00 UTC, and a release commit is a version bump nobody has built a package from yet.

just release-preflight checks local tools, required packaging files (including .packit.yaml), and whether AUR_SSH_KEY, APT_GPG_PRIVATE_KEY, and APT_GPG_PASSPHRASE are configured in GitHub secrets (via gh). COPR needs no secret — it is driven by Packit. Preflight and CI also read the public production COPR API and require its enabled chroots to equal the checked-in authority: Fedora 43/44/45 are required and Rawhide is the only optional experimental chroot. Rawhide may be present or absent; a missing required chroot or any unknown extra is release-blocking drift. Preflight goes one step further than CI and asks what production COPR actually serves: the EVR of the predecessor pinned in dist/release-matrix.json. The checker never modifies the project. Preflight always runs packit config validate --offline against .packit.yaml, in the digest-pinned Fedora container built from test/Containerfile.packit — the same real schema gate just test-copr runs, reachable without a host packit install. It has no skip path: podman is a preflight prerequisite, and without it the gate fails rather than passing unrun. just test-packit-config runs the same gate on its own.

Preflight also holds the APT compatibility window: main and legacy compatibility suites present until 0.3.0, as dist/release-matrix.json declares, and absent from the first 0.3.0 tree on. just test-apt-repo proves the published shape when run, as a clean APT client; no workflow runs it.

Package repository setup (one-time)

AUR (Arch Linux)

Automated after setup. Every stable release run attempts AUR publication after the GitHub Release becomes public. If AUR_SSH_KEY is absent, the publisher prints a skip notice and exits successfully; an invalid configured key fails after publication and needs manual recovery. Release operators must therefore verify the live AUR packages rather than infer publication from a green job.

One-time setup (~10 minutes):

1. Create an AUR account at https://aur.archlinux.org/register

2. Add your SSH public key to your AUR account at https://aur.archlinux.org/account

3. Register the package names. CI's publish-aur.sh will create any of these on first push if they don't already exist, but you can also pre-register them manually:

   REPO_ROOT="$(pwd)"
   
   # facelock (source build — default for `yay -S facelock`)
   git clone ssh://aur@aur.archlinux.org/facelock.git aur-facelock
   cd aur-facelock
   cp "$REPO_ROOT/dist/PKGBUILD" .
   cp "$REPO_ROOT/dist/facelock.install" .
   # dist/PKGBUILD ships a __SRC_SHA256__ placeholder; substitute the real
   # tarball digest before pushing or the recipe refuses to build. Download
   # first, hash the file after: piping curl into sha256sum hashes empty
   # input when the download fails, and that digest must never be published.
   TAG="$(sed -n 's/^_tag=//p' PKGBUILD)"
   curl -fSsL -o "/tmp/facelock-v${TAG}.tar.gz" \
     "https://github.com/tyvsmith/facelock/archive/v${TAG}.tar.gz" &&
     SUM="$(sha256sum "/tmp/facelock-v${TAG}.tar.gz" | cut -d' ' -f1)" &&
     sed -i "s/__SRC_SHA256__/${SUM}/" PKGBUILD
   makepkg --printsrcinfo > .SRCINFO
   git add PKGBUILD facelock.install .SRCINFO
   git commit -m "Initial commit"
   git push
   cd ..
   
   # facelock-bin (prebuilt binaries from the GitHub Release — no cargo build)
   git clone ssh://aur@aur.archlinux.org/facelock-bin.git aur-facelock-bin
   cd aur-facelock-bin
   cp "$REPO_ROOT/dist/PKGBUILD-bin" PKGBUILD
   cp "$REPO_ROOT/dist/facelock.install" .
   makepkg --printsrcinfo > .SRCINFO
   git add PKGBUILD facelock.install .SRCINFO
   git commit -m "Initial commit"
   git push
   cd ..
   
   # facelock-git (VCS package tracking main)
   git clone ssh://aur@aur.archlinux.org/facelock-git.git aur-facelock-git
   cd aur-facelock-git
   cp "$REPO_ROOT/dist/PKGBUILD-git" PKGBUILD
   cp "$REPO_ROOT/dist/facelock.install" .
   makepkg --printsrcinfo > .SRCINFO
   git add PKGBUILD facelock.install .SRCINFO
   git commit -m "Initial commit"
   git push
   

4. Generate an SSH key for CI and add the public key to your AUR account:

   ssh-keygen -t ed25519 -f aur-deploy-key -N ""
   

5. Add the private key as a GitHub repository secret named AUR_SSH_KEY:

   gh secret set AUR_SSH_KEY < aur-deploy-key
   

   Or use the web UI: https://github.com/tyvsmith/facelock/settings/secrets/actions

After this, every non-prerelease tag push automatically updates the AUR package.

COPR (Fedora)

Packit reads .packit.yaml from the released tag. Packit's documented upstream_tag_exclude filtering applies to downstream synchronization jobs, not to copr_build, so it is not a prerelease safety boundary.

The production tyvsmith/facelock job sits at trigger: release, restored by hand for the stable release. A prerelease-capable configuration parks it back at trigger: ignore, which makes an alpha-tagged config structurally incapable of selecting a release-triggered production project. just release-preflight rejects a production release job for a prerelease and rejects its absence for a stable, so the trigger moves with the version instead of drifting away from it. The deliberate stable restoration targets fedora-43-x86_64, fedora-44-x86_64, and the separate fedora-45-x86_64 branched target. Rawhide is Fedora 46 development in this matrix, not an alias for Fedora 45, and not a staging or production Packit target. A configuration that targets Rawhide for release fails the matrix check.

.packit.yaml deliberately uses JSON syntax, which is a valid YAML subset. Release guards therefore parse its jobs semantically with the Python standard library instead of comparing YAML spelling; general YAML outside that subset fails closed. The production project is tyvsmith/facelock; the prerelease staging project is tyvsmith/facelock-testing, covered below. Issue #236 still owns the remaining staging infrastructure, so changing the project's chroots or permissions belongs there, not to a release-identity change.

The COPR RPM is built from source with the spec's default %bcond_with bundled_ort mode and does not bundle ONNX Runtime. Its BuildRequires/Requires: onnxruntime use Fedora's runtime-only package; the package check asserts onnxruntime-devel is absent and creates a real ORT session from the checksum-pinned minimal model in test/fixtures/. (The ort crate feature api-20 keeps the binary compatible with Fedora's runtime.)

One-time setup (~10 minutes):

1. Create a Fedora Account at https://accounts.fedoraproject.org
2. Log in to COPR at https://copr.fedorainfracloud.org and ensure the tyvsmith/facelock project exists with the fedora-43-x86_64, fedora-44-x86_64, and fedora-45-x86_64 chroots enabled. The optional fedora-rawhide-x86_64 experimental chroot may be enabled or absent; no other chroot is allowed (Settings → Chroots).
3. Install the Packit-as-a-Service GitHub App on the repository: https://github.com/marketplace/packit-as-a-service
4. In the COPR project → Settings → Permissions, grant the packit user admin permission, and in the "allowed forge projects" field add github.com/tyvsmith/facelock. Builder permission is enough to build and not enough to edit the project, which is a distinction Packit makes for you: it reconciles the project against .packit.yaml before submitting anything, and a reconciliation it is not allowed to perform aborts every target. See "Why v0.1.4 never reached COPR" below.
5. In the COPR project → Settings, enable "Enable internet access during builds". The RPM is built from source and cargo fetches crates from crates.io during %build; COPR's build chroot is network-isolated by default, so this toggle is required or the build fails resolving crates. The toggle is half of it: COPR takes network access per build, and a Packit submission carries its own value. .packit.yaml must also declare enable_net: true on every copr_build job, because Packit's default is false and it wins over the project. See "Why v0.2.0 built nothing in COPR" below.

Step 4's allowlist and step 5's toggle apply to both channels and are checked on every pull request by python3 test/check-live-release-channels.py, which reads them off the public project response. Step 5's .packit.yaml half is not in that response and is checked by test/check-release-matrix.py instead. The permission grant is not public, so it stays a hand-confirmed step -- and it is the one that failed silently for v0.1.4.

Verify the COPR build locally before relying on it with just test-copr, which reproduces the Packit SRPM + mock from-source rebuild on a Fedora chroot and checks that the payload has no bundle while its dependencies select Fedora ORT.

Only a stable-tagged config with the deliberately restored production release trigger can populate production COPR automatically. A prerelease tag never points at production; staging below is where a candidate gets built.

Why v0.1.4 never reached COPR

Packit was installed, the release trigger was correct, and the release event reached it. Every target still failed at submission with Copr project update failed for 'tyvsmith/facelock' project., thirteen seconds after publication (#333).

Packit reconciles the COPR project against .packit.yaml before it submits anything, and it edits the project whenever the config's targets are not already a subset of the project's enabled chroots. The v0.1.4 config listed fedora-42-x86_64, which the project had never enabled and which COPR no longer offers. Editing a COPR project requires admin, and until this section was written the setup steps above asked for builder. One chroot the project did not have cost all three builds. v0.1.3 failed identically and reached COPR only because it was submitted by hand fourteen minutes later.

Two rules follow, and both are enforced:

• .packit.yaml targets must be a subset of the project's enabled chroots. The release matrix binds fedora.packit_release_targets to copr_channels.production.required_supported_chroots, and test/check-live-release-channels.py requires the live project to enable every one of them, so a target the project lacks fails before the tag.
• A COPR build that never lands must fail something. Nothing in the release run can observe Packit's submission — it happens outside the run, after publication — so the release workflow's verify-copr job polls the public COPR API for the released EVR and fails the run when it never appears. Before this existed, the chroot comparison passed on every day of the three months COPR served 0.1.3.

Packit's default fix-spec-file action rewrites Release: 1%{?dist} to 1.{timestamp}.{ref}, which would publish a v0.2.0 release as facelock-0.2.0-1.20260904220135575676.v0.2.0 rather than the facelock-0.2.0-1 the conversion table promises. The production copr_build job therefore carries update_release: false, and production's served comparison is an equality.

The mismatch would not stay cosmetic. RPM ranks the suffixed build above the canonical one:

0.2.0-1.fc44 < 0.2.0-1.20260904220135575676.master.0.g7d9ffe7.fc44

So on a machine with both the COPR repo and a directly installed RPM of the same release, COPR wins every dnf update and the version the other channels ship never takes hold. Packit's own documentation warns that an inherited release suffix breaks NVR ordering. Pinning the release is what keeps every channel's 0.2.0 the same 0.2.0.

Staging keeps the default. It builds every pull request into one project, so its NVRs have to differ from each other, and the snapshot suffix is what makes them; its comparison ends at the boundary dot instead, accepting 0.2.0-1 and 0.2.0-1.<anything> while still refusing 0.2.0-11. The flag and the comparison are one contract in test/check-release-matrix.py: neither channel can change one without the other, because either half alone reds every stable release.

What is proven, and what is not. The schema accepts the per-job flag (just test-packit-config), Packit's own config parser resolves it to False for the production job while staging stays True, and packit srpm with the flag set produces facelock-0.2.0-1.fc44.src.rpm against facelock-0.2.0-1.20260904220135575676.master.0.g7d8eef8.fc44.src.rpm without it. What no local check can reach is packit-service applying the job's flag on a real release event: just test-copr builds its SRPM through the CLI, which reads package-level config, so that lane still carries the snapshot suffix and proves the package builds rather than the EVR it will be published under. The first stable tag after this change is the proof. If that release publishes a suffixed EVR anyway, the service ignored the job-level flag and verify-copr fails on an otherwise healthy build. The response is to delete update_release from the production job and set its served_evr_exact back to false, taking the suffix on both channels again.

Do not hoist update_release to the top level to force it. Top level reaches staging too, and staging builds every pull request into one project: without the snapshot suffix two pull requests produce the same NVR. Scoping the flag to the production job is the whole reason production and staging can differ here, so the fallback for a flag the service ignores is to stop asking for the canonical EVR, not to ask for it somewhere that breaks the other channel.

just release-preflight asks the same question about the previous release: test/check-live-release-channels.py --expect-predecessor requires production COPR to serve the EVR pinned in predecessors. The v0.1.4 build was never backfilled — 0.2.0 supersedes it — so that gap is recorded in copr_channels.production.served_evr_gap and reported rather than failed. The record names both EVRs and issue #333, and it retires itself: once the predecessor pin moves past v0.1.4 the release matrix contract fails until the record is deleted.

A recovery after a failed submission is a hand-submitted build from a checkout of the tag. Packit reacts only to new release events, so re-publishing is not an option:

git checkout vX.Y.Z
packit srpm --no-update-release
copr-cli build tyvsmith/facelock facelock-X.Y.Z-1.*.src.rpm \
    -r fedora-43-x86_64 -r fedora-44-x86_64 -r fedora-45-x86_64

--no-update-release is not optional. The Packit CLI reads package-level config, not a job's, so it would otherwise apply the snapshot suffix the production job pins off and hand production an EVR its own gate refuses. packit build in-copr has no equivalent flag, which is why the recovery goes through copr-cli with an SRPM built locally. Name the three supported chroots: copr-cli build with no -r builds every chroot the project has enabled, which picks up the optional Rawhide one. This path needs a COPR API token in ~/.config/copr, which the Packit CLI did not.

Why v0.2.0 built nothing in COPR

v0.2.0 fixed everything v0.1.4 got wrong. Packit triggered on the release event, reconciled the project, and submitted all three chroots under the canonical 0.2.0-1. Every one of them then failed in %build, four minutes in, on Could not resolve host: index.crates.io.

enable_net is a per-build value, not a project one. The project toggle is the default COPR applies to a build that does not carry its own, and a Packit submission always carries its own: Packit's enable_net defaults to false and is sent on every build it creates. So the toggle was on, the build task read 'enable_net': False, and the chroot had no resolver while cargo needed crates.io. v0.1.3 is the control -- same spec, same project, and it built, because a human submitted it with copr-cli, which omits the field and lets the project default stand.

Three things had to be true at once for this to reach production, and each is now closed:

• The gate read the wrong layer. test/check-live-release-channels.py compares the public project response, which is the only place enable_net appears before a build exists, and the value there was correct. test/check-release-matrix.py now requires enable_net: true on every copr_build job in .packit.yaml, which is the layer that decides.
• The local lane hardcoded the answer. just test-copr builds the Packit SRPM and rebuilds it under mock, and it passed --enable-network unconditionally, so it modelled a chroot COPR was never going to give it. The lane now reads the flag out of .packit.yaml and gets the same network the real submission would.
• The docs asked for the toggle and stopped. Setup step 5 now names both halves.

Recovery is the hand-submitted build above. copr-cli sends no enable_net unless --enable-net is passed, so it inherits the project toggle and needs nothing else.

Staging COPR (tyvsmith/facelock-testing)

The project exists, with exactly the fedora-43-x86_64, fedora-44-x86_64, and fedora-45-x86_64 chroots, internet access during builds enabled, and github.com/tyvsmith/facelock on its Packit forge allowlist. Those three properties are contract-checked on every pull request, not just described here: test/check-live-release-channels.py reads all of them off the project response and fails on any of them. dist/release-matrix.json records the claim as copr_channels.staging.provisioned: true and holds the expected forge project in copr_channels.staging.required_forge_project.

Builder permission for the packit user is the one setup step no gate can see. COPR serves project permissions only to an authenticated owner, so the checker, which reads the public project API, cannot compare them. Confirm that one by hand in the web UI.

.packit.yaml declares a second copr_build job for it, on trigger: pull_request with manual_trigger: true. Packit therefore offers the build on a pull request and a maintainer dispatches it by hand with a /packit build comment; nothing builds into staging on its own, and a tag never publishes into staging.

python3 test/check-live-release-channels.py --channel staging now queries the real project and compares it with the checked-in authority, on every pull request in CI and again in just release-preflight. A chroot that appears or disappears in COPR fails that gate, as does internet access switched off or a forge allowlist that stops naming this repository.

The trigger and the switch move together, and test/check-release-matrix.py enforces the pairing in both directions: provisioned: true requires trigger: pull_request, and provisioned: false requires trigger: ignore. Setting the switch back without moving the trigger, or the reverse, fails the release matrix contract. The pairing is what stops a pull-request trigger from aiming every Packit run at a project that answers 404.

Provisioning was three edits, and the contract rejected any two of them without the third: the switch in dist/release-matrix.json, the trigger in .packit.yaml, and retiring the staging COPR provisioning must stay unclaimed until issue #236 creates the project assertion in test/check-release-matrix.py. That third assertion existed so provisioning could not be claimed by a config change alone; retiring it was the moment someone confirmed the project really exists. The live comparison above is what holds the claim honest from here.

Staging tolerates no optional experimental chroot. Production accepts Rawhide's presence or absence; in staging any chroot beyond the supported three is drift.

Pre-tag attestation

scripts/release-attestation.py renders and validates the document that binds a candidate to what its channels serve: the candidate commit, the EVR each channel serves per target, artifact and repository digests, signing key fingerprints, and when each channel last refreshed its repository metadata.

python3 scripts/release-attestation.py render --input facts.json --output attestation.json
python3 scripts/release-attestation.py validate --attestation attestation.json --expect expect.json

validate fails closed on a drifted candidate commit, a served EVR or digest that disagrees with the recorded expectations, a changed signing fingerprint, metadata older than metadata_max_age_seconds or stamped in the future, and on any channel carrying the production COPR identity. Gathering those facts from live staging repositories is issue #236's remaining infrastructure work; the script and its contract cases run against fixtures today.

Note: a previously published release will not retroactively build — Packit reacts only to new Release events.

The old COPR_WEBHOOK_URL GitHub secret is no longer used and can be deleted (gh secret delete COPR_WEBHOOK_URL).

APT (Debian/Ubuntu)

Automated after setup. Every stable release run attempts to build a signed APT repository before it publishes the GitHub Release. APT_GPG_PRIVATE_KEY and APT_GPG_PASSPHRASE are required; if either is absent or invalid, the stable release remains unpublished.

One-time setup (~15 minutes):

1. Generate a GPG signing key (if you don't have one):

   gpg --full-generate-key
   # Select RSA 4096, expiry 3y
   # UID: Ty Smith (Package Signing) <packages@m.tysmith.me>
   

2. Export and add the private key as a GitHub secret:

   gpg --armor --export-secret-keys "packages@m.tysmith.me" | gh secret set APT_GPG_PRIVATE_KEY
   

3. Add the passphrase as a GitHub secret:

   gh secret set APT_GPG_PASSPHRASE --body "your-passphrase"
   

   Or use the web UI: https://github.com/tyvsmith/facelock/settings/secrets/actions

The repository configuration lives in dist/apt/conf/distributions. Two codenamed suites are published:

• trixie: Debian 13 TPM build using Trixie Backports Rust/Cargo
• resolute: Ubuntu 26.04 TPM build using native Rust/Cargo

Two compatibility suites are published alongside them until 0.3.0, for clients whose source entry was written for v0.1.4:

• main: the trixie package, included by publish-apt.sh from the same validated artifact
• legacy: no package; reprepro export writes signed empty indexes so apt update keeps succeeding

Those clients must replace the suite in their Facelock source entry with their operating-system codename before 0.3.0. dist/release-matrix.json declares the window under apt_suites.compat, and check-release-matrix.py fails the first tree at or past retire_at that still carries the stanzas. Prerelease packages are never inserted into any of these stable suites. Stable publication requires exactly one suite-matching package for both codenames before signing or repository writes begin.

The APT repo is hosted at https://tysmith.me/facelock/apt/ alongside the docs site. The public keyring is at https://tysmith.me/facelock/apt/tysmith-archive-keyring.gpg. It carries one rsa4096 key: Ty Smith (Package Signing) <packages@m.tysmith.me>, fingerprint E7F8A4C424C6D59BD38536B536A81FCD934C17CE, expiring 2029-03-27. dist/release-matrix.json pins that fingerprint, uid, and expiry under apt_signing_key. test/check-release-matrix.py fails the tree if docs/quickstart.md or this file quotes a different fingerprint or drops the pinned one, so rotate the pin and both docs together.

GPG key rotation: The fetched keyring is a client's only trust root; nothing redelivers it on its own. A client that never re-fetches stops verifying the repository once the key expires (2029-03-27) or is rotated early, because apt refuses an expired or unrecognized key's signature. Renewing the expiry instead of rotating does not help: gpg --edit-key <key-id> expire writes a new self-signature onto the same exported key, and a client's already-fetched copy still carries the old one. Redistribution is required either way.

The publisher enforces the pin at sign time: it refuses to sign unless the imported key's fingerprint, uid, and expiry all match apt_signing_key, naming the mismatch. Rotating the APT_GPG_PRIVATE_KEY secret without updating the pin therefore fails the release rather than publishing a keyring the docs do not match. When the signing key changes, generate a new key, update APT_GPG_PRIVATE_KEY and APT_GPG_PASSPHRASE, update the apt_signing_key pin and both docs, and cut a new release; just test-release-matrix fails once today reaches the pinned expiry, and fails again if the pin and the docs disagree. A keyring package that lets an installed client pick up the new key through apt upgrade is tracked in #346.

Manual AUR update (fallback)

If CI is not configured or fails:

1. Download the release tarball, then compute the checksum from the file. Piping curl into sha256sum prints the digest of empty input when the download fails; downloading first prints no digest at all:

   curl -fSsL -o "facelock-v$VERSION.tar.gz" \
     "https://github.com/tyvsmith/facelock/archive/v$VERSION.tar.gz" &&
     sha256sum "facelock-v$VERSION.tar.gz"
   

2. Clone the AUR repo (first time only):

   git clone ssh://aur@aur.archlinux.org/facelock.git aur-facelock
   

3. Copy dist/PKGBUILD and dist/facelock.install into the AUR repo
4. Replace the __SRC_SHA256__ placeholder in the PKGBUILD with the real checksum from step 1
5. Generate .SRCINFO:

   cd aur-facelock
   makepkg --printsrcinfo > .SRCINFO
   

6. Commit and push to AUR:

   git add PKGBUILD facelock.install .SRCINFO
   git commit -m "Update to v$VERSION"
   git push
   

Version Sources

The canonical version is in the root Cargo.toml under [workspace.package]. The version fields synced by just release are:

The independently maintained dist/release-matrix.json records supported targets, lifecycle depth, and immutable environment identities. Release preflight, CI, and release metadata checks validate that authority; just release does not rewrite it.

The version facelock-git actually installs

dist/PKGBUILD-git's pkgver field is display only. AUR's web page and .SRCINFO show it because makepkg --printsrcinfo runs without a checkout to describe, and just release keeps it level with the release so the page does not drift. What a build installs is whatever pkgver() computes at build time:

<released pkgver>.r<commits since that tag>.g<abbreviated object name>

git describe --abbrev=7 sets a floor, not a width: the object name is seven hex characters, or more where seven would be ambiguous. So a build off v0.1.4 reads like 0.1.4.r650.ga8c48b7, and one off v0.2.0-alpha.1 like 0.2.0alpha1.r7.gdeadbee. Two properties make that version usable, and both are enforced rather than assumed:

• it must outrank the release it descends from, or pacman refuses the upgrade and every AUR helper reports the package as permanently out of date
• it must rank below the next release, or the git package blocks the real one

Four things earn that, and all four were live faults (#330):

test/release-version-contract.sh holds the recipe to that shape against a synthetic tag graph, and test/release-native-ordering.sh hands the result to vercmp inside the pinned Arch container. release_arch_git_pkgver in scripts/release-versions.sh is the one definition both read.

ONNX Runtime Bundling

The ort crate is built with feature api-20, so facelock requires ONNX Runtime 1.20 or newer at runtime. ONNX Runtime is forward-compatible, so a single build works against any runtime ≥ 1.20.

ONNX Runtime is sourced differently per channel:

• GitHub-Release .deb: bundles CPU-only ORT 1.20.1 under /usr/lib/facelock/, because ONNX Runtime is not available in Ubuntu repositories.
• GitHub-Release direct .rpm: builds the spec with --with bundled_ort, installs libonnxruntime.so.1 under %{_libdir}/facelock/, and has no system onnxruntime dependency.
• COPR RPM (built from source by Packit): leaves the spec's %bcond_with bundled_ort disabled, contains no bundled runtime, and requires Fedora's system onnxruntime package.
• Arch Linux (PKGBUILD): depends on the virtual onnxruntime capability. The exact official-repository providers are onnxruntime-cpu, onnxruntime-opt-cuda, and onnxruntime-opt-rocm; there is no package literally named onnxruntime.

The bundled ORT is a CPU-only fallback — users who install a system-wide GPU-enabled ONNX Runtime (CUDA, ROCm, OpenVINO) will have it take precedence automatically (the search order prefers system paths over the bundled copy).

The reviewed pins in .github/workflows/release.yml include the version, upstream URL, archive and library SHA-256 values, upstream commit, and MIT license identity. The download job verifies the archive before extraction and the library after extraction, then emits manifest.json, SHA256SUMS, and PROVENANCE.md beside upstream LICENSE, ThirdPartyNotices.txt, VERSION_NUMBER, and GIT_COMMIT_ID. Direct RPM assembly re-verifies those inputs and enters .github/workflows/scripts/run-networkless.sh before creating the source archive or rpmbuild tree. That wrapper uses util-linux enosys to deny socket and io_uring network syscalls, closes inherited non-stdio file descriptors, and requires its network probe to fail with ENOSYS before invoking rpmbuild; CARGO_NET_OFFLINE=true remains defense in depth. The RPM ships the reviewed inputs under its package documentation/license directories for SBOM and provenance consumers.

When upgrading the ort crate dependency, update every reviewed ORT pin and the RPM bundle filename together and, if the crate requires a higher floor, the api-NN feature in crates/facelock-face/Cargo.toml.

Rawhide may be attempted only with the digest-pinned experimental environment recorded in dist/release-matrix.json. A Rawhide system-ORT build/session smoke is best effort: absence or failure is nonblocking and can never stand in for lifecycle, upgrade, rollback, artifact, served-version, availability, or alpha release evidence. It must not publish or modify a COPR channel.

Upgrade Safety

Since facelock is a PAM module, broken releases can lock users out. Every release must:

1. Pass just check (tests + clippy + fmt)
2. Pass just test-arch-pam (Arch container PAM smoke tests)
3. Pass just test-arch-camera-free (camera-free daemon and one-shot E2E)
4. Pass just test-arch-loopback (synthetic camera, no person) or just test-arch-camera-required (a camera and a person in frame, the only run that proves real-sensor recognition) against the final release commit; just release-preflight fails until one has
5. Pass just test-rpm and just test-deb (multi-distro package validation)
6. Not change PAM auth semantics without explicit changelog entry
7. Preserve /etc/pam.d/sudo backup on install (/var/lib/facelock/pam-backups/sudo.<timestamp>)
8. Default to PAM_IGNORE on internal errors (fall through to password)

Upgrading from the last release

just test-upgrade-v014 proves that state written by v0.1.4 survives an upgrade to the candidate and a rollback back to v0.1.4. Two lanes, Debian trixie and Fedora 44, each install the real published artifact rather than a synthesized older build of the candidate.

What the lanes pin. dist/release-matrix.json carries a predecessors block holding the GitHub release id, the asset id, the SHA256 and the byte size of each predecessor artifact. The lane Containerfiles take those as build args and carry no digest of their own, so one review changes the pin everywhere. just test-upgrade-v014-pins asks the release API whether those assets are still the assets it serves, which is how a re-uploaded or substituted predecessor gets caught before a lane silently proves something about a different file.

What the lane images carry. Each image installs the runtime libraries the released binary needs before the predecessor goes on. v0.1.4 wrote its Debian control file by hand and never declared libxkbcommon0, which its own binary links, so that release cannot start on a minimal Debian 13 at all. The candidate is built from debian/control and derives the list with ${shlibs:Depends}. Nothing is masked by supplying it: candidate dependency resolution belongs to test/deb-dependency-closure.sh on a pristine suite base.

What the lanes build. Predecessor state comes from the released v0.1.4 binary, never from the candidate: plaintext rows, keyfile-encrypted rows, mixed rows, and two swtpm-sealed shapes, one PCR-bound and one not. Each shape also carries a modified config, the reviewed models, an enrollment marker, an audit log and a hand-wired PAM service, because v0.1.4 has no facelock pam subcommand and that is the shape a real upgrade finds.

What each lane proves after the upgrade. The V5 database reaches V6 with legacy rows at device_id = NULL. A known embedding still decrypts to the exact plaintext it was enrolled as, which a file hash cannot show: a preserved key and a preserved ciphertext nobody can open any more hash identically. No key artifact is replaced and none appears that was not there before. Modes converge to ADR 010 without content changing. The enrollment marker keeps its owner and mode and its content is reconciled against the database rather than preserved byte for byte — the one piece of state the upgrade is supposed to rewrite (#137). The administrator's PAM service is byte-identical, a correct password still authenticates, and a wrong one still fails.

Version ordering on a development tree. Until just release bumps the workspace, the candidate .deb built from the tree is 0.1.4-1~deb13u1, which sorts below the published 0.1.4-1. The lanes build the same payload as an upgrade-test version instead, and every run prints the version it chose and why. Once the workspace version sorts above 0.1.4 the re-versioning stops and FACELOCK_UPGRADE_TEST_VERSION becomes a no-op: the lane installs the shipped version exactly. The native comparator inside the container decides either way, so a lane can never quietly become a downgrade test. Whatever version it lands on is spelled by scripts/release-versions.sh, the same file the release workflow uses, so a pre-release candidate reaches the lane as 0.2.0~alpha.3-1~deb13u1 rather than in a Cargo spelling neither packager would ever ship. The RPM release counter it passes is local to the lane, not the series counter from "Prerelease identity conversions" above: the lane's only ordering requirement is against the published predecessor, never against a previously published prerelease.

Upgraders from v0.1.4 already have face authentication enabled. That release's pam-auth-update profile shipped Default: yes, so installing it switched Facelock on in common-auth. The packaged profile is Default: no now, which applies to fresh installs; an upgrade leaves the global stack exactly as it found it, and the lane fails if it is edited in either direction. Removing an enabled profile would take face authentication away from someone using it, so the lane treats that as the more dangerous direction, not a clean result.

Where it runs. Locally, by design: a cached run is about twenty minutes and a cold one considerably more, so packaging.yml does not carry it and a nightly-only job is the follow-up. just check runs the container-free contract (just test-upgrade-v014-contract), so a broken lane definition still fails every pull request.

Rollback. The candidate daemon starts and migrates the database before the downgrade, so the predecessor is handed the file production would hand it. V6 has no down-migration and the schema stays at 6 after the package rolls back. See docs/contracts.md for what that does and does not guarantee.