fds-os/docs/developer/reproducible-builds.md

Frozen inputs and offline rebuilds

Development reference and historical context. For current operating instructions, use the user manual. Acceptance applies only to the source and artifacts identified in each record.

Documentation index · Build host · Release signatures

M12 adds a local input snapshot, fresh restore, network-isolated build, and complete artifact comparison. Two fresh network-isolated builds passed with all 27 artifacts byte-identical. The M12 evidence page records the source/input identities, signed local release and actual archive round-trip. This result applies to the recorded inputs and documented build host; it does not claim equivalence across arbitrary host environments.

Normal builds still use Void's signed rolling repositories. A frozen build uses the selected package files and build environment from one explicit snapshot. It does not silently consult a newer repository if an input is missing.

Current builds keep the upstream reference in vendor/void-packages/ and use .host/void-packages/ for overlays and cached build state. New snapshots and restores use this separate layout. Restoring an older snapshot retains its historical layout and saved build scripts; its files and acceptance are not rewritten. See the workspace change for fresh validation of this migration.

What is frozen

tools/frozen-inputs create NEW_DIRECTORY records:

• Project-owned source, documentation, configuration and package overlays, with a content identity independent of an FDS Git commit. This works before the repository's first commit too; it does not invent a commit identifier.
• The exact pinned Void checkout in a Git bundle, preserving upstream files and the shallow-history boundary needed to read its commit timestamp after restore.
• Cargo's locked vendored sources, the selected Rust toolchain and its static ARM target. Cargo credentials and user configuration are not copied.
• The project-local Void binary build environment, static XBPS host tools and image-tool prefix. Temporary build trees, personal home directories and mount points are excluded.
• Cached build dependencies, exact third-party packages selected by each of the three rootfs profiles, and kernel/DHCP/hello source archives. Local FDS packages are rebuilt from source instead of seeding the output repository.
• Pinned Pi firmware, the VM kernel fixture, and the official EEPROM preview inputs. The preview is not a backup of a physical Pi.

lock.json records paths, kinds, modes, lengths, hashes and symlink targets. Added, missing or changed entries fail verification. The snapshot is several gigabytes. The completed offline diagnostic build occupied about 24 GB and its snapshot occupied 6.9 GB. Two restored builds need separate space; retained VM test images add further usage. Check available space before starting and keep headroom beyond these observations, which are not upper bounds. The workstation still supplies the documented Arch/Linux prerequisites, including Python, bubblewrap, Git, Make and compression tools. Their environment is recorded; this is not a replacement for a host OS.

Prepare the snapshot

Complete the first-build setup, then run the complete artifact build from the repository root:

make all

make all runs bootstrap and package construction, builds both normal SYSTEM profiles and independent recovery, then creates the four initramfs formats, production Pi boot volume, complete internal disk, signing tools and production EEPROM preview. It does not write physical media or change EEPROM settings. Use make packages to build only the FDS base packages. Builders perform their normal format, ABI and rootfs checks; the complete runtime acceptance suite remains separate.

Runtime test targets select their required profile explicitly: init, boot, console, cartridge, DATA, media and stress checks use CLI; desktop and power checks use development; recovery checks use recovery. They no longer depend on which rootfs happened to be built last. make rootfs-test intentionally checks the most recently exported archive, so use it immediately after a profile build when validating that specific export.

Fetch all locked crates, including platform-specific sources that Cargo's vendor operation requires even when they are not built for ARM:

cargo fetch --locked
./tools/frozen-inputs create "$PWD/out/inputs-0.1.0"
./tools/frozen-inputs verify "$PWD/out/inputs-0.1.0"

Use a new output directory. The command will not overwrite a previous snapshot. The snapshot operation itself is offline; missing selected packages or locked crate sources fail with an error. The subsequent fresh build establishes whether the complete dependency set was captured. Keep failed outputs for diagnosis and retry into a new directory. Run every build, snapshot and VM workflow sequentially: the normal checkout shares one Void masterdir. Snapshot creation locks rootfs/base-package assembly, but a separate standalone kernel or VM command must not be started concurrently.

Restore into two independent trees

If starting from the signed release's input archive, extract it as the ordinary build user with tar --extract --zstd --same-permissions --no-same-owner and the desired --file and new --directory paths. Mode preservation matters: default extraction can mask writable bits or remove sticky bits and cause the input lock to reject an otherwise intact archive. Verify the extracted snapshot before restoring it. The release's README provides the complete commands.

The destination parent directory must already exist, and the destination itself must not exist. Use paths without whitespace, as required by xbps-src:

./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-a"
./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-b"

Restoration verifies every locked entry first. Each new tree gets its own Void masterdir, source/download caches, Cargo directory and Rustup directory. It starts without an FDS target directory or generated FDS packages/images.

Run the builds sequentially and retain their full logs:

out/rebuild-a/tools/in-frozen-build make all >out/rebuild-a.log 2>&1
out/rebuild-b/tools/in-frozen-build make all >out/rebuild-b.log 2>&1
./tools/compare-builds out/rebuild-a out/rebuild-b \
  --report out/reproducibility-0.1.0.json

The wrapper changes to the restored project directory, uses its private Rust caches, makes the rest of the workstation read-only, and removes network access for the build and all child processes. Missing input therefore cannot be hidden by a download. FDS's xbps-src wrapper keeps dependency resolution enabled and excludes remote repositories; the custom repository filename includes local so the pinned xbps-src also retains it in its cross sysroot. Rootfs assembly uses the frozen per-profile repository. Setting FDS_OFFLINE=1 alone selects offline inputs but does not provide the wrapper's network isolation.

The comparison checks both rootfs tars, recovery rootfs, all installable images, all four initramfs formats, Pi kernel/DTB, every FDS package and the EEPROM configuration/rollback outputs. It reports each mismatch and exits nonzero if anything differs. Matching files alone do not attest how they were built; retain the isolated build logs alongside the report.

Assemble a local signed release

After both complete builds match, create or choose a signing key as described in Release signatures. Use the assembler in the first restored tree:

out/rebuild-a/tools/assemble-release \
  --inputs "$PWD/out/inputs-0.1.0" \
  --comparison-build "$PWD/out/rebuild-b" \
  --output-directory "$PWD/out/fds-os-0.1.0" \
  --key "$HOME/fds-signing/release-01.key"

The assembler verifies the snapshot again, checks that the first checkout still matches its source identity, compares the complete pair, and verifies recorded rootfs build inputs. It creates versioned image names, source and complete input archives, package inventories, comparison evidence and usage guides. It signs the manifest and immediately verifies every listed file. The private key is never copied into the release, and existing output directories are refused.

The result is a local signed release with physical validation marked deferred; it is not published or installed automatically. The supplied key defines its signing identity. Distribute its fingerprint through an independently trusted channel before someone relies on the bundled public key.

Determinism and troubleshooting

FDS uses the pinned Void commit's timestamp for package and image timestamps. Owned Rust binaries are stripped. make uses tools/cargo-build to map checkout, Rustup and Cargo registry source paths to stable /usr/src/fds paths; otherwise panic messages can embed the builder's absolute directory even in stripped binaries. The helper preserves the target's static linking flags, and its flags participate in Cargo's build cache. Direct cargo build remains useful for local development, but release construction uses the helper. This uses Rust's source-path remapping and Cargo's merged configuration arrays. The kernel has explicit build user/host/time, and filesystem UUIDs derive from content/configuration identities. Rootfs export restores target ownership and capabilities instead of preserving the builder's UID. It removes only the build-specific auxiliary linker cache and fetched XBPS repository indexes; the installed package database, keys and runtime linker cache remain available.

If verification reports changed snapshot entries, restore the original snapshot or create a new one from the intended inputs. Do not edit lock.json to hide a mismatch. If the two builds differ, use the comparison's named artifacts and each build's rootfs/package manifests to identify the first differing stage. Archive compression is not a substitute for comparing the uncompressed images. Physical performance and durability still require the actual Pi and storage.