update docs
This commit is contained in:
@@ -0,0 +1,183 @@
|
||||
# Frozen inputs and offline rebuilds
|
||||
|
||||
Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.
|
||||
|
||||
|
||||
[Documentation index](README.md) · [Build host](build-host.md) · [Release signatures](releases.md)
|
||||
|
||||
M12 adds a local input snapshot, fresh restore, network-isolated build, and
|
||||
complete artifact comparison. **Two fresh network-isolated builds passed with
|
||||
all 27 artifacts byte-identical.** The [M12 evidence page](m12-validation.md#final-local-release-acceptance)
|
||||
records the source/input identities, signed local release and actual archive
|
||||
round-trip. This result applies to the recorded inputs and documented build
|
||||
host; it does not claim equivalence across arbitrary host environments.
|
||||
|
||||
Normal builds still use Void's signed rolling repositories. A frozen build uses
|
||||
the selected package files and build environment from one explicit snapshot.
|
||||
It does not silently consult a newer repository if an input is missing.
|
||||
|
||||
## What is frozen
|
||||
|
||||
`tools/frozen-inputs create NEW_DIRECTORY` records:
|
||||
|
||||
- Project-owned source, documentation, configuration and package overlays, with
|
||||
a content identity independent of an FDS Git commit. This works before the
|
||||
repository's first commit too; it does not invent a commit identifier.
|
||||
- The exact pinned Void checkout in a Git bundle, preserving upstream files and
|
||||
the shallow-history boundary needed to read its commit timestamp after restore.
|
||||
- Cargo's locked vendored sources, the selected Rust toolchain and its static
|
||||
ARM target. Cargo credentials and user configuration are not copied.
|
||||
- The project-local Void binary build environment, static XBPS host tools and
|
||||
image-tool prefix. Temporary build trees, personal home directories and
|
||||
mount points are excluded.
|
||||
- Cached build dependencies, exact third-party packages selected by each of
|
||||
the three rootfs profiles, and kernel/DHCP/hello source archives. Local FDS
|
||||
packages are rebuilt from source instead of seeding the output repository.
|
||||
- Pinned Pi firmware, the VM kernel fixture, and the official EEPROM preview
|
||||
inputs. The preview is not a backup of a physical Pi.
|
||||
|
||||
`lock.json` records paths, kinds, modes, lengths, hashes and symlink targets.
|
||||
Added, missing or changed entries fail verification. The snapshot is several
|
||||
gigabytes. The completed offline diagnostic build occupied about 24 GB and its
|
||||
snapshot occupied 6.9 GB. Two restored builds need separate space; retained VM
|
||||
test images add further usage. Check available space before starting and keep
|
||||
headroom beyond these observations, which are not upper bounds. The workstation still supplies the documented
|
||||
Arch/Linux prerequisites, including Python, bubblewrap, Git, Make and compression
|
||||
tools. Their environment is recorded; this is not a replacement for a host OS.
|
||||
|
||||
## Prepare the snapshot
|
||||
|
||||
Complete the [first-build setup](getting-started.md), then run the complete
|
||||
artifact build from the repository root:
|
||||
|
||||
```sh
|
||||
make all
|
||||
```
|
||||
|
||||
`make all` runs bootstrap and package construction, builds both normal SYSTEM
|
||||
profiles and independent recovery, then creates the four initramfs formats,
|
||||
production Pi boot volume, complete internal disk, signing tools and production
|
||||
EEPROM preview. It does not write physical media or change EEPROM settings.
|
||||
Use `make packages` to build only the FDS base packages. Builders perform their
|
||||
normal format, ABI and rootfs checks; the complete runtime acceptance suite
|
||||
remains separate.
|
||||
|
||||
Runtime test targets select their required profile explicitly: init, boot,
|
||||
console, cartridge, DATA, media and stress checks use CLI; desktop and power
|
||||
checks use development; recovery checks use recovery. They no longer depend on
|
||||
which rootfs happened to be built last. `make rootfs-test` intentionally checks
|
||||
the most recently exported archive, so use it immediately after a profile build
|
||||
when validating that specific export.
|
||||
|
||||
Fetch all locked crates, including platform-specific sources that Cargo's vendor
|
||||
operation requires even when they are not built for ARM:
|
||||
|
||||
```sh
|
||||
cargo fetch --locked
|
||||
./tools/frozen-inputs create "$PWD/out/inputs-0.1.0"
|
||||
./tools/frozen-inputs verify "$PWD/out/inputs-0.1.0"
|
||||
```
|
||||
|
||||
Use a new output directory. The command will not overwrite a previous snapshot.
|
||||
The snapshot operation itself is offline; missing selected packages or locked
|
||||
crate sources fail with an error. The subsequent fresh build establishes whether
|
||||
the complete dependency set was captured. Keep failed outputs for diagnosis and
|
||||
retry into a new directory.
|
||||
Run every build, snapshot and VM workflow sequentially: the normal checkout
|
||||
shares one Void masterdir. Snapshot creation locks rootfs/base-package assembly,
|
||||
but a separate standalone kernel or VM command must not be started concurrently.
|
||||
|
||||
## Restore into two independent trees
|
||||
|
||||
If starting from the signed release's input archive, extract it as the ordinary
|
||||
build user with `tar --extract --zstd --same-permissions --no-same-owner` and
|
||||
the desired `--file` and new `--directory` paths. Mode preservation matters:
|
||||
default extraction can mask writable bits or remove sticky bits and cause the
|
||||
input lock to reject an otherwise intact archive. Verify the extracted snapshot
|
||||
before restoring it. The release's README provides the complete commands.
|
||||
|
||||
The destination parent directory must already exist, and the destination itself
|
||||
must not exist. Use paths without whitespace, as required by xbps-src:
|
||||
|
||||
```sh
|
||||
./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-a"
|
||||
./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-b"
|
||||
```
|
||||
|
||||
Restoration verifies every locked entry first. Each new tree gets its own Void
|
||||
masterdir, source/download caches, Cargo directory and Rustup directory. It starts
|
||||
without an FDS target directory or generated FDS packages/images.
|
||||
|
||||
Run the builds sequentially and retain their full logs:
|
||||
|
||||
```sh
|
||||
out/rebuild-a/tools/in-frozen-build make all >out/rebuild-a.log 2>&1
|
||||
out/rebuild-b/tools/in-frozen-build make all >out/rebuild-b.log 2>&1
|
||||
./tools/compare-builds out/rebuild-a out/rebuild-b \
|
||||
--report out/reproducibility-0.1.0.json
|
||||
```
|
||||
|
||||
The wrapper changes to the restored project directory, uses its private Rust
|
||||
caches, makes the rest of the workstation read-only, and removes network access
|
||||
for the build and all child processes. Missing input therefore cannot be hidden
|
||||
by a download. FDS's xbps-src wrapper keeps dependency resolution enabled and
|
||||
excludes remote repositories; the custom repository filename includes `local`
|
||||
so the pinned xbps-src also retains it in its cross sysroot. Rootfs assembly uses the frozen per-profile
|
||||
repository. Setting `FDS_OFFLINE=1` alone selects offline inputs but does not
|
||||
provide the wrapper's network isolation.
|
||||
|
||||
The comparison checks both rootfs tars, recovery rootfs, all installable images,
|
||||
all four initramfs formats, Pi kernel/DTB, every FDS package and the EEPROM
|
||||
configuration/rollback outputs. It reports each mismatch and exits nonzero if
|
||||
anything differs. Matching files alone do not attest how they were built; retain
|
||||
the isolated build logs alongside the report.
|
||||
|
||||
## Assemble a local signed release
|
||||
|
||||
After both complete builds match, create or choose a signing key as described in
|
||||
[Release signatures](releases.md). Use the assembler in the first restored tree:
|
||||
|
||||
```sh
|
||||
out/rebuild-a/tools/assemble-release \
|
||||
--inputs "$PWD/out/inputs-0.1.0" \
|
||||
--comparison-build "$PWD/out/rebuild-b" \
|
||||
--output-directory "$PWD/out/fds-os-0.1.0" \
|
||||
--key "$HOME/fds-signing/release-01.key"
|
||||
```
|
||||
|
||||
The assembler verifies the snapshot again, checks that the first checkout still
|
||||
matches its source identity, compares the complete pair, and verifies recorded
|
||||
rootfs build inputs. It creates versioned image names, source and complete input
|
||||
archives, package inventories, comparison evidence and usage guides. It signs
|
||||
the manifest and immediately verifies every listed file. The private key is
|
||||
never copied into the release, and existing output directories are refused.
|
||||
|
||||
The result is a **local** signed release with physical validation marked
|
||||
`deferred`; it is not published or installed automatically. The supplied key
|
||||
defines its signing identity. Distribute its fingerprint through an independently
|
||||
trusted channel before someone relies on the bundled public key.
|
||||
|
||||
## Determinism and troubleshooting
|
||||
|
||||
FDS uses the pinned Void commit's timestamp for package and image timestamps.
|
||||
Owned Rust binaries are stripped. `make` uses `tools/cargo-build` to map checkout,
|
||||
Rustup and Cargo registry source paths to stable `/usr/src/fds` paths; otherwise
|
||||
panic messages can embed the builder's absolute directory even in stripped
|
||||
binaries. The helper preserves the target's static linking flags, and its flags
|
||||
participate in Cargo's build cache. Direct `cargo build` remains useful for local
|
||||
development, but release construction uses the helper. This uses Rust's
|
||||
[source-path remapping](https://doc.rust-lang.org/rustc/command-line-arguments.html#--remap-path-prefix-remap-source-names-in-output)
|
||||
and Cargo's [merged configuration arrays](https://doc.rust-lang.org/cargo/reference/config.html).
|
||||
The kernel has explicit build user/host/time,
|
||||
and filesystem UUIDs derive from content/configuration identities. Rootfs export
|
||||
restores target ownership and capabilities instead of preserving the builder's
|
||||
UID. It removes only the build-specific auxiliary linker cache and fetched XBPS
|
||||
repository indexes; the installed package database, keys and runtime linker
|
||||
cache remain available.
|
||||
|
||||
If verification reports changed snapshot entries, restore the original snapshot
|
||||
or create a new one from the intended inputs. Do not edit `lock.json` to hide a
|
||||
mismatch. If the two builds differ, use the comparison's named artifacts and
|
||||
each build's rootfs/package manifests to identify the first differing stage.
|
||||
Archive compression is not a substitute for comparing the uncompressed images.
|
||||
Physical performance and durability still require the actual Pi and storage.
|
||||
Reference in New Issue
Block a user