FDS/OS 1.0 fixes

2026-09-23 03:12:45 +08:00
parent 8a4788fca8
commit 6bfcce8070
23 changed files with 403 additions and 40 deletions
@@ -44,7 +44,8 @@ Unknown names are left alone, including ad hoc diagnostic experiments.
 - User-created cartridge images, installed software trees and personal emulator sessions
  such as `out/my-emulator/`, including their persistent DATA overlays.
 - `out/logs/`, `out/manifests/`, packages, downloads, `out/cache/`, `.host/`, the
-  Void checkout/build container, and Cargo/Rustup caches outside this checkout.
+  writable Void checkout/build container under `.host/void-packages/`, and
+  Cargo/Rustup caches outside this checkout.
 - Git-tracked files and any recognized workspace containing a `.fds-keep` entry.

 To retain an older generated workspace for investigation, put a marker in it
@@ -10,6 +10,8 @@ the inputs named in those reports; a changed source tree needs fresh checks.

 ## Design and build references

+- [Clean upstream and writable Void workspace](void-workspace.md)
+
 - [Master plan](master-plan.md)
 - [Build host and dependencies](build-host.md)
 - [Development workflow](development.md)
@@ -112,7 +112,7 @@ equivalent of:
 ```sh
 export PATH="$PWD/.host/xbps/usr/bin:$PATH"
 export XBPS_ARCH=x86_64
-cd vendor/void-packages
+cd .host/void-packages
 ./xbps-src -A x86_64 binary-bootstrap
 ```

@@ -131,7 +131,7 @@ To reproduce the two builds individually from the repository root:
 ```

 The latter command sets the local XBPS PATH, checks the Void pin, and executes
-this command within the submodule:
+this command within the writable build checkout:

 ```sh
 ./xbps-src -a aarch64 pkg hello
@@ -186,9 +186,11 @@ tests accumulate. [Cleanup](cleanup.md) explains the retained current images,
 release/input archives and caches. Full image builds and VM suites need tens of
 GiB per working tree, beyond the initial M0 smoke test's requirements.

+- `vendor/void-packages/`: clean, pinned upstream submodule; no generated overlays.
+- `.host/void-packages/`: independent local build checkout at the same pin.
 - `.host/xbps/`: project-local host tools, not installed into `/usr`.
- `vendor/void-packages/masterdir-x86_64/`: glibc build container.
- `vendor/void-packages/hostdir/`: download caches, cross packages, and build output.
+- `.host/void-packages/masterdir-x86_64/`: glibc build container.
+- `.host/void-packages/hostdir/`: download caches, cross packages, and build output.
 - `out/packages/`: exported aarch64 test package and its repository index.
 - `out/manifests/void-build-packages.txt`: packages remaining installed after the
  build. Cross toolchains and temporary dependencies removed by xbps-src remain
@@ -197,12 +199,20 @@ GiB per working tree, beyond the initial M0 smoke test's requirements.
 - `out/manifests/artifacts.sha256`: validation artifact digests.
 - `out/logs/`: actual command logs, including logs from failed attempts.

-Place FDS package overlays at `packages/fds-*/template`. `tools/prepare-void`
-copies only directories containing a template and refuses to replace tracked
+Place FDS package overlays at `packages/fds-*/template`. `tools/prepare-void-workspace`
+clones the pinned upstream locally without shared Git objects or network access.
+`tools/prepare-void` copies directories containing a template into that writable checkout and refuses to replace tracked
 upstream packages. These copies are visible inside the build container. Base,
 base-files, init, CLI, kernel and Dasung overlays are active; future directories
 without templates remain inactive.

+Bootstrap migrates matching old generated copies, configuration, `hostdir` and
+`masterdir-*` out of the submodule using renames, preserving cached downloads
+and container files. Unknown files, staged edits, changed copies or destination
+conflicts stop migration for explicit reconciliation. Python 3 is used for
+checked migration and locking; it is already a documented FDS host prerequisite.
+The workstation builder also rejects submodules as writable build targets.
+
 If an overlay changes and preparation reports a stale copy, inspect and preserve
 any unique edits before moving the specific generated copy out of `srcpkgs/` and
 retrying. Modified upstream tracked files cause immediate failure. Conflicting
@@ -453,7 +453,10 @@ Record:
 VOID_PACKAGES_COMMIT
 ```

-Inject FDS packages through an overlay.
+Keep this submodule clean. Prepare a writable checkout at `.host/void-packages`
+from the same pinned commit. Inject FDS packages from `packages/` into that
+build checkout; keep generated sources, caches and build containers outside
+the submodule.

 Required relationship:

@@ -16,6 +16,13 @@ Normal builds still use Void's signed rolling repositories. A frozen build uses
 the selected package files and build environment from one explicit snapshot.
 It does not silently consult a newer repository if an input is missing.

+Current builds keep the upstream reference in `vendor/void-packages/` and use
+`.host/void-packages/` for overlays and cached build state. New snapshots and
+restores use this separate layout. Restoring an older
+snapshot retains its historical layout and saved build scripts; its files and
+acceptance are not rewritten. See [the workspace change](void-workspace.md) for
+fresh validation of this migration.
+
 ## What is frozen

 `tools/frozen-inputs create NEW_DIRECTORY` records:
@@ -94,14 +94,14 @@ normal. Never use a broad reset or clean command without checking what it remove
 For `etc/conf differs from config/xbps-src.conf`, compare the two:

 ```sh
-diff -u vendor/void-packages/etc/conf config/xbps-src.conf
+diff -u .host/void-packages/etc/conf config/xbps-src.conf
 ```

 Put the desired settings in `config/xbps-src.conf`, then follow
 [the explicit synchronization procedure](development.md#change-build-configuration-deliberately).
 A `diff` exit status of 1 means differences were found, not that the comparison failed.

-For `Stale overlay`, compare the named `vendor/void-packages/srcpkgs/fds-*`
+For `Stale overlay`, compare the named `.host/void-packages/srcpkgs/fds-*`
 directory with its source under `packages/`. Preserve any unique edits in the
 source first. Move that specific generated copy into a backup directory under
 `out/`, outside `srcpkgs/`, then run `./tools/prepare-void` again. Do not move or
@@ -121,7 +121,7 @@ space before assuming that a quiet download is a deadlock:
 ```sh
 tail -n 40 out/logs/bootstrap.log
 tail -n 40 out/logs/xbps-hello.log
-du -sh out/downloads vendor/void-packages/hostdir
+du -sh out/downloads .host/void-packages/hostdir
 df -h .
 ```

@@ -0,0 +1,77 @@
+# Clean upstream and writable Void workspace
+
+FDS keeps upstream source separate from its build state:
+
+| Path | Ownership and purpose |
+| --- | --- |
+| `vendor/void-packages/` | Clean upstream Git submodule at `VOID_PACKAGES_COMMIT` |
+| `packages/` | Authoritative FDS base-package sources, including Dasung |
+| `examples/software/*/void/` | Authoritative example PROGRAM package sources |
+| `.host/void-packages/` | Generated writable checkout, package copies, local configuration, caches and build container |
+
+`make bootstrap` creates the build checkout locally at the pinned commit. It
+copies Git objects independently; there is no network fetch for this step, no
+Git alternates dependency and no hardlinks into the upstream source. Normal
+package bootstrap still uses its configured repositories.
+
+`tools/prepare-void` copies FDS package overlays into the writable checkout and
+retains the existing checks against replacing upstream packages or overwriting
+changed generated copies. Workstation builds default to the same checkout.
+Explicitly selecting a Git submodule with `--void-packages` is rejected before
+source files or build state can be written there.
+
+The workspace helper migrates old, matching generated files and build caches by
+renaming them. It compares every file, mode and symlink in each known source
+copy first. Unknown upstream files, staged changes, independent edits and
+conflicting destinations stop migration. Run builds and migration sequentially.
+The workspace lock prevents two preparations from changing the layout together.
+Python is already a build-host prerequisite; this introduces no new dependency.
+
+For this checkout, the 58 staged additions matched the main-repository sources.
+Their local commit `47580322e801ef195c9bf279fd1fe4d8a1f723e9` and both staged
+patches were saved under `.host/backups/void-overlays-20260922-134611/` before
+unstaging those copies. The parent gitlink was restored to upstream commit
+`02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1`, matching the declared pin. All eleven
+package directories, `hostdir`, `masterdir-x86_64` and `etc/conf` were moved into
+the build checkout. No upstream tracked source was changed.
+
+An explicit future pin update advances the writable checkout without replacing
+its caches. Git refuses updates that would overwrite local files. Continue to
+record the parent gitlink and `VOID_PACKAGES_COMMIT` together.
+
+Frozen-input creation uses the separate build checkout. New restores recreate
+that layout. Older saved projects retain their
+original layout and scripts; the frozen 0.1.0 release is untouched. Restore
+compatibility tests do not imply a new reproducibility or physical-hardware claim.
+
+## Validation
+
+Fresh checks for this change passed:
+
+- `make bootstrap`, `make smoke-test`, and `make check`: migrated container use,
+  ARM/glibc package compilation, static-musl compilation, clean upstream state,
+  exact overlay/cache preservation, shallow pin updates, rejection of independent
+  edits and source conflicts, and cleanup protections.
+- `make workstation-test`: `out/workstation-images.ckkodex6`; real Void source
+  builds from `.host/void-packages`, image validation and confirmed file write/readback.
+  Selecting the upstream submodule as a build target is rejected without writes.
+- `make emulator-test`: `out/emu-test.op5azhlm`; full twelve-bay PROGRAM/DATA,
+  foreground terminal, eject/unplug, legacy reading and service-restart checks.
+- `make release-contract-test`: `out/m12-release-contracts.wmsfrcl0`; both separate
+  and legacy frozen-layout restores, retained source caches, local offline
+  repositories and existing tamper/release-comparison checks.
+- Documentation links, formatting, Python syntax and a final empty upstream Git
+  status, including ignored files, were checked.
+
+Logs are in `out/logs/void-workspace/`; source and log hashes, pins, native binary
+hashes and fixture results are recorded in `out/manifests/void-workspace-acceptance.json`.
+The optional host-native ARM execution shortcut was skipped in smoke-test. ARM
+execution was exercised by the full emulator suite. Physical hardware was not used.
+
+The first emulator attempt (`out/emu-test.463emq8u`) failed on an empty cartridge
+IPC response during initial discovery in its admin VM. The complete rerun passed
+without target-system changes. SYSTEM, kernel and initramfs hashes matched the
+previous accepted inputs. The cause of that first response failure was not
+established; its command/serial logs are retained. Future failures now collect
+live service diagnostics before VM shutdown, without retrying failed commands
+or relaxing their acceptance conditions.
@@ -34,6 +34,9 @@ make check
 ```

 Bootstrap prepares pinned Rust and XBPS tools and the Void build container.
+The upstream reference stays clean in `vendor/void-packages/`. Generated package
+copies, build configuration and caches live in `.host/void-packages/`; edit
+FDS package sources in `packages/`, not in either generated checkout.
 The smoke test checks the AArch64 build toolchains; `make check` validates the
 build guardrails. Stop and resolve any failed command before continuing. Logs
 are in `out/logs/`. Repeated preparation reuses verified downloads and caches.
@@ -26,7 +26,10 @@ normal Void `template` and any `files/` or `patches/` directories it uses. Paths
 are relative to this recipe. The [included hello package](../examples/software/hello/void/template)
 is a complete example.

-Custom sources are copied into an untracked `srcpkgs/PACKAGE` directory. Existing
+Custom sources are copied into `srcpkgs/PACKAGE` inside the selected writable
+build checkout. FDS uses `.host/void-packages/`; its `vendor/void-packages/`
+submodule remains unchanged. The builder rejects a Git submodule as its build
+checkout. Existing
 tracked Void sources are never overwritten. If a generated source copy differs,
 inspect it, preserve any independent edits, then remove that generated copy and
 retry. An unchanged copy can be reused.
@@ -18,7 +18,7 @@ export PATH="$PWD/out/workstation:$PATH"

 | Prerequisite | Used for |
 | --- | --- |
-| Prepared Void `void-packages` checkout and native XBPS utilities | Build source packages and install their AArch64 runtime dependencies |
+| Prepared writable Void `void-packages` checkout and native XBPS utilities | Build source packages and install their AArch64 runtime dependencies |
 | `mkfs.erofs`, `fsck.erofs` | Create and inspect read-only payload filesystems |
 | `bwrap` | Inspect filesystems and install package trees in a user namespace |
 | `xz` | Read older software cartridges and package/source archives |
@@ -31,12 +31,14 @@ share the same Void checkout.

 Prepare a Void source checkout using its [upstream instructions](https://github.com/void-linux/void-packages/blob/02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1/README.md).
 Complete `xbps-src binary-bootstrap` for the workstation architecture before
-using it. The workstation itself need not run Void. Select that checkout with
+using it. Keep this writable checkout outside Git submodules. The workstation itself need not run Void. Select that checkout with
 `--void-packages /path/to/void-packages`; native `xbps-*` tools must be in PATH,
 or selected with `--xbps-bin /path/to/bin`. Builds target AArch64/glibc.

 For the complete FDS build workstation, [bootstrap](getting-started.md) already
-prepares `vendor/void-packages` and `.host/xbps/usr/bin`. Reuse its local tools:
+keeps `vendor/void-packages` as a clean upstream reference and prepares the
+writable `.host/void-packages` checkout plus `.host/xbps/usr/bin`. The native
+builder defaults to `.host/void-packages`. Reuse its local tools:

 ```sh
 ./tools/prepare-image-tools