update docs

2026-09-22 13:23:34 +08:00
parent 99bc3d15c5
commit 8a4788fca8
126 changed files with 7198 additions and 2425 deletions
@@ -0,0 +1,349 @@
+# Build software cartridges and run FDS in QEMU
+
+Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.
+
+
+This guide uses a **Linux workstation**, not the Raspberry Pi. It takes you from
+source software to a cartridge disk image, runs that image in FDS, and explains
+how to write the same complete image to USB. All commands run from the checkout
+root unless marked as guest commands.
+
+The two native host tools are `fds-cartridge` and `fds-emulator`. They use typed
+Clap interfaces; append `--help` to any command or subcommand. The target's `fds`
+command remains a static AArch64 executable. The new host tools are never part
+of the Pi base image. Dasung support remains in the base boot bundle.
+
+The accepted local `out/fds-os-0.1.0/` release predates this extension. Leave that
+release intact and build current images for the emulator. See
+[workstation acceptance](workstation-validation.md) for current evidence.
+
+For accumulated build and test images, use `make clean-preview` and `make clean`.
+Current images, native tools, latest test fixtures and personal emulator sessions
+are kept; obsolete automated test sessions are disposable. See
+[Cleanup](cleanup.md) for the exact scope and how to preserve an older test run.
+
+## Install and build the workstation tools
+
+Use a normal Linux account with Rust/Cargo, a C linker, Bash and Make. The
+repository's `rust-toolchain.toml` selects the Rust version. Runtime prerequisites
+are:
+
+| Tool | Purpose |
+| --- | --- |
+| `qemu-system-aarch64` | Runs the FDS ARM kernel and userspace using portable TCG |
+| `qemu-img` | Creates a separate writable overlay for DATA; source images stay unchanged |
+| `mkfs.erofs`, `fsck.erofs` from erofs-utils | Build and verify cartridge partitions; tested with 1.9.4 |
+| `xz` | Compress/decompress software tarballs with a decompression memory limit |
+| `bwrap` from bubblewrap | Restricts filesystem inspection to a private writable staging directory |
+| A software-specific cross compiler | Builds AArch64 applications; unnecessary for portable scripts |
+| `sfdisk`, `tar`, `mke2fs`, Python 3 | Independent integration tests; not needed to boot an existing image |
+
+Install these with your distribution's package manager. Unprivileged user
+namespaces must work for bubblewrap. Building **these host tools** does not need
+Void, XBPS, an Arch host, a Pi, root, or a running FDS guest:
+
+```sh
+make workstation
+export PATH="$PWD/out/workstation:$PATH"
+fds-cartridge doctor
+fds-emulator doctor
+```
+
+Current acceptance ran on Arch Linux x86_64. The tools use native Linux
+interfaces and distribution-provided utilities; other distributions and ARM
+workstations have not yet been exercised by this acceptance run.
+
+The outputs are native binaries in `out/workstation/`. The build explicitly
+selects the workstation architecture instead of the workspace's default ARM
+target. Rust dependencies are pinned in `Cargo.lock`. `tar` 0.4.46 and its
+`filetime` dependency provide archive decoding/creation; FDS additionally checks
+entry types, paths, counts, sizes, executable architecture and hashes.
+
+For this repository's **existing Arch/Void setup**, reuse its local tools:
+
+```sh
+make workstation
+./tools/prepare-image-tools
+./tools/in-void xbps-install -y qemu-img
+out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools doctor
+out/workstation/fds-emulator doctor --qemu-runner tools/in-void
+```
+
+The install command above changes only the project-local build container. The
+optional runners accept a program name followed by its arguments; omitting them
+uses programs in the workstation's `PATH`. Set the image runner on every
+cartridge create, inspect or preview invocation that needs it. A QEMU runner is
+saved in the session and reused for later DATA insertions.
+
+## Build two software bundles
+
+Each software recipe declares an id, display name, version, target architecture,
+root directory, and named executable commands. A trusted `[build]` section may
+run a workstation compiler. It is executed only by `software build`, never by
+`software pack`, inspection, or a cartridge insertion.
+
+The [hello recipe](../../examples/software/hello/software.toml) builds an AArch64
+C executable. Install an AArch64 glibc cross compiler as `aarch64-linux-gnu-gcc`,
+then run:
+
+```sh
+fds-cartridge software build examples/software/hello/software.toml out/demo-hello
+fds-cartridge software pack examples/software/report/software.toml out/demo-report
+fds-cartridge software inspect out/demo-hello
+```
+
+If your cross compiler has a different name, edit the recipe's `build.command`.
+With this checkout's existing Void cross toolchain, the equivalent is:
+
+```sh
+mkdir -p examples/software/hello/root/bin
+./tools/in-void aarch64-linux-gnu-gcc -O2 examples/software/hello/hello.c \
+  -o examples/software/hello/root/bin/hello
+out/workstation/fds-cartridge software pack \
+  examples/software/hello/software.toml out/demo-hello
+out/workstation/fds-cartridge software pack \
+  examples/software/report/software.toml out/demo-report
+```
+
+Use **one** of those hello workflows. Outputs must not already exist: choose
+new output names when rebuilding, then update your cartridge recipe. The example
+bundles each contain `software.toml` and `<id>.tar.xz`. The archive contains the
+contents of the software root, such as `bin/hello`; it has no leading `root/`.
+
+For your own application, copy a recipe and populate its root with `bin/`,
+`lib/` and `share/` as needed. C/C++ glibc builds target AArch64; static Rust
+programs should use `aarch64-unknown-linux-musl`. Set `architecture = "any"` only
+for scripts/data without ELF files. An x86_64 executable is rejected. Shared
+libraries must be compatible with the target SYSTEM's glibc, or bundled as
+appropriate. There is no cross-cartridge dependency resolver.
+
+## Assemble the cartridge image
+
+The example cartridge recipe groups the two bundles into two payload partitions:
+
+```sh
+fds-cartridge create examples/software/cartridge.toml out/demo-tools.img
+fds-cartridge inspect out/demo-tools.img
+```
+
+For the project-local EROFS tools, use:
+
+```sh
+out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools \
+  create examples/software/cartridge.toml out/demo-tools.img
+```
+
+The resulting complete disk image has **three GPT partitions**:
+
+| Partition | GPT name | Contents |
+| --- | --- | --- |
+| 1 | `FDS_METADATA` | Cartridge identity and the software catalogue |
+| 2 | `FDS_PAYLOAD02` | `bundles/demo.hello.tar.xz` |
+| 3 | `FDS_PAYLOAD03` | `bundles/demo.report.tar.xz` |
+
+To put both programs in one payload partition, use one `[[payload]]` entry with
+`bundles = ["../../out/demo-hello", "../../out/demo-report"]`. This makes a
+two-partition image. Each payload group creates exactly one partition. The first
+partition identifies every software bundle and its payload partition.
+
+Creation finishes only after verifying both GPT tables, all EROFS partitions,
+every archive and the catalogue. Inspection prints JSON, including offsets,
+lengths, SHA-256 and available commands. Repeating an unchanged recipe produces
+the same image bytes. See [the format](software-format.md) for limits and rules.
+
+## Boot current FDS in the emulator
+
+You need three matching current artifacts: the FDS Pi kernel, its initramfs, and
+a SYSTEM image containing the guest software runtime. Build these on the OS
+build workstation using [the build guide](getting-started.md). For an already
+bootstrapped checkout:
+
+```sh
+make rootfs PROFILE=cli
+make system-card PROFILE=cli
+make initramfs
+make workstation
+out/workstation/fds-emulator --session out/my-emulator start \
+  --qemu-runner tools/in-void
+```
+
+If QEMU is installed directly on your Linux workstation, omit `--qemu-runner`.
+To use images copied from another build machine, supply their paths:
+
+```sh
+fds-emulator --session out/my-emulator start \
+  --kernel /path/to/kernel_2712.img \
+  --initramfs /path/to/fds-initramfs.img \
+  --system /path/to/fds-system-cli.img
+```
+
+Create `out/` first if using a fresh directory. The session directory must be
+**new**, private, and short enough for Unix sockets (under 90 bytes including its
+absolute parent path). Start waits for the FDS prompt and verifies the emulator
+bay configuration. It uses two emulated CPUs and 1024 MiB RAM by default;
+`--memory-mib 2048` increases RAM. `--timeout` adjusts the boot deadline.
+
+The session records inputs, virtual devices, serial output and QEMU diagnostics.
+QEMU stays running after the command exits. SYSTEM and software images are
+read-only. No physical disks, host network interface, or Pi monitor are attached.
+
+## Insert, use and remove cartridges
+
+From the workstation:
+
+```sh
+fds-emulator --session out/my-emulator insert 01 out/demo-tools.img
+fds-emulator --session out/my-emulator guest -- fds bay 01
+fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.hello:hello
+fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.report:report
+fds-emulator --session out/my-emulator guest -- tail -20 /run/log/cartridged/current
+fds-emulator --session out/my-emulator eject 01
+```
+
+Insertion is asynchronous: `fds bay 01` may briefly show `EMPTY` before USB
+storage discovery completes. Retry the status command until it reports
+`MOUNTED READ ONLY`, or inspect its error. The catalogue lists exact commands.
+`fds run` starts a managed process and prints its PID; its output goes to the
+cartridge service log. Packages are verified and extracted into temporary,
+read-only RAM filesystems on first run. They are not compiled on the guest.
+
+All twelve bays, numbered `01`–`12`, are available. Simultaneously mounted
+PROGRAM cartridges need distinct cartridge IDs; inserting duplicate IDs is
+reported as a guest error. `eject` asks FDS to stop
+managed consumers, release mounts and declare `SAFE`, then removes the virtual
+USB device. If FDS refuses, QEMU leaves the cartridge attached. To deliberately
+simulate an accidental pull, use:
+
+```sh
+fds-emulator --session out/my-emulator unplug 01
+```
+
+A forced unplug can lose DATA writes. The guest must then detect the removal
+and clean up its processes and mounts. `status` reports the actual QEMU devices
+and block nodes as well as the recorded image paths.
+
+Use the interactive shell if preferred:
+
+```sh
+fds-emulator --session out/my-emulator console
+```
+
+At the `FDS>` prompt, type `fds bays`, `fds bay 01`, or `fds run 01 --
+demo.hello:hello`. **Ctrl-] detaches**; it does not stop the VM. Use a second
+workstation terminal for insertion/unplug. Detach the console before `guest`,
+`eject`, or ordinary `stop`, because they also need exclusive serial access.
+
+Finish with:
+
+```sh
+fds-emulator --session out/my-emulator stop
+```
+
+This invokes native FDS shutdown. `stop --force` cuts virtual power without that
+sequence. Logs and DATA overlays remain in the session directory. Choose a new
+session directory to boot again; stopped sessions are retained for inspection,
+not resumed from RAM snapshots.
+
+## Writable DATA images
+
+Insert an existing FDS DATA disk image with the same `insert` command. The
+emulator creates a unique `data-<bay>-<id>.qcow2` overlay in the session directory;
+all guest writes go there. The original image remains unchanged. Safe eject
+retains the overlay, while inserting the original image again starts a fresh
+one. Software cartridges remain read-only.
+
+After eject or shutdown, export an overlay if you need the changed DATA contents:
+
+```sh
+qemu-img convert -f qcow2 -O raw /path/to/session/data-02-IDENTIFIER.qcow2 out/saved-data.img
+```
+
+Keep the original backing image at its recorded path until conversion completes.
+Never convert or edit an image while it is attached to a running VM.
+
+## Write the completed image to USB
+
+The burn flow always starts with the complete image created above. It does not
+construct partitions directly on a drive. Identify the intended **whole USB
+drive**, unmount it, then make a preview as your normal user:
+
+```sh
+fds-cartridge preview out/demo-tools.img /dev/sdX out/usb-preview.json
+```
+
+Replace `/dev/sdX` with the actual whole USB drive. The JSON records its model,
+size, insertion identity, exact image hash and an exact `confirmation` string.
+Review these before copying the full phrase into the write command:
+
+```sh
+sudo /absolute/path/to/fds-cartridge write out/usb-preview.json \
+  --confirm 'COPY THE EXACT confirmation VALUE FROM THE PREVIEW'
+```
+
+The writer rechecks source and target identity, requires a USB whole disk,
+rejects mounted/protected storage, writes the **full image**, flushes and verifies
+readback. On larger drives it relocates the backup GPT to the end. A replaced
+USB drive or changed image requires a fresh preview. This destroys the selected
+drive's existing contents. No physical drive has been written by the automated
+acceptance tests.
+
+To rehearse on a disposable file without root or USB hardware:
+
+```sh
+truncate -s 64M out/disposable-usb.img
+fds-cartridge preview out/demo-tools.img out/disposable-usb.img \
+  out/file-preview.json --file-target
+fds-cartridge write out/file-preview.json --confirm 'COPY THE EXACT confirmation VALUE'
+fds-cartridge inspect out/disposable-usb.img
+```
+
+The file must be at least as large as the source image. Use `--image-tool-runner`
+for preview/inspection if EROFS tools are provided by the local wrapper.
+
+## Troubleshooting
+
+| Symptom | What to do |
+| --- | --- |
+| Required executable not found | Run the corresponding `doctor`; install the named host utility or select an explicit runner |
+| Bubblewrap namespace failure | Enable unprivileged user namespaces according to your workstation policy; do not run creation/inspection as root |
+| Output already exists | Use a new bundle/image/preview/session name; creation does not overwrite outputs |
+| Unknown `fds.emulator` option or missing emulator settings | Rebuild both the current initramfs and SYSTEM; the frozen 0.1.0 images predate this feature |
+| Console in use | Detach with Ctrl-] before guest commands, safe eject, or shutdown |
+| Guest command timed out | Inspect `console.log`; the command may still be running, so do not blindly repeat a write |
+| Incomplete insertion | Run `unplug BAY` to reconcile the recorded intent with actual QEMU devices, then reinsert |
+| Software digest/path/architecture error | Rebuild the bundle and cartridge on the workstation; the guest will not run an invalid archive |
+| Cache limit exceeded | Reduce the software bundle; one runtime tree including inode overhead is limited to 256 MiB |
+| Safe eject blocked | Close unmanaged processes or extra mounts using the media, then retry; `unplug` is only for deliberate failure simulation |
+| VM will not boot | Read session `console.log` and `qemu.log`; verify all three supplied boot artifacts belong to the current build |
+
+QEMU exercises the actual Linux/FDS software path. It does not emulate Pi
+firmware, RP1, USB power sequencing, the physical twelve-bay wiring, or Dasung
+power recovery. Those acceptance checks remain hardware procedures.
+
+## Physical Pi acceptance procedure (deferred)
+
+After assembling and calibrating the physical bay map, use a disposable USB
+cartridge and the current matching SYSTEM/initramfs. Perform these checks on the
+Pi; the VM results do not replace them:
+
+1. Build the hello/report cartridge on the workstation, record its image hash,
+   and use preview/confirmation to write that entire image to the chosen USB drive.
+2. Insert it in a calibrated bay. Record `fds --json bay BAY`; confirm both
+   software entries and their declared payload partitions appear.
+3. Run `demo.hello:hello` and `demo.report:report`. Save their output from the
+   cartridge log and confirm UID 1000. Check `/proc/self/mountinfo` for read-only
+   payload and software-cache mounts.
+4. Run `fds eject BAY`, verify SAFE, then remove and reinsert the cartridge in
+   another calibrated bay. Repeat both commands. Record any USB enumeration or
+   I/O errors.
+5. With a disposable cartridge and a deliberately long-running test program,
+   test surprise removal separately. Verify its managed processes terminate and
+   all of that bay's software mounts disappear. Do not use valuable DATA for
+   this failure test.
+6. Exercise native shutdown with active software, cold boot, and a real USB
+   power cycle. Save actual observations and kernel/service logs. Follow the
+   separate [Dasung hardware procedure](dasung.md) for monitor recovery.
+
+Record the Pi, hub, USB drive and kernel versions with the result. Hardware
+latency, electrical behavior and data durability remain unverified until these
+measurements are performed on the assembled machine.