felis/fds-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update docs

Browse Source
This commit is contained in:
felis
2026-09-22 13:23:34 +08:00
parent 99bc3d15c5
commit 8a4788fca8
126 changed files with 7198 additions and 2425 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/workstation.md
+146 -255
View File
@@ -1,166 +1,133 @@
# Build software cartridges and run FDS in QEMU
# Build software cartridges and use the emulator
This guide uses a **Linux workstation**, not the Raspberry Pi. It takes you from
source software to a cartridge disk image, runs that image in FDS, and explains
how to write the same complete image to USB. All commands run from the checkout
root unless marked as guest commands.
This walkthrough runs on a Linux workstation. It builds applications from Void
source packages, creates a complete cartridge image, runs it in FDS, and explains
how to write it to USB. Commands run from the FDS checkout root unless marked
as guest commands.
The two native host tools are `fds-cartridge` and `fds-emulator`. They use typed
Clap interfaces; append `--help` to any command or subcommand. The target's `fds`
command remains a static AArch64 executable. The new host tools are never part
of the Pi base image. Dasung support remains in the base boot bundle.
## Prepare the tools
The accepted local `out/fds-os-0.1.0/` release predates this extension. Leave that
release intact and build current images for the emulator. See
[workstation acceptance](workstation-validation.md) for current evidence.
## Install and build the workstation tools
Use a normal Linux account with Rust/Cargo, a C linker, Bash and Make. The
repository's `rust-toolchain.toml` selects the Rust version. Runtime prerequisites
are:
| Tool | Purpose |
| --- | --- |
| `qemu-system-aarch64` | Runs the FDS ARM kernel and userspace using portable TCG |
| `qemu-img` | Creates a separate writable overlay for DATA; source images stay unchanged |
| `mkfs.erofs`, `fsck.erofs` from erofs-utils | Build and verify cartridge partitions; tested with 1.9.4 |
| `xz` | Compress/decompress software tarballs with a decompression memory limit |
| `bwrap` from bubblewrap | Restricts filesystem inspection to a private writable staging directory |
| A software-specific cross compiler | Builds AArch64 applications; unnecessary for portable scripts |
| `sfdisk`, `tar`, `mke2fs`, Python 3 | Independent integration tests; not needed to boot an existing image |
Install these with your distribution's package manager. Unprivileged user
namespaces must work for bubblewrap. Building **these host tools** does not need
Void, XBPS, an Arch host, a Pi, root, or a running FDS guest:
The native tools are `fds-cartridge` and `fds-emulator`. Build them with Rust,
a C linker and Make; the repository selects its pinned Rust toolchain:
```sh
make workstation
mkdir -p out
export PATH="$PWD/out/workstation:$PATH"
fds-cartridge doctor
fds-emulator doctor
```
Current acceptance ran on Arch Linux x86_64. The tools use native Linux
interfaces and distribution-provided utilities; other distributions and ARM
workstations have not yet been exercised by this acceptance run.
| Prerequisite | Used for |
| --- | --- |
| Prepared Void `void-packages` checkout and native XBPS utilities | Build source packages and install their AArch64 runtime dependencies |
| `mkfs.erofs`, `fsck.erofs` | Create and inspect read-only payload filesystems |
| `bwrap` | Inspect filesystems and install package trees in a user namespace |
| `xz` | Read older software cartridges and package/source archives |
| `qemu-system-aarch64`, `qemu-img` | Run the emulator and create writable DATA overlays |
The outputs are native binaries in `out/workstation/`. The build explicitly
selects the workstation architecture instead of the workspace's default ARM
target. Rust dependencies are pinned in `Cargo.lock`. `tar` 0.4.46 and its
`filetime` dependency provide archive decoding/creation; FDS additionally checks
entry types, paths, counts, sizes, executable architecture and hashes.
Install tools through your Linux distribution's package manager. Enable
unprivileged user namespaces for bubblewrap. Software builds run as an ordinary
user and execute trusted Void build templates on the workstation. Run source builds sequentially when they
share the same Void checkout.
For this repository's **existing Arch/Void setup**, reuse its local tools:
Prepare a Void source checkout using its [upstream instructions](https://github.com/void-linux/void-packages/blob/02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1/README.md).
Complete `xbps-src binary-bootstrap` for the workstation architecture before
using it. The workstation itself need not run Void. Select that checkout with
`--void-packages /path/to/void-packages`; native `xbps-*` tools must be in PATH,
or selected with `--xbps-bin /path/to/bin`. Builds target AArch64/glibc.
For the complete FDS build workstation, [bootstrap](getting-started.md) already
prepares `vendor/void-packages` and `.host/xbps/usr/bin`. Reuse its local tools:
```sh
make workstation
./tools/prepare-image-tools
./tools/in-void xbps-install -y qemu-img
out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools doctor
out/workstation/fds-emulator doctor --qemu-runner tools/in-void
fds-cartridge --image-tool-runner tools/in-image-tools doctor
fds-emulator doctor --qemu-runner tools/in-void
```
The install command above changes only the project-local build container. The
optional runners accept a program name followed by its arguments; omitting them
uses programs in the workstation's `PATH`. Set the image runner on every
cartridge create, inspect or preview invocation that needs it. A QEMU runner is
saved in the session and reused for later DATA insertions.
An optional runner receives a utility name followed by its arguments. Image
runners select EROFS utilities; XBPS runners provide the package-installation
namespace. Without runners, the tools use native utilities and bubblewrap.
## Build two software bundles
## Build the example cartridge
Each software recipe declares an id, display name, version, target architecture,
root directory, and named executable commands. A trusted `[build]` section may
run a workstation compiler. It is executed only by `software build`, never by
`software pack`, inspection, or a cartridge insertion.
The repository includes hello and system-report Void source packages. Each
software recipe names a package, its source-template directory and its public
commands. A cartridge recipe groups those software recipes into payloads.
The [hello recipe](../examples/software/hello/software.toml) builds an AArch64
C executable. Install an AArch64 glibc cross compiler as `aarch64-linux-gnu-gcc`,
then run:
With native tools and a prepared checkout:
```sh
fds-cartridge software build examples/software/hello/software.toml out/demo-hello
fds-cartridge software pack examples/software/report/software.toml out/demo-report
fds-cartridge software inspect out/demo-hello
```
If your cross compiler has a different name, edit the recipe's `build.command`.
With this checkout's existing Void cross toolchain, the equivalent is:
```sh
mkdir -p examples/software/hello/root/bin
./tools/in-void aarch64-linux-gnu-gcc -O2 examples/software/hello/hello.c \
-o examples/software/hello/root/bin/hello
out/workstation/fds-cartridge software pack \
examples/software/hello/software.toml out/demo-hello
out/workstation/fds-cartridge software pack \
examples/software/report/software.toml out/demo-report
```
Use **one** of those hello workflows. Outputs must not already exist: choose
new output names when rebuilding, then update your cartridge recipe. The example
bundles each contain `software.toml` and `<id>.tar.xz`. The archive contains the
contents of the software root, such as `bin/hello`; it has no leading `root/`.
For your own application, copy a recipe and populate its root with `bin/`,
`lib/` and `share/` as needed. C/C++ glibc builds target AArch64; static Rust
programs should use `aarch64-unknown-linux-musl`. Set `architecture = "any"` only
for scripts/data without ELF files. An x86_64 executable is rejected. Shared
libraries must be compatible with the target SYSTEM's glibc, or bundled as
appropriate. There is no cross-cartridge dependency resolver.
## Assemble the cartridge image
The example cartridge recipe groups the two bundles into two payload partitions:
```sh
fds-cartridge create examples/software/cartridge.toml out/demo-tools.img
fds-cartridge --void-packages /path/to/void-packages \
create examples/software/cartridge.toml out/demo-tools.img
fds-cartridge inspect out/demo-tools.img
```
For the project-local EROFS tools, use:
With the project-local Void and image tools, use this equivalent command:
```sh
out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools \
fds-cartridge --image-tool-runner tools/in-image-tools \
--xbps-tool-runner tools/in-void --xbps-bin "$PWD/.host/xbps/usr/bin" \
create examples/software/cartridge.toml out/demo-tools.img
fds-cartridge --image-tool-runner tools/in-image-tools inspect out/demo-tools.img
```
The resulting complete disk image has **three GPT partitions**:
Choose one route. Output paths must be new. The command builds each package
with `xbps-src`, installs the package and runtime dependencies into private
staging trees, verifies their architecture and integrity, and writes the complete
image. Source build logs go to standard error; the final result is JSON.
| Partition | GPT name | Contents |
| --- | --- | --- |
| 1 | `FDS_METADATA` | Cartridge identity and the software catalogue |
| 2 | `FDS_PAYLOAD02` | `bundles/demo.hello.tar.xz` |
| 3 | `FDS_PAYLOAD03` | `bundles/demo.report.tar.xz` |
The example image contains:
To put both programs in one payload partition, use one `[[payload]]` entry with
`bundles = ["../../out/demo-hello", "../../out/demo-report"]`. This makes a
two-partition image. Each payload group creates exactly one partition. The first
partition identifies every software bundle and its payload partition.
| Partition | Contents |
| --- | --- |
| 1: `FDS_METADATA` | Cartridge identity and software catalogue |
| 2: `FDS_PAYLOAD02` | `programs/demo.hello/`, including `usr/bin/hello` and libraries |
| 3: `FDS_PAYLOAD03` | `programs/demo.report/`, including `usr/bin/report` and dependencies |
Creation finishes only after verifying both GPT tables, all EROFS partitions,
every archive and the catalogue. Inspection prints JSON, including offsets,
lengths, SHA-256 and available commands. Repeating an unchanged recipe produces
the same image bytes. See [the format](software-format.md) for limits and rules.
There are no new xz software bundles. Programs are ready to run directly from
EROFS. Inspection validates both GPT tables, filesystem contents and software
hashes, then prints partition and command details as JSON.
## Boot current FDS in the emulator
## Build a reusable installed-software directory
You need three matching current artifacts: the FDS Pi kernel, its initramfs, and
a SYSTEM image containing the guest software runtime. Build these on the OS
build workstation using [the build guide](getting-started.md). For an already
bootstrapped checkout:
To separate package compilation from cartridge assembly:
```sh
make rootfs PROFILE=cli
make system-card PROFILE=cli
make initramfs
make workstation
out/workstation/fds-emulator --session out/my-emulator start \
--qemu-runner tools/in-void
fds-cartridge --xbps-tool-runner tools/in-void \
--xbps-bin "$PWD/.host/xbps/usr/bin" \
software build examples/software/hello/software.toml out/demo-hello
fds-cartridge software inspect out/demo-hello
```
If QEMU is installed directly on your Linux workstation, omit `--qemu-runner`.
To use images copied from another build machine, supply their paths:
The output contains `software.toml` and `root/`. A cartridge's `sources` list can
name this directory instead of a source recipe. For example:
```toml
format = 2
id = "my.tools"
name = "My tools"
version = "1.0"
[[payload]]
sources = ["../out/demo-hello"]
```
Paths are relative to the cartridge recipe. Put several sources in one list to
share a partition, or add `[[payload]]` sections for separate partitions. See
[the format reference](software-format.md) for complete source metadata and limits.
## Start FDS
Build matching boot inputs using [Build and start FDS](getting-started.md), or
copy a kernel, initramfs and SYSTEM image from an FDS build machine.
```sh
fds-emulator --session out/my-emulator start --qemu-runner tools/in-void
```
Omit `--qemu-runner` when QEMU is installed directly on the workstation. To use
copied boot images:
```sh
fds-emulator --session out/my-emulator start \
@@ -169,173 +136,97 @@ fds-emulator --session out/my-emulator start \
--system /path/to/fds-system-cli.img
```
Create `out/` first if using a fresh directory. The session directory must be
**new**, private, and short enough for Unix sockets (under 90 bytes including its
absolute parent path). Start waits for the FDS prompt and verifies the emulator
bay configuration. It uses two emulated CPUs and 1024 MiB RAM by default;
`--memory-mib 2048` increases RAM. `--timeout` adjusts the boot deadline.
The session directory must be new and private. Keep its absolute path short
(under 90 bytes) for Unix sockets. Start waits for the FDS prompt and uses two
emulated CPUs and 1024 MiB RAM; `--memory-mib 2048` increases memory. SYSTEM and
PROGRAM images are read-only. The session holds logs and any DATA overlays.
The session records inputs, virtual devices, serial output and QEMU diagnostics.
QEMU stays running after the command exits. SYSTEM and software images are
read-only. No physical disks, host network interface, or Pi monitor are attached.
## Insert, use and remove cartridges
## Insert and run software
From the workstation:
```sh
fds-emulator --session out/my-emulator insert 01 out/demo-tools.img
fds-emulator --session out/my-emulator guest -- fds bay 01
fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.hello:hello
fds-emulator --session out/my-emulator guest -- hello 'Hello from the emulator'
fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.report:report
```
Discovery is asynchronous. Wait until the bay reports `MOUNTED READ ONLY` before
running commands. Direct commands return their output and exit status to the
caller. `fds run` starts a background job; read its output with:
```sh
fds-emulator --session out/my-emulator guest -- tail -20 /run/log/cartridged/current
```
Open an interactive shell with `fds-emulator --session out/my-emulator console`.
At `FDS>`, use `hello`, `report`, `fds bays` or other guest commands. **Ctrl-]**
detaches without stopping the VM. Detach before using `guest`, `eject` or ordinary
`stop`, since they also need the serial console.
All twelve virtual bays accept images. Simultaneously inserted PROGRAM cartridges
must have distinct cartridge IDs. Duplicate identities produce a guest error.
Command-name collisions are resolved as described in [Using cartridges](cartridges.md).
## Eject and stop
```sh
fds-emulator --session out/my-emulator eject 01
```
Insertion is asynchronous: `fds bay 01` may briefly show `EMPTY` before USB
storage discovery completes. Retry the status command until it reports
`MOUNTED READ ONLY`, or inspect its error. The catalogue lists exact commands.
`fds run` starts a managed process and prints its PID; its output goes to the
cartridge service log. Packages are verified and extracted into temporary,
read-only RAM filesystems on first run. They are not compiled on the guest.
All twelve bays, numbered `01`–`12`, are available. Simultaneously mounted
PROGRAM cartridges need distinct cartridge IDs; inserting duplicate IDs is
reported as a guest error. `eject` asks FDS to stop
managed consumers, release mounts and declare `SAFE`, then removes the virtual
USB device. If FDS refuses, QEMU leaves the cartridge attached. To deliberately
simulate an accidental pull, use:
```sh
fds-emulator --session out/my-emulator unplug 01
```
A forced unplug can lose DATA writes. The guest must then detect the removal
and clean up its processes and mounts. `status` reports the actual QEMU devices
and block nodes as well as the recorded image paths.
Use the interactive shell if preferred:
```sh
fds-emulator --session out/my-emulator console
```
At the `FDS>` prompt, type `fds bays`, `fds bay 01`, or `fds run 01 --
demo.hello:hello`. **Ctrl-] detaches**; it does not stop the VM. Use a second
workstation terminal for insertion/unplug. Detach the console before `guest`,
`eject`, or ordinary `stop`, because they also need exclusive serial access.
Finish with:
```sh
fds-emulator --session out/my-emulator status
fds-emulator --session out/my-emulator stop
```
This invokes native FDS shutdown. `stop --force` cuts virtual power without that
sequence. Logs and DATA overlays remain in the session directory. Choose a new
session directory to boot again; stopped sessions are retained for inspection,
not resumed from RAM snapshots.
Eject asks FDS to stop programs and release storage before removing the virtual
USB device. If FDS refuses, the cartridge stays attached. `stop` performs native
FDS shutdown. Session logs remain available; create a new session to boot again.
## Writable DATA images
`unplug BAY` simulates a physical pull, and `stop --force` cuts virtual power.
Use these only for deliberate failure simulation; DATA writes can be lost.
Insert an existing FDS DATA disk image with the same `insert` command. The
emulator creates a unique `data-<bay>-<id>.qcow2` overlay in the session directory;
all guest writes go there. The original image remains unchanged. Safe eject
retains the overlay, while inserting the original image again starts a fresh
one. Software cartridges remain read-only.
## Save emulator DATA
After eject or shutdown, export an overlay if you need the changed DATA contents:
Inserting a DATA image creates a writable `data-<bay>-<id>.qcow2` overlay in the
session directory. Eject retains it. Reinserting the original image creates a
fresh overlay. To export the changed contents after eject or shutdown:
```sh
qemu-img convert -f qcow2 -O raw /path/to/session/data-02-IDENTIFIER.qcow2 out/saved-data.img
qemu-img convert -f qcow2 -O raw /path/to/session/data-02-ID.qcow2 out/saved-data.img
```
Keep the original backing image at its recorded path until conversion completes.
Never convert or edit an image while it is attached to a running VM.
Keep its backing image at the recorded path until export completes. Never convert
an overlay while a running VM is using it. Personal session directories are
preserved by `make clean`; keep them under your own names, such as `my-emulator`.
## Write the completed image to USB
## Write a cartridge to USB
The burn flow always starts with the complete image created above. It does not
construct partitions directly on a drive. Identify the intended **whole USB
drive**, unmount it, then make a preview as your normal user:
Identify the intended whole USB drive, unmount its partitions, then preview:
```sh
fds-cartridge preview out/demo-tools.img /dev/sdX out/usb-preview.json
```
Replace `/dev/sdX` with the actual whole USB drive. The JSON records its model,
size, insertion identity, exact image hash and an exact `confirmation` string.
Review these before copying the full phrase into the write command:
Replace `/dev/sdX` with the actual destination. The JSON identifies the drive,
capacity, image hash and exact confirmation phrase. Check those values before
writing: this replaces the selected drive's contents.
```sh
sudo /absolute/path/to/fds-cartridge write out/usb-preview.json \
--confirm 'COPY THE EXACT confirmation VALUE FROM THE PREVIEW'
```
The writer rechecks source and target identity, requires a USB whole disk,
rejects mounted/protected storage, writes the **full image**, flushes and verifies
readback. On larger drives it relocates the backup GPT to the end. A replaced
USB drive or changed image requires a fresh preview. This destroys the selected
drive's existing contents. No physical drive has been written by the automated
acceptance tests.
The writer rechecks identity, writes the entire disk image, flushes it and verifies
readback. On a larger drive it moves the backup GPT to the end. A changed image
or reinserted drive requires a fresh preview. Use the image-tool runner for
preview and inspection if your EROFS utilities are in the local container.
To rehearse on a disposable file without root or USB hardware:
To rehearse with a disposable regular file:
```sh
truncate -s 64M out/disposable-usb.img
fds-cartridge preview out/demo-tools.img out/disposable-usb.img \
out/file-preview.json --file-target
truncate -s "$(stat -c %s out/demo-tools.img)" out/disposable-usb.img
fds-cartridge preview out/demo-tools.img out/disposable-usb.img out/file-preview.json --file-target
fds-cartridge write out/file-preview.json --confirm 'COPY THE EXACT confirmation VALUE'
fds-cartridge inspect out/disposable-usb.img
```
The file must be at least as large as the source image. Use `--image-tool-runner`
for preview/inspection if EROFS tools are provided by the local wrapper.
## Troubleshooting
| Symptom | What to do |
| --- | --- |
| Required executable not found | Run the corresponding `doctor`; install the named host utility or select an explicit runner |
| Bubblewrap namespace failure | Enable unprivileged user namespaces according to your workstation policy; do not run creation/inspection as root |
| Output already exists | Use a new bundle/image/preview/session name; creation does not overwrite outputs |
| Unknown `fds.emulator` option or missing emulator settings | Rebuild both the current initramfs and SYSTEM; the frozen 0.1.0 images predate this feature |
| Console in use | Detach with Ctrl-] before guest commands, safe eject, or shutdown |
| Guest command timed out | Inspect `console.log`; the command may still be running, so do not blindly repeat a write |
| Incomplete insertion | Run `unplug BAY` to reconcile the recorded intent with actual QEMU devices, then reinsert |
| Software digest/path/architecture error | Rebuild the bundle and cartridge on the workstation; the guest will not run an invalid archive |
| Cache limit exceeded | Reduce the software bundle; one runtime tree including inode overhead is limited to 256 MiB |
| Safe eject blocked | Close unmanaged processes or extra mounts using the media, then retry; `unplug` is only for deliberate failure simulation |
| VM will not boot | Read session `console.log` and `qemu.log`; verify all three supplied boot artifacts belong to the current build |
QEMU exercises the actual Linux/FDS software path. It does not emulate Pi
firmware, RP1, USB power sequencing, the physical twelve-bay wiring, or Dasung
power recovery. Those acceptance checks remain hardware procedures.
## Physical Pi acceptance procedure (deferred)
After assembling and calibrating the physical bay map, use a disposable USB
cartridge and the current matching SYSTEM/initramfs. Perform these checks on the
Pi; the VM results do not replace them:
1. Build the hello/report cartridge on the workstation, record its image hash,
and use preview/confirmation to write that entire image to the chosen USB drive.
2. Insert it in a calibrated bay. Record `fds --json bay BAY`; confirm both
software entries and their declared payload partitions appear.
3. Run `demo.hello:hello` and `demo.report:report`. Save their output from the
cartridge log and confirm UID 1000. Check `/proc/self/mountinfo` for read-only
payload and software-cache mounts.
4. Run `fds eject BAY`, verify SAFE, then remove and reinsert the cartridge in
another calibrated bay. Repeat both commands. Record any USB enumeration or
I/O errors.
5. With a disposable cartridge and a deliberately long-running test program,
test surprise removal separately. Verify its managed processes terminate and
all of that bay's software mounts disappear. Do not use valuable DATA for
this failure test.
6. Exercise native shutdown with active software, cold boot, and a real USB
power cycle. Save actual observations and kernel/service logs. Follow the
separate [Dasung hardware procedure](dasung.md) for monitor recovery.
Record the Pi, hub, USB drive and kernel versions with the result. Hardware
latency, electrical behavior and data durability remain unverified until these
measurements are performed on the assembled machine.
See [troubleshooting](troubleshooting.md) for failed builds, namespace errors,
busy consoles, invalid media and blocked ejects.