fds-os/docs/developer/workstation.md

# Build software cartridges and run FDS in QEMU

Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.


This page records the earlier archive-based software prototype. For installed
Void software trees, the Arch `fds-tools` package, and reusable emulator sessions,
follow the [current walkthrough](../workstation.md).

This historical guide uses a **Linux workstation**, not the Raspberry Pi. It takes you from
source software to a cartridge disk image, runs that image in FDS, and explains
how to write the same complete image to USB. All commands run from the checkout
root unless marked as guest commands.

The two native host tools are `fds-cartridge` and `fds-emulator`. They use typed
Clap interfaces; append `--help` to any command or subcommand. The target's `fds`
command remains a static AArch64 executable. The new host tools are never part
of the Pi base image. Dasung support remains in the base boot bundle.

The accepted local `out/fds-os-0.1.0/` release predates this extension. Leave that
release intact and build current images for the emulator. See
[workstation acceptance](workstation-validation.md) for current evidence.

For accumulated build and test images, use `make clean-preview` and `make clean`.
Current images, native tools, latest test fixtures and personal emulator sessions
are kept; obsolete automated test sessions are disposable. See
[Cleanup](cleanup.md) for the exact scope and how to preserve an older test run.

## Install and build the workstation tools

Use a normal Linux account with Rust/Cargo, a C linker, Bash and Make. The
repository's `rust-toolchain.toml` selects the Rust version. Runtime prerequisites
are:

| Tool | Purpose |
| --- | --- |
| `qemu-system-aarch64` | Runs the FDS ARM kernel and userspace using portable TCG |
| `qemu-img` | Creates a separate writable overlay for DATA; source images stay unchanged |
| `mkfs.erofs`, `fsck.erofs` from erofs-utils | Build and verify cartridge partitions; tested with 1.9.4 |
| `xz` | Compress/decompress software tarballs with a decompression memory limit |
| `bwrap` from bubblewrap | Restricts filesystem inspection to a private writable staging directory |
| A software-specific cross compiler | Builds AArch64 applications; unnecessary for portable scripts |
| `sfdisk`, `tar`, `mke2fs`, Python 3 | Independent integration tests; not needed to boot an existing image |

Install these with your distribution's package manager. Unprivileged user
namespaces must work for bubblewrap. Building **these host tools** does not need
Void, XBPS, an Arch host, a Pi, root, or a running FDS guest:

```sh
make workstation
export PATH="$PWD/out/workstation:$PATH"
fds-cartridge doctor
fds-emulator doctor
```

Current acceptance ran on Arch Linux x86_64. The tools use native Linux
interfaces and distribution-provided utilities; other distributions and ARM
workstations have not yet been exercised by this acceptance run.

The outputs are native binaries in `out/workstation/`. The build explicitly
selects the workstation architecture instead of the workspace's default ARM
target. Rust dependencies are pinned in `Cargo.lock`. `tar` 0.4.46 and its
`filetime` dependency provide archive decoding/creation; FDS additionally checks
entry types, paths, counts, sizes, executable architecture and hashes.

For this repository's **existing Arch/Void setup**, reuse its local tools:

```sh
make workstation
./tools/prepare-image-tools
./tools/in-void xbps-install -y qemu-img
out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools doctor
out/workstation/fds-emulator doctor --qemu-runner tools/in-void
```

The install command above changes only the project-local build container. The
optional runners accept a program name followed by its arguments; omitting them
uses programs in the workstation's `PATH`. Set the image runner on every
cartridge create, inspect or preview invocation that needs it. A QEMU runner is
saved in the session and reused for later DATA insertions.

## Build two software bundles

Each software recipe declares an id, display name, version, target architecture,
root directory, and named executable commands. A trusted `[build]` section may
run a workstation compiler. It is executed only by `software build`, never by
`software pack`, inspection, or a cartridge insertion.

The [hello recipe](../../examples/software/hello/software.toml) builds an AArch64
C executable. Install an AArch64 glibc cross compiler as `aarch64-linux-gnu-gcc`,
then run:

```sh
fds-cartridge software build examples/software/hello/software.toml out/demo-hello
fds-cartridge software pack examples/software/report/software.toml out/demo-report
fds-cartridge software inspect out/demo-hello
```

If your cross compiler has a different name, edit the recipe's `build.command`.
With this checkout's existing Void cross toolchain, the equivalent is:

```sh
mkdir -p examples/software/hello/root/bin
./tools/in-void aarch64-linux-gnu-gcc -O2 examples/software/hello/hello.c \
  -o examples/software/hello/root/bin/hello
out/workstation/fds-cartridge software pack \
  examples/software/hello/software.toml out/demo-hello
out/workstation/fds-cartridge software pack \
  examples/software/report/software.toml out/demo-report
```

Use **one** of those hello workflows. Outputs must not already exist: choose
new output names when rebuilding, then update your cartridge recipe. The example
bundles each contain `software.toml` and `<id>.tar.xz`. The archive contains the
contents of the software root, such as `bin/hello`; it has no leading `root/`.

For your own application, copy a recipe and populate its root with `bin/`,
`lib/` and `share/` as needed. C/C++ glibc builds target AArch64; static Rust
programs should use `aarch64-unknown-linux-musl`. Set `architecture = "any"` only
for scripts/data without ELF files. An x86_64 executable is rejected. Shared
libraries must be compatible with the target SYSTEM's glibc, or bundled as
appropriate. There is no cross-cartridge dependency resolver.

## Assemble the cartridge image

The example cartridge recipe groups the two bundles into two payload partitions:

```sh
fds-cartridge create examples/software/cartridge.toml out/demo-tools.img
fds-cartridge inspect out/demo-tools.img
```

For the project-local EROFS tools, use:

```sh
out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools \
  create examples/software/cartridge.toml out/demo-tools.img
```

The resulting complete disk image has **three GPT partitions**:

| Partition | GPT name | Contents |
| --- | --- | --- |
| 1 | `FDS_METADATA` | Cartridge identity and the software catalogue |
| 2 | `FDS_PAYLOAD02` | `bundles/demo.hello.tar.xz` |
| 3 | `FDS_PAYLOAD03` | `bundles/demo.report.tar.xz` |

To put both programs in one payload partition, use one `[[payload]]` entry with
`bundles = ["../../out/demo-hello", "../../out/demo-report"]`. This makes a
two-partition image. Each payload group creates exactly one partition. The first
partition identifies every software bundle and its payload partition.

Creation finishes only after verifying both GPT tables, all EROFS partitions,
every archive and the catalogue. Inspection prints JSON, including offsets,
lengths, SHA-256 and available commands. Repeating an unchanged recipe produces
the same image bytes. See [the format](software-format.md) for limits and rules.

## Boot current FDS in the emulator

You need three matching current artifacts: the FDS Pi kernel, its initramfs, and
a SYSTEM image containing the guest software runtime. Build these on the OS
build workstation using [the build guide](getting-started.md). For an already
bootstrapped checkout:

```sh
make rootfs PROFILE=cli
make system-card PROFILE=cli
make initramfs
make workstation
out/workstation/fds-emulator --session out/my-emulator start \
  --qemu-runner tools/in-void
```

If QEMU is installed directly on your Linux workstation, omit `--qemu-runner`.
To use images copied from another build machine, supply their paths:

```sh
fds-emulator --session out/my-emulator start \
  --kernel /path/to/kernel_2712.img \
  --initramfs /path/to/fds-initramfs.img \
  --system /path/to/fds-system-cli.img
```

Create `out/` first if using a fresh directory. The session directory must be
**new**, private, and short enough for Unix sockets (under 90 bytes including its
absolute parent path). Start waits for the FDS prompt and verifies the emulator
bay configuration. It uses two emulated CPUs and 1024 MiB RAM by default;
`--memory-mib 2048` increases RAM. `--timeout` adjusts the boot deadline.

The session records inputs, virtual devices, serial output and QEMU diagnostics.
QEMU stays running after the command exits. SYSTEM and software images are
read-only. No physical disks, host network interface, or Pi monitor are attached.

## Insert, use and remove cartridges

From the workstation:

```sh
fds-emulator --session out/my-emulator insert 01 out/demo-tools.img
fds-emulator --session out/my-emulator guest -- fds bay 01
fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.hello:hello
fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.report:report
fds-emulator --session out/my-emulator guest -- tail -20 /run/log/cartridged/current
fds-emulator --session out/my-emulator eject 01
```

Insertion is asynchronous: `fds bay 01` may briefly show `EMPTY` before USB
storage discovery completes. Retry the status command until it reports
`MOUNTED READ ONLY`, or inspect its error. The catalogue lists exact commands.
`fds run` starts a managed process and prints its PID; its output goes to the
cartridge service log. Packages are verified and extracted into temporary,
read-only RAM filesystems on first run. They are not compiled on the guest.

All twelve bays, numbered `01`–`12`, are available. Simultaneously mounted
PROGRAM cartridges need distinct cartridge IDs; inserting duplicate IDs is
reported as a guest error. `eject` asks FDS to stop
managed consumers, release mounts and declare `SAFE`, then removes the virtual
USB device. If FDS refuses, QEMU leaves the cartridge attached. To deliberately
simulate an accidental pull, use:

```sh
fds-emulator --session out/my-emulator unplug 01
```

A forced unplug can lose DATA writes. The guest must then detect the removal
and clean up its processes and mounts. `status` reports the actual QEMU devices
and block nodes as well as the recorded image paths.

Use the interactive shell if preferred:

```sh
fds-emulator --session out/my-emulator console
```

At the `FDS>` prompt, type `fds bays`, `fds bay 01`, or `fds run 01 --
demo.hello:hello`. **Ctrl-] detaches**; it does not stop the VM. Use a second
workstation terminal for insertion/unplug. Detach the console before `guest`,
`eject`, or ordinary `stop`, because they also need exclusive serial access.

Finish with:

```sh
fds-emulator --session out/my-emulator stop
```

This invokes native FDS shutdown. `stop --force` cuts virtual power without that
sequence. Logs and DATA overlays remain in the session directory. Choose a new
session directory to boot again; stopped sessions are retained for inspection,
not resumed from RAM snapshots.

## Writable DATA images

Insert an existing FDS DATA disk image with the same `insert` command. The
emulator creates a unique `data-<bay>-<id>.qcow2` overlay in the session directory;
all guest writes go there. The original image remains unchanged. Safe eject
retains the overlay, while inserting the original image again starts a fresh
one. Software cartridges remain read-only.

After eject or shutdown, export an overlay if you need the changed DATA contents:

```sh
qemu-img convert -f qcow2 -O raw /path/to/session/data-02-IDENTIFIER.qcow2 out/saved-data.img
```

Keep the original backing image at its recorded path until conversion completes.
Never convert or edit an image while it is attached to a running VM.

## Write the completed image to USB

The burn flow always starts with the complete image created above. It does not
construct partitions directly on a drive. Identify the intended **whole USB
drive**, unmount it, then make a preview as your normal user:

```sh
fds-cartridge preview out/demo-tools.img /dev/sdX out/usb-preview.json
```

Replace `/dev/sdX` with the actual whole USB drive. The JSON records its model,
size, insertion identity, exact image hash and an exact `confirmation` string.
Review these before copying the full phrase into the write command:

```sh
sudo /absolute/path/to/fds-cartridge write out/usb-preview.json \
  --confirm 'COPY THE EXACT confirmation VALUE FROM THE PREVIEW'
```

The writer rechecks source and target identity, requires a USB whole disk,
rejects mounted/protected storage, writes the **full image**, flushes and verifies
readback. On larger drives it relocates the backup GPT to the end. A replaced
USB drive or changed image requires a fresh preview. This destroys the selected
drive's existing contents. No physical drive has been written by the automated
acceptance tests.

To rehearse on a disposable file without root or USB hardware:

```sh
truncate -s 64M out/disposable-usb.img
fds-cartridge preview out/demo-tools.img out/disposable-usb.img \
  out/file-preview.json --file-target
fds-cartridge write out/file-preview.json --confirm 'COPY THE EXACT confirmation VALUE'
fds-cartridge inspect out/disposable-usb.img
```

The file must be at least as large as the source image. Use `--image-tool-runner`
for preview/inspection if EROFS tools are provided by the local wrapper.

## Troubleshooting

| Symptom | What to do |
| --- | --- |
| Required executable not found | Run the corresponding `doctor`; install the named host utility or select an explicit runner |
| Bubblewrap namespace failure | Enable unprivileged user namespaces according to your workstation policy; do not run creation/inspection as root |
| Output already exists | Use a new bundle/image/preview/session name; creation does not overwrite outputs |
| Unknown `fds.emulator` option or missing emulator settings | Rebuild both the current initramfs and SYSTEM; the frozen 0.1.0 images predate this feature |
| Console in use | Detach with Ctrl-] before guest commands, safe eject, or shutdown |
| Guest command timed out | Inspect `console.log`; the command may still be running, so do not blindly repeat a write |
| Incomplete insertion | Run `unplug BAY` to reconcile the recorded intent with actual QEMU devices, then reinsert |
| Software digest/path/architecture error | Rebuild the bundle and cartridge on the workstation; the guest will not run an invalid archive |
| Cache limit exceeded | Reduce the software bundle; one runtime tree including inode overhead is limited to 256 MiB |
| Safe eject blocked | Close unmanaged processes or extra mounts using the media, then retry; `unplug` is only for deliberate failure simulation |
| VM will not boot | Read session `console.log` and `qemu.log`; verify all three supplied boot artifacts belong to the current build |

QEMU exercises the actual Linux/FDS software path. It does not emulate Pi
firmware, RP1, USB power sequencing, the physical twelve-bay wiring, or Dasung
power recovery. Those acceptance checks remain hardware procedures.

## Physical Pi acceptance procedure (deferred)

After assembling and calibrating the physical bay map, use a disposable USB
cartridge and the current matching SYSTEM/initramfs. Perform these checks on the
Pi; the VM results do not replace them:

1. Build the hello/report cartridge on the workstation, record its image hash,
   and use preview/confirmation to write that entire image to the chosen USB drive.
2. Insert it in a calibrated bay. Record `fds --json bay BAY`; confirm both
   software entries and their declared payload partitions appear.
3. Run `demo.hello:hello` and `demo.report:report`. Save their output from the
   cartridge log and confirm UID 1000. Check `/proc/self/mountinfo` for read-only
   payload and software-cache mounts.
4. Run `fds eject BAY`, verify SAFE, then remove and reinsert the cartridge in
   another calibrated bay. Repeat both commands. Record any USB enumeration or
   I/O errors.
5. With a disposable cartridge and a deliberately long-running test program,
   test surprise removal separately. Verify its managed processes terminate and
   all of that bay's software mounts disappear. Do not use valuable DATA for
   this failure test.
6. Exercise native shutdown with active software, cold boot, and a real USB
   power cycle. Save actual observations and kernel/service logs. Follow the
   separate [Dasung hardware procedure](dasung.md) for monitor recovery.

Record the Pi, hub, USB drive and kernel versions with the result. Hardware
latency, electrical behavior and data durability remain unverified until these
measurements are performed on the assembled machine.