diff --git a/Asset-images-fds-control.png b/Asset-images-fds-control.png new file mode 100644 index 0000000..e53bd26 Binary files /dev/null and b/Asset-images-fds-control.png differ diff --git a/Cartridges.md b/Cartridges.md new file mode 100644 index 0000000..b2b9692 --- /dev/null +++ b/Cartridges.md @@ -0,0 +1,102 @@ +--- +include_toc: true +--- + +# Use cartridges and run programs + +A bay number identifies a physical slot, from `01` through `12`. It stays tied +to the slot rather than the order in which Linux discovers USB disks. + +| Cartridge | Role | +| --- | --- | +| SYSTEM | Supplies the read-only operating system used at boot | +| PROGRAM | Supplies applications and their runtime dependencies | +| DATA | Stores your writable files at `/data` | +| ENVIRONMENT | Requests an installed desktop profile, such as WindowMaker | + +## Inspect a bay + +Insert the cartridge, then run: + +```sh +fds bays +fds bay 01 +``` + +USB discovery takes place asynchronously. `fds bay` shows its state, identity, +mount location, running process count and software catalogue. `fds --json bay 01` +returns structured details, including command aliases. `fds rescan` refreshes +the inventory. No application starts merely because a PROGRAM cartridge is inserted. + +| State | Meaning and next action | +| --- | --- | +| EMPTY | No cartridge is detected | +| MOUNTED READ ONLY | Contents are available; PROGRAM commands can run | +| MOUNTED READ WRITE | DATA is active at `/data` | +| SAFE | Storage is released; remove the cartridge | +| PROTECTED | This cartridge supplies the running SYSTEM; shut down before removal | +| UNCONFIGURED | The physical slot needs a bay mapping | +| ERROR / AMBIGUOUS | Read the detail; correct the media or mapping before use | + +## Run a software command + +For the included hello/report cartridge: + +```sh +hello 'Hello from my cartridge' +report +fds run 01 -- demo.hello:hello 'Hello from my cartridge' +``` + +Direct commands run in the foreground with your terminal, input/output streams, +arguments, working directory and exit code. Pipes and redirection work normally. +`fds run` starts a managed background program and prints its PID; its output is +written to `/run/log/cartridged/current`. Both forms run as the ordinary FDS user +and are tracked for cartridge removal. + +New cartridges run directly from read-only payload partitions. They require no +program extraction or compilation on FDS. The stable `/run/fds/bin` directory is +already in the console and terminal PATH, so existing shells see inserted commands. + +If cartridges export the same command name, the lowest numbered bay wins. +Within a bay, the first software selector in lexical order wins. System commands +appear earlier in PATH and retain their usual meanings. To select a specific +cartridge command, use its qualified alias: + +```sh +b01:demo.hello:hello 'Explicit bay and software' +fds run 01 -- demo.hello:hello +``` + +Legacy PROGRAM media uses `b01:COMMAND` for a qualified direct alias. Commands +are removed on eject/unplug; a shared name falls back to the next available bay. +After the final cartridge is removed, a shell may remember the old executable +path; `hash -r` clears Bash's command cache. + +## Eject safely + +```sh +cd "$HOME" +fds eject 01 +fds bay 01 +``` + +Eject stops managed programs, flushes writable storage and releases its mounts. +Remove the cartridge after **SAFE** is reported. If eject fails, close files, +shells or extra mounts using that cartridge and retry. Never treat a timeout as +permission to pull writable media. Use `fds poweroff` to stop the whole computer; +swap the running SYSTEM only after shutdown. + +The [control panel](Desktop) provides the same safe-eject action. For writable +storage behavior, see [DATA and files](Data). + +## Configure physical bay numbering + +Run `fds topology` with one identifiable USB device inserted in each slot in +turn. Record the controller/hub topology for both USB 2 and USB 3 connections, +then map those stable identities in the machine's `bays.toml`. Do not map +`/dev/sda` names or enumeration order. The emulator supplies its own known map. + +Use [machine settings](Internal-Storage) to validate, export and install the +configuration. The detailed wiring/calibration procedure is in the +[hardware engineering reference](Developer-Cartridges). diff --git a/Cleanup.md b/Cleanup.md new file mode 100644 index 0000000..6e117a9 --- /dev/null +++ b/Cleanup.md @@ -0,0 +1,95 @@ +--- +include_toc: true +--- + +# Reclaim build space + +Run these from the repository root after stopping builds and test VMs: + +```sh +make clean-preview +make clean +``` + +The preview lists every selected directory, its combined allocated footprint, +and the generated directories being kept. It does not delete files. `make clean` +recalculates the selection and deletes it without a confirmation prompt. Repeat +it whenever old build/test workspaces accumulate. It needs Python 3, Git and GNU +`du`, already used by the build host; it needs no Rust compilation, downloads, +root privileges or new Python packages. + +## What is removed + +- Old generated rootfs, kernel, initramfs, SYSTEM, boot, recovery and internal + image workspaces that are no longer referenced by published output links. +- Old generated integration-test directories, including their disk images, + disposable test DATA overlays and local diagnostic logs. The current results + referenced by `*-latest` links or workstation current-pointer files are kept. +- Old generated package staging copies and service compilation directories. +- This checkout's `target/` Rust compilation output. Exported executables in + `out/` and `out/workstation/` remain available. The next Rust build recompiles + as needed using the retained Cargo source cache. + +The command recognizes the specific temporary-directory names produced by the +repository's builders and tests. It never treats all of `out/` as disposable. +Unknown names are left alone, including ad hoc diagnostic experiments. + +## What is kept + +- Current rootfs archives for every profile, kernel, initramfs, boot, recovery, + internal disk and SYSTEM images, through their published symlinks. Links in + `out/manifests/` and links within retained workspaces also retain their targets. +- Latest published test runs, the current workstation image/emulator fixtures, + the current Dasung s6 database, and the newest `m9-images.*` fixture needed by + the media tests. These can still occupy tens of GiB. +- Signed releases such as `out/fds-os-0.1.0/`, input snapshots such as + `out/inputs-m12-v5/`, and independent restored trees such as + `out/rebuild-m12-v5-a/`. These remain available for release reproduction. +- User-created cartridge images, installed software trees and personal emulator sessions + such as `out/my-emulator/`, including their persistent DATA overlays. +- `out/logs/`, `out/manifests/`, packages, downloads, `out/cache/`, `.host/`, the + writable Void checkout/build container under `.host/void-packages/`, and + Cargo/Rustup caches outside this checkout. +- Git-tracked files and any recognized workspace containing a `.fds-keep` entry. + +To retain an older generated workspace for investigation, put a marker in it +before cleaning: + +```sh +touch out/m8-vm.YOUR_RUN/.fds-keep +make clean-preview +``` + +Replace `YOUR_RUN` with the actual directory suffix. Remove that marker when the +workspace is disposable again. Reserve the generated names for the build/test +tools; store personal sessions and images under your own names. + +## Build-space management + +Image builds and VM suites retain workspaces for diagnosis and preserve the last +successful published image. Repeat `make clean-preview` and `make clean` after +completed build batches. Cleanup is explicit; there is no automatic retention +limit during a build. Full image builds require tens of GiB per independent tree. + +## Running safely and rebuilding + +Use cleanup sequentially, just like the existing shared-container builds. It +holds the existing rootfs/base-package/image-tool locks, excludes another +cleanup, and refuses mounted candidate paths or observable active build/VM +processes. These checks are safeguards, not permission to start a separate build +while cleanup is running. It never follows a candidate symlink into another +directory, and refuses a symlinked `out/` entirely. Only the checkout containing +the script is eligible; there is no arbitrary deletion-path option. + +Existing published images and native tools remain usable after cleanup. To +refresh exported binaries, run `make tooling` or `make workstation`. To rebuild +the smoke-test artifacts and rerun build checks: + +```sh +make smoke-test +make check +``` + +After cleanup, run the smoke test before `make check`, since the checks need its +rebuilt executable. Cleanup implementation and validation details are in the +[developer notes](Developer-Cleanup). diff --git a/Dasung.md b/Dasung.md new file mode 100644 index 0000000..a7156e1 --- /dev/null +++ b/Dasung.md @@ -0,0 +1,199 @@ +--- +include_toc: true +--- + +# Dasung Paperlike display + +FDS includes `dasungd` for the configured Dasung Paperlike 13K grayscale monitor. +The controller starts during early boot and remains supervised in the base system, +including console-only use. No GUI or ENVIRONMENT cartridge is required. + +For installation, disk preparation and first power-on, start with +[Run FDS on a Raspberry Pi 5](Hardware-Setup). + +The supplied profile uses the dedicated monitor's EDID and USB companion identity, +with raw mode 1 and contrast 4 as startup defaults. It is specific to that monitor; +do not use its identity as a blanket match for other CH340 USB devices. + +## HDMI plus USB-C on the Pi + +Use the Pi's **micro-HDMI output** for video to the monitor's HDMI input. Use a +separate **USB-A-to-USB-C data cable** from a Pi host port to the monitor for +controller access, and power the monitor with its supplied DC adapter. The +Pi's own USB-C connector is its power input; it does not output USB-C video. +See the [wiring table](Hardware-Setup#2-wire-hdmi-video-usb-control-and-power-separately), +[official Pi display guidance](https://www.raspberrypi.com/documentation/computers/getting-started.html#display), +and [DASUNG's power guidance](https://shop.dasung.com/pages/q-a). + +HDMI video and USB controller access are separate checks. The daemon talks USB +regardless of the video cable, but the HDMI-plus-USB arrangement still needs to +expose the expected UART and companion bridge on this monitor. HDMI alone does +not carry the daemon's control protocol. A charge-only cable or a USB-C lead +connected only to a charger cannot provide host USB control. + +The image currently supplies a firmware EDID override for 3200 × 2400 at about +37 Hz, recorded on the original workstation. DASUNG says some HDMI hosts need +27 Hz instead; that does not establish which mode the Pi and this cable will +accept. The forced profile can prevent using the monitor's native HDMI mode +list. Use the diagnostic procedure below if needed; neither Pi HDMI compatibility +nor physical power-cycle recovery is recorded as passed yet. +[Manufacturer HDMI note](https://shop.dasung.com/pages/dasung-paperlike-13k-the-worlds-first-13-3-inch-37hz-3k-e-ink-monitor). + +## Inspect the controller + +The controller's socket is restricted to root. Use the local `RECOVERY#` console +described in [Enter recovery](Recovery#enter-recovery); the ordinary `FDS>` +account cannot access it. On FDS recovery: + +```sh +dasungd status +dasungd query +``` + +`status` reports whether the monitor is connected and responsive, the age of its +last reply and cached parameters. `query` requests current values. To refresh +the display or adjust contrast: + +```sh +dasungd refresh +dasungd set contrast 4 +dasungd set mode 1 --save +dasungd forget mode +``` + +A successful set means the command was sent; use query/status to read back the +result. The grayscale model uses raw numeric modes. Saved overrides in +`/run/dasungd/settings.json` survive a daemon restart during the same boot, but +not power loss. Image configuration defaults are in `/etc/dasungd.toml`. + +## Diagnose a connection problem + +Inspect `/run/log/dasungd/current` and the reported status. Discovery requires +the configured DRM EDID and matching USB companion topology. An ambiguous match +is rejected. Set `usb_path` only after identifying the intended device topology. + +At `RECOVERY#`, start with: + +```sh +lsusb +lsusb -t +dasungd status +dasungd query +tail -80 /run/log/dasungd/current +``` + +The existing profile expects UART `1a86:7523` alongside SPI bridge `1a86:5512` +under the same hub. If neither appears, check monitor power, its USB data port, +the cable and the Pi host connection. If only one appears, preserve the USB tree +and log: the cable/input combination has not yet satisfied the current profile. +Do not disable companion checks or match every CH340 device to hide the failure. + +`status` separates `connected` from `responsive` and includes `usb_error`, +`display_error`, `displays`, `rx_frames` and `last_rx_ms_ago`. `query` queues +requests; run `status` again after replies arrive. Keepalive packets being sent +are not themselves evidence that the monitor answered. + +Capture the actual DRM connector names and EDID at `RECOVERY#`: + +```sh +mkdir /tmp/dasung-hdmi +cat /proc/cmdline > /tmp/dasung-hdmi/cmdline.txt +for fds_connector in /sys/class/drm/card*-*; do + [ -f "$fds_connector/status" ] || continue + fds_connector_name=${fds_connector##*/} + printf '%s: ' "$fds_connector_name" + cat "$fds_connector/status" + cat "$fds_connector/modes" + cp "$fds_connector/status" "/tmp/dasung-hdmi/$fds_connector_name.status" + cp "$fds_connector/modes" "/tmp/dasung-hdmi/$fds_connector_name.modes" + cp "$fds_connector/edid" "/tmp/dasung-hdmi/$fds_connector_name.edid" +done +``` + +Use a new directory for each capture and save it before shutdown using +[machine diagnostics](Internal-Storage#save-and-retrieve-diagnostics). +An empty EDID on a disconnected connector is expected. The `modes` file lists +available resolution names, not proof of the active refresh rate or picture +quality. With `drm.edid_firmware` active, the EDID you read may be the supplied +override; capture again without it to learn what HDMI actually advertises. + +The configured DRM serial is `L56051794302`. Native HDMI EDID might identify the +display differently from the recorded USB-C profile. With `require_display=true`, +an identity mismatch can leave USB control disconnected even when HDMI video +works. Retain the native EDID, USB topology and errors for a deliberate profile +update; do not silently remove identity checks. Config changes must ultimately +be rebuilt into both early boot and base/recovery images. The read-only +`/etc/dasungd.toml` is not a persistent settings editor. + +## Try the monitor's own HDMI EDID + +Use this when the stock forced timing gives a blank or unstable picture, or when +temporarily connecting a different HDMI display. The override currently applies +to connected DRM outputs without a connector qualifier, so attaching another +screen while retaining it is not an independent native-mode test. + +Follow the offline edit procedure below. In the disk's **single-line** +`cmdline.txt`, remove only this token for the EDID experiment: + +```text +drm.edid_firmware=edid/dasung-paperlike13k-37hz.bin +``` + +For verbose diagnosis with a USB keyboard and display console, a complete +command line based on the current production template is: + +```text +console=tty1 rdinit=/init ro loglevel=7 vt.global_cursor_default=0 fds.boot=normal fds.debug=1 +``` + +For the serial boot mode, retain `console=ttyAMA10,115200` after `console=tty1` +instead. Use `fds.boot=recovery` in place of `fds.boot=normal` when intentionally +booting recovery; never append a second conflicting `fds.boot` option. Do not add +`fds.emulator=1` on real hardware or replace stage0 with a Raspberry Pi OS +`root=…` command line. + +Reboot and repeat the DRM/USB capture. Removing the override allows the monitor's +HDMI EDID to supply modes; it does **not** guarantee 27 Hz or fix missing USB +control. Record any change in picture and controller status. Restore the saved +command line to return to the shipped profile. Keep both captures so a future +HDMI profile change can be based on observed timings and identity. + +### Edit the BOOT command line offline + +Shut FDS down and disconnect Pi power. Connect NVMe to the Linux workstation, or +boot the separate maintenance SD and access the inactive NVMe there. Identify +the same disk by model/serial used when flashing it. Mount only its first +partition, labeled `FDS_BOOT`; FDS does not normally mount it at +`/boot/firmware` as Raspberry Pi OS does. + +On that **workstation or maintenance OS**, replacing the partition path below: + +```sh +lsblk -p -o NAME,MODEL,SERIAL,SIZE,TYPE,PARTLABEL,MOUNTPOINTS +sudo mkdir -p /mnt/fds-boot +sudo mount -o nosuid,nodev,noexec \ + /dev/disk/by-id/REPLACE_WITH_VERIFIED_NVME-part1 /mnt/fds-boot +sudo cp -n /mnt/fds-boot/cmdline.txt /mnt/fds-boot/cmdline.before-hardware.txt +sudoedit /mnt/fds-boot/cmdline.txt +cat /mnt/fds-boot/cmdline.txt +sudo umount /mnt/fds-boot +``` + +Check the identity before the mount and stop on any failed step. If the partition +was automounted, unmount that instance first. Keep `/mnt/fds-boot` dedicated to +this operation. The no-clobber copy retains an existing backup; if repeating the +experiment, inspect that backup and skip the copy step if it already exists. +Do not assume an older backup contains your latest settings. Keep `cmdline.txt` +on one line and record its final contents with the +test session. Safely disconnect the enclosure before moving NVMe back to the Pi. + +To undo the edit, mount the same verified partition again, restore +`cmdline.before-hardware.txt` over `cmdline.txt`, and unmount before rebooting. +These edits change the flashed copy only; they do not change the source images +or their recorded hashes. Reflashing restores the image's original command line. + +FDS uses the daemon for USB control. Its configuration disables the original +workstation-specific display hotplug procedure. Video mode and cabling are part +of the machine's boot/display configuration. The detailed protocol, build +rationale and physical display procedures are in the +[developer reference](Developer-Dasung). diff --git a/Data.md b/Data.md new file mode 100644 index 0000000..a853249 --- /dev/null +++ b/Data.md @@ -0,0 +1,56 @@ +--- +include_toc: true +--- + +# DATA and persistent files + +FDS keeps SYSTEM read-only. Store documents, source code and other persistent +files on a DATA cartridge. With one healthy DATA cartridge inserted, FDS mounts +it at `/data` and gives the `fds` user access to its files. + +```sh +fds bays +fds bay 02 +mkdir -p /data/projects +printf 'My first FDS file\n' >/data/projects/hello.txt +cat /data/projects/hello.txt +``` + +Confirm that the bay reports `MOUNTED READ WRITE` before writing. Only one DATA +cartridge is active at a time. To select DATA explicitly: + +```sh +fds data use 02 +``` + +If another DATA is active, eject it first. In recovery, DATA starts read-only and +requires this explicit activation before it can be used for ordinary writes. + +## Programs using DATA + +Run a system command under the DATA bay's process tracking when it should stop +as part of ejecting that DATA: + +```sh +fds run 02 -- /usr/bin/tail -f /data/application.log +``` + +This background command uses `/data` as its working directory. Its output goes +to the cartridge service log. A directly launched PROGRAM is tracked under its +PROGRAM bay; close it if it keeps files open on a different DATA cartridge. + +## Finish a session + +```sh +cd "$HOME" +fds eject 02 +``` + +Wait for SAFE, then remove the cartridge. `fds poweroff` performs the storage +shutdown sequence for all active bays. Unexpected power loss or pulling a drive +during a write can damage files; [recovery](Recovery) explains checking DATA. + +In the emulator, writes go to a separate session overlay. Eject preserves that +overlay, and reinserting the original image creates a new overlay. See +[exporting emulator DATA](Workstation#save-emulator-data) before moving or +deleting a session. diff --git a/Desktop.md b/Desktop.md new file mode 100644 index 0000000..99a625a --- /dev/null +++ b/Desktop.md @@ -0,0 +1,71 @@ +--- +include_toc: true +--- + +# WindowMaker and FDS Control + +WindowMaker provides the FDS desktop. Its default appearance is grayscale on +all displays: white backgrounds, black borders and selected titles, gray widgets, +Terminus text, and no animations. No E-Ink monitor detection or special cartridge +is needed to select this theme. + +## Open the desktop + +At the FDS console: + +```sh +fds profile activate windowmaker +fds profiles +``` + +FDS opens a terminal and **FDS Control**. You can reopen the panel from the +WindowMaker root menu's **FDS Control** item, or run `fds-control` in a terminal. +An ENVIRONMENT cartridge requesting the `windowmaker` profile can also start +the desktop. The Dasung controller runs independently of this session. + +## Manage cartridges in the panel + +![FDS Control with a PROGRAM cartridge and its terminal output](Asset-images-fds-control.png) + +1. Select a bay in the left column. The right column shows its identity, state, + running process count and available software commands. +2. Select a command, then choose **Run in terminal**. The terminal stays open + after the command exits so you can read its output; close it when finished. +3. Choose **Safe eject** to stop that bay's programs and release its storage. + Wait for **SAFE** before removing the cartridge. +4. Use **Rescan** to refresh discovery. The panel also checks for changes + automatically and updates the display when the visible state changes. + +For an inactive DATA cartridge, the primary button becomes **Use DATA**. Activate +it there to make it available at `/data`; eject another active DATA first. + +Errors appear in the status area. Actions are unavailable when their prerequisites +are missing, such as running an application from an empty bay or ejecting the +active SYSTEM. The panel uses the same cartridge service as `fds`. + +Keyboard controls: **Tab** moves focus between bays, programs and buttons; +**Up/Down** changes the selected bay or program; **Enter** activates the focused +control; **R** rescans; **Esc** closes the panel. Closing the panel does not eject cartridges or close +application terminals; use Safe eject when you want to release a cartridge. + +## Return to the console + +Choose **Return to console** in the root menu, or run: + +```sh +fds profile deactivate +``` + +This closes the desktop session. It does not shut down FDS. Use `fds poweroff` +when you want to turn the computer off. + +## Customize the appearance + +System defaults are installed in `/etc/WindowMaker/`. Your files in +`$HOME/GNUstep/Defaults/` override those defaults. The session copies the FDS +settings into a new home once and preserves existing preferences. Use +WindowMaker's settings tools or edit those preference files to change fonts, +colors and behavior. Plain `wmaker` also reads the FDS system defaults. + +The root menu includes Ethernet on/off controls. Equivalent console commands +are `fds network on`, `fds network off`, and `fds profiles` to inspect the result. diff --git a/Developer-Architecture.md b/Developer-Architecture.md new file mode 100644 index 0000000..43149cd --- /dev/null +++ b/Developer-Architecture.md @@ -0,0 +1,174 @@ +--- +include_toc: true +--- + +# Understanding FDS/OS + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Roadmap](Developer-Roadmap) · [Glossary](Developer-Glossary) + +FDS/OS is designed for a Raspberry Pi 5 portable computer whose operating system, +user data, applications, and optional environments can be carried on removable +cartridges. Its central design question is: what must stay with the machine, and +what should move with the user? + +**Status:** M0–M12 software checks cover the cross-build, configured ARM rootfs, +native init, Pi kernel/initramfs/SYSTEM images, ordinary-user console, cartridge +management, writable DATA, desktop/network activation and verified media writing. +Ordered shutdown, twelve-device virtual stress and independent recovery have +passed software acceptance. M12 internal storage, development tools and signature +verification have also passed software checks. The signed local 0.1.0 release +passed two complete offline builds and archive verification. Physical Pi boot +and wiring remain deferred. Use +[Your first build](Developer-Getting-Started) and [Boot images](Developer-Boot) to try the VM. + +## Three things to keep separate + +1. **The build workstation** is an x86_64 Arch Linux computer. It downloads tools, + compiles ARM software, checks artifacts, assembles the rootfs, and boots the ARM VM. +2. **The machine firmware layer** lives on the Pi's internal NVMe. It supplies the + kernel, early boot program, recovery environment, and hardware configuration. +3. **The removable operating environment** comes from cartridges. SYSTEM supplies + the OS root filesystem; DATA supplies persistent user files; other cartridges + supply programs, configuration, or physical USB hardware. + +```text +Build workstation + Arch x86_64 + ├── Void build container → aarch64 glibc packages + └── Rust toolchain → static AArch64 FDS executables + │ + └── Pi kernel + initramfs + SYSTEM → ARM VM checks + +Pi 5 machine (planned) + Internal NVMe Removable bays + ├── FDS_BOOT ───────────────→ SYSTEM: read-only OS + ├── FDS_RECOVERY DATA: user files + └── FDS_INTERNAL PROGRAM / ENVIRONMENT / hardware + machine configuration +``` + +The Void build container on the workstation is **not** the Pi's root filesystem. +It may contain build dependencies that must never appear in the final FDS image. +Likewise, a successful ARM package build does not prove that a Pi can boot it. + +## Why cartridges change the operating model + +A conventional writable root filesystem combines the installed OS, local changes, +and often user data on one disk. FDS separates their lifetimes: + +- A SYSTEM image is built and validated as a unit. Updating means preparing and + swapping to a new SYSTEM cartridge; retaining the previous one enables rollback. +- A DATA cartridge follows the user. With no DATA cartridge, the home is + temporary memory-backed storage, so it does not survive power loss. +- A PROGRAM cartridge supplies a self-contained application image. It does not + install arbitrary files into SYSTEM. +- An ENVIRONMENT cartridge selects a known configuration such as WindowMaker. + In version 1, the desktop software itself is already present in SYSTEM. + +This requires explicit safe-eject behavior: removable does not mean safe to pull +while an application is writing. See [Cartridges](Developer-Cartridges). + +## Two library strategies + +Both software groups target **Linux on AArch64**. They differ in how they obtain +the C runtime and other libraries, not in which processor runs them. + +| Software group | Build target | Runtime approach | Reason | +| --- | --- | --- | --- | +| Ordinary Linux packages | Void `aarch64` | glibc, usually shared libraries | Compatibility with the GNU userland, Xorg, WindowMaker, compilers, and third-party packages | +| FDS-owned control tools | Rust `aarch64-unknown-linux-musl` | Static musl by default | Early boot and control programs can run without the installed glibc loader or version | + +M0 demonstrates these with GNU hello and `fds-smoketest`, respectively. A static +FDS executable still needs a compatible Linux kernel; static linkage does not +turn it into firmware or a standalone OS. Future dependencies using C/C++ FFI +will need their own musl cross-build assessment. + +The project does not switch the whole distribution to musl. XBPS target packages +remain `aarch64`, not `aarch64-musl`. The downloaded **host** XBPS tools happen to +be static musl for x86_64 so they can run on Arch; their libc is independent of +the target package ABI. See [Packages](Developer-Packages). + +## Workstation software and emulation + +Native Linux `fds-cartridge` builds software from trusted recipes, packages xz +tarballs, and creates a complete GPT image with metadata partition 1 plus payload +partitions. It verifies that image before a separately confirmed whole-USB write. +Native `fds-emulator` boots the FDS kernel/initramfs/SYSTEM in QEMU and exposes +twelve virtual USB bays. Writable DATA uses separate overlays; source images +stay unchanged. These tools are host programs, independent of the target's ABI. + +The guest performs bounded verification and temporary extraction, then runs a +selected software command as UID 1000. It does not compile software or run +installation hooks from cartridges. See [the walkthrough](Developer-Workstation) and +[format contract](Developer-Software-Format). + +## Storage and writable state + +M4 builds the actual FAT32 boot partition and GPT/EROFS SYSTEM cartridge. M5 +provides the temporary home. M6 adds managed read-only cartridge mounts. The +M12 internal BOOT/RECOVERY/INTERNAL builder and persistent settings are described +in [Internal storage](Developer-Internal-Storage). + +| Location | Intended content | Write policy | +| --- | --- | --- | +| Internal `FDS_BOOT` (FAT32) | Pi boot files, kernel, DTBs, initramfs | Managed machine boot content | +| Internal `FDS_RECOVERY` (EROFS) | Independent recovery OS | Read-only recovery image | +| Internal `FDS_INTERNAL` (ext4) | Bay map, hardware configuration, diagnostics | Persistent machine state; no ordinary user files | +| Cartridge `FDS_SYSTEM` (EROFS) | OS root filesystem | Read-only; replace image instead of in-place updates | +| Software `FDS_METADATA` + `FDS_PAYLOAD02`… (EROFS) | Catalogue and xz software archives | Read-only; extracted executable caches are temporary | +| Cartridge `FDS_DATA` (ext4) | User home and/or `/data` | Persistent user writes | +| `/run`, `/tmp`, `/var/tmp`, `/var/log` | Runtime state, temporary files, logs | tmpfs; volatile | +| Home without DATA | Temporary user session | tmpfs; volatile | + +Writable DATA uses `/data`; `/home/fds` stays temporary. +The table describes the final storage roles. Use the image builders rather than +applying this layout to an existing workstation disk. + +## Boot and service responsibilities + +The Pi boots its kernel and small initramfs from internal NVMe. A static Rust +`fds-stage0` locates a partition named `FDS_SYSTEM`, mounts its EROFS filesystem, +and transfers control to native s6 in the new root. + +The console should become usable as soon as its own prerequisites are ready. +It must not wait for the network, desktop, all 12 USB devices, or a complete +cartridge catalog. `fds-cartridged` handles hotplug, physical bay mapping and +metadata, and activates known desktop profiles from validated ENVIRONMENT media. + +See [Boot](Developer-Boot) for the sequence, [Services](Developer-Services) for responsibilities, +and [Performance](Developer-Performance) for targets and how they will be measured. + +## Trust and ownership boundaries + +Cartridge metadata describes a cartridge; it is not permission to run supplied +scripts as root. Known profiles and hardware mappings belong to the OS. Ordinary +programs should run with ordinary user privileges. An unrecognized device should +not become trusted merely because it appears in a USB bay. + +Package customization belongs in project-owned `packages/fds-*` overlays. +Upstream Void tracked files remain unchanged at a recorded source commit. +M0 checks this boundary automatically. Source pins and artifact hashes improve +traceability. Exact selected input snapshots, offline reproduction and local +release signing have passed [M12 acceptance](Developer-M12-Validation). The +independent [recovery environment](Developer-Recovery) is implemented and tested. + +## Where implementation starts and stops + +Today the active sources also include `rust/dasungd/`, its package overlay, and +`s6/source/` for the base monitor service. `make dasung` builds its static ARM +executable, XBPS package, and a service database for validation. The monitor +service is independent of GUI and cartridge profiles. M2 includes these in a rootfs with native s6 init and a generic ARM VM test. +M3 adds the FDS CLI and shared Rust contracts. M4 adds the Pi kernel, Rust +initramfs, GPT/EROFS SYSTEM and FAT32 boot partition. Its actual stage0 handoff +and media insertion/recovery paths run in a generic ARM VM. Physical Pi +validation remains deferred. M5 adds the ordinary console and boot tracing; M6 +adds the cartridge daemon. M7–M11 add writable DATA, desktop/network profiles, +verified media writing, ordered shutdown and virtual USB stress. M12 adds recovery, +internal machine storage and release preparation. + +Use [Boot images](Developer-Boot) for the current builders and their acceptance boundary. +See [M1 rootfs](Developer-Rootfs) to use the current filesystem archive. Consult the [roadmap](Developer-Roadmap) for acceptance +boundaries and the [master plan](Developer-Master-Plan) for the complete specification. diff --git a/Developer-Boot.md b/Developer-Boot.md new file mode 100644 index 0000000..fac2e3b --- /dev/null +++ b/Developer-Boot.md @@ -0,0 +1,186 @@ +--- +include_toc: true +--- + +# Boot images and stage0 + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Implementation status](Developer-Implementation-Status) + +FDS keeps the machine's boot files on internal NVMe and its operating system on a +removable SYSTEM cartridge. The Pi firmware loads the kernel and a small initial +filesystem. Its Rust `fds-stage0` program finds the partition named `FDS_SYSTEM`, +mounts it read-only, and hands control to native s6. Device names such as `sda` +are never used as cartridge identities. + +The software boot path has passed on an ARM virtual machine using the actual +FDS Pi kernel. That does not verify Pi firmware, NVMe, RP1, display output, USB +power or physical boot speed. Those checks still require physical testing; follow +the [real-hardware setup guide](Hardware-Setup). M5 now provides an +ordinary-user `FDS>` console with a temporary home +and measured boot events. Password login remains locked; this is an intentional +local single-user session. The M2 test VM separately offers a temporary root +development shell. See [native init](Developer-Init) and [performance reports](Developer-Performance). + +## Build on the workstation + +Use the [Arch host setup](Developer-Build-Host) first. In addition to its original tools, +install host `lz4`, `zstd`, `util-linux`, `e2fsprogs` and `libarchive` if missing. +They supply initramfs compression, independent GPT inspection and the initial +VM's filesystem/archive tools; these are build/test dependencies. +Python 3.14+ reads zstd-compressed XBPS archives. Filesystem builders are obtained +as verified Void packages in `.host/image-tools`, separate from the build container. + +From the repository root, run sequentially: + +```sh +make bootstrap +make rootfs PROFILE=cli +make initramfs +make system-card PROFILE=cli +make boot-volume +make rootfs-test +make init-test +make boot-test +``` + +The rootfs build includes the FDS CLI, base Dasung daemon, native s6 database, and +the kernel's matching modules. Its first kernel cross-build is substantial; +progress is recorded in `out/logs/xbps-fds-kernel.log`. `make kernel` also builds +that kernel independently. Later calls reuse its package only when recorded +source inputs and every exported payload hash still match. To force a kernel +rebuild, run `./tools/build-kernel --rebuild`. This source/payload cache is not +yet a complete lock of rolling binary build dependencies. +No command here opens or writes a physical target disk. +On a new build host, `make init-test` installs QEMU's system emulator into the +project-local Void environment before `make boot-test` uses it. + +| Output | Meaning | +| --- | --- | +| `out/kernel/boot/kernel_2712.img` | Pi 5 kernel with boot-critical drivers built in | +| `out/kernel/usr/lib/modules/` | Matching optional drivers | +| `out/fds-initramfs.img` | Uncompressed default initramfs; a symlink to a versioned build | +| `out/initramfs/initramfs.cpio*` | Uncompressed, gzip, legacy-LZ4 and Zstandard variants | +| `out/rootfs-aarch64.tar` | Configured userspace archive with target ownership and capabilities | +| `out/fds-system-cli.img` | Whole GPT cartridge image, containing an EROFS `FDS_SYSTEM` partition | +| `out/fds-boot.img` | **512 MiB FAT32 partition image**, labeled `FDS_BOOT`; not a whole NVMe disk | +| `out/boot-volume/manifest.json` | Boot image hash, firmware input and every boot file hash | +| `out/m4-vm-latest/` | Logs and exact inputs from the last passing full boot matrix | + +`make internal-image` combines BOOT, independent RECOVERY and INTERNAL settings +into `out/fds-internal.img`; see [Internal storage](Developer-Internal-Storage) for the +full procedure. The FAT partition image alone is not that complete disk layout. +Use the [media tools](Developer-Media-Tools) for confirmed removable-cartridge writes. + +## What stage0 does + +The initial filesystem contains static-musl stage0 and Dasung executables, +configuration, the monitor's EDID, mountpoint directories and `/dev/console`. +There is no shell, BusyBox, Python, glibc, or udev in it. Stage0 mounts proc, sysfs +and devtmpfs itself and binds its kernel-event socket before scanning devices. + +If a single SYSTEM partition is present, stage0 verifies its block-device identity, +mounts EROFS read-only, and checks the FDS identity and native init executables. +It stops and reaps the early Dasung process before the base service starts, +moves the kernel filesystems into the new root, removes the old in-memory root, +and executes the s6 launcher. It refuses this path outside root PID 1 on ramfs/tmpfs. + +Without SYSTEM, the console displays a missing-media prompt and keeps processing +device events. Inserting media resumes boot without rebooting or polling sleeps. +When more than one matching partition is visible, stage0 refuses to choose one. +Its waiting console accepts: + +```text +list Show the discovered kernel partition identities +rescan Repeat discovery immediately +recovery Select FDS_RECOVERY instead of FDS_SYSTEM +reboot Reboot from early userspace +poweroff Power off from early userspace +``` + +An invalid filesystem or incomplete FDS root remains at the prompt. `list` is a +diagnostic view and can show technical device paths; physical bay labels arrive +with the cartridge daemon. Version 1 uses explicit recovery rather than an +interactive choice between multiple SYSTEMs. Detection applies to devices visible +at selection time; a cartridge inserted after root handoff cannot replace root. + +## Pi display and development output + +`image/pi5/config.txt` selects the Pi 5 kernel, initramfs, external PCIe and VC4 +KMS, disables HDMI audio, and avoids automatic camera/DSI setup. It does not +overclock the Pi or PCIe link. The pinned Pi kernel retains 16 KiB pages and +disables the CH341 SPI driver required by the Dasung protection policy. + +The production command line uses the display console and quiet kernel output. +It loads the Paperlike 13K's recorded EDID from the initramfs before SYSTEM exists. +The unqualified `drm.edid_firmware` setting applies this EDID to connected DRM +outputs: this image is intended for the user's dedicated Dasung display. The +actual adapter, connector, mode acceptance and picture still need Pi validation. +Early USB control and the later base daemon share the same monitor identity. + +For serial diagnosis, build: + +```sh +make boot-volume BOOT_MODE=development +``` + +This replaces the convenience output link with a separate development build; +previous versioned directories remain. Development enables kernel/daemon output +and makes the Pi 5 `ttyAMA10` debug UART the primary userspace console. That is +the Pi 5 debug connector, not an assumption about the 40-pin GPIO UART. Rebuild +with `BOOT_MODE=production` to restore the display-console configuration. + +`fds.boot=recovery` on the kernel command line selects `FDS_RECOVERY` immediately. +`fds.emulator=1` is reserved for the [workstation emulator](Developer-Workstation): +the SYSTEM loader requires QEMU virt and its fixed xHCI controller before +enabling the twelve virtual bay mappings. Default Pi boot is unchanged. +`fds.debug=1` exposes the early controller's diagnostics. Duplicate or unknown +`fds.*` options fail clearly rather than being silently ignored. + +## What the automated test proves + +`make boot-test` appends a test-only checker to a copy of the exported rootfs; +the original stage-2 script and compiled service graph still run. It checks: + +- All four initramfs formats reach native s6 on read-only EROFS. +- Missing SYSTEM waits; a virtual USB insertion resumes boot. +- Two SYSTEM partitions remain unselected; the console can enter recovery. +- Explicit recovery mode selects RECOVERY even when SYSTEM is also present. +- Invalid filesystems and foreign OS roots are refused; recovery remains usable. +- Kernel page size, root ownership, ping capability, temporary filesystems, + single Dasung ownership, service start/restart/stop and orderly power-off. +- Input disk hashes remain unchanged; the VM has no network or host-device access. + +QEMU uses its `max` ARM CPU because the Pi kernel needs 16 KiB page support; +the earlier M2 Cortex-A72 fixture cannot run this kernel. These are compatibility +and behavior checks, not Pi performance measurements. Test RECOVERY is a relabeled +fixture for selection; the independent recovery product is tracked separately. + +Reference material: [Pi kernel builds](https://www.raspberrypi.com/documentation/computers/linux_kernel.html), +[boot configuration](https://www.raspberrypi.com/documentation/computers/config_txt.html), +[Pi UARTs](https://www.raspberrypi.com/documentation/computers/configuration.html#configuring-uarts), +and [Linux initramfs](https://docs.kernel.org/filesystems/ramfs-rootfs-initramfs.html). + +## Use the ordinary FDS console + +After building the CLI rootfs, initramfs and SYSTEM image: + +```sh +make console-vm +``` + +The ARM virtual machine boots the real stage0 and read-only SYSTEM image. At +`FDS>` you are the ordinary `fds` user (UID 1000), with a temporary writable home. +Try `fds info`, `fds inspect /FDS/CARTRIDGE.TOML`, and `fds boot-profile`. +The console deliberately uses local autologin; both account passwords are locked. +Use `fds poweroff` for an orderly shutdown through native s6. Enter `exit` to +restart the console. Press **Ctrl-a, then x** to close QEMU immediately. +Closing QEMU this way ends the test immediately; it is not a graceful-shutdown test. +No host USB devices or physical disks are attached to this VM. + +Run `make console-test` for automated console/readiness checks and +`make performance-test` for the software optimization comparison. Read +[M5 validation](Developer-M5-Validation) and [measurement boundaries](Developer-Performance) +before interpreting VM timing as hardware evidence. diff --git a/Developer-Build-Host.md b/Developer-Build-Host.md new file mode 100644 index 0000000..eda7004 --- /dev/null +++ b/Developer-Build-Host.md @@ -0,0 +1,268 @@ +--- +include_toc: true +--- + +# M0: Arch Linux x86_64 build host + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Troubleshooting](Developer-Troubleshooting) + +This is the technical reference for the build foundation used by later milestones. +The [rootfs guide](Developer-Rootfs) adds assembly, Python and isolated ARM execution requirements. If you are +setting up the repository for the first time, follow [Your first build](Developer-Getting-Started) +for an ordered walkthrough with expected output. For editing, rebuilding, and +configuration changes after setup, use [Development](Developer-Development). + +## Scope and pinned inputs + +M0 validates the cross-build environment. The scoped [Dasung integration](Developer-Dasung) +adds a controller package and service definitions without implementing a full OS. +M1 adds the configured FDS rootfs; M2 adds native init and the generic ARM VM. +M3 adds static FDS tools. M4 cross-builds the Pi kernel and creates initramfs, +SYSTEM GPT/EROFS and a FAT32 boot partition; see [Boot images](Developer-Boot). +M6 adds cartridge management and M7 adds writable DATA. M8 adds the optional +WindowMaker stack and on-demand Ethernet. M12 adds the [complete internal disk](Developer-Internal-Storage), +independent recovery and persistent machine settings. The +[signature tools](Developer-Releases) have passed acceptance; the +[frozen-input workflow](Developer-Reproducible-Builds) and signed local 0.1.0 release +have passed two complete offline builds and artifact/archive verification. The [desktop guide](Developer-Desktop) explains the +new dependencies and the development image used for virtual display tests. +See [Native init](Developer-Init) for the additional test dependencies. An ARM host +and a connected Pi are not required. + +| Input | Pin | +| --- | --- | +| void-packages | Git submodule and `VOID_PACKAGES_COMMIT`: `02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1` | +| Static XBPS host tools | Versioned `0.60.4_1.x86_64-musl` URL and a SHA-256 digest stored in this repository | +| Rust | `rust-toolchain.toml`: 1.98.0, minimal profile, rustfmt | +| Rust target | `aarch64-unknown-linux-musl` | +| Rust crate dependencies | Smoketest has none; all workspace dependencies, including shared Clap CLI parsing, are pinned in `Cargo.lock`, fetched at bootstrap, and built with `--locked --offline` | +| XBPS test package | `srcpkgs/hello/template` in the pinned Void tree: GNU hello 2.12.3_1 | +| Pi kernel | `packages/fds-kernel/template`: 6.12.87, pinned Raspberry Pi source archive and FDS config delta | +| Pi firmware | `config/pi-firmware.conf`: signed 20260715_1 package with a pinned SHA-256 | + +The musl ABI of the static host tools does not determine the target ABI. Scripts +explicitly use `XBPS_ARCH=x86_64` for the glibc build container, then run +`./xbps-src -a aarch64 pkg hello` to build the ARM glibc package. Local ARM package +indexing also explicitly sets `XBPS_ARCH=aarch64`. + +## Generic Linux workstation tools and desktop additions + +`make workstation` builds the Arch `fds-tools` package; `make workstation-install` +installs it with pacman. `make workstation-binaries` builds native `fds-cartridge` +and `fds-emulator` on generic Linux. Python 3 and Git resolve build versions; +Dasung builds install these tools in the project-local Void container as well. Runtime +requirements are QEMU, erofs-utils, bubblewrap and native XBPS with a prepared +Void source checkout for software creation. `fds-cartridge` invokes xbps-src, +installs runtime dependencies and creates direct EROFS trees. The Rust tar reader +and xz utility remain for legacy cartridges and package/source archives. See +[the current workstation guide](Workstation). + +The target `fds-control` panel uses x11rb 0.13.2's pure-Rust core X11 connection. +It adds no shared GUI toolkit or background service. Its protocol, hostname and +OS-access dependencies are locked and included in Rust license notices. Both +the panel and foreground launcher are packaged in fds-cli as static ARM binaries. +See [the revision rationale](Developer-Current-Revision). + +## Host dependencies + +Run as an ordinary user on x86_64 Arch Linux with unprivileged user namespaces. +Rootfs construction now requires Linux 6.7+ with namespaced `binfmt_misc` support. +The M5 build uses a private handler and static QEMU for child execution; it does +not register an ARM interpreter in the workstation's global registry. +The checkout path must not contain whitespace, a limitation of upstream +xbps-src. Allow several GB of disk space and provide a network connection. + +```sh +sudo pacman -S --needed bash coreutils binutils git curl make file tar xz gzip zstd \ + bubblewrap rustup ca-certificates findutils diffutils grep sed gawk util-linux +``` + +Alternatively, run `./tools/bootstrap-host --install-deps`. Only this explicit +option invokes sudo/pacman. Default `make bootstrap` checks dependencies and +exits nonzero when one is missing, without installing host system packages. + +Dependency rationale: + +- bash, coreutils, findutils, diffutils, grep, sed, gawk, and util-linux provide + the build scripts and upstream build system with their required shell tools. +- git pins upstream source; curl and CA certificates download HTTPS inputs. +- tar, xz, gzip, and zstd extract archives and XBPS packages. +- file and binutils inspect ELF artifacts; make provides the build entry points. +- bubblewrap runs the build container as an ordinary user. +- rustup installs the pinned Rust toolchain and target. rustfmt checks source + formatting and is not a runtime dependency. + +Rust's bundled `rust-lld` and target-provided static musl/CRT are sufficient for +the current pure-Rust smoketest. No Zig, Docker, OpenSSL crate, or additional +musl runtime is required. The imported Dasung controller bundles libusb and uses a Void musl C cross +compiler inside the container; see [its dependency rationale](Developer-Dasung#dependencies-and-source-provenance). +Glibc C libraries cannot simply be reused in static-musl binaries. + +## Exact commands + +Run sequentially from the repository root: + +```sh +make bootstrap +make smoke-test +make check +``` + +Bootstrap initializes the documentation submodule if it is missing and the +pinned Void submodule, verifies and extracts XBPS, checks +the tools' x86_64 static ELF properties, installs the pinned Rust toolchain and +target, fetches the locked workspace crates, copies `config/xbps-src.conf` into the local Void checkout, then runs the +equivalent of: + +```sh +export PATH="$PWD/.host/xbps/usr/bin:$PATH" +export XBPS_ARCH=x86_64 +cd .host/void-packages +./xbps-src -A x86_64 binary-bootstrap +``` + +`binary-bootstrap` installs upstream **base-chroot** as a compilation environment. +It is separate from the FDS `fds-base` target rootfs built in M1. Builds run through +bubblewrap without starting Void services on Arch. Repeated bootstrap reuses the +masterdir and verified downloads. The script checks the container architecture, +installed base-chroot package, and execution of GCC inside the container. + +To reproduce the two builds individually from the repository root: + +```sh +./tools/cargo-build --locked --offline --release \ + --target aarch64-unknown-linux-musl -p fds-smoketest +./tools/build-package hello +``` + +The latter command sets the local XBPS PATH, checks the Void pin, and executes +this command within the writable build checkout: + +```sh +./xbps-src -a aarch64 pkg hello +``` + +xbps-src installs the aarch64 GNU cross toolchain that runs on x86_64, together +with the target glibc sysroot. No Pi compilation is needed. The pinned template's +checksum verifies the package source archive. + +## Validation + +```sh +file target/aarch64-unknown-linux-musl/release/fds-smoketest +readelf -hW target/aarch64-unknown-linux-musl/release/fds-smoketest +readelf -lW target/aarch64-unknown-linux-musl/release/fds-smoketest +readelf -dW target/aarch64-unknown-linux-musl/release/fds-smoketest +readelf -VW target/aarch64-unknown-linux-musl/release/fds-smoketest +ldd target/aarch64-unknown-linux-musl/release/fds-smoketest +``` + +Rust acceptance requires ELF64, little endian, AArch64, executable type EXEC/DYN, +static linkage reported by `file`, no `PT_INTERP`, no `DT_NEEDED`, and no +`GLIBC_*` version requirements. Static PIE is also acceptable. The pinned Rust +target, self-contained linking configuration, and compile-time target checks +establish musl provenance; an ELF's static label alone does not identify its libc. + +`ldd` is supplemental diagnostic output. A static executable normally produces +`not a dynamic executable` with exit status 1. **An x86_64 host's ldd alone cannot +prove that an ARM executable is static.** + +XBPS validation checks architecture=aarch64 in the local package index, then +extracts `usr/bin/hello` from the generated package. It requires AArch64, +`/lib/ld-linux-aarch64.so.1`, and `libc.so.6`. An architecture suffix in a package +filename is not sufficient evidence. + +`make check` verifies rejection of the wrong architecture, dynamic executables +passed as static, a static-musl executable passed as a glibc package, non-ELF +input, missing artifacts, invalid arguments, Void pin drift, and modified +upstream files. It also tests overlay protection and runs rustfmt. Pipelines +enable pipefail so `tee` cannot hide a failed build. + +Optionally install `sudo pacman -S --needed qemu-user` to execute the ARM Rust +binary during smoke-test and compare its output. When qemu-aarch64 is absent, +the script explicitly reports SKIP. M0's required acceptance is actual +cross-compilation plus ELF/package inspection. Static inspection is not an ARM +runtime or Pi boot test. + +## Inputs, caches, and overlays + +Use `make clean-preview` followed by `make clean` when repeated builds and VM +tests accumulate. [Cleanup](Developer-Cleanup) explains the retained current images, +release/input archives and caches. Full image builds and VM suites need tens of +GiB per working tree, beyond the initial M0 smoke test's requirements. + +- `vendor/void-packages/`: clean, pinned upstream submodule; no generated overlays. +- `fds-os.wiki/`: authoritative documentation submodule, also read by packaging. +- `.host/void-packages/`: independent local build checkout at the same pin. +- `.host/xbps/`: project-local host tools, not installed into `/usr`. +- `.host/void-packages/masterdir-x86_64/`: glibc build container. +- `.host/void-packages/hostdir/`: download caches, cross packages, and build output. +- `out/packages/`: exported aarch64 test package and its repository index. +- `out/manifests/void-build-packages.txt`: packages remaining installed after the + build. Cross toolchains and temporary dependencies removed by xbps-src remain + traceable through cache digests and build logs. +- `out/manifests/void-package-inputs.sha256`: digests of cached `.xbps` inputs. +- `out/manifests/artifacts.sha256`: validation artifact digests. +- `out/logs/`: actual command logs, including logs from failed attempts. + +Place FDS package overlays at `packages/fds-*/template`. `tools/prepare-void-workspace` +clones the pinned upstream locally without shared Git objects or network access. +`tools/prepare-void` copies directories containing a template into that writable checkout and refuses to replace tracked +upstream packages. These copies are visible inside the build container. Base, +base-files, init, CLI, kernel and Dasung overlays are active; future directories +without templates remain inactive. + +Bootstrap migrates matching old generated copies, configuration, `hostdir` and +`masterdir-*` out of the submodule using renames, preserving cached downloads +and container files. Unknown files, staged edits, changed copies or destination +conflicts stop migration for explicit reconciliation. Python 3 is used for +checked migration and locking; it is already a documented FDS host prerequisite. +The workstation builder also rejects submodules as writable build targets. + +If an overlay changes and preparation reports a stale copy, inspect and preserve +any unique edits before moving the specific generated copy out of `srcpkgs/` and +retrying. Modified upstream tracked files cause immediate failure. Conflicting +local build configuration also causes failure and must be reconciled explicitly. +See [configuration and overlay troubleshooting](Developer-Troubleshooting#configuration-or-overlay-conflict) +for the procedure. + +Updating Void requires an explicit change to both the submodule gitlink and +`VOID_PACKAGES_COMMIT`, followed by validation and a commit containing both. +Do not fetch master and silently change the pin. Replacing the static host XBPS +archive also requires updating both its versioned URL and checksum. + +## Known limitations + +1. Source templates, Rust, and XBPS tools are pinned; **the complete Void binary + repository is not snapshotted**. Initial bootstrap, xbps-src build-container + updates, and missing build dependencies still use signed rolling repositories. + Cached inputs, version inventories, and digests support auditing but cannot + guarantee future upstream retention. M12 preserves the exact selected + packages, sources and build tools through the accepted + [snapshot and offline workflow](Developer-Reproducible-Builds). Its two fresh builds + produced 27 matching artifacts. This freezes the release inputs, not the + entire rolling Void repository; the original M0 check alone does not + establish reproducibility. +2. Bootstrap supports x86_64 Arch Linux. Other host distributions and ARM hosts + have not been validated. Unprivileged user namespaces must be available. +3. M4 tests the actual Pi kernel and initramfs-to-EROFS handoff in a generic ARM + VM. This does not emulate Pi firmware, RP1, physical bays or the display. + Media burning and physical boot-performance measurements remain separate work. +4. `opt-level=z` is the plan's initial setting, not a proven fastest setting. + Compare optimization levels later using real programs and hardware. +5. During the initial upstream bootstrap, bubblewrap's mounted `/tmp` produced + `rm: cannot remove '//tmp': Device or resource busy` in the cleanup phase. + Upstream returned success; the subsequent container execution, cross build, + and public bootstrap command passed. The upstream source was left unchanged. + +## Upstream references + +- [Pinned Void xbps-src usage and foreign host documentation](https://github.com/void-linux/void-packages/blob/02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1/README.md) +- [Pinned aarch64 cross profile](https://github.com/void-linux/void-packages/blob/02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1/common/cross-profiles/aarch64.sh) +- [Void static XBPS](https://docs.voidlinux.org/xbps/troubleshooting/static.html) +- [Official static tool SHA-256 checksums](https://repo-default.voidlinux.org/static/sha256sums.txt) +- [Rust aarch64-unknown-linux-musl](https://doc.rust-lang.org/rustc/platform-support/aarch64-unknown-linux-musl.html) +- [Rust self-contained linking](https://doc.rust-lang.org/rustc/codegen-options/index.html#link-self-contained) diff --git a/Developer-CLAP-Validation.md b/Developer-CLAP-Validation.md new file mode 100644 index 0000000..c28b547 --- /dev/null +++ b/Developer-CLAP-Validation.md @@ -0,0 +1,135 @@ +--- +include_toc: true +--- + +# Clap command-line migration validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Command usage](Developer-Tooling) · [M12 evidence](Developer-M12-Validation) + +Software acceptance completed on 2026-09-21. Every Rust command-line interface +uses the shared, locked Clap 4.6.7 dependency. The migration adds no third-party +crate versions or shared runtime libraries. Dasung already used Clap; its +workspace dependency now shares the same definition. The dependency-free +smoketest has no argument interface. + +## Implemented behavior + +- Typed commands cover `fds`, its inspect/eject/power aliases, stage0, boottrace, + cartridge/profile services, media creation/writing, and release signing. +- `fds burn` and `fds-burn` share one grammar. Format options are specific to + the selected class, with duplicate and conflicting options rejected. +- Clap generates command-specific help, versions, required-option errors and + typed bay/size validation. Global `--json` works around nested commands. +- `fds run BAY -- EXECUTABLE ARGS...` preserves every child argument, including + `--help`, `--json`, empty strings, spaces and an additional `--`. +- Private s6 flags and default daemon/PID-1 behavior are preserved. Stage0 + enters its emergency path on argument errors as PID 1 instead of exiting. +- VM client and write/stress helpers now build through Cargo and use Clap. + No project-owned Rust source reads or slices a raw process argument vector. + Kernel command-line files and metadata remain separate domain parsers. + +A regression test caught that Clap's broad argument/subcommand conflict rule +also rejected `fds-inspect --json image IMAGE`. The final implementation uses +Clap's typed conversion hook to reject only simultaneous target/image forms; +it accepts global options before or after the image subcommand. This behavior +passed host parser tests and execution of the actual ARM binary. + +## Build and runtime evidence + +The final tooling run passed **48 Rust unit tests**, ELF checks for every ARM +executable/alias, real ARM help/version/invalid-input tests, strict metadata +checks and stage0 diagnostic fixtures. Evidence: `out/logs/clap-tooling-test.log`. +The independent Dasung simulator/service and host/ARM/OpenSSL signing suites +passed in `out/logs/clap-dasung-test.log` and +`out/logs/clap-signing-test.log` (`out/m12-signing.9xgbool5/`). + +`make bootstrap`, `make smoke-test`, and `make check` passed. The complete +`make all` build passed in `out/logs/clap-all-final.log`; the earlier +`clap-all.log` attempt was intentionally stopped before fixing the inspection +option-placement regression. The final `make check` also passed in +`out/logs/clap-check-final.log`. + +All suites below passed against the rebuilt images. Each has its own +`out/logs/clap-.log` and retained VM/image evidence: + +| Make target | Evidence directory | +| --- | --- | +| `init-test` | `out/m2-vm.lsptD3/` | +| `boot-test` | `out/m4-vm.4VJHfm/` | +| `console-test` | `out/m5-vm.ktln7za0/` | +| `cartridge-test` | `out/m6-vm.nv3oq41q/` | +| `data-test` | `out/m7-vm.yv53pjkk/` | +| `desktop-test` | `out/m8-vm.q8nwbhgc/` | +| `media-image-test` | `out/m9-images.9ts4w_qb/` | +| `media-test` | `out/m9-vm.06yddrko/`, `out/m9-system.a9n8xxb_/` | +| `power-test` | `out/m10-vm.yk62sod6/` | +| `stress-test` | `out/m11-vm.vg5dicrw/`, `out/m11-faults.pd52oxs7/` | +| `recovery-test` | `out/m12-recovery.0604mqgm/` | +| `internal-test` | `out/m12-internal.uj0rbf4i/` | +| `clock-test` | `out/m12-clock.f54asrop/` | +| `development-test` | `out/m12-development.mu29x4mr/` | + +These checks include actual SYSTEM creation/write/readback/boot, confirmed DATA +repair, interrupted-checker quarantine, persistent settings across reboot, +window manager and DHCP control, sustained writes, all twelve overlapping I/O +intervals, and injected write/flush/readback/worker/daemon failures. They use +regular-file images and isolated ARM VMs. No physical media was written. + +## Executable size and startup measurements + +Clap increases executable sizes. All eight production Rust programs (including +Dasung) remain statically linked and contain no personal build-path prefix. +`out/clap-binary-sizes.json` records the final hashes and these sizes: + +| Executable | Before, bytes | After, bytes | +| --- | ---: | ---: | +| `fds` | 900,720 | 1,097,960 | +| `fds-burn` | 821,344 | 990,312 | +| `fds-stage0` | 474,336 | 633,880 | +| `fds-boottrace` | 442,152 | 606,672 | +| `fds-cartridged` | 917,328 | 1,064,832 | +| `fds-profile` | 527,880 | 687,904 | +| `fds-release` | 434,720 | 595,776 | + +Five alternating QEMU TCG boot pairs compare the previous complete image set +with the rebuilt images, using the same Pi kernel. The previous images also +precede Rust source-path normalization; this is an image-level comparison, +not an isolated microbenchmark of the parser. Each measurement uses Linux +CLOCK_BOOTTIME from kernel start to the first ordinary-user console. + +| Image set | Five samples, ms | Median, ms | +| --- | --- | ---: | +| before | 1958.754852, 2031.634456, 1998.790409, 1951.843302, 1961.052514 | 1961.052514 | +| clap | 1917.447815, 1909.301394, 1923.776234, 1955.579658, 2053.676925 | 1923.776234 | + +The median change was **−37.276280 ms**, below the +100 ms regression threshold. +This does not establish a speedup on the Pi. Input hashes, all sample reports +and serial logs are in `out/clap-console-pairs.qyyoppcj/`; the replay script is +`out/clap-boot-comparison.py`. + +The updated stage0 was also built at all three optimization levels and booted +three times per level (`out/rust-profiles.8hnnusn8/comparison.json`): + +| Optimization | Stage0 bytes | Median kernel-to-console, ms | +| --- | ---: | ---: | +| `z` | 633,880 | 1922.658965 | +| `2` | 730,568 | 1938.769685 | +| `3` | 757,168 | 1950.010662 | + +Retain `z`; these VM samples do not establish a physical-hardware winner. The +separate twelve-device test observed a +180.457 ms median versus empty bays. +That comparison adds twelve emulated USB devices and is not the before/after +Clap comparison. Its readiness-gate test still proved that cartridge service +readiness does not block the console. Physical Pi boot and shutdown targets, +USB electrical behavior, display recovery and flash durability remain deferred. + +## Release status + +The requested CLI migration is complete and included in the signed local +0.1.0 release. Fresh v5 offline builds produced 27 byte-identical artifacts, +which also match the normal images used by every runtime suite above. The old +v4 candidate was stopped before acceptance. See [final M12 evidence](Developer-M12-Validation#final-local-release-acceptance) +and the [67-section implementation ledger](Developer-Implementation-Status). diff --git a/Developer-Cartridges.md b/Developer-Cartridges.md new file mode 100644 index 0000000..f075cdc --- /dev/null +++ b/Developer-Cartridges.md @@ -0,0 +1,204 @@ +--- +include_toc: true +--- + +# Using cartridges and configuring bays + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Boot images](Developer-Boot) · [Services](Developer-Services) + +M6 adds the base `fds-cartridged` service, twelve bay states, USB hardware +recognition, strict metadata inspection, read-only storage mounts and eject. +Its [ARM VM acceptance checks passed](Developer-M6-Validation). [Writable DATA integration](Developer-Data) has passed M7 software acceptance; +[desktop/program activation](Developer-Desktop) passed M8 and [media creation tools](Developer-Media-Tools) +passed M9 software acceptance. [M10 ordered shutdown](Developer-Power) has passed software acceptance. + +## Inspect the current machine + +At the FDS console: + +```sh +fds bays +fds bay 3 +fds cartridge 3 +fds --json bays +``` + +The ordinary `fds` user can use these commands. `bays` shows all twelve bays, +including empty and unconfigured ones. `cartridge` includes validated metadata +and the current mount path. `--json` exposes a versioned machine-readable report. +`fds rescan` explicitly refreshes inventory; normal insert/remove events trigger +refresh automatically. The console does not wait for the inventory scan. + +| State | Meaning and next action | +| --- | --- | +| UNCONFIGURED | No measured controller/port mapping exists for this bay | +| EMPTY | Configured bay with no detected USB device | +| HARDWARE | USB device without storage; catalog name or VID/PID is shown | +| UNRECOGNIZED STORAGE | Storage exists but no named FDS partition is available | +| MOUNTED READ ONLY | Manifest validated; filesystem available at the displayed path | +| MOUNTED READ WRITE | Active DATA is writable at `/data`; eject before removal | +| PROTECTED | The active SYSTEM root; eject is refused | +| AMBIGUOUS | Multiple devices or named partitions match; nothing is selected arbitrarily | +| ERROR | Inspection, mounting or metadata validation failed; read the diagnostic | +| SAFE | The requested unmount succeeded; remove the cartridge | + +An unrecognized storage state may briefly appear while the kernel discovers its +partitions. It is not permission to remove a device being used elsewhere. +Errors and ambiguous media never produce SAFE. An additional mount of the same filesystem also blocks managed eject until +it is unmounted. + +## Calibrate the physical bay map + +The shipped `/etc/fds/bays.toml` is deliberately empty: no Pi bay wiring has yet +been measured. This produces UNCONFIGURED states, not invented assignments. + +1. On the assembled machine, insert one known USB device into one physical bay. +2. Run `fds topology`. This diagnostic explicitly shows controller/port paths; + ordinary bay commands omit those implementation details. +3. Record that path and repeat for each bay. Test both USB 2 and USB 3 devices, + because their companion root hubs can have different logical port paths. +4. Put the measured map in your machine configuration directory and build the + [internal image](Developer-Internal-Storage), or use `fds machine install DIRECTORY` + from recovery and reboot. Machine settings persist independently of SYSTEM. + The packaged `/etc/fds/` files are fallback defaults for missing/invalid internal + storage; `fds machine status` shows which source is active. +5. Verify all twelve devices together, in different insertion orders and after + reboot. That physical acceptance remains deferred. + +A **syntax example only**, using identities that must be replaced by observations: + +```toml +[front] +hub = "platform/example-controller:usb2/1" +[front.ports] +1 = 1 +2 = 2 +3 = 3 +4 = 4 +5 = 5 +6 = 6 + +[front_superspeed] +hub = "platform/example-controller:usb3/1" +[front_superspeed.ports] +1 = 1 +2 = 2 +3 = 3 +4 = 4 +5 = 5 +6 = 6 +``` + +`hub` names the controller and parent port chain. Its numbered `ports` map the +next downstream port to a bay number from 1 through 12. Additional groups can +map the rear hub. Explicit USB 2/3 aliases may refer to the same bay; duplicate +ports, overlapping parent/child mappings, and duplicate bays within one hub are +rejected. If two devices simultaneously match aliases for one bay, it becomes +AMBIGUOUS. A hub inserted into a cartridge bay with multiple downstream devices +also needs an explicit future composite-device policy; M6 refuses to guess. + +The identity excludes the Linux USB bus number and `/dev/sdX` enumeration order. +It includes the controller's sysfs path, USB protocol generation and port chain. +Moving a hub to another controller or changing wiring requires recalibration. + +## Storage formats and metadata + +| Class | GPT partition name | Filesystem and current behavior | +| --- | --- | --- | +| SYSTEM | FDS_SYSTEM | EROFS; current root is protected, additional media read-only | +| DATA | FDS_DATA | ext4; inspected read-only first, then activated at `/data` under the M7 policy | +| PROGRAM | FDS_METADATA + FDS_PAYLOAD02… | Metadata and xz bundles in `1+m` EROFS partitions; see [software format](Developer-Software-Format) | +| Legacy PROGRAM | FDS_PROGRAM | Existing single EROFS application tree; still readable | +| ENVIRONMENT | FDS_ENVIRONMENT | EROFS; declarative selection of a trusted built-in profile | +| UTILITY | FDS_UTILITY | EROFS; inspection only, no automatic actions | +| HARDWARE | None required | VID/PID, optional serial, device/interface class and bay | + +Storage contains `/FDS/CARTRIDGE.TOML`. For example: + +```toml +format = 1 +[cartridge] +id = "fds.windowmaker" +name = "WINDOW SYSTEM" +class = "environment" +version = "0.1" +[media] +writable = false +[activation] +profile = "windowmaker" +``` + +The manifest class must match the GPT partition name. SYSTEM, PROGRAM and +ENVIRONMENT are read-only; DATA declares writable media during the initial read-only inspection. Unknown keys, root commands, unsafe identifiers, control +characters, symlink metadata and nonregular metadata files are rejected. +The parser accepts at most 64 KiB. No `/FDS/autorun.sh` is run. + +Inspect a standalone manifest with `fds inspect /path/to/CARTRIDGE.TOML`. +For mounted storage, `fds cartridge N` reports the daemon's validated copy. +Media is inspected in a root-only staging directory, then published at +`/run/fds/media/NN` after validation, with nodev, nosuid and noexec. Device +major/minor and kernel disk sequence are checked before mounting an opened +block-device descriptor, preventing stale enumeration names from selecting a +replacement disk. + +## Eject a mounted cartridge + +```sh +fds eject 2 +``` + +Leave any shell working directory inside that cartridge and close files first. +SAFE is emitted only after an ordinary unmount succeeds. Busy mounts return an +error; forced or lazy unmount is never used to declare safe removal. Once SAFE, +the daemon keeps that insertion unmounted until removal and reinsertion, including +across a service restart. Its root-owned volatile marker is tied to the kernel +disk sequence, so a replacement disk is not mistaken for ejected media. +The active SYSTEM cannot be ejected. Swap SYSTEM only after shutting down. + +Pulling a mounted cartridge without eject is surprise removal. The daemon clears +its inventory and detaches a vanished read-only mount where needed; that cleanup +is not a successful eject. [M7 DATA handling](Developer-Data) extends ejection to writable DATA, consumer +tracking and syncfs. The temporary `/home/fds` remains usable without DATA and is lost at +power-off. + +## Hardware recognition + +Edit `packages/fds-cartridged/files/hardware-catalog.toml`, then rebuild SYSTEM. +Entries name devices, never executable actions: + +```toml +[[device]] +name = "MY SERIAL ADAPTER" +vendor = "1234" +product = "5678" +# Optional exact restrictions: +serial = "UNIT-1" +class = "02" +``` + +Replace these example IDs with observed values. Device or interface class may +match `class`. Multiple matching catalog entries produce an error; unknown +hardware remains identified by VID/PID without being mistaken for empty media. +Dasung control remains in the independent base monitor service. + +## Implementation and diagnostics + +The daemon subscribes to kernel USB/block events before taking its first snapshot. +A receive-buffer overflow causes a fresh snapshot. IPC uses a bounded Unix socket +at `/run/fds/control.sock`, mode 0660 and group `fds`, with peer UID checks for +root and the ordinary FDS user. Slow clients have individual deadlines and do not +block other clients. No asynchronous runtime or new external Rust dependency is +introduced: the existing std, libc, serde, serde_json and toml components suffice. + +Root diagnostics: `s6-svstat /run/service/cartridged` and +`cat /run/log/cartridged/current`. Logs are bounded and volatile. Restart through +`s6-rc -l /run/s6-rc -d change cartridged`, then the corresponding `-u` command. +Close users of cartridge mounts first; restart cleanup refuses busy leftovers. + +On the build host, `make cartridge-test` runs fixtures and full virtual USB tests. +These exercise actual kernel events and mounts but cannot establish physical +wiring, USB power stability, or Pi port behavior. M11 expands to twelve-device +stress, and physical calibration must be recorded separately. diff --git a/Developer-Cleanup.md b/Developer-Cleanup.md new file mode 100644 index 0000000..46027c8 --- /dev/null +++ b/Developer-Cleanup.md @@ -0,0 +1,135 @@ +--- +include_toc: true +--- + +# Reclaim build space + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +Run these from the repository root after stopping builds and test VMs: + +```sh +make clean-preview +make clean +``` + +The preview lists every selected directory, its combined allocated footprint, +and the generated directories being kept. It does not delete files. `make clean` +recalculates the selection and deletes it without a confirmation prompt. Repeat +it whenever old build/test workspaces accumulate. It needs Python 3, Git and GNU +`du`, already used by the build host; it needs no Rust compilation, downloads, +root privileges or new Python packages. + +## What is removed + +- Old generated rootfs, kernel, initramfs, SYSTEM, boot, recovery and internal + image workspaces that are no longer referenced by published output links. +- Old generated integration-test directories, including their disk images, + disposable test DATA overlays and local diagnostic logs. The current results + referenced by `*-latest` links or workstation current-pointer files are kept. +- Old generated package staging copies and service compilation directories. +- This checkout's `target/` Rust compilation output. Exported executables in + `out/` and `out/workstation/` remain available. The next Rust build recompiles + as needed using the retained Cargo source cache. + +The command recognizes the specific temporary-directory names produced by the +repository's builders and tests. It never treats all of `out/` as disposable. +Unknown names are left alone, including ad hoc diagnostic experiments. + +## What is kept + +- Current rootfs archives for every profile, kernel, initramfs, boot, recovery, + internal disk and SYSTEM images, through their published symlinks. Links in + `out/manifests/` and links within retained workspaces also retain their targets. +- Latest published test runs, the current workstation image/emulator fixtures, + the current Dasung s6 database, and the newest `m9-images.*` fixture needed by + the media tests. These can still occupy tens of GiB. +- Signed releases such as `out/fds-os-0.1.0/`, input snapshots such as + `out/inputs-m12-v5/`, and independent restored trees such as + `out/rebuild-m12-v5-a/`. These remain available for release reproduction. +- User-created cartridge images, software bundles and personal emulator sessions + such as `out/my-emulator/`, including their persistent DATA overlays. +- `out/logs/`, `out/manifests/`, packages, downloads, `out/cache/`, `.host/`, the + Void checkout/build container, and Cargo/Rustup caches outside this checkout. +- Git-tracked files and any recognized workspace containing a `.fds-keep` entry. + +To retain an older generated workspace for investigation, put a marker in it +before cleaning: + +```sh +touch out/m8-vm.YOUR_RUN/.fds-keep +make clean-preview +``` + +Replace `YOUR_RUN` with the actual directory suffix. Remove that marker when the +workspace is disposable again. Reserve the generated names for the build/test +tools; store personal sessions and images under your own names. + +Cleanup leaves historical acceptance reports intact. Reports mentioning deleted +older workspaces are historical records; their files cannot be rechecked until +rebuilt, and they never prove that changed source code passed acceptance. + +## Why the directory became so large + +Image builders and integration tests deliberately retain their working +directories so failed runs can be inspected and a failed build cannot replace a +good published image. Repeated suites therefore accumulate complete rootfs +trees, archives and several disk images per case. Before this target existed, +this checkout's `out/` measured approximately **900 GiB** of allocated file +blocks after repeated M0–M12 and workstation validation runs. That is accumulated +history, not the size of a single OS image. + +`make clean` removes superseded workspaces; it does not impose automatic +retention during a build. Use the preview periodically. For complete builds, +allow tens of GiB per independent tree plus space for VM tests and saved inputs; +the small initial smoke test's space requirement does not cover the full suite. + +The preview uses allocated blocks, rather than adding the apparent capacities +of sparse VM disks. Hardlinks are counted once within the selection. Reflinks, +filesystem snapshots and hardlinks into retained directories can reduce the +physical space actually recovered. The command also prints the observed change +in filesystem free space; other processes and delayed filesystem accounting can +affect that number. + +## Running safely and rebuilding + +Use cleanup sequentially, just like the existing shared-container builds. It +holds the existing rootfs/base-package/image-tool locks, excludes another +cleanup, and refuses mounted candidate paths or observable active build/VM +processes. These checks are safeguards, not permission to start a separate build +while cleanup is running. It never follows a candidate symlink into another +directory, and refuses a symlinked `out/` entirely. Only the checkout containing +the script is eligible; there is no arbitrary deletion-path option. + +Existing published images and native tools remain usable after cleanup. To +refresh exported binaries, run `make tooling` or `make workstation`. To rebuild +the smoke-test artifacts and rerun build checks: + +```sh +make smoke-test +make check +``` + +`make check` needs the smoke-test binary in `target/`, so run the smoke test first +after cleaning. `make clean-test` independently exercises cleanup selection, +preserved releases/data, repeated runs, symlinks, tracked files, read-only guest +directories, hardlinks, locks, live processes and mount rejection using tiny +temporary repositories. It does not build OS images or boot VMs. + +## Recorded cleanup validation + +On the development checkout, cleanup removed 576 disposable directories with +a 638.00 GiB allocated footprint. The filesystem reported 605.86 GiB more free +space immediately afterward; `out/` dropped from about 900 GiB to 268 GiB. +The retained space includes independent release rebuilds, input snapshots and +current image/test workspaces. A second preview selected zero directories. + +All 52 recorded current image/tool and frozen release file hashes remained +identical, and all 33 published links remained intact. `make bootstrap`, +`make smoke-test`, `make check` and all nine cleanup fixture tests passed. +The smoke test skipped its optional ARM execution because host `qemu-aarch64` +was unavailable; its ELF checks passed. This validates host cleanup and does +not add any physical Pi or monitor acceptance evidence. The local command log +is `out/logs/cleanup.log`, with a summary in +`out/manifests/cleanup-acceptance.json`. diff --git a/Developer-Current-Revision.md b/Developer-Current-Revision.md new file mode 100644 index 0000000..1094e81 --- /dev/null +++ b/Developer-Current-Revision.md @@ -0,0 +1,126 @@ +--- +include_toc: true +--- + +# Cartridge and desktop revision + +This work replaces the custom software bundle workflow and separates the user +manual from development history. The accepted 0.1.0 release stays unchanged. + +## Required outcomes + +1. Professional user documentation contains installation, usage and reference + material. Milestone history, measurements, test gaps and implementation notes + are grouped as `Developer-*` pages in the wiki. +2. `fds-cartridge` builds Void source packages with `xbps-src`, installs their + runtime dependencies on the workstation, and writes installed program trees + directly into EROFS payloads. New cartridges require no guest extraction. + Existing cartridge reading stays available. +3. The grayscale E-Ink WindowMaker appearance is the default desktop theme on + every display, with normal user customization still possible. +4. Validated cartridge commands become available in existing shells through a + stable PATH directory. Direct launches retain terminal behavior, arguments, + exit status, dependency paths and managed eject/unplug cleanup. Command name + collisions have deterministic, documented behavior. +5. A native Rust X11 control panel with a matching retro appearance manages bay + status, cartridge details, program launch and safe eject from WindowMaker. + +## Implementation and acceptance + +- Replace creation of custom xz bundles with typed Clap source-package options, + installed-tree metadata, deterministic integrity checks and direct mounts. +- Verify real Void builds, dependency installation and resulting image contents; + retain corruption, filesystem and write/readback rejection coverage. +- Add managed PATH launch registration and exercise foreground commands, + collisions, eject, surprise removal and service restart in the ARM guest. +- Package and integrate the X11 panel and defaults. Exercise the actual window, + input actions and rendered appearance in the guest desktop. +- Rebuild CLI and development images. Run workstation/emulator, rootfs/init and + relevant cartridge/DATA/desktop/shutdown regressions against changed sources. +- Rewrite and relocate documentation after the final interfaces are verified. + Keep hardware-specific validation procedures and limits in developer notes. +- Use `make clean-preview` and `make clean` between completed validation batches + to avoid retaining hundreds of GiB of obsolete images again. + +Current status: all five changes are implemented and the fresh host/ARM VM +acceptance below has passed. Physical checks remain a separate hardware task. + +## Fresh acceptance + +The validation logs are retained under `out/logs/cartridge-desktop-revision/`. +`out/manifests/cartridge-desktop-revision.json` records the actual working-tree +source hashes, artifact hashes, fixture identities and log hashes. The source +revision is uncommitted; HEAD alone does not identify these changes. +These checks ran against this revision, independently of the frozen release. + +| Check | Evidence and scope | +| --- | --- | +| Build host | `make bootstrap`, `make smoke-test`, `make check`; all passed. The optional host-native `qemu-aarch64` shortcut was skipped; ARM execution uses the prepared Void environment. | +| Root filesystems | `make rootfs PROFILE=cli` and `PROFILE=development`, with `make rootfs-test` for each: passed. Archives `rootfs-build.9f2sNj` and `rootfs-build.PIumlA` contain byte-identical copies of the exported FDS tools and Dasung daemon. | +| Boot / tools | `make init-test`, `out/m2-vm.ATiMG0`: native s6 PID 1, service control and orderly poweroff passed. `make tooling-test` passed shared contracts and real ARM diagnostic execution. Both SYSTEM images and initramfs were rebuilt. | +| Rust | Host tests for common contracts, software, workstation tools, cartridge service, CLI and X11 panel; static-musl ARM compilation passed. | +| Software creation | `make workstation-test`, `out/workstation-images.f6tq1mya`: actual xbps-src builds, installed dependencies, EROFS trees, deterministic image bytes, malformed-content rejection and confirmed file write/readback. | +| Emulator / foreground commands | `out/emu-test.tqxiaglf`: direct PATH and qualified names, literal arguments, pipes, exit status, cwd permissions, interactive input, Ctrl-C, suspend/resume, terminal restoration, collisions, unplug, daemon restart and legacy format-1 archive compatibility passed. | +| Cartridge control | `make cartridge-test`, `out/m6-vm.e5j5n3q5`: active SYSTEM protection, unprivileged IPC, bounded clients, mounts, busy/safe eject, hotplug, removal and restart passed. | +| DATA | `make data-test`, `out/m7-vm.oiam2j2z`: sustained writes, privilege drop, descendant cleanup, sync/unmount, persistence, multiple candidates, interrupted media and surprise removal passed. | +| Desktop | `make desktop-test`, `out/m8-vm.zgk_yiyn`: real ARM Xvfb/WindowMaker, authentication, UID 1000 panel, global grayscale defaults, mouse launch of legacy and Void programs, keyboard rescan, safe eject and session cleanup passed. ENVIRONMENT and isolated DHCP regressions also passed. | +| Shutdown | `make power-test`, `out/m10-vm.rx6kz1zd`: ordered poweroff/reboot, writeback, active preparation, quarantine and refusal on unsafe storage passed. | +| Documentation | User instructions and development history are separated; all 66 Markdown documents had balanced code fences and valid relative file links. The actual panel screenshot was visually reviewed. | + +The [desktop screenshot](Asset-images-fds-control.png) comes from the ARM guest, +not a mockup. Additional shell/where commands in its example cartridge belong +to the test fixture; the ordinary example publishes hello and report. + +## Validation limits + +The host used for acceptance is x86_64 Arch Linux with the prepared Void runner. +The native tool interface supports other Linux workstations with the documented +prerequisites, but this run does not establish coverage of every distribution. + +No physical Raspberry Pi boot, USB bay calibration, flash power-loss durability, +Dasung power-cycle recovery, E-Ink refresh quality or hardware latency was tested. +Use the existing hardware procedures for those checks. VM timings are VM +observations only. The independent s6/Dasung base integration remains mandatory. +The saved 0.1.0 release and its frozen inputs were not modified. + +## Implementation notes + +The X11 panel uses `x11rb` 0.13.2 and its pure-Rust connection, with core X11 +requests and no optional extensions or C GUI toolkit. The protocol companion +crate generates requests; `gethostname` and `rustix` support X11 authentication +and OS access. Versions are locked in Cargo.lock, and their licenses are included +by the existing Rust notice generator. This keeps the GUI compatible with the +static-musl target. Reference: https://docs.rs/x11rb/0.13.2/x11rb/. + +Foreground commands pass their already-open stdio descriptors to the privileged +service. The service creates the child, joins its root-owned bay cgroup, drops to +UID/GID 1000, and only then resolves the requested working directory. A pidfd and +exit status return to the launcher; interactive sessions use a PTY proxy. No +writable cgroup descriptor or permission to migrate arbitrary PIDs is exposed +to a client. This matters because Linux cgroup writes use open-file credentials. +Reference: Linux 6.12 kernel/cgroup/cgroup.c, cgroup_procs_write_start(). + +The first ARM compilation exposed musl/glibc differences in msghdr field widths; +portable field casts fixed those. Final acceptance uses rebuilt binaries and +images, rather than the intermediate builds that preceded that correction. + +The first emulator run found that Clap's multicall implementation uses +`Path::file_stem()`, truncating dotted aliases such as `b01:demo.report:shell`. +The launcher now lets Clap parse the invocation path as a typed positional and +uses a second typed external-subcommand parser for explicit `fds-program` calls. +Application help/version flags and dotted names have a regression test. This +keeps the no-manual-argument-parser rule while preserving the actual alias. + +The initial X11 click test raced the desktop terminal appearing above the panel. +The test now waits for that terminal and raises the panel before each real mouse +click. Actual program output, consumer tracking and SAFE state are still checked; +window existence alone is not treated as successful launch. + +The shutdown suite now publishes `out/m10-vm-latest` after successful completion. +That pointer was also installed for the successful run above. Cleanup retains +that evidence using the same published-link rule as the other runtime suites. + +After validation, `make clean` removed 52 obsolete workspaces and Rust build +output. The filesystem reported 62.17 GiB of newly available space. Current +images, latest runtime fixtures, caches, personal outputs and frozen release +inputs were retained; the complete cleanup log accompanies this revision. diff --git a/Developer-Dasung-Validation.md b/Developer-Dasung-Validation.md new file mode 100644 index 0000000..d6b79a4 --- /dev/null +++ b/Developer-Dasung-Validation.md @@ -0,0 +1,76 @@ +--- +include_toc: true +--- + +# Dasung integration validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Integration and usage](Developer-Dasung) + +Validated on 2026-09-20 on the x86_64 Arch development host. This records the FDS +port of the existing local `dasungd`; it is separate from the historical live +monitor validation in [the imported source](http://gitea.home.arpa/felis/fds-os/src/branch/main/rust/dasungd/VALIDATION.md). + +This is the original integration record. The subsequent rootfs build, emulated +ARM execution, and integration rerun are recorded in [M1 validation](Developer-M1-Validation). + +## Commands completed successfully + +```sh +make bootstrap +make smoke-test +make check +make dasung +make dasung-test +``` + +Bootstrap now fetches the locked workspace dependencies. A fresh empty Cargo +cache had exposed that resolving the new workspace member also affects an offline +build of the dependency-free smoketest; the bootstrap fetch addresses that setup +requirement. The original dependency versions were retained; the workspace lock +adds the smoketest member to the imported controller lock. + +## Checks performed + +- Static AArch64 ELF validation of both the built daemon and the executable + extracted from its `aarch64` XBPS package. No dynamic interpreter, shared-library + requirements, or glibc symbol requirements were present. +- Six imported unit tests: framing/reassembly, malformed streams, parameter and + IPC validation, saved-setting reload, and exact EDID identity/checksum/clock. +- Pseudo-terminal monitor tests: late attachment, keepalive, fragmented replies, + invalid settings, saved-setting replay, unplug/replug, duplicate-instance + exclusion, reply watchdog recovery, restart, and forgetting settings. +- Native s6 execution of the packaged service graph in an isolated container with + a read-only root and no exposed physical USB or display devices. Startup, + logging, socket permissions, restart, and clean stop passed with no monitor. + This used the x86_64 test build of the same source, not ARM emulation. +- Package contents, exact config/profile, mandatory base package selection, + base boot-bundle membership, and compilation of packaged service sources. +- The original M0 ELF/source/overlay protections and Rust formatting checks. + +## Artifact identities + +```text +7a3daf7e7beadde7d4b0991a08af04952495e1ec155bd4a2e3605fb903dcc782 out/dasungd +9c33dc3808549016f388f409658abc3ba37ff0b8d05973dcf1d5f9f8dbd89e94 out/packages/fds-dasungd-0.1.0_1.aarch64.xbps +``` + +Package dependencies are coreutils, execline, s6, s6-rc, and eudev. The static +executable does not add a shared musl, libusb, or libudev runtime dependency. +The packet keepalive/pacing and confirmed 304210 kHz EDID are retained. + +Local generated evidence: build log (`out/logs/dasung-build-final.log`, workstation-only evidence), +test log (`out/logs/dasung-test-final.log`, workstation-only evidence), and +artifact manifest (`out/manifests/dasung-artifacts.sha256`, workstation-only evidence). These outputs are +ignored by Git and must be regenerated in a fresh checkout. + +## Not verified + +There is no complete rootfs, native PID 1 deployment, Pi kernel/initramfs, or +bootable image yet. ARM runtime execution, Pi video/cabling, physical image quality, +early-boot display handoff, and corrected cold-power recovery were not tested. +The workstation's live display service and configuration were not changed. +Saved overrides use volatile `/run` until persistent machine storage is integrated. +The service test is not a boot or shutdown benchmark. diff --git a/Developer-Dasung.md b/Developer-Dasung.md new file mode 100644 index 0000000..0206e78 --- /dev/null +++ b/Developer-Dasung.md @@ -0,0 +1,194 @@ +--- +include_toc: true +--- + +# Dasung Paperlike 13K base-system integration + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Services](Developer-Services) + +FDS includes the Rust **dasungd** controller from the project's earlier +**Fix Dasung monitor black screen** task. It targets the user's **Paperlike 13K +grayscale**, serial `L56051794302`, with the confirmed 3200 × 2400 timing at about +37 Hz. This is core display support: its package is required for the base SYSTEM +image, and its service belongs to the base boot bundle, including console-only use. +It does not require an ENVIRONMENT cartridge, WindowMaker, or a graphical session. + +**Implementation boundary:** the daemon, ARM package, and s6 service definitions +are included by the [rootfs assembler](Developer-Rootfs). [M2 native init](Developer-Init) starts +the controller in the base boot graph, including in the ARM VM with no monitor. +The [M4 boot path](Developer-Boot) starts an early copy before SYSTEM exists, then stops +and reaps it before base s6 starts the normal daemon. Software handoff is tested +in a VM; physical Pi display and monitor recovery remain unverified. The final +s6 database is compiled at image build time. + +## Build it on the workstation + +Complete `make bootstrap` as described in [Your first build](Developer-Getting-Started), +then run from the repository root, as your regular user: + +```sh +make dasung +make dasung-test +``` + +The first command prepares the additional C cross compiler inside the existing +Void build container, fetches the crates pinned in `Cargo.lock`, and cross-builds +static-musl `dasungd`. It verifies the ELF, builds the `fds-dasungd` XBPS overlay, +indexes the output, and compiles the actual s6 source into a validation database. +The second runs unit tests and a simulated serial monitor, then inspects the +package and compiles the service definitions extracted from that package. It also +runs that s6 graph with a native test binary, a read-only container root, and no +physical device access to check startup, logging, permissions, restart, and stop. + +Python 3 is needed **only on the host for the simulator**. If absent on Arch, +install `python` with pacman. No Python interpreter is shipped for the daemon. +The first additional toolchain download is approximately 243 MB and needs roughly +1 GB of extra container space. Builds still depend on rolling Void binary inputs. +Run these commands sequentially with other FDS builds; they share the container. + +| Output | Purpose | +| --- | --- | +| `out/dasungd` | Static AArch64 executable; no target glibc, musl, libusb, or libudev shared library dependency | +| `out/packages/fds-dasungd-0.1.0_1.aarch64.xbps` | Target package with executable, config, exact EDID, s6 source, udev rule, and licenses | +| `out/manifests/dasung-s6-database.txt` | Path to the service database compiled on the host for validation | +| `out/manifests/dasung-artifacts.sha256` | Executable and package hashes | +| `out/manifests/dasung-build-packages.txt` | Installed build-container input versions | +| `out/logs/dasung-build.log`, `dasung-checks.log` | Build and test evidence | + +The standalone validation database contains only the Dasung subset. The M2 +image assembler merges it with the init package and compiles the full current +graph while building the rootfs. No daemon is started on the workstation by these targets; +the integration test uses its own temporary socket and pseudo-terminal. + +## What goes into the base system + +`image/base-packages.list` makes `fds-dasungd` a mandatory hardware package. +`packages/fds-dasungd/template` is the active FDS-owned XBPS overlay. The upstream +Void tree remains pinned and unchanged. Source is maintained at `rust/dasungd/`; +the original task directory is not required for future builds. + +The package installs: + +- `/usr/bin/dasungd` and `/etc/dasungd.toml`. +- `/usr/lib/firmware/edid/dasung-paperlike13k-37hz.bin`. +- `/etc/s6-rc/source/dasungd`, its runtime-directory and log services, and the + `boot/contents.d/dasungd` membership entry. +- `/usr/lib/udev/rules.d/99-dasung-spi.rules`, adapted to eudev syntax. +- Documentation and the controller/libusb licenses. + +s6 supervises the foreground daemon. A short oneshot creates private writable +runtime/log directories; it does not wait for the display. The daemon starts with +no monitor attached and retries discovery. Its initial service transition does +not wait for a monitor reply, a desktop, the network, or all cartridge bays. +The log service rotates a small log under `/run/log/dasungd`. + +The init package adds a dependency on `runtime-fs` so the controller starts with +its kernel interfaces mounted. The service runs as root because it claims USB interfaces. Its socket +is under a root-owned mode-0750 directory; runtime control is restricted to root. +A future FDS CLI can expose selected operations through an explicit policy. +No world-writable USB rule, systemd unit, or runit service is installed. + +## Confirmed profile and Pi-specific choices + +| Property | Recorded value | +| --- | --- | +| Monitor | Paperlike 13K grayscale; `L56051794302` | +| Resolution | 3200 × 2400 | +| Pixel clock | **304210 kHz** (304.21 MHz) | +| Horizontal active / sync start / sync end / total | 3200 / 3248 / 3280 / 3360 | +| Vertical active / sync start / sync end / total | 2400 / 2423 / 2427 / 2447 | +| Sync polarities | Positive horizontal and vertical | +| EDID SHA-256 | `b6c1e0a8d315d9cf5c2fb83784a177530bcc085c2d0724dc7478a9c12449e4f4` | +| Control transport | USB UART `1a86:7523`, with companion SPI bridge `1a86:5512` | +| Startup parameters | Raw mode 1 and contrast 4, previously read back on this unit | +| Keepalive | `20 01` every two seconds; minimum 150 ms between command writes | + +The video timing was confirmed on the original AMD workstation, **not on a Pi**. +The original USB-C video connection is not a Pi cabling prescription. Pi connector, +cable/adapter, mode acceptance, and picture stability require hardware validation. +For the current HDMI video plus USB control procedure, including separate monitor +power and a reversible native-EDID experiment, use the [display guide](Dasung). +Desktop 2× scaling is separate from EDID timing; a console uses its own font size. + +FDS configures `display.enabled=false` and `hotplug="none"`: the daemon handles +USB control without applying the source project's AMD debugfs disconnect/reconnect +procedure. The Pi boot/display integration must supply the appropriate KMS/firmware +EDID configuration for its real connector. The Pi boot-volume builder now supplies +an unqualified firmware EDID override for this dedicated monitor; it does not +assume `DP-2` or a numbered HDMI connector. See the [boot guide](Developer-Boot) for +the exact command line and the remaining physical mode test. + +The daemon requires the configured monitor's DRM EDID and the matching USB +companion topology before claiming a device. Multiple matching UARTs are rejected; +`usb_path` can disambiguate a verified Pi topology. Do not substitute a blanket +match for all CH340 adapters or copy this unit's identity to a different monitor. + +The original monitor's SPI driver interaction caused a dark picture after a power +cycle. `image/kernel/dasung.config` requires `CONFIG_SPI_CH341=n`; the Pi kernel +builder enforces this before boot, including the initramfs environment. The scoped +udev rule is secondary protection: it cannot prevent a built-in or already-loaded +SPI driver from probing. The daemon also reserves the companion interface without +sending SPI data and refuses to take it from an already-bound kernel driver. + +## Runtime usage once installed in an FDS image + +These are target commands, **not commands to run against the workstation's live +monitor during the build**: + +```text +dasungd status +dasungd query +dasungd refresh +dasungd set contrast 4 +dasungd set mode 1 --save +dasungd forget mode +``` + +`status` returns JSON including `connected`, `responsive`, reply age, and cached +parameters. A successful set means a packet was sent; use query/status to check +observed values. It is not proof of physical image quality. Color-model mode names +are not assigned to this grayscale monitor's raw numeric modes. + +The default socket is `/run/dasungd/control.sock`. External programs can use its +newline-delimited JSON protocol; the daemon remains the only USB owner: + +```json +{"op":"status"} +{"op":"set","parameter":"contrast","value":4,"save":true} +``` + +In FDS's current payload, saved overrides live in `/run/dasungd/settings.json`. +They survive a daemon restart **within the same boot**, but not power loss. +Mode 1/contrast 4 are persistent image configuration defaults. Once persistent +machine-state mounts are implemented, point `state_file` at that internal writable +storage to retain user overrides across boots. Do not put it on read-only SYSTEM +or make the base display daemon depend on a removable DATA cartridge. + +## Dependencies and source provenance + +The imported Rust dependencies are retained and pinned by the workspace lockfile: +`rusb` wraps USB access; `libc` supports Linux serial/locking calls; `ctrlc` handles +termination; `clap` provides CLI parsing; `serde`, `serde_json`, and `toml` support +configuration and the local socket; `anyhow` supplies contextual errors. +Transitive versions are recorded in `Cargo.lock`. + +The `vendored` feature builds libusb from the pinned `libusb1-sys` crate. The Void +`cross-aarch64-linux-musl` toolchain supplies C headers/compiler for that library; +Rust's bundled musl target alone was enough for the old pure-Rust smoke-test, but +is insufficient for this C dependency. `LIBUSB_NO_PKG_CONFIG` and +`LIBUDEV_NO_PKG_CONFIG` prevent accidental host/shared-library linkage. libusb uses +its Linux netlink backend. No libusb/libudev shared runtime package is needed. + +The target package depends on GNU coreutils for directory preparation, execline +for service launch, s6/s6-rc for supervision, and eudev for device rules. These +match the planned base userspace. The container's s6-rc is a build-time compiler, +not a target PID 1 running on Arch. + +[Import provenance](http://gitea.home.arpa/felis/fds-os/src/branch/main/rust/dasungd/IMPORT.md) records the original source hashes. +[Original validation](http://gitea.home.arpa/felis/fds-os/src/branch/main/rust/dasungd/VALIDATION.md) records the live handoff and +remaining physical test. Protocol and reconnect tests use simulated hardware; +Pi boot, image quality, physical disconnect/reconnect, and corrected cold-power +recovery must still be verified on the actual target. No boot benchmark is claimed. diff --git a/Developer-Data.md b/Developer-Data.md new file mode 100644 index 0000000..add0536 --- /dev/null +++ b/Developer-Data.md @@ -0,0 +1,145 @@ +--- +include_toc: true +--- + +# Writable DATA and managed programs + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Cartridges](Developer-Cartridges) · [Services](Developer-Services) + +M7 implements writable DATA. Its sustained-write, safe-eject and surrounding +image regression checks [passed in the ARM VM](Developer-M7-Validation). A DATA +cartridge uses a GPT partition named `FDS_DATA`, ext4, and a validated +`/FDS/CARTRIDGE.TOML` declaring class `data` and `writable = true`. +Use the [M9 media tools](Developer-Media-Tools) to create new DATA media. The virtual +tests construct disposable image files and never format a workstation disk. + +## Where your files live + +`/home/fds` remains a temporary home, whether DATA is present or absent. Its +contents disappear at power-off. Persistent user files belong under `/data`. +Separating these paths lets you eject DATA without hiding the active shell's +home directory or substituting another cartridge underneath it. + +After inserting a DATA cartridge into a configured bay: + +```sh +fds bays +fds cartridge 2 +ls /data +``` + +If exactly one valid DATA candidate is available and none is active, the daemon +activates it at `/data`. The filesystem must grant UID/GID 1000 the intended +write permissions; insertion does not recursively change ownership of user files. +M9's DATA formatter supplies that ownership for fresh media. + +An active DATA session stays attached to its current cartridge when another is +inserted. If a discovery snapshot contains multiple candidates with none active, +they remain read-only until you select one: + +```sh +fds data use 4 +``` + +Eject the existing active DATA before selecting a replacement. The daemon never +changes `/data` underneath an active session. EMPTY, ERROR and SAFE states retain +the meanings in the [cartridge guide](Developer-Cartridges); MOUNTED READ WRITE means +that the displayed DATA filesystem is available for user writes. + +## Run a program that participates in eject + +Ordinary console commands work as usual. They are not automatically killed when +you request eject: an open file or working directory may make eject fail as busy. +For a background task that should stop when its DATA is ejected, use: + +```sh +fds run 2 -- /usr/bin/bash -c 'date > /data/managed-example.txt' +``` + +This starts a **background** managed program and prints its process ID. It has no +interactive input. Output goes to the bounded, volatile cartridge service log: + +```sh +tail /run/log/cartridged/current +fds cartridge 2 +``` + +The executable path must be absolute. The child starts with `/data` as its working +directory, the ordinary FDS UID/GID, no supplementary groups, no effective +capabilities, a small explicit environment, and `no_new_privs`. Programs cannot +gain privileges through setuid executables. The limit is 256 managed processes +per bay. This interface accepts healthy writable DATA; the [desktop guide](Developer-Desktop) +also explains M8's explicit PROGRAM-media launch interface. + +A root-owned Linux cgroup tracks the program and its descendants, including +children that outlive their parent. Cgroups are kernel process tracking, not a +new init system or runtime package. The configured kernel already includes the +required cgroup v2 and process-limit support. The daemon sets up its hierarchy +independently of the console. See the [kernel cgroup interface](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html). + +## Eject and remove DATA + +```sh +cd "$HOME" +fds eject 2 +``` + +Eject checks for additional mounts, stops managed consumers, flushes the DATA +filesystem with `syncfs`, and performs a normal unmount. Only successful completion +produces SAFE. A retained filesystem descriptor observes writeback errors during +the session. M10 additionally makes the filesystem read-only and checks writeback +again before closing that descriptor for unmount, preventing an error-reporting +gap when a busy mount needs a later retry. +Linux's [syncfs interface](https://man7.org/linux/man-pages/man2/syncfs.2.html) +reports filesystem errors that must not be ignored. + +Managed programs receive TERM first. The daemon waits on actual cgroup exit +notifications; programs that have not exited after a one-second deadline are +killed as a group, including newly forked descendants. A further bounded exit +check must succeed. These are failure deadlines, not delays imposed on a program +that has already stopped. Requesting eject therefore ends those background jobs; +it does not promise that unfinished application work is completed. + +An unmanaged busy shell/file, extra mount, failed flush or failed unmount prevents +SAFE. Close the reported use and retry. A recorded DATA writeback fault requires +investigation or recovery; a later successful call is not used to erase it and +pretend the failed write succeeded. The daemon never uses lazy/forced unmount to +claim successful eject. A successfully ejected insertion remains unmounted across +a daemon restart until removal/reinsertion. + +M10 records writable sessions and faults outside the daemon process. If it crashes +before verified unmount, the same insertion stays quarantined and read-only on +restart. A later flush cannot reconstruct the lost error history. See +[Shutdown and recovery records](Developer-Power#data-recovery-records) for the current +implementation and its acceptance status. + +Pulling DATA during writes is an error. The daemon stops its managed consumers, +cleans up the vanished mount and logs that writes may have been lost. This path +never produces SAFE and makes no filesystem-cleanliness promise. Use recovery +inspection/repair before trusting media that was removed during I/O. + +## Build and verify on the workstation + +After changing DATA code or image configuration, build fresh inputs in order: + +```sh +make rootfs PROFILE=cli +make initramfs +make system-card +make data-test +``` + +The test boots disposable virtual media, writes continuously, verifies ordinary +user privileges and descendant shutdown, checks busy and additional mounts, +reopens persisted data, tests multiple candidates and simulates surprise removal. +After safe eject it extracts the ext4 partition into an ordinary file, runs +read-only `e2fsck -fn`, and compares the sustained-write payload byte-for-byte. +No host mount or physical USB device is used. + +Run `make cartridge-test`, `make init-test`, `make boot-test` and +`make console-test` for the surrounding regressions. Test logs and VM evidence +are written under `out/`. Real flash-controller caches, battery loss, USB power +and physical Pi eject/shutdown latency remain hardware acceptance work. diff --git a/Developer-Desktop.md b/Developer-Desktop.md new file mode 100644 index 0000000..8029a47 --- /dev/null +++ b/Developer-Desktop.md @@ -0,0 +1,219 @@ +--- +include_toc: true +--- + +# Desktop, programs, and networking + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Cartridge guide](Developer-Cartridges) · [DATA guide](Developer-Data) + +M8 software checks passed in the ARM virtual machine. The [validation report](Developer-M8-Validation) +records desktop, program and networking evidence. Raspberry Pi video output and +Dasung display quality remain deferred physical checks. + +## What starts when you boot + +FDS opens its ordinary-user console first. Xorg, WindowMaker, and the DHCP client +are installed in SYSTEM but are absent from the boot bundle. Dasung control +remains in the base boot bundle, including when no desktop is running. + +The default graphical backend is Xorg on virtual terminal 2, using its built-in +modesetting driver. The physical keyboard and pointer use libinput. Console +access remains on the original terminal. There is no display manager. Physical +VC4 output, monitor resolution, and E-Ink appearance still require Pi testing. + +## Start and stop the desktop + +At `FDS>`: + +```sh +fds profiles +fds profile activate windowmaker +fds profiles +``` + +Activation is asynchronous. The second status command reports the selected +profile and an activation-to-ready duration once WindowMaker is ready. This +measurement starts at the activation request, so time spent deciding when to +start the desktop is excluded. `fds boot-profile` also records the first desktop +readiness event for the boot. Neither measure includes firmware or power-on. + +The desktop opens an FDS terminal. Right-click its white background to open the +menu, launch another terminal, enable/disable Ethernet, or return to the console. +To stop it from either a terminal or the console: + +```sh +fds profile deactivate +``` + +Stopping the desktop closes its terminal and all processes descended from its +session. Save work first. Files in `/home/fds` survive desktop restarts during the +same boot; the home directory is still temporary and disappears at shutdown. +Use [DATA](Developer-Data) for persistent files. Removing an active DATA cartridge also +stops the desktop before flushing DATA, because desktop applications may use it. + +`fds-profile status`, `fds-profile activate windowmaker`, and +`fds-profile deactivate` provide the standalone static helper interface. + +## ENVIRONMENT cartridges + +An ENVIRONMENT cartridge is a read-only EROFS partition named `FDS_ENVIRONMENT`. +Its `FDS/CARTRIDGE.TOML` uses the [documented manifest format](Developer-Cartridges), with: + +```toml +[activation] +profile = "windowmaker" +``` + +When exactly one eligible ENVIRONMENT cartridge is present, the daemon requests +WindowMaker after console readiness. The graphical runtime comes from SYSTEM; +the cartridge provides a declarative request, never a privileged executable. +Unknown profile names do not execute anything. If several eligible cartridges +arrive together, select a profile explicitly instead of relying on bay ordering. + +Eject its bay with `fds eject N`. The desktop stops before SAFE is reported. +Surprise removal also stops its associated desktop. Manually deactivating while +the cartridge remains inserted suppresses automatic reactivation until removal +and reinsertion. A manually started desktop is independent of ENVIRONMENT media. +A cartridge-service restart reevaluates mounted media and retires old optional +sessions; this is a maintenance operation, not a way to preserve desktop jobs. + +## E-Ink defaults + +`fds-eink` installs the WindowMaker policy and Terminus fonts: a white background, +black text and borders, grayscale controls, outline movement/resizing, no +animations, no blinking decorations, no app-icon bounce, no dock, no compositor, +and no periodically updating clock. X11 compositing is disabled. The terminal +uses black on white and a nonblinking cursor. + +Defaults are copied into the ordinary user's `~/GNUstep/Defaults` at the first +session. Later sessions preserve edits there. They are temporary unless you +explicitly store a copy on DATA. The authoritative image defaults live in +`/usr/share/fds/eink`; edit their package sources and rebuild SYSTEM to change the +machine-wide defaults. + +Font caches and X11 font indexes are generated while building the image. +An active X server's keyboard map is session state, generated only on desktop +activation. That is separate from global boot-time cache generation. + +## Launch a software cartridge + +Build current software cartridges on a [Linux workstation](Developer-Workstation). They +contain metadata plus xz software bundles in payload partitions. In FDS: + +```sh +fds bay 4 +fds run 4 -- demo.hello:hello +fds eject 4 +``` + +Use the software id and command listed by `fds bay`. The guest verifies and +extracts the bundle into a read-only temporary cache, then starts it as UID 1000. +There are no guest software build hooks. + +### Legacy PROGRAM compatibility + +Existing PROGRAM media contains a read-only EROFS partition named `FDS_PROGRAM`, metadata +in `FDS/CARTRIDGE.TOML`, and its self-contained application files: + +```text +app/bin/editor +app/lib/ +app/share/ +``` + +FDS validates and mounts it at `/run/fds/apps/`. Insertion does not +run anything, and the mount initially disallows execution. To launch an executable +named `editor` from bay 4: + +```sh +fds cartridge 4 +fds run 4 -- editor +``` + +Use a simple executable name from `app/bin`, followed by that program's arguments. +The daemon enables read-only execution and starts it as UID 1000, with no added +privileges. Executable paths must remain within the cartridge's `app` directory. +Programs receive `FDS_APP`, a PATH including `app/bin`, `LD_LIBRARY_PATH` including +`app/lib`, and `XDG_DATA_DIRS` including `app/share`. Bundled libraries can also use +an executable-relative RPATH. FDS does not resolve dependencies across cartridges. + +Programs are background jobs. Graphical programs can use the active authenticated +X display. Standard output/error goes to the bounded, temporary cartridge log: + +```sh +tail /run/log/cartridged/current +fds bay 4 +fds eject 4 +``` + +The bay status reports managed processes. Eject stops the whole process group, +including forked descendants, before unmounting. The existing DATA form remains +`fds run N -- /absolute/system/command arguments`, using `/data` as its working +directory. DATA itself remains mounted with execution disabled. + +## Ethernet on demand + +External networking is off until explicitly requested or a mapped USB Ethernet cartridge +produces a real network interface. Interfaces are associated with their USB +ancestors; a descriptive label alone does not start networking. Wi-Fi setup is +not implemented by this Ethernet policy. The local loopback interface is always +up for applications on the same machine; it does not enable external traffic. + +```sh +fds network on +fds profiles +ip -brief address +ip route +fds network off +``` + +Enabling networking starts DHCP without waiting for a lease. An absent server +cannot delay `FDS>`. Explicit activation selects available Ethernet interfaces; +automatic activation selects interfaces under mapped cartridge USB devices. +Stopping networking stops DHCP, brings the managed interfaces down, and clears +temporary DNS settings. This initial profile uses IPv4 DHCP; it disables kernel +IPv6 autoconfiguration on those interfaces. After an explicit stop, an inserted Ethernet +cartridge remains suppressed until it is removed, or you enable networking again. + +Leases, DNS settings, and logs live under `/run`; nothing makes SYSTEM writable. +`/etc/resolv.conf` points to `/run/fds/resolv.conf`. Inspect +`/run/log/network/current` when an address is missing. + +## Development and troubleshooting + +The production console image includes the real desktop stack. The development +image adds Xvfb and X11 diagnostics for a virtual display without a connected GPU: + +```sh +make rootfs PROFILE=development +make desktop-test +``` + +The test creates its own disposable image selecting Xvfb; the normal development +image still selects Xorg. Never mistake an Xvfb pass for Pi graphics verification. +Changing the SYSTEM image profile requires building that rootfs first; the image +builder rejects a profile label that disagrees with its embedded identity. + +If desktop activation fails, `fds profiles` reports it. Read +`/run/log/xserver/current` and `/run/log/desktop/current`, then deactivate before +retrying. Xorg's detailed log is `/run/log/xserver/Xorg.0.log`. X access requires +an authority cookie readable only by root and the FDS group; do not use `xhost +` +or disable access control as a workaround. + +Dependency rationale: Xorg supplies the display server and modesetting driver; +libinput supplies keyboard/pointer input; WindowMaker supplies window management; +Terminus and fontconfig supply the typography; xterm supplies a terminal; the static FDS helper +observes X11 property events for readiness without polling delays; xset/xsetroot apply the +static session settings. Xvfb, xdotool, xwd, xwininfo, xdpyinfo, xprop, and xauth are +optional development diagnostics. `fds-dhcpcd` builds the pinned upstream DHCP +client with privilege separation and volatile state paths, omitting its runit +service files. All FDS control helpers remain static Rust/musl binaries. + +The readiness observer uses the [X11 core protocol](https://xorg.freedesktop.org/archive/X11R7.7/doc/xproto/x11protocol.html) +and establishes its property subscription before launching the window manager. +Font caches are generated with the image's normalized timestamp, following +[fontconfig's reproducible-cache support](https://fontconfig.pages.freedesktop.org/fontconfig/fontconfig-user.html), +so exporting the filesystem does not invalidate their directory timestamps. diff --git a/Developer-Development.md b/Developer-Development.md new file mode 100644 index 0000000..900ec31 --- /dev/null +++ b/Developer-Development.md @@ -0,0 +1,319 @@ +--- +include_toc: true +--- + +# Working on FDS/OS + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Troubleshooting](Developer-Troubleshooting) + +This guide is for someone who has completed the first build and wants to change +or inspect the working code. Run commands from the repository root, as a normal +user. M0–M12 software acceptance is complete; consult [AGENTS.md](http://gitea.home.arpa/felis/fds-os/src/branch/main/AGENTS.md) +and the [roadmap](Developer-Roadmap) before adding components. + +## Choose the smallest useful build + +| You changed or want to do | Run | +| --- | --- | +| See the available entry points | `make help` | +| Prepare a new checkout or restore missing host tools | `make bootstrap` | +| Edit the Rust smoketest | The Rust build and validation sequence below | +| Build/test the base display controller | `make dasung`, then `make dasung-test`; see [Dasung](Developer-Dasung) | +| Change the CLI, shared contracts or stage0 | `make tooling`, then `make tooling-test`; rebuild initramfs and run `make boot-test` for boot-path changes | +| Change kernel configuration | `make kernel`, rebuild rootfs and boot images, then `make boot-test`; see [Boot images](Developer-Boot) | +| Change USB discovery, mapping or media writing | Build CLI images, then `make stress-test`; see [Stress tests](Developer-Stress-Testing) | +| Change shutdown or DATA safety | Rebuild both profiles; run `make power-test` against development, plus `make data-test` and `make init-test`; see [Shutdown](Developer-Power) | +| Change init or boot services | `make rootfs PROFILE=cli`, `make rootfs-test`, then `make init-test`; see [Native init](Developer-Init) | +| Change base packages or rootfs assembly | `make rootfs PROFILE=cli`, `make rootfs-test`, then `make init-test`; see [Rootfs](Developer-Rootfs) | +| Build the native development toolset | `make rootfs PROFILE=development`, `make rootfs-test`, `make system-card PROFILE=development`, then `make development-test` | +| Change internal storage or machine settings | Rebuild CLI and recovery, then `make internal-test`; see [Internal storage](Developer-Internal-Storage) | +| Change release signing or verification | `make signing-test`; see [Release signatures](Developer-Releases) | +| Rebuild the test XBPS package | `./tools/build-package hello` | +| Verify an M0 build-system change | `make bootstrap`, then `make smoke-test`, then `make check` | +| Check Rust formatting and validation protections | `make check`, after building the Rust program | +| Read or edit documentation | Check examples, links, and claims against the implementation | + +Run builds sequentially. The scripts share one Void masterdir and package cache; +starting bootstrap and package builds concurrently is not a supported workflow. +`make packages` builds every FDS base package. `make all` builds the complete +artifact set; two independent frozen offline builds passed with matching output. See +[Frozen inputs and offline rebuilds](Developer-Reproducible-Builds). Use `make clean-preview` +and `make clean` to remove obsolete workspaces and Rust compilation output while +keeping published images, latest test fixtures, saved releases, inputs and +caches. See [Cleanup](Developer-Cleanup) for retention rules and rebuilding afterward. +There is no generic `make test` target. + +## Edit and rebuild the Rust program + +The active program is [rust/fds-smoketest/src/main.rs](http://gitea.home.arpa/felis/fds-os/src/branch/main/rust/fds-smoketest/src/main.rs). +It is deliberately tiny: it establishes that the intended target and static +linking work. `rust/dasungd/`, `rust/fds-common/`, `rust/fds-cli/`, and +`rust/fds-stage0/`, `rust/fds-boottrace/`, and `rust/fds-cartridged/` are also active. Directories absent from the workspace member +list remain reserved. Use the [tooling guide](Developer-Tooling) for the FDS programs, +[Cartridges](Developer-Cartridges) for bay and metadata changes, and [DATA](Developer-Data) +for writable storage and managed jobs. + +Use the shared `clap` workspace dependency for every Rust command-line interface. +Declare commands and options with `Parser`, `Subcommand`, `Args`, and typed value +parsers; do not match or consume raw argument vectors. Shared media syntax lives +in `rust/fds-burn/src/cli.rs`; installed FDS aliases use Clap's multicall support +in `rust/fds-cli/src/cli.rs`. Keep authorization and filesystem validation in the +operation layer after parsing. Add parser tests when changing required options, +conflicts, private s6 flags, aliases, or child-argument forwarding. + +The disposable Rust VM workloads in `tests/helpers/` are a Cargo workspace +member and use the same locked Clap dependency. They are never installed in +the base packages. Their stress workload uses release optimization level 2; +production executables retain the size-oriented release settings. + +After editing the program: + +```sh +cargo fmt --all +cargo build --locked --offline --release \ + --target aarch64-unknown-linux-musl -p fds-smoketest +./tools/verify-elf target/aarch64-unknown-linux-musl/release/fds-smoketest aarch64 static +make check +``` + +Cargo writes to `target/aarch64-unknown-linux-musl/release/`. It does **not** refresh +`out/fds-smoketest` or its manifest. Run `make smoke-test` when you need a freshly +exported artifact and matching checksums. + +If you installed optional QEMU, execute the freshly built file with: + +```sh +qemu-aarch64 target/aarch64-unknown-linux-musl/release/fds-smoketest +``` + +The smoke-test contract expects exactly `FDS/OS M0: aarch64 static-musl OK`. +Changing that text requires updating the runtime assertion and documentation as +part of the same change. The crate also rejects the wrong architecture, libc, +or missing static CRT at compile time. Do not bypass those checks to make an +x86_64 build appear to pass. + +The release profile and target configuration live in [Cargo.toml](http://gitea.home.arpa/felis/fds-os/src/branch/main/Cargo.toml) +and [.cargo/config.toml](http://gitea.home.arpa/felis/fds-os/src/branch/main/.cargo/config.toml). Rust and target versions are +selected through [rust-toolchain.toml](http://gitea.home.arpa/felis/fds-os/src/branch/main/rust-toolchain.toml). This M0 crate has +no external crate dependencies. `--offline` works after bootstrap has installed +the toolchain; it is not a promise that the complete build pipeline is offline. + +## Build and inspect a package + +```sh +./tools/build-package hello +``` + +This checks the Void source pin, prepares configuration and overlays, invokes +xbps-src's aarch64 cross build, and copies the resulting main package into +`out/packages/`. Its log is `out/logs/xbps-hello.log`. Follow +[Packages](Developer-Packages) for indexing, querying, and extracting the output. + +`hello` is the verified M0 package. The helper accepts other template names, but +its exporter currently looks only for that main package in the top-level +`hostdir/binpkgs/`. Packages emitted into subrepositories and separately named +subpackages need additional handling; the helper is not yet a universal package +export interface. + +For diagnosis, this is the underlying command in its correct host environment: + +```sh +( + export PATH="$PWD/.host/xbps/usr/bin:$PATH" + export XBPS_ARCH=x86_64 + cd vendor/void-packages + ./xbps-src -a aarch64 pkg hello +) +``` + +The subshell keeps the directory and environment changes local. Prefer the FDS +helper for normal work: it also checks the pin and exports the output. The raw +command does neither. Do not run `xbps-install` against the Arch host or install +an ARM package into the x86_64 build container. + +## Change build configuration deliberately + +The source of truth is [config/xbps-src.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/xbps-src.conf). +`XBPS_MAKEJOBS` controls package build parallelism; the current value is 4. +`XBPS_CHROOT_CMD=bwrap` selects the supported container backend. + +`tools/prepare-void` copies this file to the ignored +`vendor/void-packages/etc/conf`. If the two differ, it stops instead of silently +overwriting local settings. After intentionally editing the checked-in file, +review the difference: + +```sh +diff -u vendor/void-packages/etc/conf config/xbps-src.conf +``` + +A difference makes `diff` exit 1; that is expected. Once the checked-in version +contains every setting you intend to keep, synchronize and prepare: + +```sh +cp config/xbps-src.conf vendor/void-packages/etc/conf +./tools/prepare-void +``` + +This overwrites the local generated configuration, so reconcile any intentional +local edits before copying. Re-run the three M0 validation commands after a build +configuration change. A mirror change does not lock the packages served by that +mirror; see [input limitations](Developer-Build-Host#known-limitations). + +## How future package overlays fit + +FDS-owned package sources belong in `packages/fds-*/`, outside the upstream +submodule. An overlay becomes active only when its directory contains a +`template`. `fds-base`, `fds-base-files`, `fds-init`, `fds-dasungd`, `fds-cli`, `fds-kernel`, +`fds-cartridged`, `fds-eink`, and `fds-dhcpcd` are active. + +For active overlays and future templates, `tools/prepare-void` copies each +active directory into `vendor/void-packages/srcpkgs/`. It allows an identical +repeat copy, rejects a name that collides with a tracked upstream package, and +stops if a previous generated copy differs. Edit the source under `packages/`, +then reconcile the specific stale generated copy. Never put the only copy of an +FDS change inside `vendor/`. + +The [troubleshooting guide](Developer-Troubleshooting#configuration-or-overlay-conflict) +explains how to preserve and inspect a conflicting copy. Empty future directories +are intentional; adding a fake template to silence a missing-package error does +not implement that milestone. + +## Preserve logs and identify inputs + +The normal log filenames are reused. Before reproducing a failure that you need +to retain, save the existing logs: + +```sh +log_archive=$(mktemp -d "$PWD/out/saved-logs.XXXXXX") +cp -a out/logs/. "$log_archive/" +printf 'Saved logs to %s\n' "$log_archive" +``` + +After a successful smoke-test: + +```sh +cat VOID_PACKAGES_COMMIT +rustc --version +cargo --version +cat out/manifests/void-build-packages.txt +sha256sum -c out/manifests/artifacts.sha256 +``` + +The package inventory lists what remains installed in the build container after +xbps-src cleans up temporary dependencies. Cached package digests and build logs +provide additional input records. These records do not constitute a complete, +immutable upstream package repository snapshot. + +## Source updates and changes for review + +Keep upstream tracked files unchanged. Updating Void is a deliberate change to +both the submodule gitlink and `VOID_PACKAGES_COMMIT`, followed by the full M0 +checks. Updating host XBPS requires a matching archive URL and checksum. Updating +Rust requires considering the pinned toolchain, target, and generated artifacts +together. Do not use an unreviewed `git pull` inside the submodule as a repair. + +For a change report, state what changed, which commands actually passed, and what +was skipped. Include the failing command and relevant log when reporting a bug. +Do not describe ELF inspection as execution, QEMU userspace execution as a Pi +boot, or a target timing as a measured result. Use English for documentation, +comments, diagnostics, and project discussions. + +## Desktop verification + +Build `make rootfs PROFILE=development`, then run `make desktop-test`. This +executes the actual ARM WindowMaker, terminal, cartridge daemon, and DHCP client +in disposable VMs. Its Xvfb backend and root test console exist only in generated +fixtures. The normal SYSTEM configuration uses Xorg and the ordinary FDS user. + +VM helpers place QEMU in a private network namespace. The test's virtual DHCP +server can advertise DNS while remaining unable to reach host interfaces or the +Internet. QEMU diagnostics have separate `*.stderr.log` files, keeping host +emulator messages out of guest serial/JSON responses. Screenshots and measured +activation timestamps are retained under `out/m8-vm.*`. + +To return to a console image after desktop development, rebuild +`make rootfs PROFILE=cli` and `make system-card PROFILE=cli`. A SYSTEM image cannot +be relabeled as another profile without rebuilding the matching filesystem. + +## Use the development SYSTEM + +`profiles/development.list` extends the same immutable base with native aarch64 +**glibc** development tools. FDS control programs retain their static-musl build. +Build and verify the profile on the x86_64 workstation: + +```sh +make rootfs PROFILE=development +make rootfs-test +make system-card PROFILE=development +make development-test +``` + +These commands do not install compilers on the Pi's read-only SYSTEM at runtime. +The profile must be built into a new SYSTEM image. Store source code and build +outputs on an active DATA cartridge, or under `/tmp` for disposable experiments. +The ordinary user's home is temporary in this profile too. + +At native startup, FDS advances an older wall clock to the image's fixed source +timestamp in `/usr/share/fds/build-epoch`. A newer kernel/RTC clock is retained. +This prevents immutable compiler and CMake inputs from appearing to be in the +future when a machine boots without a usable RTC. It requires no network, +time server or waiting. `/run/fds/clock.json` records which case occurred. +The floor is an approximate lower bound, **not the current date**. Set the date +explicitly from a trusted time source when accurate timestamps or certificate +validity matter. Physical Pi RTC behavior remains to be tested. Boot-duration +measurements continue to use `CLOCK_BOOTTIME`, independent of wall-clock changes. +`make development-test` checks the older-clock case while compiling real projects; +`make clock-test` checks preservation of a newer clock in a disposable VM. + +| Selected package(s) | Purpose | +| --- | --- | +| `gcc`, `glibc-devel` | Native C/C++ compilation, libc headers and startup files; GCC selects its matching support libraries and C++ headers | +| `make`, `cmake`, `meson`, `ninja`, `pkg-config` | Common project build systems and dependency discovery; Meson brings its Python runtime | +| `rust`, `cargo` | Native Rust development and package builds; Rust selects its standard library and LLVM runtime dependencies | +| `git` | Local source history and optional remote checkout when networking is explicitly enabled | +| `gdb`, `strace` | Interactive debugging and syscall inspection | +| `vim` | Terminal editing, with no graphical session required | +| Xvfb and the existing X11 diagnostic tools | Virtual desktop integration tests; they remain opt-in | + +No development package is added to the boot service bundle. The profile uses +GCC for C/C++ and Rust for Rust; additional compilers and physical performance +profilers can be considered separately when there is a concrete need. + +For a disposable first C program at `FDS>`: + +```sh +mkdir -p /tmp/hello +cd /tmp/hello +printf '#include \nint main(void) { puts("Hello from FDS"); }\n' >hello.c +cc -g -o hello hello.c +./hello +gdb ./hello +``` + +Within GDB, use `break main`, `run`, `next`, and `quit`. For persistent work, +use `/data/projects` after selecting a healthy DATA cartridge. Stop programs and +leave that directory before `fds eject BAY`. + +For a dependency-free Rust example: + +```sh +cargo new --vcs none /tmp/hello-rust +cd /tmp/hello-rust +cargo run --offline +``` + +`--offline` avoids contacting a registry. Projects with third-party crates need +those sources prepared beforehand or an explicit network connection. To preserve +Cargo downloads, put `CARGO_HOME` on DATA and stop related builds before ejecting. +An ordinary development program uses the target's glibc environment; rebuilding +the FDS distribution and its static control plane still follows the documented +x86_64 cross-build workflow. + +`make development-test` runs native C, C++ and Rust compiler/linker executions, +CMake/Meson/Ninja/Make builds, GDB and strace, a local Git commit, and a headless Vim +edit in a disposable ARM VM. It verifies read-only SYSTEM, absent optional +services and native shutdown. Test results are recorded in +[M12 validation](Developer-M12-Validation); they do not establish Pi compiler throughput. diff --git a/Developer-EEPROM.md b/Developer-EEPROM.md new file mode 100644 index 0000000..43ddc77 --- /dev/null +++ b/Developer-EEPROM.md @@ -0,0 +1,156 @@ +--- +include_toc: true +--- + +# Pi 5 EEPROM configuration + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Boot images](Developer-Boot) · [Recovery](Developer-Recovery) · [Implementation ledger](Developer-Implementation-Status) + +M12 now provides an offline configuration workflow using the pinned official +Raspberry Pi tool and a real Pi 5 firmware image. The host checks passed; +application to a physical Pi, boot order, PMIC behavior and timing remain deferred. +`tools/configure-pi-eeprom` only creates files. It never reads or writes a hardware +EEPROM, invokes a firmware updater, or reboots a machine. + +## Preview a profile on the build workstation + +From the repository root: + +```sh +./tools/configure-pi-eeprom --profile production +./tools/configure-pi-eeprom --profile development +make eeprom-test +``` + +The first invocation downloads three checksum-pinned inputs into +`.host/eeprom/`: the official configuration script, the Pi 5 preview firmware and +its upstream license. Later calls verify and reuse them. `FDS_OFFLINE=1` requires +all inputs to exist and refuses a network download. + +The command prints its new `out/eeprom-PROFILE.*` directory. It contains: + +| File | Purpose | +| --- | --- | +| `base.bin` | Exact input firmware retained unchanged | +| `original.conf` | Configuration to restore | +| `configured.conf` | Reviewed profile merged with the supplied settings | +| `configured.bin` | Real EEPROM image with the new configuration | +| `rollback.bin` | Same base firmware with `original.conf` restored | +| `review.diff` | Human-readable configuration changes | +| `manifest.json` | Input identity, selected settings and output SHA-256 hashes | +| `LICENSE` | Complete upstream tool and firmware notices | + +Default inputs are **preview defaults, not a backup of your Pi**. A rollback image +restores the configuration supplied to this command on the supplied base firmware. +It cannot reconstruct a different firmware version or settings that were never +provided. Retain the actual machine's configuration and matching firmware input +before applying anything to that machine. + +## Profile behavior + +| Setting | Production | Development | +| --- | --- | --- | +| `BOOT_ORDER` | `0xf6`: NVMe, repeat | `0xf16`: NVMe, SD, repeat | +| `BOOT_UART` | `0` | `1` | +| `NET_INSTALL_ENABLED` | `0` | `0` | +| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | +| `POWER_OFF_ON_HALT` | `1` | `1` | +| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | + +Boot order is read from the right. Both profiles avoid scanning the twelve USB +cartridges for firmware boot. Development retains an SD rescue path. Both disable +network-install keyboard detection; Raspberry Pi documents that this detection +adds USB initialization and enumeration work. These settings do not establish a +measured FDS boot improvement. [Official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). + +On Pi 5, the power-off setting requests PMIC standby on halt; the dedicated power +button remains the wake mechanism. The wait setting leaves cold power-on boot +enabled. Whether the assembled computer and attached hardware behave as intended +still needs a physical test. [Official power settings](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#POWER_OFF_ON_HALT). + +The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf) and +[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf). No GPIO wake option or +unmeasured HDMI tuning is added. + +## Preserve the machine's settings + +In a Raspberry Pi maintenance environment with the official EEPROM utilities, +collect its configuration and version information before changing firmware: + +```sh +rpi-eeprom-config > pi-current.conf +rpi-eeprom-update > pi-eeprom-status.txt +``` + +Retain the matching original firmware image separately. The status report helps +identify it; it is not a binary backup. Copy the saved inputs to the workstation, +then prepare a new directory: + +```sh +./tools/configure-pi-eeprom \ + --profile production \ + --base-image /absolute/path/to/original-pi5-firmware.bin \ + --current-config /absolute/path/to/pi-current.conf \ + --output-directory /absolute/path/to/new-eeprom-review +``` + +The base must be a regular 2 MiB Pi 5 image, not a device node. The output directory +must not already exist. The helper preserves unrelated configuration lines and +conditional sections. It replaces all occurrences of the six managed settings, +including conditional overrides, with the selected profile's final `[all]` +settings. Review that change in `review.diff`; the exact original conditional +configuration remains in `original.conf` and `rollback.bin`. + +The upstream parser reads the generated images back before success is reported. +This checks file construction and configuration roundtrip, not hardware +compatibility. No secure-boot key, fuse, customer signature or OTP setting is +modified by this helper. + +## Apply and roll back during physical testing + +This step is deliberately outside the host/VM acceptance run. Use the Pi's +maintenance environment and the official installed EEPROM utilities. Once the +reviewed files and saved original inputs are available there, the upstream +configuration interface is: + +```sh +sudo rpi-eeprom-config --apply ./configured.conf ./base.bin +``` + +The updater may program EEPROM immediately or schedule an update depending on +its platform and update method. Follow its output and verify the resulting +configuration/version after the required restart. To restore the supplied +configuration on the same supplied firmware: + +```sh +sudo rpi-eeprom-config --apply ./original.conf ./base.bin +``` + +If the firmware version itself changed, use the saved matching original firmware +for the rollback operation. Keep the maintenance SD and an external EEPROM rescue +route available; internal recovery cannot repair an EEPROM that prevents internal +boot. The official [EEPROM update and recovery guide](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom) +describes that physical workflow. FDS has not yet tested it on your Pi. + +## Inputs, dependencies and software evidence + +The input lock is [inputs.json](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/inputs.json), pinned to +`raspberrypi/rpi-eeprom` commit `2fee426f27b6c54d3f5b6f36efd9a2fe1286a45d` and +Pi 5 preview firmware `pieeprom-2026-09-12.bin`. Every download is SHA-256 checked; +a changed cached input is rejected. The +[official source tool](https://github.com/raspberrypi/rpi-eeprom/blob/2fee426f27b6c54d3f5b6f36efd9a2fe1286a45d/rpi-eeprom-config) +is stored unchanged in the cache. Void's tracked source remains unchanged. + +The host workflow uses existing Python and curl. It does not add a target daemon, +package or Rust dependency. The upstream parser's optional signing dependencies +are not needed for configuration-only operations. + +`make eeprom-test` passed production/development roundtrips, unchanged firmware +payloads, repeatable binary output, refusal to overwrite an existing directory, +preservation of custom settings, exact rollback of conditional settings, and +rejection of malformed images, device nodes and oversized configuration files. +Initial evidence is `out/m12-eeprom.zi144zib/`, with log +`out/logs/m12-eeprom-check.log`. The complete local release acceptance is recorded in [M12 validation](Developer-M12-Validation). diff --git a/Developer-Getting-Started.md b/Developer-Getting-Started.md new file mode 100644 index 0000000..bd87dac --- /dev/null +++ b/Developer-Getting-Started.md @@ -0,0 +1,240 @@ +--- +include_toc: true +--- + +# Your first build + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Troubleshooting](Developer-Troubleshooting) + +This walkthrough takes you from an FDS Git checkout to two verified ARM Linux +artifacts. It explains the output as you go. You do not need a Raspberry Pi. + +At the end you will have a small Rust program built with static musl and GNU hello +packaged for aarch64 glibc. This proves that the workstation can build the two +kinds of software FDS/OS needs. It does not produce a bootable OS image. + +## 1. Check the machine and checkout + +Use an x86_64 Arch Linux workstation as a normal user. Check it with: + +```sh +uname -m +cat /etc/os-release +id -u +``` + +Expect `x86_64`, `ID=arch`, and a user ID other than `0`. The bootstrap script +currently rejects other host platforms and root execution. + +Enter your FDS checkout. For the existing development workspace: + +```sh +cd /home/felis/source/fds +pwd +git status --short +``` + +On another workstation, substitute its checkout directory. It must contain +`Makefile`, `Cargo.toml`, `tools/`, `.gitmodules`, and Git metadata. Avoid spaces +in the path. The configured repository is on the local Gitea server; follow +[the current checkout instructions](Getting-Started#prepare-the-workstation) +to initialize the documentation submodule too. + +You need HTTPS access to GitHub, Void repositories, Rust distribution servers, +and package source hosts. Initial downloads total hundreds of MB. Plan for +several GB of free disk space; the verified workstation used roughly 1.6 GB for +the Void masterdir and 455 MB for its hostdir cache, before Rustup and other +outputs. These are observations, not fixed requirements or a build-time promise. + +```sh +df -h . +``` + +## 2. Install host prerequisites + +On an otherwise maintained Arch installation, install the required packages: + +```sh +sudo pacman -S --needed bash coreutils binutils git curl make file tar xz gzip zstd \ + bubblewrap rustup ca-certificates findutils diffutils grep sed gawk util-linux +``` + +This is the host package installation step. Subsequent build commands run as your +normal user. The alternative `./tools/bootstrap-host --install-deps` runs the same +package installation and then continues bootstrap; choose one route. + +The [dependency reference](Developer-Build-Host#host-dependencies) explains why each +package is needed. This project uses Rustup to select its pinned Rust compiler, +not an arbitrary system Rust version. If pacman reports an existing Rust package +conflict, resolve the host package choice before continuing; do not force file +replacement. + +Check that bubblewrap can create a user namespace: + +```sh +bwrap --ro-bind / / --unshare-user --uid 0 --gid 0 true +``` + +Success produces no output and exits normally. If it fails, use +[Troubleshooting](Developer-Troubleshooting#bubblewrap-or-user-namespace-failure). +This is a host capability requirement, not something a Pi can fix. + +## 3. Prepare the build environment + +From the repository root: + +```sh +make bootstrap +``` + +This performs these steps: + +1. Initializes `vendor/void-packages` at the recorded Git commit. +2. Downloads the pinned static XBPS archive, verifies its SHA-256, and extracts + the host tools into `.host/xbps/`. +3. Copies the checked-in Void build configuration into the local upstream checkout. +4. Installs Rust 1.98.0, rustfmt, and `aarch64-unknown-linux-musl` through Rustup. +5. Fetches the locked workspace crates needed for Cargo to resolve the Dasung member, + including when building the dependency-free smoketest offline. +6. Creates the x86_64 glibc Void build container and verifies that GCC runs inside it. + +The Void build container supplies compilers and package tools; it is not the +future FDS root filesystem. Rustup changes your user toolchain installation. +XBPS host tools stay inside this repository, and no Pi storage is written. + +The final success marker is: + +```text +PASS: M0 host bootstrap; run make smoke-test to build both aarch64 artifacts +``` + +The command saves its output in `out/logs/bootstrap.log`. Re-running bootstrap +reuses the checkout and downloads, but still checks their pins and integrity. +If it exits nonzero, stop here and resolve that failure before running smoke-test. + +## 4. Cross-build both artifacts + +```sh +make smoke-test +``` + +First Cargo builds `rust/fds-smoketest` for ARM, using the static musl libraries +and linker bundled with the Rust toolchain. Then xbps-src cross-compiles GNU +hello and packages it for aarch64 glibc. The first package build downloads the +cross compiler, target libraries, and build dependencies such as texinfo. + +The script inspects actual ELF files and local package metadata. It checks: + +| Artifact | Required result | +| --- | --- | +| Rust smoketest | AArch64, static linkage, no dynamic interpreter or shared library requirement | +| GNU hello in the XBPS package | AArch64, glibc loader and `libc.so.6` dependency | + +A successful run contains these messages; other compiler output appears between them: + +```text +PASS: aarch64 static ELF +PASS: aarch64 glibc ELF +PASS: XBPS package hello-2.12.3_1 architecture=aarch64 (glibc) +PASS: M0 smoke test complete +``` + +Without QEMU, it also reports: + +```text +SKIP: ARM execution (optional qemu-aarch64 not installed); ELF verification passed +``` + +That skip means the ARM program was compiled and inspected, but not executed. +It does not hide a failed compiler or ELF check. If QEMU is installed, execution +must pass; an emulator failure is a real smoke-test failure. + +## 5. Run the protection and formatting checks + +```sh +make check +``` + +Run this after smoke-test: it expects the built Rust executable to exist. It +checks that bad inputs are rejected, that the Void pin and upstream files are +protected, that overlays cannot replace upstream packages, and that Rust source +is formatted. Several `PASS: rejects ...` lines are expected; they mean the +negative tests worked. The final commands include: + +```text +PASS: M0 guardrail checks complete +cargo fmt --all -- --check +``` + +Rustfmt is normally silent on success. `make check` does not rebuild the ARM +program. After a source change, rebuild first, as described in +[Development](Developer-Development#edit-and-rebuild-the-rust-program). + +## 6. Inspect what you built + +```sh +ls -lh out/fds-smoketest out/packages/ +file out/fds-smoketest +./tools/verify-elf out/fds-smoketest aarch64 static +sha256sum -c out/manifests/artifacts.sha256 +``` + +`file` should include `ARM aarch64` and `statically linked` (or `static-pie linked`). +The checksum command should report `OK` for both artifacts. Hashes verify the +files against this run's manifest; they are not release signatures. + +For ELF details, use: + +```sh +readelf -hW out/fds-smoketest +readelf -lW out/fds-smoketest +readelf -dW out/fds-smoketest +``` + +The header names AArch64. There should be no `INTERP` segment or `NEEDED` library. +An x86_64 host's `ldd` may say `not a dynamic executable` and return 1; that message +is expected here, but is insufficient by itself to prove a foreign ELF is static. +The [package guide](Developer-Packages#inspect-the-built-package) walks through the +corresponding inspection of GNU hello. + +## 7. Optionally execute the ARM Rust program + +The [Arch qemu-user package](https://archlinux.org/packages/extra/x86_64/qemu-user/) +provides user-mode emulation. Install it if you want to execute the ARM program +on the x86_64 workstation: + +```sh +sudo pacman -S --needed qemu-user +qemu-aarch64 out/fds-smoketest +``` + +Expected output: + +```text +FDS/OS M0: aarch64 static-musl OK +``` + +Because this executable is static, this command needs no ARM rootfs or glibc +sysroot. Explicitly invoking `qemu-aarch64` also avoids needing automatic binfmt +registration. Re-running `make smoke-test` records its own QEMU check when the +emulator is on PATH. + +This executes one Linux userspace program. It does not emulate the Pi's firmware, +USB bays, display, battery, or operating-system boot. QEMU execution was not +performed during the initial M0 validation; see the [recorded results](Developer-M0-Validation). + +## 8. Decide what to do next + +- To build the configured OS filesystem and try its ARM programs, follow [Rootfs](Developer-Rootfs): `make rootfs PROFILE=cli`, then `make rootfs-test`. +- To boot native s6 and open a development console, follow [Native init and ARM VM](Developer-Init): `make init-test`, then `make vm`. +- To build the included Dasung monitor package, follow [Dasung](Developer-Dasung). +- To edit the existing Rust program, follow [Development](Developer-Development). +- To understand the `.xbps` artifact, follow [Packages](Developer-Packages). +- To understand the intended removable operating system, read [Architecture](Developer-Architecture) and [Cartridges](Developer-Cartridges). +- To see when bootable images become possible, read [Roadmap](Developer-Roadmap). + +Bootstrap, smoke-test, and check complete the M0 foundation. Continue with the +M1 guide for a configured rootfs archive. There is still no flashing step. diff --git a/Developer-Glossary.md b/Developer-Glossary.md new file mode 100644 index 0000000..cc10a41 --- /dev/null +++ b/Developer-Glossary.md @@ -0,0 +1,59 @@ +--- +include_toc: true +--- + +# Glossary + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) + +| Term | Meaning in this repository | +| --- | --- | +| **M0** | Milestone zero: prepare and verify the workstation's cross-build environment. M1 adds the rootfs; M2 adds native init and ARM VM boot. | +| **Host / build workstation** | The x86_64 Arch Linux computer running the build. | +| **Target** | The platform being built for: AArch64 Linux on the Pi. | +| **x86_64** | The processor architecture of the supported build workstation. | +| **AArch64 / aarch64 / arm64** | Names commonly used for 64-bit ARM. Void uses `aarch64` for its glibc package target. | +| **Cross-compilation** | Compiling on one platform to produce code for another. An x86_64 build can produce an ARM executable that does not run directly on its host. | +| **Toolchain** | Compiler, linker, and related tools used to build software. | +| **Sysroot** | Target headers and libraries made available to a cross compiler. It is not necessarily a bootable filesystem. | +| **ABI** | Application binary interface: the machine-level conventions that compiled programs and libraries must agree on. Matching CPU architecture alone is insufficient. | +| **libc** | The C runtime library. FDS uses glibc for conventional packages and musl inside its static Rust tools. | +| **glibc** | GNU's C library, used by the main FDS userspace and Void `aarch64` packages. | +| **musl** | An alternative C library used for static FDS-owned programs. It does not replace the system's glibc package set. | +| **Static linking** | Including needed library code in an executable at build time, so it does not need those shared libraries at startup. It still needs the operating system/kernel. | +| **Dynamic linking / loader** | Loading shared libraries when a program starts. The ARM glibc loader is `/lib/ld-linux-aarch64.so.1`. | +| **ELF** | The executable/library file format inspected by `file` and `readelf`. The validator checks architecture and runtime requirements. | +| **Rust target triple** | A name such as `aarch64-unknown-linux-musl` specifying the compiler's target platform and library environment. | +| **Cargo / Rustup** | Cargo builds Rust projects; Rustup installs/selects Rust toolchains and target components. | +| **QEMU user mode** | Executes a foreign-architecture Linux program on the host. It does not boot or emulate the entire Pi in the M0 example. | +| **XBPS** | Void's package-management tools and package format, reused by FDS. It is separate from Arch's pacman. | +| **xbps-src** | Void's source-package build system. It creates packages using recipes called templates. | +| **Template** | An xbps-src package recipe describing source, dependencies, build actions, and packaging. | +| **Submodule / gitlink** | A separate Git repository embedded at a path; the parent records the exact upstream commit it expects. | +| **Pin** | An explicit version, commit, or digest chosen for an input. A source pin alone does not freeze every downloaded dependency. | +| **Overlay** | An FDS-owned package directory copied into the build tree without modifying tracked upstream packages. | +| **masterdir** | The generated Void build-container filesystem under `vendor/void-packages/masterdir-x86_64/`. This is not the target OS root. | +| **hostdir** | xbps-src's generated cache/output directory under `vendor/void-packages/hostdir/`. | +| **bubblewrap / bwrap** | The host tool used to isolate the build environment with Linux namespaces and mounts. No target init system runs inside this M0 container. | +| **Manifest** | A generated record of package versions or file checksums used to inspect a particular build. | +| **Rootfs** | The root filesystem used by a running OS, including programs, libraries, and configuration. `make rootfs` builds its archive. | +| **Initramfs** | A small early userspace filesystem loaded with the kernel, used before switching to the main root filesystem. | +| **PID 1 / init** | The first userspace process and the foundation of system startup, supervision, and shutdown. FDS uses native s6. | +| **s6 / s6-rc** | FDS's process-supervision and service-dependency tools. The service database is compiled while building the image. | +| **EROFS** | The read-only filesystem format used for SYSTEM and recovery images. | +| **tmpfs** | A memory-backed filesystem for temporary state. Its contents do not persist through power loss. | +| **GPT / PARTNAME** | Disk partition metadata and a partition's name. Stage0 discovers SYSTEM using `FDS_SYSTEM`, rather than a changing `/dev/sdX` name. | +| **NVMe** | The internal solid-state storage interface/device. The FDS internal image holds boot, recovery, and machine configuration. | +| **SYSTEM / DATA** | Cartridge roles: operating-system image and persistent user files, respectively. | +| **PROGRAM / ENVIRONMENT** | Cartridge roles: application payload and activation/configuration of a known environment. | +| **Bay topology** | Mapping a physical USB hub/port path to the user-facing BAY 01–12 labels. | +| **Hotplug** | Inserting or removing a device while the machine is running. Detection does not by itself make removal safe. | +| **sysfs / netlink** | Linux interfaces used by FDS to inspect devices and receive cartridge events. | +| **Profile** | A known OS configuration or service selection activated by a cartridge. It is not an arbitrary root script supplied by media. | +| **Safe eject** | Stopping users of a cartridge, flushing pending writes, unmounting it, then reporting it safe to remove. Use `fds eject BAY`. | + +For how these pieces fit together, read [Architecture](Developer-Architecture). For +commands that currently work, follow [Your first build](Developer-Getting-Started). diff --git a/Developer-Guide.md b/Developer-Guide.md new file mode 100644 index 0000000..6ffbd94 --- /dev/null +++ b/Developer-Guide.md @@ -0,0 +1,89 @@ +--- +include_toc: true +--- + +# Developer notes + +This is the engineering side of FDS: how the pieces are built, why choices were +made, and what we actually measured. The [user manual](Home) describes +how to use the current interfaces without the milestone history. + +Start with [the active cartridge/desktop revision](Developer-Current-Revision). Its +acceptance is separate from the frozen 0.1.0 release. Old test reports describe +the inputs named in those reports; a changed source tree needs fresh checks. + +## Design and build references + +- [Editing the Gitea wiki](Gitea-Wiki) +- [Clean upstream and writable Void workspace](Developer-Void-Workspace) + +- [Master plan](Developer-Master-Plan) +- [Build host and dependencies](Developer-Build-Host) +- [Development workflow](Developer-Development) +- [Workstation tooling plan](Developer-Workstation-Tooling-Plan) +- [Architecture](Developer-Architecture), [packages](Developer-Packages), [rootfs](Developer-Rootfs), [init](Developer-Init), [services](Developer-Services) +- [Reproducible builds and frozen releases](Developer-Reproducible-Builds) +- [Dasung integration and physical monitor procedure](Developer-Dasung) +- [Physical bay calibration and stress procedures](Developer-Stress-Testing) +- [Implementation ledger](Developer-Implementation-Status) and [roadmap](Developer-Roadmap) + +## Acceptance history + +- [clap-validation](Developer-CLAP-Validation) +- [dasung-validation](Developer-Dasung-Validation) +- [m0-validation](Developer-M0-Validation) +- [m1-validation](Developer-M1-Validation) +- [m10-validation](Developer-M10-Validation) +- [m11-validation](Developer-M11-Validation) +- [m12-validation](Developer-M12-Validation) +- [m2-validation](Developer-M2-Validation) +- [m3-validation](Developer-M3-Validation) +- [m4-validation](Developer-M4-Validation) +- [m5-validation](Developer-M5-Validation) +- [m6-validation](Developer-M6-Validation) +- [m7-validation](Developer-M7-Validation) +- [m8-validation](Developer-M8-Validation) +- [m9-validation](Developer-M9-Validation) +- [workstation-validation](Developer-Workstation-Validation) + +## Earlier guides and working notes + +These retain useful implementation details and the historical interfaces that +went with them. Use the user manual for current commands, especially software +cartridge creation: new payloads contain installed Void package trees. + +- [architecture](Developer-Architecture) +- [boot](Developer-Boot) +- [build-host](Developer-Build-Host) +- [cartridges](Developer-Cartridges) +- [cleanup](Developer-Cleanup) +- [dasung](Developer-Dasung) +- [data](Developer-Data) +- [desktop](Developer-Desktop) +- [development](Developer-Development) +- [eeprom](Developer-EEPROM) +- [getting-started](Developer-Getting-Started) +- [glossary](Developer-Glossary) +- [implementation-status](Developer-Implementation-Status) +- [init](Developer-Init) +- [internal-storage](Developer-Internal-Storage) +- [m4-work](Developer-M4-Work) +- [master-plan](Developer-Master-Plan) +- [media-tools](Developer-Media-Tools) +- [packages](Developer-Packages) +- [performance](Developer-Performance) +- [power](Developer-Power) +- [recovery](Developer-Recovery) +- [releases](Developer-Releases) +- [reproducible-builds](Developer-Reproducible-Builds) +- [roadmap](Developer-Roadmap) +- [rootfs](Developer-Rootfs) +- [services](Developer-Services) +- [software-format](Developer-Software-Format) +- [stress-testing](Developer-Stress-Testing) +- [tooling](Developer-Tooling) +- [troubleshooting](Developer-Troubleshooting) +- [workstation-tooling-plan](Developer-Workstation-Tooling-Plan) +- [workstation](Developer-Workstation) + +- [Git-derived build versions and Arch packaging](Developer-Versioning) diff --git a/Developer-Implementation-Status.md b/Developer-Implementation-Status.md new file mode 100644 index 0000000..bb51582 --- /dev/null +++ b/Developer-Implementation-Status.md @@ -0,0 +1,141 @@ +--- +include_toc: true +--- + +# Implementation through M12 + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +This is the software acceptance ledger for the 67 sections of the +[master plan](Developer-Master-Plan). The user authorized implementation through M12 +and deferred tests requiring the physical Raspberry Pi or attached hardware. +The current release checkpoint is recorded in [M12 validation](Developer-M12-Validation). + +**M0–M12 software acceptance is complete.** The signed local 0.1.0 release +passed two fresh network-isolated builds with all 27 artifacts identical. +Those artifacts also match the complete post-Clap VM acceptance. Signature +verification and actual source/input archive round-trips passed; see +[final release evidence](Developer-M12-Validation#final-local-release-acceptance). + +The final command-line migration uses shared, locked Clap for every Rust +command-line interface, including the VM helpers. Its 48 Rust tests, actual ARM +CLI checks and complete runtime acceptance are recorded in +[Clap validation](Developer-CLAP-Validation). The dependency-free smoketest has no +argument interface. Kernel command-line files and cartridge metadata use their +own strict format parsers; they are not process argument parsers. + +## Workstation software and emulator extension + +Current sources extend the frozen 0.1.0 checkpoint with native Linux +`fds-cartridge` and `fds-emulator` tools. The software format uses metadata plus +`m` payload partitions containing xz tarballs. Software builds and complete +image creation run on the workstation; the guest verifies, extracts and runs +selected commands as UID 1000. See [usage](Developer-Workstation), +[format](Developer-Software-Format), and [current acceptance](Developer-Workstation-Validation). +Historical M0–M12 and Clap reports describe their recorded artifacts; they do +not by themselves validate these new sources. + +## Requirement coverage + +“Verified” below means software, artifact or isolated ARM VM evidence. It does +not mean the assembled Pi has been tested. The detailed milestone reports retain +the commands, artifacts, failures that led to fixes, and measured results. + +| Section | Implementation and evidence | Physical or later boundary | +| --- | --- | --- | +| 1. Project objective | Pi 5 images, all cartridge classes, native s6, Dasung base support and 12-device virtual stress; [overview](Overview), [stress evidence](Developer-M11-Validation) | Assembled FP-85, battery, display and all physical bays deferred | +| 2. ABI strategy | Every profile uses aarch64 glibc packages; all eight production Rust programs are static ARM musl; rootfs and ELF audits pass | Actual Pi execution deferred | +| 3. Limited musl scope | No target musl runtime package or second general-purpose package repository; glibc compiler/desktop acceptance passes | None in software scope | +| 4. Native init | s6-linux-init hands PID 1 to s6-svscan; s6-rc database compiled during image construction; [native init evidence](Developer-M2-Validation) | Pi startup timing deferred | +| 5. Packages | Nine FDS base packages selected and audited; forbidden init/runtime packages rejected; [packages](Developer-Packages) | None in software scope | +| 6. Build host | x86_64 Arch bootstrap, glibc cross-package and static Rust checks; [build host](Developer-Build-Host) | Other host distributions are not validated | +| 7. Repository layout | Complete build/runtime components and public interfaces; location choices documented below | No placeholder component claims | +| 8. Void upstream | Pinned commit, unchanged tracked upstream files, protected overlays, exact frozen package inputs; [offline builds](Developer-Reproducible-Builds) | Normal online bootstrap still uses rolling repositories | +| 9. Rust workspace | All required tools and aliases built; release/static settings and shared dependencies; [tooling](Developer-Tooling) | None in software scope | +| 10. Dependencies | Locked synchronous dependencies, Clap policy and all 68 third-party crate notices; z/2/3 builds and repeated boot comparison | Physical optimization winner deferred | +| 11. Boot architecture | Pi kernel → static stage0 → read-only SYSTEM → native s6 and ordinary-user console; [boot evidence](Developer-M4-Validation) | Firmware, RP1 and physical NVMe/USB handoff deferred | +| 12. Internal NVMe | BOOT/RECOVERY/INTERNAL GPT layout; root-private atomic settings and bounded diagnostic snapshots persist across VM reboot; [internal storage](Developer-Internal-Storage) | Actual NVMe and power-loss durability deferred | +| 13. SYSTEM | Named GPT partition and EROFS, strict manifest/architecture checks, immutable update by replacement; [media evidence](Developer-M9-Validation) | Physical cartridge write/boot deferred | +| 14. Stage0 | Direct Linux/sysfs/netlink discovery, mount and root handoff; mandatory early Dasung process; nine-case real-kernel VM matrix | Physical display keepalive/recovery deferred | +| 15. Missing SYSTEM | Explicit wait state, event-driven insertion/resume and independent recovery verified | Physical insertion latency deferred | +| 16. Multiple SYSTEMs | Ambiguity enters recovery instead of choosing enumeration order; explicit recovery verified | Interactive selector is a later-version option | +| 17. Kernel | Pinned Pi kernel package, built-in critical driver delta verified after Kconfig resolution, modules/DTBs packaged | Actual RP1, PCIe, graphics and USB support deferred | +| 18. Initramfs | Minimal static early userspace; raw/gzip/lz4/zstd construction, extraction and boot checks | Pi decompression/timing comparison deferred | +| 19. Firmware | Production/development EEPROM previews, exact readback and rollback preparation; [EEPROM](Developer-EEPROM) | No EEPROM was read from or applied to a physical Pi | +| 20. Filesystems | SYSTEM stays read-only; volatile logs, temporary directories and state; root ownership is explicitly root:root 0755 | Physical persistence/durability deferred | +| 21. User data | Persistent DATA at `/data`, temporary `/home/fds`, unprivileged sessions; [DATA evidence](Developer-M7-Validation) | Home persistence is not claimed | +| 22. Service graph | Native boot bundle includes console, devices, cartridge and mandatory Dasung services; optional bundles excluded | Hardware service behavior deferred | +| 23. Fast boot | Console depends on runtime filesystem/hostname, not cartridge scan, DHCP or desktop; blocked-service VM tests pass | Pi timing targets deferred | +| 24. eudev | Native supervision and trigger without settle barriers; console opens with eudev blocked | Physical hotplug races deferred | +| 25. Cartridge daemon | Static event-driven daemon, private metadata probes, bounded IPC, mount lifecycle and restart behavior; [cartridge evidence](Developer-M6-Validation) | Real device-controller behavior deferred | +| 26. Bay topology | Controller/protocol/port identity, strict map validation, 12 virtual bays independent of block/bus enumeration | User must calibrate the physical wiring | +| 27. Metadata | Strict bounded TOML, six class values, architecture and payload validation, unknown fields rejected | None in software scope | +| 28. Hardware cartridges | Filesystem-free VID/PID/class catalog and input/Ethernet recognition; no catalog autorun | Catalog starts empty until physical devices are identified | +| 29. ENVIRONMENT | Validated declarative activation of the trusted WindowMaker profile | GUI runtime on the cartridge itself is a later-version option | +| 30. WindowMaker | SYSTEM includes Xorg, WindowMaker, packaged caches/fonts; authenticated X11 and transitions verified; [desktop evidence](Developer-M8-Validation) | Real graphics/input path deferred | +| 31. E-Ink | White/black/grayscale profile, Terminus and reduced decorations/animations; mandatory Dasung in base and early boot | Picture quality, refresh and power-cycle recovery deferred | +| 32. PROGRAM | Metadata-first GPT software bundles plus legacy reading; xz verification, read-only runtime caches and UID 1000 consumers; see [current evidence](Developer-Workstation-Validation) | Physical application cartridge behavior deferred | +| 33. CLI | Complete typed Clap command hierarchy, aliases, nested JSON, help/errors and unchanged child-argument forwarding; [usage](Developer-Tooling) | None in software scope | +| 34. Media writer | Static create/inspect/burn/eject, insertion-bound confirmation, protected disks and full readback; actual virtual write then boot | Physical throughput, cache and power-loss behavior deferred | +| 35. Installation model | Immutable SYSTEM images contain packages; replacement cartridge provides update/rollback | Physical swap/boot deferred | +| 36. Profiles | CLI, development and independent recovery rootfs/images; profile-specific package audits and runtime checks | None in software scope | +| 37. Build interface | Complete `make all`, packages, kernel, initramfs, rootfs, SYSTEM, boot and internal outputs; [offline builds](Developer-Reproducible-Builds) | No command silently flashes hardware | +| 38. Cross compilation | Static Rust, glibc XBPS and Pi kernel cross-built on x86_64; generated target files inspected/executed under emulation | Native Pi build performance deferred | +| 39. QEMU | Packaged ARM OS, real kernel, disposable USB/NVMe media and serial evidence across all runtime suites | QEMU does not emulate the Pi firmware/display/electrical system | +| 40. Boot targets | Actual CLOCK_BOOTTIME reports and repeated comparisons; [performance](Developer-Performance) | No physical cold-boot target is claimed | +| 41. Boottrace | T0–T6 event model, first-console event retention, reports and persisted diagnostic exports | Firmware timing and physical display events deferred | +| 42. Regression rule | +100 ms investigation threshold, alternating before/after boots and z/2/3 comparison; [Clap measurements](Developer-CLAP-Validation) | VM samples are not Pi performance claims | +| 43. Boot blacklist | No systemd, runit, BusyBox, shell PID 1, boot-time cache compilation, settle or sleep-based readiness barriers | Dasung protocol pacing/backoff is not a console readiness barrier | +| 44. Network | Ethernet recognition and on-demand native DHCP bundle; console remains usable without DHCP | Physical adapter/link testing deferred | +| 45. Logging | Runtime logs stay volatile; explicit bounded diagnostics can be stored on internal media or exported; [recovery](Developer-Recovery) | Physical storage durability deferred | +| 46. Shutdown | Ordered desktop/jobs/network/DATA/services stop with native halt/reboot; busy and writeback-error paths remain powered on; [power evidence](Developer-M10-Validation) | Real power removal deferred | +| 47. Shutdown timing | Idle/workload/error-path VM measurements retained without artificial waits | Sub-second Pi/battery/flash targets deferred | +| 48. Eject | Consumers stopped, DATA syncfs/unmount before SAFE, quarantine across failure/restart and insertion identity checks | Physical LEDs are optional later work; electrical/cache durability deferred | +| 49. Recovery | Independent EROFS boots without usable SYSTEM, read-only DATA defaults, root-confirmed repair, media replacement and diagnostics; [M12 evidence](Developer-M12-Validation) | Physical recovery/EEPROM path deferred | +| 50. Security | No cartridge root autorun; unprivileged managed applications, strict metadata/IPC, protected media operations and tested failure paths | Distribution signatures do not implement secure boot | +| 51. Initial UI | Direct ordinary-user FDS console without artificial delays, display manager, splash or automatic desktop | Physical presentation/input latency deferred | +| 52. Development | Native GCC/C++, Rust/Cargo, Make/CMake/Meson/Ninja, Git, GDB, strace, Vim and display tools execute in the development image | Optional example tools such as LLVM are not mandatory; Pi throughput deferred | +| 53. M0 | [Host validation](Developer-M0-Validation), repeated after Clap | Optional host `qemu-aarch64` smoke execution skipped; separate ARM execution suites pass | +| 54. M1 | [Rootfs validation](Developer-M1-Validation), all three current profile audits | Physical boot deferred | +| 55. M2 | [Native s6 validation](Developer-M2-Validation), repeated after Clap | Physical Dasung behavior deferred | +| 56. M3 | [Tooling acceptance](Developer-Tooling), complete Clap migration and static ARM tests | None in software scope | +| 57. M4 | [Boot matrix](Developer-M4-Validation), repeated after Clap | Physical Pi boot deferred | +| 58. M5 | [Console/performance evidence](Developer-M5-Validation), current repeated Clap comparison | Physical speed targets deferred | +| 59. M6 | [Cartridge acceptance](Developer-M6-Validation), repeated after Clap | Real bay calibration deferred | +| 60. M7 | [DATA acceptance](Developer-M7-Validation), repeated after Clap | Physical storage durability deferred | +| 61. M8 | [Desktop/network acceptance](Developer-M8-Validation), repeated after Clap | Physical display/input/link behavior deferred | +| 62. M9 | [Media acceptance](Developer-M9-Validation), repeated after Clap | Physical write/readback deferred | +| 63. M10 | [Shutdown acceptance](Developer-M10-Validation), repeated after Clap | Physical power/timing deferred | +| 64. M11 | [Twelve-device stress and fault injection](Developer-M11-Validation), repeated after Clap | Simultaneous physical devices and electrical tests deferred | +| 65. M12 | [Final release evidence](Developer-M12-Validation), signatures, input lock, independent offline builds, recovery, internal settings and guides | Hardware qualification and publication are separate actions | +| 66. Implementation rules | Buildable milestones, justified dependencies, strict ABI/init policy, stable topology and measured regressions | No fabricated physical benchmarks | +| 67. First task | Original M0 setup/negative checks completed and repeated; complete scope explicitly authorized afterward | Historical M0-only gate was superseded by the user's through-M12 request | + +## Interface and policy decisions + +| Design sketch | Implemented interface | +| --- | --- | +| `image/build-rootfs` | `tools/build-rootfs`, exposed by `make rootfs` | +| `tools/qemu-test` | `make init-test` for native init and `make boot-test` for the Pi-kernel/stage0 matrix | +| `tools/burn` | Packaged static `fds-burn` and `fds burn` | +| `profiles/cli.toml`, `development.toml`, `recovery.toml` | `profiles/*.list` select packages; immutable profile marker selects recovery policy | +| `profiles/windowmaker.toml` | Built-in `windowmaker` runtime profile with a native s6 desktop bundle | + +DATA uses `/data`, one of the two alternatives permitted by section 21. +`/home/fds` remains temporary, avoiding replacement of a live shell's home on +insertion/eject. Internal boot reports and cartridge inventories are explicit +diagnostic snapshots, not authoritative live state or automatic boot writes. +An interactive multi-SYSTEM selector, moving GUI runtime onto ENVIRONMENT media, +and optional eject LEDs remain later-version options in the specification. + +## Hardware work intentionally deferred + +The next acceptance phase needs the assembled computer: EEPROM backup/application +and rollback, NVMe-to-USB boot, RP1/PCIe/USB behavior, actual bay calibration and +all twelve devices, display picture/refresh/cold-power recovery, removable battery +and power behavior, physical storage durability, cold boot and shutdown timing. +Use [stress testing](Developer-Stress-Testing), [Dasung](Developer-Dasung), [EEPROM](Developer-EEPROM), +[internal installation](Developer-Internal-Storage) and [performance](Developer-Performance) +to collect that evidence. No physical target disk was flashed, and no Pi EEPROM +or monitor was changed by the software acceptance work. diff --git a/Developer-Init.md b/Developer-Init.md new file mode 100644 index 0000000..43e203d --- /dev/null +++ b/Developer-Init.md @@ -0,0 +1,194 @@ +--- +include_toc: true +--- + +# M2: native s6 init and the ARM development VM + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Rootfs build](Developer-Rootfs) · [Services](Developer-Services) + +M2 adds the system's real init path. The kernel enters the generated execline +launcher at `/sbin/init`; that immediately executes `s6-linux-init`, which prepares +`/run` and replaces itself with `s6-svscan`. The supervisor remains +PID 1. Bash is used for some stage-2 configuration tasks, as an ordinary child +process. No shell loop, systemd, runit, or BusyBox owns PID 1. + +You can boot and inspect this system on the x86_64 workstation using a complete +ARM QEMU machine. This is a generic development VM, not a Raspberry Pi image. +It does not test Pi firmware, a cartridge boot path, or the physical monitor. + +## Build and verify + +First complete the [build-host setup](Developer-Getting-Started) and the +[rootfs prerequisites](Developer-Rootfs#1-prepare-the-workstation). M2 additionally uses +the Arch `e2fsprogs` and `libarchive` packages to create an ext4 VM disk directly +from the tar, without mounting it or requiring root: + +```sh +sudo pacman -S --needed e2fsprogs libarchive +make rootfs PROFILE=cli +make rootfs-test +make init-test +``` + +`make init-test` installs `qemu-system-aarch64` inside the project-local Void build +container when missing. Void packages its emulator with display, audio, and other +libraries, even though this test uses only a serial console. Those dependencies +stay in the build container; they do not become target packages or host services. + +The test also downloads a signed generic ARM kernel, pinned by version and SHA-256 +in `config/vm-test.conf`. Its AHCI, ext4, serial console and PCI drivers are built +in and devtmpfs is automounted by the kernel, so no test initramfs or replacement stage 0 is needed. The kernel package is +approximately 161 MB; only its boot files are extracted. Its module collection is +not installed into FDS. The separate [M4 boot test](Developer-Boot) uses the actual +FDS Pi kernel and its initramfs-to-EROFS handoff. + +The VM uses software CPU emulation, 1 GiB of RAM, two virtual CPUs, no network, +and no host USB, display, filesystem sharing, or block-device passthrough. +Its root filesystem is mounted read-only. QEMU discards any block changes in a +temporary disk overlay, preserving the generated test disk. + +Successful output includes: + +```text +FDS_M2_PID1: /usr/bin/s6-svscan +FDS_M2_FILESYSTEM: read-only root, writable tmpfs, root ownership and capabilities +FDS_M2_BOOT: getty, eudev and Dasung supervised without a monitor +FDS_M2_SERVICE: s6-rc start, echo, supervised restart, stop and start passed +FDS_M2_PASS +PASS: M2 ARM boot, native s6 PID 1, service control, and orderly power-off +``` + +The test must finish within 180 seconds. This is a failure deadline, not a boot +performance target. A panic, failed assertion, missing marker, timeout, or missing +power-off message makes the command fail. Retained logs explain the result. + +## Open a real booted ARM shell + +After `make init-test` succeeds: + +```sh +make vm +``` + +The same guest checks run, then the VM opens a temporary root shell on its serial +console. Try: + +```sh +readlink /proc/1/exe +findmnt / +findmnt /run +s6-rc -l /run/s6-rc -a list +s6-svstat /run/service/dasungd +dasungd status +``` + +The rootfs's ordinary getty still requires authentication and root remains locked. +The VM hook stops getty before opening its shell so they do not compete for the +serial console; `s6-svstat` therefore reports getty down during that shell session. +The password-free shell exists only in the disposable VM test overlay, when +`make vm` explicitly requests it. The checker and shell hook do not ship in the +normal rootfs archive. Type `exit` to request orderly VM power-off. Ctrl+C stops +the emulator immediately if the guest is stuck; it does not shut down your host. + +The shell's `/run`, `/tmp`, `/var/tmp`, and `/var/log` writes disappear at power-off. +Changes to `/etc` or `/usr` fail because SYSTEM is read-only. Rebuild the rootfs +and rerun `make init-test` after changing packages or init configuration; `make vm` +otherwise boots the previous successful test disk. + +## Control the test service + +The `test-echo` service is an actual Unix-socket echo server, supervised by s6. +It is included in the service database but excluded from the normal boot bundle. +In the VM shell: + +```sh +s6-rc -l /run/s6-rc -u change test-echo +s6-svstat -o up,ready,pid /run/service/test-echo +printf 'hello from ARM\n' | s6-ipcclient /run/fds-test/echo.sock s6-ioconnect -t 3000 +s6-svc -wR -T 5000 -r /run/service/test-echo +s6-rc -l /run/s6-rc -d change test-echo +``` + +Its socket is root-only, under `/run/fds-test` with mode 0700. The server sends an +actual readiness notification when it is listening. The automated test checks +the reply, confirms the PID changes on restart, stops the service, and starts it +again. No arbitrary boot delay substitutes for readiness. + +## What starts at boot + +| Component | Role and dependencies | +| --- | --- | +| Native stage 1 | Consume the kernel-provided `/dev`, mount `/run`; start s6 supervision and the catch-all logger | +| `runtime-fs` | Mount proc, sysfs, devpts and volatile temporary/log filesystems | +| `hostname` | Apply the image's `/etc/hostname` | +| `console` | Create volatile login accounting files after runtime mounts | +| `getty` | Offer a login on `/dev/console`, after console and hostname setup | +| `eudevd` | Process device events; readiness requires a successful control request | +| `udev-trigger` | Request initial subsystem and device events after eudev readiness | +| `dasungd-runtime`, `dasungd-log`, `dasungd` | Prepare private state/log paths and supervise the base monitor controller | + +Console startup does not depend on eudev enumeration, a monitor connection, the +network, or a cartridge scan. No global `udevadm settle` is used. The eudev package +has no native s6 notification, so `s6-notifyoncheck` performs a bounded check of +its control channel. It signals readiness only after a successful response. + +Cartridge discovery, the FDS CLI, desktop activation and on-demand Ethernet were +added after the original M2 milestone. The current image includes their real +implementations; see [Services](Developer-Services), [Cartridges](Developer-Cartridges) and +[Desktop](Developer-Desktop). This page focuses on the native init mechanism and its +generic test fixture. SSH remains outside the default boot bundle. + +## Files and build-time work + +| Location | Meaning | +| --- | --- | +| `packages/fds-init/` | Init skeleton, runtime mount/accounting helpers, and package template | +| `s6/source/` | Complete current service graph, split between init, Dasung and cartridge/desktop packages | +| `/etc/s6-linux-init/current/` | Generated native init environment and stage scripts in the image | +| `/etc/s6-rc/compiled/` | Compiled service database, produced during image construction | +| `/run/service/`, `/run/s6-rc/` | Writable supervision and live service state | +| `/run/uncaught-logs/` | Catch-all service log, also copied to the console | +| `/run/log/dasungd/` | Rotating monitor-controller log | +| `out/m2-vm-latest/serial.log` | Last successful full ARM boot test | +| `out/m2-vm-latest/rootfs.ext4` | Disposable test disk; not a Pi or SYSTEM cartridge image | + +`s6-linux-init-maker` and `s6-rc-compile` run during the rootfs build under ARM +emulation. Boot only reads their output. The VM test preserves that exact compiled +database. Its only init hook runs the original stage-2 script to completion, then +runs the checker as a child process. Normal `out/rootfs-aarch64.tar` contains +neither the checker nor the development-shell hook. + +The s6-generated template contains FIFOs. Archive validation permits them only +inside its `run-image` directory; those templates are copied into writable `/run` +at boot. Before entering `/sbin/init`, the kernel or later stage 0 must provide +a mounted devtmpfs at `/dev`. Stage 1 deliberately does not mount it twice. +Services never place their live FIFOs or status files on read-only SYSTEM. + +M10 adds cartridge-aware preparation before s6-rc stops the remaining services. +Its native shutdown hook stays alive if DATA cannot be verified and unmounted. +`s6-linux-init-shutdownd` still performs final process cleanup and the kernel +power action. See [Shutdown and reboot](Developer-Power) for usage, blocked-request +recovery, and the current M10 acceptance status. + +## Troubleshooting + +- **`make vm` asks you to run `make init-test`:** no successful VM disk has been + published yet. Build the rootfs, then run the test. +- **Kernel pin/hash failure:** do not substitute an unverified kernel. The exact + configured package must remain available or be deliberately updated and retested. +- **Disk creation reports missing tar support:** use an `e2fsprogs` build with + libarchive support and install `libarchive`. No loop mount or sudo fallback is used. +- **No guest success marker:** read `out/logs/init-checks.log` and the printed + `out/m2-vm.*` directory's `serial.log`. Failed test workspaces are retained. +- **Root login is denied on the ordinary getty:** this is expected with the locked + base accounts. Use the explicit development VM shell, not a default password. +- **Dasung says disconnected:** the VM intentionally has no physical monitor. + Use [Dasung integration](Developer-Dasung) for its eventual hardware validation. + +Upstream references: [s6-linux-init](https://skarnet.org/software/s6-linux-init/overview.html), +[init generation](https://skarnet.org/software/s6-linux-init/s6-linux-init-maker.html), +[s6-rc initialization](https://skarnet.org/software/s6-rc/s6-rc-init.html), and +[ext4 creation](https://man7.org/linux/man-pages/man8/mke2fs.8.html). diff --git a/Developer-Internal-Storage.md b/Developer-Internal-Storage.md new file mode 100644 index 0000000..1534af3 --- /dev/null +++ b/Developer-Internal-Storage.md @@ -0,0 +1,280 @@ +--- +include_toc: true +--- + +# Internal storage and machine settings + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Boot images](Developer-Boot) · [Recovery](Developer-Recovery) + +The internal NVMe supplies the Pi's firmware boot files, independent recovery, +and machine settings. SYSTEM remains a separate removable cartridge. Ordinary +user files belong on DATA, not internal storage. No physical disk is written by +any build command below; Pi/NVMe boot and power-loss tests remain deferred. + +## Build the complete disk image + +Build the component images first, in this order: + +```sh +make kernel +make initramfs +make boot-volume BOOT_MODE=production +make recovery +make internal-image +``` + +The result is `out/fds-internal.img`, a **complete GPT disk image**. Its versioned +build directory contains `layout.json`, source payload checksums, and readback +verification. This differs from `out/fds-boot.img` (a raw FAT partition) and +`out/fds-recovery.img` (a raw EROFS partition). + +| Partition | Format | Default allocation | Purpose | +| --- | --- | --- | --- | +| 1: FDS_BOOT | FAT32, EFI system type | 512 MiB | Pi firmware, kernel, DTBs, configuration and initramfs | +| 2: FDS_RECOVERY | EROFS | 1 GiB | Complete independent maintenance system | +| 3: FDS_INTERNAL | ext4 | 256 MiB | Machine settings and explicitly saved diagnostics | + +The builder pads the recovery partition without changing its EROFS contents, +aligns partitions to 1 MiB, writes both GPT copies, verifies each payload, and +checks the table independently with `sfdisk`. Internal ext4 is fully initialized +before deployment. It contains root-owned `config/` and private `diagnostics/`. +Unused capacity beyond this initial layout is not automatically expanded. + +For a 2 GiB recovery allocation or larger settings partition: + +```sh +./image/build-internal --recovery-mib 2048 --internal-mib 512 +``` + +Output directories supplied with `--output-directory` must already exist and be +empty. The builder accepts ordinary image files, never a host block-device +output. Final hardware provisioning and identity checks belong to the physical +acceptance procedure; do not confuse a partition payload with the complete disk. + +## Install the internal disk when hardware is available + +This is a **deferred physical procedure**. The image and virtual NVMe workflow +are software-tested; writing and booting the user's actual NVMe still require +the hardware. Installation erases the selected disk, including any existing +machine settings. Retain backups before replacing an existing installation. + +Use an NVMe enclosure or another Linux machine that can access the target drive +while the Pi is off. First verify a downloaded release using a separately trusted +public key as described in [Release signatures](Developer-Releases). Its complete disk +is named `fds-internal-0.1.0.img`; a local build uses `out/fds-internal.img`. +Use the complete disk image, not the separate BOOT or RECOVERY payload. + +List disks before and after connecting the intended drive: + +```sh +lsblk -d -o NAME,PATH,MODEL,SERIAL,SIZE,TRAN,LOG-SEC +ls -l /dev/disk/by-id/ +``` + +Match the physical model, serial and capacity. Choose a persistent **whole-disk** +`/dev/disk/by-id/` path without a `-partN` suffix; never guess a `/dev/sdX` name. +Unmount every target partition and disable any swap on it. Do not select the +workstation's system disk. The current images require 512-byte logical sectors +and a disk at least as large as the image; 4 KiB logical-sector media is not +supported by this layout. + +Open Bash (`bash`), replace both paths below, then run the block. It checks the +selected disk again, requires a typed confirmation, writes and reads back the +complete image, and relocates the backup GPT when the disk is larger than the +image. It does not expand any partition or filesystem. + +```bash +( + set -euo pipefail + fds_image="$PWD/out/fds-internal.img" + fds_disk=/dev/disk/by-id/REPLACE_WITH_THE_TARGET_DISK + [[ -f "$fds_image" && -s "$fds_image" && -b "$fds_disk" ]] + [[ $(lsblk -dnro TYPE "$fds_disk") == disk ]] + [[ $(sudo blockdev --getss "$fds_disk") == 512 ]] + fds_bytes=$(stat -Lc %s "$fds_image") + (( $(sudo blockdev --getsize64 "$fds_disk") >= fds_bytes )) + lsblk -p -o NAME,TYPE,MODEL,SERIAL,SIZE,MOUNTPOINTS "$fds_disk" + if lsblk -nrpo MOUNTPOINTS "$fds_disk" | grep '[^[:space:]]' >/dev/null; then + echo 'Target has mounted filesystems or swap; stop and release them first.' >&2 + exit 1 + fi + if lsblk -nrpo TYPE "$fds_disk" | grep -Ev '^(disk|part)$' >/dev/null; then + echo 'Target has device-mapper or other active descendants; stop.' >&2 + exit 1 + fi + read -r -p "Type ERASE $fds_disk to erase this disk: " fds_confirmation + [[ "$fds_confirmation" == "ERASE $fds_disk" ]] + sudo dd if="$fds_image" of="$fds_disk" bs=4M conv=fsync status=progress + sudo blockdev --flushbufs "$fds_disk" + sudo cmp -n "$fds_bytes" "$fds_image" "$fds_disk" + sudo sfdisk --lock=yes --relocate gpt-bak-std "$fds_disk" + sudo sfdisk --verify "$fds_disk" + sudo blockdev --flushbufs "$fds_disk" +) +``` + +Every command must succeed. The byte comparison occurs before relocating GPT, +because relocation intentionally changes disk-table headers. GNU `dd`'s `fsync` +flushes output before it returns; `sfdisk`'s `gpt-bak-std` moves the backup header +to the end of the target. See the [GNU dd manual](https://www.gnu.org/s/coreutils/manual/html_node/dd-invocation.html) +and [sfdisk manual](https://man7.org/linux/man-pages/man8/sfdisk.8.html). +These commands use the already documented coreutils, diffutils and util-linux +host tools; they do not add a target daemon. + +Confirm that the disk shows `FDS_BOOT`, `FDS_RECOVERY` and `FDS_INTERNAL` in that +order using `lsblk -o NAME,PARTLABEL,FSTYPE,MOUNTPOINTS`. Unmount anything the +desktop automatically mounted, then safely disconnect the enclosure and install +the NVMe in the powered-off Pi. Follow [EEPROM preparation](Developer-EEPROM) for the +separately reviewed NVMe boot settings. + +Prepare the **first SYSTEM cartridge on the workstation** using the same guarded +write block, with its two path assignments changed to the chosen SYSTEM image +and a different, empty USB cartridge disk. For a local build select +`out/fds-system-cli.img` or `out/fds-system-development.img`; release files are +`fds-system-cli-0.1.0.img` and `fds-system-development-0.1.0.img`. They are +alternative complete GPT images, each containing one `FDS_SYSTEM` partition. +Readback and backup-GPT relocation apply to this disk too. Check its partition +label, safely disconnect it, and insert exactly one SYSTEM cartridge in the Pi +before normal boot. Subsequent cartridge creation and updates can use FDS's +confirmed [media workflow](Developer-Media-Tools) after bay calibration. + +On the first physical boot, check `fds info`, `fds machine status`, and +`fds bays`. The supplied bay map is empty until calibration. Follow +[physical acceptance](Developer-Stress-Testing) to measure ports, test the Dasung +display, and record actual boot/shutdown behavior before relying on the machine. + +## Configure the machine before building + +Copy `config/machine/` to your own directory. It contains three files: + +- `machine.toml`: `format = 1` and a short human-readable `name`. +- `bays.toml`: the measured controller/port map described in + [Cartridges and bay calibration](Developer-Cartridges). +- `hardware-catalog.toml`: optional USB identification names using the same schema + as the base catalog. Entries are data and cannot run commands. + +The supplied bay map is deliberately empty because the physical wiring has not +been measured. Do not invent Pi USB paths. USB 2 and USB 3 companion ports need +explicit aliases for the same bay. + +```sh +cp -a config/machine out/my-machine +# Edit the three files in out/my-machine using your editor. +make internal-image MACHINE_CONFIG=out/my-machine +``` + +The builder compiles a native host copy of `fds` and uses the same strict parser +as the target system. You can also validate or pack settings yourself: + +```sh +cargo build --locked --offline --release --target x86_64-unknown-linux-gnu -p fds-cli +./target/x86_64-unknown-linux-gnu/release/fds machine validate out/my-machine +./target/x86_64-unknown-linux-gnu/release/fds machine pack out/my-machine out/my-machine.json +``` + +The three source files become one atomic `config/machine.json` document on +FDS_INTERNAL. Names, lengths, bay aliases, catalog fields and unknown keys are +validated before use. Do not put passwords or private keys in this configuration: +its active snapshot is readable by the local FDS user. + +## What happens at boot + +The `machine-config` native s6 oneshot precedes `cartridged`. It does **not** +precede the console or Dasung controller. It accepts exactly one non-removable +NVMe disk with the three named partitions in the order above. USB lookalikes are +ignored; multiple eligible NVMe disks are rejected instead of choosing by name. + +The settings partition must be clean ext4 with the expected label. Loading uses +`ro,noload,nosuid,nodev,noexec` in a private mount namespace, validates the entire +settings bundle, copies it into `/run/fds/machine/`, and unmounts. No filesystem +repair, journal replay, cache compilation or persistent write occurs during +normal boot. The cartridge service uses that snapshot throughout this boot, +including after a service restart. + +If internal storage is missing, unclean, invalid or ambiguous, the service records +an explanation and uses the immutable image's `/etc/fds/` defaults. The console +still opens. Check the actual source before treating bays as calibrated: + +```sh +fds machine status +fds --json machine status +fds machine export /tmp/current-machine +``` + +Export creates a new directory with the three editable source files. It never +overwrites an existing directory. The status source is `internal_nvme` or +`image_defaults`. A temporarily unavailable NVMe is not adopted later in the same +boot; resolve the issue and reboot to load its settings. This prevents changing +bay identities underneath active cartridge operations. + +## Update settings from recovery + +Bring the edited source directory on a DATA cartridge. At the local root +`RECOVERY#` console, identify its bay with `fds bays`; healthy DATA is mounted +read-only under `/run/fds/media/NN`. For example, if it is in BAY 02: + +```sh +fds machine validate /run/fds/media/02/my-machine +fds machine install /run/fds/media/02/my-machine +fds reboot +``` + +Installation is restricted to root in the recovery image. It writes the complete +validated bundle atomically, saves the old bytes as `config/previous.json`, +flushes and unmounts the internal filesystem, and reports success only after +those steps. A reboot activates the new settings. The currently running bay map +is unchanged, so active media does not move to a different bay mid-operation. +Keep a copy of your previous source directory on DATA or the build host to +reinstall it if the new calibration is wrong. + +An invalid existing JSON document can be replaced this way. Wrong ownership, +symlinks, an unclean filesystem or a damaged directory require offline filesystem +maintenance first. Recovery does not automatically repair internal NVMe; its +`fds recovery repair` command is deliberately limited to DATA cartridges. + +## Save and retrieve diagnostics + +Logs and boot records stay in RAM by default. Root may explicitly save a file +of up to 16 MiB. Saved names cannot contain paths, and existing names are refused. +These operations mount internal ext4 only for the operation and then unmount it. +For example, from recovery: + +```sh +fds --json boot-profile >/tmp/boot.json +fds machine store boot-first.json /tmp/boot.json +fds --json bays >/tmp/cartridge-inventory.json +fds machine store cartridges-first.json /tmp/cartridge-inventory.json +bash /usr/share/fds/capture-hardware /tmp/hardware-capture +tar -C /tmp -czf /tmp/hardware-capture.tar.gz hardware-capture +fds machine store hardware-first.tar.gz /tmp/hardware-capture.tar.gz +fds machine fetch boot-first.json /tmp/retrieved-boot.json +``` + +Use distinct names for subsequent sessions. A failed flush or unmount is an +error, not a successful save. These are machine diagnostics; do not use this +facility as ordinary user storage. The saved cartridge inventory is a persistent +diagnostic snapshot, including metadata already inspected during this boot. +Retrieve it with `fds machine fetch cartridges-first.json /tmp/saved-inventory.json`. +The daemon's live metadata cache remains volatile and is rebuilt from currently +attached devices; saved snapshots are never used to authorize media actions. +Together, explicit boot reports, inventory snapshots and hardware captures provide +the boot history, cached metadata and diagnostics assigned to FDS_INTERNAL in +master-plan section 12, without adding internal writes to startup or shutdown. + +## Dependencies and validation + +No new target package or Rust crate is required. The host image-tool prefix adds +`e2fsprogs` for ext4 creation, inspection and validation; it already supplies FAT +and EROFS tools. A private unprivileged user namespace gives created files root +ownership without requiring a root build session. + +`make internal-test` exercises the actual packaged runtime in ARM VMs with +virtual NVMe. Acceptance evidence belongs in [M12 validation](Developer-M12-Validation); +a passing VM does not verify the Pi EEPROM, PCIe path or physical flash durability. + +The Linux [ext4 mount documentation](https://www.kernel.org/doc/html/latest/admin-guide/ext4.html) +explains why read-only loading also disables journal replay. Filesystem creation +options follow the upstream [mke2fs manual](https://man7.org/linux/man-pages/man8/mke2fs.8.html). diff --git a/Developer-M0-Validation.md b/Developer-M0-Validation.md new file mode 100644 index 0000000..6c61473 --- /dev/null +++ b/Developer-M0-Validation.md @@ -0,0 +1,201 @@ +--- +include_toc: true +--- + +# M0 validation report + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Roadmap](Developer-Roadmap) + +This is the initial acceptance record, not a setup tutorial. Later documentation +improvements do not change the build results or skips recorded here. Linked logs +and artifacts are generated local files and are not included in a fresh checkout. + +Validated on 2026-09-20 in `/home/felis/source/fds` on an x86_64 Arch Linux +Framework host, running Linux 7.2.6-arch2-1. + +**M0 is complete:** the host bootstrapped an upstream Void build container, +cross-built an aarch64 glibc XBPS package, and built a static-musl AArch64 Rust +executable. M1 remains a proposal. + +All project-owned documentation, source comments, diagnostics, and ongoing +development context use English. The complete master plan retains its 67 +numbered sections. The pinned upstream submodule has no tracked modifications. + +## Changes + +- Added the repository skeleton, milestone instructions, master plan, architecture + contracts, and build documentation. +- Pinned `vendor/void-packages` to + `02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1` using a Git submodule and + `VOID_PACKAGES_COMMIT`. +- Added versioned, checksum-verified static XBPS host tools and a bubblewrap + bootstrap workflow for ordinary Arch users. +- Added FDS overlay preparation with upstream collision and stale-copy checks. +- Added the Rust workspace, Rust 1.98.0 pin, static musl linking configuration, + dependency-free `fds-smoketest`, and release settings from the plan. +- Added `make bootstrap`, `make smoke-test`, `make check`, package export, + ELF inspection, failure-path checks, and build input manifests. +- Reserved future component directories without implementing later milestones. + +The initial M0 file inventory appears at the end of this report; guides added +after acceptance are not included in that historical inventory. + +## Exact build commands + +These public entry points all completed with exit status 0: + +```sh +cd /home/felis/source/fds +make bootstrap +make smoke-test +make check +``` + +The underlying cross-build commands are: + +```sh +cargo build --locked --offline --release \ + --target aarch64-unknown-linux-musl -p fds-smoketest + +export PATH="$PWD/.host/xbps/usr/bin:$PATH" +export XBPS_ARCH=x86_64 +cd vendor/void-packages +./xbps-src -A x86_64 binary-bootstrap +./xbps-src -a aarch64 pkg hello +``` + +The first bootstrap created the build environment on this host. The final +public command runs reused verified downloads and that environment. Large +archives were prefetched from official servers with SHA-256 verification; +XBPS also verified package signatures. This was not a test of a pristine clone +on a second machine. + +## Smoke-test output + +Actual excerpts from `out/logs/smoke-test.log`: + +```text +Void commit: 02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1 +rustc 1.98.0 (88d9e12ae 2026-08-18) +cargo 1.98.0 (797e8a9bc 2026-08-05) +PASS: x86_64 static ELF +PASS: aarch64 static ELF +ldd (host diagnostic, exit 1): not a dynamic executable +PASS: aarch64 package exported to out/packages/ +PASS: aarch64 glibc ELF +PASS: XBPS package hello-2.12.3_1 architecture=aarch64 (glibc) +SKIP: ARM execution (optional qemu-aarch64 not installed); ELF verification passed +PASS: M0 smoke test complete +``` + +Rust `file` output identifies an `ELF 64-bit LSB executable, ARM aarch64`, +`statically linked, stripped`. `readelf` confirms no interpreter, shared library +dependency, or glibc symbol version requirement. The extracted GNU hello ELF +uses `/lib/ld-linux-aarch64.so.1` and requires `libc.so.6`. + +`make check` passed its ABI rejection, invalid-input, Void pin, dirty-upstream, +overlay collision, stale-overlay, configuration protection, and formatting +checks. An initial package-index architecture error was corrected by explicitly +setting `XBPS_ARCH=aarch64` when indexing the exported package; the full +smoke-test then passed. + +Actual logs and inventories: + +- Bootstrap log (`out/logs/bootstrap.log`, workstation-only evidence) +- Smoke-test log (`out/logs/smoke-test.log`, workstation-only evidence) +- Package build log (`out/logs/xbps-hello.log`, workstation-only evidence) +- Guardrail and formatting checks (`out/logs/check.log`, workstation-only evidence) +- Build package inventory (`out/manifests/void-build-packages.txt`, workstation-only evidence) +- Cached package input digests (`out/manifests/void-package-inputs.sha256`, workstation-only evidence) +- Artifact digests (`out/manifests/artifacts.sha256`, workstation-only evidence) + +Generated output is ignored by Git. These links refer to this validated checkout; +rerun the commands to recreate the output in another checkout. + +## Artifacts + +| Artifact | Verified ABI | SHA-256 | +| --- | --- | --- | +| `out/fds-smoketest` | AArch64, static musl | `3c719a386cfef5b67f90488d9c67075c835c1a42d4db0efac0b475267dfc25ee` | +| `out/packages/hello-2.12.3_1.aarch64.xbps` | aarch64 package, glibc executable | `2703e81cff78d3d11b1a36f323c0af25691b258effcd60fe7f6d074ca8cdf1d8` | + +## Known limitations + +- QEMU execution was skipped because qemu-aarch64 is not installed. No ARM + hardware, Pi boot, bay topology, E-Ink, or boot/shutdown timing was tested. +- Void source, Rust, and host XBPS are pinned. Rolling binary dependencies are + audited through cached digests and version inventories, but a complete binary + repository snapshot and reproducible release are still future work. +- This is a cross-build foundation; there is no bootable FDS image yet. +- Initial upstream bubblewrap cleanup reported a busy `/tmp` mount point while + returning success. Subsequent container execution and cross-build checks + passed. See [the build guide](Developer-Build-Host#known-limitations). + +## Next milestone proposal + +M1 should implement `fds-base` and `fds-base-files`, assemble an aarch64 glibc +rootfs with GNU userspace and XBPS, and generate `out/rootfs-aarch64.tar`. +Acceptance should audit package contents, confirm glibc/coreutils/binutils, +require no musl runtime package, and exclude BusyBox, systemd, and runit. +Begin M1 only after an explicit request. + +## Initial M0 changed-file inventory + +These are the paths recorded at M0. The manual now lives in the +`fds-os.wiki` submodule; this historical inventory retains its original paths. + +```text +.cargo/config.toml +.gitignore +.gitmodules +AGENTS.md +Cargo.lock +Cargo.toml +Makefile +README.md +VOID_PACKAGES_COMMIT +config/host-tools.conf +config/xbps-src.conf +docs/architecture.md +docs/boot.md +docs/build-host.md +docs/cartridges.md +docs/m0-validation.md +docs/master-plan.md +docs/packages.md +docs/performance.md +docs/recovery.md +docs/services.md +image/.gitkeep +out/.gitkeep +packages/fds-base-files/.gitkeep +packages/fds-base/.gitkeep +packages/fds-cartridged/.gitkeep +packages/fds-cli/.gitkeep +packages/fds-eink/.gitkeep +packages/fds-kernel/.gitkeep +profiles/.gitkeep +rust-toolchain.toml +rust/fds-boottrace/.gitkeep +rust/fds-burn/.gitkeep +rust/fds-cartridged/.gitkeep +rust/fds-cli/.gitkeep +rust/fds-common/.gitkeep +rust/fds-smoketest/Cargo.toml +rust/fds-smoketest/src/main.rs +rust/fds-stage0/.gitkeep +s6/source/.gitkeep +tests/hardware/.gitkeep +tests/integration/m0-checks +tests/unit/.gitkeep +tools/bootstrap-host +tools/build-package +tools/lib.sh +tools/prepare-void +tools/smoke-test +tools/verify-elf +vendor/void-packages +``` diff --git a/Developer-M1-Validation.md b/Developer-M1-Validation.md new file mode 100644 index 0000000..dbc5366 --- /dev/null +++ b/Developer-M1-Validation.md @@ -0,0 +1,132 @@ +--- +include_toc: true +--- + +# M1 root filesystem validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Build and usage](Developer-Rootfs) · [Roadmap](Developer-Roadmap) + +Validated on **2026-09-21 (Asia/Shanghai)** on the x86_64 Arch workstation. +The archive build timestamp is `2026-09-20T16:58:35Z`. This is userspace and +archive validation, not a Pi boot or native PID 1 result. + +## Artifact produced + +| Item | Recorded result | +| --- | --- | +| Output | `out/rootfs-aarch64.tar` | +| Archive size | 346,705,920 bytes | +| Configured packages | 108 | +| ARM ELF paths inspected | 1,167 | +| SHA-256 | `b2f0472d49aeb1565d124bc30f250cb304dc265be4d049f1ac9e8cb533742a34` | +| Void source pin | `02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1` | +| Emulation tools | PRoot 5.2.0_1 and QEMU aarch64 userspace 11.1.1_1 | + +The successful build is retained in `out/rootfs-build.Ewfkl2/`. Convenience links +point to its archive, staging root, and manifests. Future rebuilds may replace +the convenience links; this report describes the result above. + +Selected package versions: + +```text +fds-base-0.1.0_1 +fds-base-files-0.1.0_1 +fds-dasungd-0.1.0_1 +glibc-2.41_1 +coreutils-9.4_3 +binutils-2.44_2 +xbps-0.60.7_1 +s6-2.15.0.0_1 +s6-rc-0.6.1.1_1 +``` + +Every selected package archive is retained with its SHA-256 in the build +directory. Remote packages were verified against Void's RSA keys; local FDS +packages are unsigned development builds. The package archives total 71,371,822 +bytes, excluding signature files. This is input retention, not a complete +upstream repository snapshot or a reproducible-release claim. + +## Acceptance completed + +```sh +make rootfs PROFILE=cli +make rootfs-test +``` + +Both commands exited successfully. Checks established: + +- `fds-base` selects GNU userspace, glibc, XBPS and the mandatory Dasung package. + Neither the Void `base-system` nor its `base-files` package is installed. +- Every installed package is `aarch64` or `noarch` and configured. All scanned + ELF files are little-endian 64-bit ARM. Dynamic interpreters resolve to glibc. + Dasung is statically linked and has no dynamic interpreter or shared-library + dependency. No BusyBox, systemd, runit, or musl runtime package is present. +- The target's package setup ran under QEMU. English UTF-8, CA certificates, + the udev hardware database, the dynamic linker cache, and the Dasung s6 database + were generated before export. `_uuidd` was created and root remains locked. +- ARM GNU coreutils/binutils, glibc, shell pipelines, locale lookup, XBPS queries, + package integrity checking, Dasung configuration/EDID checking, and s6 database + queries succeeded. Tests do not open a real USB or DRM device. +- Exported ownership and permissions were checked, including locked account + files, sticky temporary directories, the tty group for `wall`/`write`, the + xbuilder group for `xbps-uchroot`, and portable `CAP_NET_RAW` for ping. +- GNU tar restored the exported ping capability successfully inside a private + user namespace, verifying the archive's actual extended-attribute encoding. +- A fresh extraction of the exported archive passed the same userspace checks. +- Failure tests rejected a forbidden package, an unconfigured package, a musl + package ABI, an injected x86_64 executable, a missing Dasung boot entry, and an + unsupported image profile. + +The build leaves upstream Void tracked files unchanged. The FDS layout manifest +avoids changing upstream lint rules. The tests use the exported tar rather than +trusting the original staging tree alone. + +## Regression checks + +The following also exited successfully after the M1 changes: + +```sh +make bootstrap +make smoke-test +make check +make dasung-test +``` + +The Dasung rerun passed all six Rust unit tests, the serial simulator cases, +package inspection, and the s6 boot/logger/socket/restart/stop check with a +read-only root and no monitor. Its physical-hardware checks remain skipped. +The existing M0 smoke script still reports its optional host-PATH QEMU skip: +the emulator is installed only inside the project-local build container. M1's +explicit container-based ARM runtime tests passed independently. + +Shell syntax checks were updated to recognize shell files by their interpreter, +so adding Python image tools does not make the older test suites parse Python +as shell. Python syntax, documentation file links, English project context, +and upstream tracked-file cleanliness were also checked. + +## Emulation and rootless-build limitations addressed + +The packaged PRoot version does not translate every modern syscall reliably. +The helper therefore makes the ARM filesystem the actual root with bubblewrap, +and uses direct QEMU for XBPS integrity checking and capability operations. +PRoot is used for child-process execution during package configuration and shell +sessions, not as evidence of successful ownership or capability changes. + +Rootless configuration emits expected group-change failures for `wall`, `write`, +and `xbps-uchroot`. Archive generation restores their intended numeric groups +and the archive tests verify the result. Other package payload ownership comes +from the retained XBPS headers. No host user ID is exported as the target root. + +## Remaining boundaries + +- M2: native s6 PID 1, complete boot graph, runtime mounts and booted service control. +- M4 and later: Pi firmware/kernel/initramfs, EROFS SYSTEM image, console, and media creation. +- Hardware: monitor picture, cold-power recovery, USB hotplug under Pi boot, and 12-bay behavior. +- Performance: no boot, shutdown, or monitor readiness benchmark was measured. + +The rootfs is not flashable, does not provide `fds` or a cartridge daemon, and +does not provision a login password. See [Rootfs](Developer-Rootfs) for the working +emulated-shell workflow and the [roadmap](Developer-Roadmap) for later acceptance gates. diff --git a/Developer-M10-Validation.md b/Developer-M10-Validation.md new file mode 100644 index 0000000..2b9932b --- /dev/null +++ b/Developer-M10-Validation.md @@ -0,0 +1,118 @@ +--- +include_toc: true +--- + +# M10 shutdown validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Shutdown usage](Developer-Power) · [Implementation ledger](Developer-Implementation-Status) + +M10 software acceptance passed on 2026-09-21 in isolated ARM virtual machines. +The complete shutdown suite, CLI packaging and earlier-milestone regressions +passed. Physical Pi timing, flash-controller durability, +power rails and battery behavior remain deferred. + +## Verified shutdown behavior + +The ordinary FDS user can request `fds poweroff` or `fds reboot`. New cartridge +operations freeze before the daemon stops the desktop, managed programs and +networking. The original DATA filesystem descriptor checks writeback, the +filesystem becomes read-only, a second check catches remount writeback, and normal +unmount completes before native s6 stops the remaining services and halts or +reboots. Dasung remains supervised until that final service stop. + +`make power-test` passed all six cases in `out/m10-vm.tituqzh7/`: + +| Case | Observed result | VM preparation-to-kernel-action time | +| --- | --- | --- | +| Idle | Ordinary-user poweroff; unprivileged internal hook refused | 540.733 ms | +| Active workload | GUI, DHCP and TERM-ignoring descendants stopped; DATA verified; actual reboot | 1835.329 ms | +| Direct native poweroff | Busy DATA blocks the native hook until resolved; cancellation refused | 414.091 ms | +| Active media preview | Shutdown and new confirmation refused; cancellation leaves disk unchanged | 289.099 ms after retry | +| Crashed DATA service | Same insertion stays quarantined across further restarts; no false SAFE | 281.307 ms after removing fixture | +| Attached-device writeback error | Real EIO retained across restarts; SAFE and shutdown refused | 344.777 ms after removing fixture | + +These are measurements inside generic QEMU with the Pi kernel, not Pi or +physical power-off benchmarks. They do not establish the sub-second idle or +1.5-second normal-DATA hardware targets. The workload deliberately ignores TERM, +so the manager must wait its bounded termination deadline and confirm actual +exit before proceeding. + +Each case records ordered `CLOCK_BOOTTIME` events and captures the final native +shutdown record. The test waits for the kernel's halt/reboot message and QEMU's +successful exit; closing the emulator does not count as a completed shutdown. +The full command log is `out/logs/m10-final-power.log`. + +## DATA integrity and failure handling + +The workload writes at least 128 MiB through a rotating 64 MiB file. After reboot, +independent host `e2fsck` finds a clean filesystem, SHA-256 matches all 64 MiB, and +an ordinary-user marker retains its exact contents. Open writable descriptors +and a shell working directory both block shutdown. Explicit resume enables new +operations but does not recreate stopped jobs. + +Before writable activation, a root-private insertion record is created under +`/run/fds/data-sessions`. A service crash cannot discard the original descriptor's +writeback history and then issue SAFE based on a fresh descriptor. The same +insertion becomes read-only and quarantined; a known error retains its diagnostic. +Records survive service restarts during this boot, not power loss or a different +insertion. The separate DATA regression passed in `out/m7-vm.vmhkmyqk/`. + +The writeback test uses QEMU's `blkdebug` to return persistent EIO for writes to a +known allocated file block. GPT, manifest and journal metadata remain outside +that injected sector. The device stays attached; Linux reports a real writeback +error and `syncfs` fails. The daemon refuses eject, keeps the fault after restart, +and blocks shutdown through another restart. No filesystem-cleanliness claim is +made for this deliberately damaged image. Removing the test fixture merely lets +the remaining system shut down; it never turns the failed insertion SAFE. + +See [QEMU's fault-injection documentation](https://www.qemu.org/docs/master/devel/testing/blkdebug.html). +A one-shot fault is insufficient evidence by itself because the guest may retry +it successfully; the acceptance case verifies the actual guest-visible EIO. + +## Native guard and dependencies + +The native shutdown hook remains alive while preparation is blocked, including +when an administrator calls the upstream `poweroff` directly. A nonzero hook +exit would let native shutdown continue, so watcher failure uses a blocked +signal wait. Normal waiting uses inotify and process/cgroup exit events, with no +sleep-based assumptions. Once native shutdown starts it cannot be cancelled. + +`fds-power` is a static alias of `fds`. The CLI now directly declares the already +locked `libc` crate for inotify, identity and signal operations. No new external +crate or target package is introduced. `fds-init` explicitly depends on +`fds-cartridged`, since the native hook requires its DATA checks. DHCP and its +privilege-separated children now share a managed cgroup. The final native grace +period is zero after managed exits and DATA verification have completed. + +## Regression evidence + +The development archive passed rootfs rejection checks, native PID 1/service +control and actual poweroff, desktop/network transitions, tooling/static-ELF +checks and `make check`. Logs are `out/logs/m10-final-*.log`; desktop evidence is +`out/m8-vm.1c20rcpb/`, native init is `out/m2-vm.5ARJNL/`. The final CLI image also passed native init (`out/m2-vm.ysuhZ5/`), all nine +stage0 cases (`out/m4-vm.e3SBsX/`), and the ordinary-user console checks +(`out/m5-vm.4l796uhp/`). Cartridge regression passed in `out/m6-vm.p7gfwnj2/`, DATA regression in +`out/m7-vm.82fqc9wa/`, and confirmed media writes in `out/m9-vm.5a58kou6/`. + +The development runtime tested here is `out/rootfs-build.WEtGnr/`, with +191 packages and 1406 AArch64 ELF files. Its rootfs SHA-256 is +`9de03100b998cac2a754fc8e05acbad0bd3f45846dd637ba4c597ea3ba61f5bb`. +Its SYSTEM is `out/system-build.dd3e74_2/system.img`, SHA-256 +`bef3b01bc8bc68ca673aa6a0ec73892ddaa2901864eed204745749915ee34b18`. +The subsequent explicit init-to-cartridge package dependency was verified in the +final CLI build; the development archive already includes that daemon through +its base package. Runtime binaries and service behavior are identical. +The CLI archive has 180 packages and 1394 AArch64 ELF files, and is 499527680 bytes. + +Kernel-to-console was 1821.039788 ms versus M9's 1854.858231 ms: **−33.818443 ms**. +The actual ARM comparison passed without a regression exception; see +`out/logs/m10-boot-comparison.log`. This is a VM observation, not a Pi target result. + +| Artifact | Location | SHA-256 | +| --- | --- | --- | +| CLI rootfs | `out/rootfs-build.0AzReg/rootfs-aarch64.tar` | `ebfc58171aeee6f737c2a93a0f3a318d4f02eb915edb3b5a3cf0f51848863930` | +| CLI SYSTEM | `out/system-build.c7q19iil/system.img` | `81dc8083777b497f02a290a15eeb246465689d0deaff97d761aeb882d140b44b` | +| FAT32 boot partition | `out/boot-build.42abbf_h/boot.fat` | `0972b5973d1fefc377ec4c800043fb8e4aa3013140f95218cdebcf38dcd22a49` | diff --git a/Developer-M11-Validation.md b/Developer-M11-Validation.md new file mode 100644 index 0000000..0c959da --- /dev/null +++ b/Developer-M11-Validation.md @@ -0,0 +1,114 @@ +--- +include_toc: true +--- + +# M11 stress validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Stress-test usage and physical procedure](Developer-Stress-Testing) · [Implementation ledger](Developer-Implementation-Status) + +M11 software acceptance passed on 2026-09-21. The expanded twelve-device suite +passed in `out/m11-vm.o1c_tqiv/`, including the explicit readiness gate and +diagnostic collector. The public `make stress-test` passed all six initial media +failure cases in `out/m11-faults.g5uw_jao/`. The strengthened final +`make media-fault-test` passed in `out/m11-faults.hfyqoui3/`, verifying the actual +virtual-device byte counter before each interruption. `make check` then passed. Physical Pi/hub/battery tests remain deferred. + +## Twelve-device evidence + +The complete runs used the packaged M10 CLI rootfs and actual Pi kernel in +an isolated generic ARM VM. A protected virtual SYSTEM root and twelve USB disks +were present: one DATA and eleven distinct PROGRAM cartridges. Every identity +matched its bay through cold boot, three simultaneous hotplug/removal cycles, +rotated placements and reverse device insertion order. + +Eleven direct readers each verified 32 MiB while DATA wrote and flushed 64 MiB. +All twelve measured I/O intervals overlapped. QEMU block counters confirmed the +corresponding reads/writes, so cached file reads alone could not satisfy the test. +A subsequent managed writer and its TERM-ignoring descendant ran during native +reboot. Independent host `e2fsck` passed; the entire 64 MiB payload matched its +expected SHA-256. Another boot with a different device declaration order retained +correct mapping and data, and completed native poweroff. Read-only cartridge +image hashes were unchanged. + +Kernel logs from normal stress/reboot contained no detected unexpected USB reset, +controller failure, I/O error or filesystem error. This only describes the virtual +hardware used in the test. It says nothing about physical voltage, hub firmware, +cables, RP1 or a storage controller's cache guarantees. + +## Startup observations + +Three paired samples used the same SYSTEM and virtual controller, with either +zero or twelve USB disks. The first complete run observed: + +| USB population | Kernel-to-console samples | Median | +| --- | --- | --- | +| Empty | 1907.961445, 1761.020483, 1847.460910 ms | 1847.460910 ms | +| Twelve | 1968.638224, 1998.194820, 1927.705171 ms | 1968.638224 ms | + +The median device-load difference was **+121.177314 ms**, above the 100 ms review +threshold. Stage0 discovery remained approximately 23–27 ms in both scenarios; +the increased work appears mainly between s6 start and console readiness. +The unchanged getty graph has no cartridge-readiness dependency. Concurrent USB +and cartridge processing competes with startup in this two-vCPU emulator. This +is an explanation of the controlled device-load comparison, not a claim that a +code change has been proven faster or that Pi timing targets passed. + +Earlier complete cold-boot pairs from interrupted test attempts observed ++122.544124 ms and +76.064782 ms; their raw reports remain in +`out/m11-vm.jcvwpxel/` and `out/m11-vm.w4oyzwje/`. They are not full acceptance runs. +The spread is retained rather than discarding slower observations. The expanded +suite then passed with medians 1900.735990 ms (empty) and 1957.588750 ms (twelve): +**+56.852760 ms**. All samples and the actual ARM comparison are stored in +`out/m11-vm.o1c_tqiv/boot-comparison.json` and `boot-review.log`. The blocked-daemon +case reached the real UID-1000 prompt with no cartridge control socket, then +completed enumeration and native shutdown after an explicit event released the +daemon. This verifies console independence under the same twelve-device load. + +## Test-harness findings + +The AArch64 Linux `O_DIRECT` flag differs from the x86_64 value. An initial +fixture incorrectly requested directory-only opens and failed before I/O. The +corrected ARM-only test helper performs aligned direct reads; byte checks and +block counters then passed. No target runtime change was needed. + +A burst of asynchronous device-deletion events could arrive while a new QMP +monitor connection was being greeted. The shared VM helper now accepts those +events before the greeting, without treating them as command responses. Checked +serial capture executes a command once and retries only a corrupted transfer. +It preserves kernel diagnostics instead of suppressing error messages. + +## Media failure acceptance + +Actual-device tests passed for write EIO, flush EIO and readback EIO. +Each leaves a failed operation across daemon restart, with no SAFE marker. +Cancellation, worker death and daemon death during confirmed writing also passed +in `out/m11-faults.g5uw_jao/`. The strengthened run in +`out/m11-faults.hfyqoui3/` additionally requires at least 64 MiB in QEMU's actual +write counter while the transfer remains incomplete, retaining that counter and +job state in an interruption record. All three strengthened interruption cases +passed: each had accepted 128 MiB into the guest write path and delivered 64 MiB +to the virtual disk, with a 270532608-byte source still incomplete. Cancellation +retained its diagnostic, worker death retained a failed-completion error, and +daemon death retained a restart error. Each remained failed after another +restart, with the daemon usable and no SAFE marker. + +Final logs are `out/logs/m11-final-stress-retry.log`, +`out/logs/m11-final-faults.log` and `out/logs/m11-final-check.log`. The final failure +run records input hashes and QEMU 11.1.1 identity beside its images. Runtime +packages and boot artifacts remain the verified M10 builds; M11 adds tests and +physical-test tooling rather than changing the target service behavior. + +## Physical acceptance + +The [physical procedure](Developer-Stress-Testing#prepare-the-physical-session) covers +bay calibration, cold boot, rotated placement, concurrent I/O, orderly reboot and +poweroff, power behavior and external timing. `tools/capture-hardware` collects +read-only local diagnostics with per-command exit statuses and checksums. It +reports collection success separately from hardware acceptance. All 17 capture +commands returned success in the ARM VM, and the resulting private directory +and checksums passed inspection. The blank physical session template remains +explicitly `not_run`. None of those +physical tests has been performed in this milestone. diff --git a/Developer-M12-Validation.md b/Developer-M12-Validation.md new file mode 100644 index 0000000..d982d1c --- /dev/null +++ b/Developer-M12-Validation.md @@ -0,0 +1,414 @@ +--- +include_toc: true +--- + +# M12 production preparation: software evidence + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Implementation ledger](Developer-Implementation-Status) · [EEPROM](Developer-EEPROM) · [Recovery](Developer-Recovery) + +M12 **software acceptance is complete**, including the signed local FDS/OS +0.1.0 release. Physical Pi tests, EEPROM application, display behavior and +storage power-loss behavior are deferred. Earlier checkpoints below preserve +the development history; the final acceptance identifies the released bytes. + +## Final local release acceptance + +Completed on 2026-09-21. The local release is +`out/fds-os-0.1.0/README.md` (`out/fds-os-0.1.0/README.md`, workstation-only evidence). +It contains 38 signed manifest entries, versioned images and packages, +profile inventories, source and complete build-input archives, guides and +reproducibility evidence. It has not been published or installed on hardware. + +- Frozen source SHA-256: `8947e57552f5db834fbb6bcce6d79623bf0605072b42b4ccf04d1e4fe7bb9f29`. +- Input-lock SHA-256: `6fa630446f91a7a0a2b85324b7a258ea6351978ff8f8a4cf9970f73f8d24fe2c`. +- Manifest SHA-256: `0c223b4996258efbe33db780ab0d05019c35e36be291499a9548c27371c451a4`. +- Signing public-key fingerprint (decoded 32 bytes): `d5c95e8c1a079ed523b859c0ee78b8e849bbf387b3719dcdae1661ae4f7eb267`. +- Signing public-key text-file SHA-256: `30d832747e95dca2c7ad3f5a117d2069a2d94e6d3eee2dd385caea688159e677`. + +Two newly restored trees, `out/rebuild-m12-v5-a` and `out/rebuild-m12-v5-b`, +ran `tools/in-frozen-build make all` sequentially with no network access. +Both started without FDS build outputs. The complete 27-artifact comparison +passed, covering all five installable images, three rootfs tars, four initramfs +formats, kernel/DTB, four EEPROM preview files and all nine FDS packages. +Logs: `out/logs/m12-offline-v5-{a,b}.log`; comparison: +`out/m12-reproducibility-v5.json` (`out/m12-reproducibility-v5.json`, workstation-only evidence). + +Every one of those artifacts also matches the normal build used by the complete +[Clap runtime acceptance](Developer-CLAP-Validation). The identity proof is +`out/m12-tested-to-release-v5.json` (`out/m12-tested-to-release-v5.json`, workstation-only evidence), +and `out/m12-clap-acceptance-index.json` records the 21 acceptance logs and hashes. +This establishes exact-byte coverage of the released images without repeating +the same VM suite against identical images. Physical tests remain deferred. + +The first restored tree additionally passed `make smoke-test`, `make check` +and `make signing-test` inside the isolated wrapper. A second full comparison +after those checks still passed. Logs are `out/logs/m12-v5-{smoke-test,check,signing-test}.log`; +the comparison is `out/m12-reproducibility-v5-after-checks.json`. +The optional direct host `qemu-aarch64` smoke run is reported as skipped when +absent; the separate ARM signing and OS runtime suites execute real ARM code. + +The assembler verified source identity and rootfs build inputs before signing. +Verification against the separately retained public key checked every manifest +entry. OpenSSL independently verified the actual release signature. Both actual +release archives were extracted with mode-preserving flags; the complete input +lock and standalone source identity passed. Evidence: +`out/m12-release-acceptance-v5.json` (`out/m12-release-acceptance-v5.json`, workstation-only evidence), +with logs in `out/logs/m12-v5-*.log` and extracted files under +`out/m12-release-acceptance.mzkfsvev`. + +The private signing key is outside the checkout, snapshot and release output. +Its corresponding trusted public key is +`/home/felis/.local/share/fds/signing-0.1.0-20260921/release-01.pub`. +On this workstation, from the checkout: + +```sh +out/rebuild-m12-v5-a/target/x86_64-unknown-linux-gnu/release/fds-release verify \ + out/fds-os-0.1.0 \ + --key /home/felis/.local/share/fds/signing-0.1.0-20260921/release-01.pub +``` + +| Image | Bytes | SHA-256 | +| --- | ---: | --- | +| `fds-boot-0.1.0.img` | 536,870,912 | `1466bbac3f782115bce78b48376ab90a7364416379700e944c529280b7253703` | +| `fds-internal-0.1.0.img` | 1,881,145,344 | `5197cd5edbbb9f689f12d233371cf0079ac50b36a4d3ca3cfe4c38cd320fc468` | +| `fds-recovery-0.1.0.img` | 491,593,728 | `4b0c9dad4336535d5b90c35cacd1500528836008be3384bdcd22b078b8a25f5b` | +| `fds-system-cli-0.1.0.img` | 493,879,296 | `03233a1118471cd8550ef2813fa99d371ef00b23edde5a3ade161d371e36c177` | +| `fds-system-development-0.1.0.img` | 1,989,148,672 | `9110f966212b59186e60ae7f88b6500fb0909478e6cbbb17735eebc969092c1e` | +| `kernel_2712-0.1.0.img` | 26,751,488 | `9e811bc55aa8e0b6a7233c6c84978244f2affee7bc808f3913dcd2fab3cd8013` | + +The [67-section ledger](Developer-Implementation-Status) records implementation, +evidence and physical deferrals for the entire master plan. Main-checkout status +documentation was updated after acceptance; the frozen trees and signed archives +retain the exact checkpoint documentation used for their source identity. +Consequently, their historical “verification in progress” wording predates this +record. Runtime code, configuration and image bytes were not changed by this +final documentation update. Use the preserved inputs to reproduce the exact +release images. The signed `RELEASE-NOTES.md` also corrects an archived guide's +fingerprint terminology: the tool hashes decoded key bytes, not the text file. +The standalone release verification guide includes that correction. The original +assembler output is retained at `out/fds-os-0.1.0-checkpoint/`; final packaging +adds these signed release notes and updated standalone guidance without changing +any image, package, source archive or input archive. + +## EEPROM configuration and rollback + +`make eeprom-test` passed against the actual pinned 2 MiB Pi 5 EEPROM image. +Evidence is in `out/m12-eeprom.5b97ph36/`; the public run is recorded in +`out/logs/m12-eeprom-final.log`. `make check` also passed afterward. + +The tests cover both production and development settings, exact configuration +readback, unchanged firmware payloads, identical repeated outputs, preservation +of unrelated conditional settings, exact rollback configuration, existing-output +refusal, and rejection of invalid firmware/device/config inputs. The tool never +invokes an update/apply command. The default firmware is a preparation input, +not a backup of the user's Pi. + +See the [EEPROM guide](Developer-EEPROM) for the source commit, hashes, complete profile +settings, generated files, and the deferred physical application/rollback steps. + +## Independent recovery + +The recovery implementation has its own rootfs profile, EROFS builder, local +maintenance console, read-only cartridge defaults, and explicit DATA check/repair +commands. It retains native s6 and the base Dasung controller. No additional Rust +crate or target package was needed: repair uses the existing e2fsprogs and kernel +loop support while preserving the physical disk reservation. + +`make recovery`, `make rootfs-test` and the complete `make recovery-test` passed. +The final runtime evidence is `out/m12-recovery.rlvmytrz/`, linked from +`out/m12-recovery-latest`. Logs are +`out/logs/m12-recovery-identity-build.log`, +`out/logs/m12-recovery-identity-rootfs.log`, and +`out/logs/m12-recovery-final-acceptance.log`. + +| Artifact | Verified result | +| --- | --- | +| Recovery rootfs | `out/rootfs-build.D5TCGs/rootfs-aarch64.tar`, 499,609,600 bytes; 180 packages and 1,394 AArch64 ELF files | +| Rootfs SHA-256 | `0611c0f8f24d04a733401063f9f52c7bc08f6389cb707601df083390fe063ae1` | +| Recovery EROFS | `out/recovery-build.9ukiaknh/recovery.erofs`, 490,598,400 bytes | +| EROFS SHA-256 | `7a219ca803325e4d60c3acf1b4cffe8717e7a7ec4e65377f48c900c131910bcd` | + +The tests use the packaged recovery image and Pi kernel. The only rootfs change +for virtual cartridge tests is the observed virtual bay map; no replacement +runtime binaries or simulated success responses are inserted. + +Verified cases: + +- Two independent EROFS constructions from the same tar produce identical bytes. +- Recovery boots with SYSTEM absent and with an unreadable SYSTEM. Its root is + read-only, PID 1 is native s6, the maintenance console is root, and Dasung is + supervised without needing the physical monitor. +- DATA remains read-only; ENVIRONMENT and Ethernet insertion do not automatically + start the desktop or DHCP. An ordinary user cannot invoke recovery repairs. +- A real ext4 inode reference-count error is detected without changing DATA. + Wrong confirmations and busy mounts are rejected. Confirmed repair fixes the + error, flushes/invalidate caches, and passes another read-only check. The + original test file survives, and a separate host e2fsck also passes. +- Failed checks survive daemon restart as quarantine. Confirmation from a previous + insertion is rejected. Clean checks preserve every byte of the DATA image. +- Recovery writes a complete known-good SYSTEM image to a disposable virtual USB + device, verifies it by readback, inspects it, and excludes it from DATA repair. + A separate VM boots that written image to the ordinary UID 1000 FDS console. + The source SYSTEM is the previously verified M10 CLI image; this proves the + recovery replacement workflow, not a new release-signing claim. +- Killing the daemon during a real checker kills the child and preserves DATA. + An explicit retry succeeds. Twenty partition-table rereads and a subsequent + service restart preserve the unmounted SAFE state; temporary loops are gone. +- The diagnostic collector completes all 17 commands and verifies its output + checksums. Successful cases power off through native s6. + +The interruption/rescan case also passed three consecutive focused runs in +`out/m12-recovery-rescan.rlr9u2aq/`, `out/m12-recovery-rescan.cvzye8z0/`, and +`out/m12-recovery-rescan.wj0jlfkq/`. This caught and fixed a real bug: temporary +partition disappearance during a reread previously cleared the eject record. +The daemon now retains that record while the same physical disk insertion is +present; a new insertion still invalidates it. + +The normal CLI rebuild and regression checks also passed: `make rootfs PROFILE=cli`, +`make rootfs-test`, `make system-card PROFILE=cli`, `make init-test`, +`make console-test`, `make data-test`, `make tooling-test`, and `make check`. +Evidence is in `out/logs/m12-cli-*.log`, `out/logs/m12-tooling.log`, and +`out/logs/m12-check.log`. The normal-image root check emitted +`FDS_M12_RECOVERY_BOUNDARY`, confirming that recovery repair remains unavailable +outside the recovery profile, even to root. + +That CLI rootfs is `out/rootfs-build.HqBtpz/rootfs-aarch64.tar` +(SHA-256 `af132511a6784eced9e52e971f010df6717e3caed751d14017ca9e924196e8b3`), +and its SYSTEM image is `out/system-build.psymtwbr/system.img` +(SHA-256 `5bc69f31d74a4c07596b696a72138cd24327ff4bc467f428717a56ce44c6bd27`). +Native init passed in `out/m2-vm.GyJsNg`, console checks in `out/m5-vm.h5ygfe1j`, +and DATA checks in `out/m7-vm.nt_tsdfs`. Console readiness was 1,853.221941 ms, +32.182153 ms above the earlier M10 CLI baseline of 1,821.039788 ms; this is below +the 100 ms investigation threshold. These artifacts precede the internal-settings +implementation below. No physical timing or media-durability claim follows from +these VM measurements. + +## Internal NVMe and machine settings + +Implementation and current usage are described in [Internal storage](Developer-Internal-Storage). +`make internal-test` passed in `out/m12-internal.vqr74fcs/`, linked from +`out/m12-internal-latest`. The public log is +`out/logs/m12-internal-final-acceptance.log`; rootfs evidence is in +`out/logs/m12-machine-final-rootfs.log`. The test uses packaged CLI and recovery +images, the actual Pi kernel, and disposable virtual NVMe devices. No runtime +binary or configuration file is injected into the booted OS. + +Verified cases: + +- Two complete 1,881,145,344-byte internal GPT images are byte-identical, with + independent FAT, EROFS, ext4, GPT and payload checks. Their SHA-256 is + `0d464ba18b0988a79a75ea54b6d81c5fa01acfea567b3d0a79f9faf2a428dfc6`. +- `make internal-image` publishes the same hash at `out/internal-build.xxzvw_3l/internal.img`, + linked from `out/fds-internal.img`; its log is `out/logs/m12-internal-published.log`. +- The actual loader selects one correctly laid-out NVMe and leaves every byte + unchanged during read-only loading. The private mount is gone afterward. +- Root recovery installs validated settings atomically, preserves previous bytes, + flushes/unmounts storage, and keeps this boot's active mapping unchanged even + across a cartridge-daemon restart. +- A separate reboot loads the saved machine name, bay map and hardware catalog. + The real virtual keyboard is recognized in its assigned bay with its saved name. +- Boot diagnostics survive reboot and are retrieved byte-for-byte. Non-root + writes, path traversal, duplicates, existing retrieval outputs and oversized + diagnostic files are rejected. Independent post-write e2fsck passes. +- BOOT and RECOVERY payload hashes remain unchanged after settings/diagnostic + writes. Ordinary user data is not placed on internal storage. +- A complete USB lookalike cannot supply settings. Two eligible NVMes boot the + normal UID 1000 console with explicit ambiguity/fallback status. +- Invalid JSON, writable settings, symlinks and unclean ext4 leave the disk bytes + unchanged, record an error and retain a usable recovery console with image + defaults. No automatic filesystem repair or journal replay is performed. + +An earlier test used unsupported PCI hotplug on QEMU's root PCI bus; the final +ambiguity fixture instead boots the ordinary SYSTEM with both NVMes present. +Earlier partial runs remain available, but are not counted as acceptance. + +The settings trust checks exposed omitted `/` metadata in the rootfs archive. +EROFS had synthesized a host-owned mode-0777 root directory. The archive now +records root:root mode 0755 explicitly, both OS builders reject old archives, +and rootfs/VM tests check the result. `out/m12-root-metadata-repro-audit.json` +compares two otherwise-identical recovery builds; only the new `.` entry differs. +The build-specific auxiliary linker cache has been removed. + +The tested recovery rootfs is `out/rootfs-build.P4WWwz/rootfs-aarch64.tar` +(SHA-256 `9d0e54533399da79616853e8233d2abb8efbc6b2d51772bd86f2669134c957b4`), +and its EROFS is `out/recovery-build.1ukd0xgl/recovery.erofs` +(SHA-256 `1c1930c86960f8e208408f4a0c244e4e5ae876263bab234b25307d4154ef3f91`). +The updated CLI rootfs is `out/rootfs-build.BP1Ylv/rootfs-aarch64.tar` +(SHA-256 `58eb153416972dca1c29a87d7b356638b8b1c08df56231470c18ee50fbad2d29`), +and SYSTEM is `out/system-build.o9d6affu/system.img` +(SHA-256 `cb544eb25f18b6572f78fc245818bd2cbfa18423b44d57b5af251069f2e6498e`). + +The complete recovery regression also passed against these updated artifacts in +`out/m12-recovery.idrwxslu/` (`out/logs/m12-machine-recovery-regression.log`). It +repeats the missing/damaged SYSTEM, actual DATA repair, interrupted-checker, +partition-reread, full SYSTEM burn/readback, diagnostics and replacement-boot +checks described above. This time the source SYSTEM is the updated CLI image +listed here, including the settings loader and corrected filesystem-root metadata. + +The updated normal CLI has already passed rootfs, SYSTEM construction, native +init, console independence and DATA regression checks. Logs are +`out/logs/m12-machine-cli-*.log`; native init evidence is `out/m2-vm.5FHPFv`, +console evidence is `out/m5-vm.ifuo9ei6`, and DATA evidence is +`out/m7-vm.6xcpnj97`. The measured VM console time was 1,865.948488 ms, +44.908700 ms above the earlier M10 CLI baseline, below the 100 ms review threshold. + +## Build-host regression + +After the host image-tool changes, `make bootstrap`, `make smoke-test`, and +`make check` all passed. Logs are `out/logs/m12-foundation-bootstrap.log`, +`out/logs/m12-foundation-smoke.log`, and `out/logs/m12-foundation-check.log`. +The smoke test cross-built and inspected GNU hello as aarch64 glibc and the Rust +smoketest as static aarch64-musl. Optional direct ARM execution was skipped because +the workstation lacks `qemu-aarch64`; the separate ARM VM/runtime suites above +still executed the actual OS tools. Upstream tracked Void files remain unchanged. + +## Development profile + +The native GNU C/C++ and Rust toolchains, build systems, Git, GDB, strace and Vim +are selected and documented in [Development](Developer-Development). The expanded image +passed `make development-test` in `out/m12-development.r6mrgm0b/`. +The rootfs is `out/rootfs-build.p8DKvd/rootfs-aarch64.tar` (2,012,047,360 bytes; +245 packages and 1,731 AArch64 ELF files), and its SYSTEM is +`out/system-build.jnubdw1v/system.img`. Logs are +`out/logs/m12-development-clock-{build,rootfs,system,acceptance}.log`. + +The ordinary UID 1000 console compiled and ran C/C++ with CMake/Ninja, C with +Meson/Ninja and Make, and a dependency-free Rust project with offline Cargo. +GDB stopped at `main`, reported a backtrace and continued; strace captured the +program's write; a local Git commit and headless Vim edit also succeeded. +SYSTEM remained read-only and unchanged, optional services stayed off, and +native shutdown completed. Desktop/network acceptance then passed in +`out/m8-vm.4c6_jj4h/`, and the complete shutdown/DATA error suite passed in +`out/m10-vm.q_ewzl4g/`; see `out/logs/m12-development-{desktop,power}.log`. + +This test found a real startup issue: the VM's wall clock started at Unix second +1 while CMake's immutable inputs had the source timestamp 1,789,909,701. +Ninja consequently tried to regenerate its manifest repeatedly. Native startup +now advances only an older clock to the fixed image timestamp, without a time +server or waiting. The compiler acceptance verifies that path; a separate +`make clock-test` passed in `out/m12-clock.l39g3uhx/`, preserving a guest clock +set to 2030 before the actual startup code runs. Non-root clock changes are +rejected and boot measurements still use `Linux CLOCK_BOOTTIME`. +`out/logs/m12-clock-final.log` records that check. This is not physical RTC +validation and the floor does not supply the accurate current date. + +The first corrected-clock run also exposed a test transport mistake: interactive +Bash consumed the literal tab in a generated Makefile. Test files now cross the +serial console as base64, preserving their exact bytes. + +## Release signature tool + +`make signing-test` passed in `out/m12-signing.md4r2kbo/`, recorded in +`out/logs/m12-signing-public.log`. It builds both x86_64 and static-musl ARM +executables, checks the RFC 8032 Ed25519 vector, and independently signs/verifies +the exact manifest domain with OpenSSL. Host and ARM signatures match OpenSSL's +bytes. Wrong and weak keys, altered signature/manifest/artifact bytes, wrong +sizes, missing files, symlinks, FIFOs, path traversal, duplicates, unknown fields, +oversized manifests, insecure private-key permissions and output overwrites are +rejected. These are disposable test keys, not a signed OS release. + +The verifier is included in the base CLI package for all profiles. Its guide and +dependency rationale are in [Release signatures](Developer-Releases), and the build now +collects license notices for all 68 resolved third-party ARM workspace crates, +including the vendored libusb notices. `make bootstrap`, `make smoke-test` and +`make check` passed after adding the signing dependencies, with logs under +`out/logs/m12-signing-{bootstrap,smoke,check}.log`. The later package-notice and +offline-build interface changes are being rebuilt and checked separately. + +## Repeated console timing checkpoint + +After the clock floor, packaged signer/notices and deterministic package epoch +changes, one console run measured 2,103.545197 ms. This exceeded the 100 ms +investigation threshold against the earlier single M10 sample. Five alternating +pairs then compared the preceding internal-settings CLI image with the updated +CLI image, using the same kernel and initramfs for every run. Evidence and image +hashes are in `out/m12-console-pairs.yuoz7unr/comparison.json`; all ten serial logs +are retained beside it. The execution log is `out/logs/m12-console-pairs.log`. + +| Image | Five console-ready samples, ms | Median, ms | +| --- | --- | ---: | +| Before | 1884.195231, 1909.087577, 1960.273690, 2055.400846, 1908.411769 | 1909.087577 | +| Updated | 1938.522887, 1913.188789, 1935.906888, 1958.748315, 1943.363749 | 1938.522887 | + +The median increased by 29.435310 ms, below the threshold. The initial updated +sample was outside the repeated updated range. This is workstation/QEMU evidence +including scheduling noise, not physical Pi performance; no artificial delay or +boot barrier was added. This checkpoint precedes Rust source-path normalization. + +## Complete build and release interface checkpoint + +`make packages` now uses the same complete base-package builder as rootfs +assembly. It passed in `out/logs/m12-packages-public.log`, including a fresh +cross-build of the Pi kernel. The preceding `make bootstrap`, `make smoke-test` +and `make check` passed in `out/logs/m12-offline-foundation-{bootstrap,smoke,check}.log`. +`make all` now sequences the complete image set. Its first complete run passed +in `out/logs/m12-all-first.log`. Subsequent rootfs and foundation checks passed, +as did the complete boot matrix (`out/m4-vm.kn9Hku/`), native development tools +(`out/m12-development.hpk59nul/`) and internal-storage acceptance +(`out/m12-internal.8z6nffpa/`). The corresponding logs are +`out/logs/m12-all-{recovery-rootfs,check,boot,development,internal}.log`. +These results precede source-path normalization in Rust release builds. + +The frozen-input, restore, network-isolation, comparison and signed-assembly +interfaces are documented in [Offline rebuilds](Developer-Reproducible-Builds). +`make release-contract-test` passed in `out/m12-release-contracts.yg33ry5r/` +(`out/logs/m12-release-contracts.log`). Its small fixtures verify rejection of +changed input bytes/modes/paths/symlinks, invalid lock metadata, differing or +missing artifacts, comparison of one tree with itself, changed input locks, +existing output reports/directories and private-key symlinks. Snapshot creation +also refuses an active base-package build. These contract checks do not stand +in for two actual complete offline builds. + +A settings-size regression test also passed: individually valid TOML comments +can expand during JSON escaping. The machine configuration now rejects an +encoded bundle exceeding its loader limit before installation writes any data. +All 15 shared-contract tests passed in `out/logs/m12-machine-encoding-tests.log`. + +## Earlier release checkpoints + +The user-requested [Clap migration](Developer-CLAP-Validation) is complete: 48 Rust +tests, ARM CLI/alias execution, the complete image build, all runtime suites, +signing and Dasung checks, and repeated startup measurements passed. All Rust +command-line handling now uses the shared locked Clap dependency. The v4 +offline candidate was deliberately stopped during kernel compilation to include +this change; its second tree was not built and it has no accepted comparison. +The accepted v5 pair uses this updated source checkpoint. + +`make release-contract-test` also passed with the actual archive writer and the +documented extraction flags in `out/logs/m12-release-contracts-v4.log`: repeated +archives match, and extraction preserves ordinary executable modes, sticky +directories, symlinks and the complete frozen lock. Default tar extraction had +stripped those permissions in a small diagnostic fixture. The v3 candidate was +stopped during kernel compilation before acceptance so that the corrected +archive instructions and the completed user-guide audit could be frozen +together. It is not a failed runtime test or an accepted release build. + +The first real snapshot exposed two restore defects, now corrected: shallow +Void bundles need their Git boundary file, and xbps-src's cross repository +configuration retains only custom filenames containing `local`. The dependency +that revealed the second issue was already captured; no network fallback was +added. `out/logs/m12-offline-v1-a.log` and `out/logs/m12-offline-v2-a.log` retain +the failed attempts. The resumed v2 tree is a diagnostic probe, not a clean +release acceptance run. + +The fresh offline v2 compilation matched the normal build byte-for-byte for all +seven core FDS binaries, Dasung, the Pi kernel/DTB, and the kernel/Dasung packages. +Records are `out/m12-offline-v2-rust-comparison.json` and +`out/m12-offline-v2-native-comparison.json`. Personal absolute build paths are +absent from all eight release executables after path normalization; see +`out/m12-remap-binaries.json`. Bootstrap, smoke-test and checks passed in +`out/logs/m12-remap-{bootstrap,smoke,check}.log`. + +The documented larger-disk GPT relocation was checked on a disposable regular +file in `out/m12-internal-relocation.j29f1rx6/`. After extending the image by +512 MiB and relocating its backup GPT, independent partition-table verification +passed and every partition's start, size and payload SHA-256 remained unchanged. +This verifies the image operation only; the physical write/readback and Pi boot +procedure in [Internal storage](Developer-Internal-Storage) remains deferred. + +Those checkpoints preceded the final clean v5 pair. The acceptance record at +the top of this page closes the software release work with actual complete +builds, comparisons, signing, archive verification and the master-plan audit. diff --git a/Developer-M2-Validation.md b/Developer-M2-Validation.md new file mode 100644 index 0000000..a0dbf45 --- /dev/null +++ b/Developer-M2-Validation.md @@ -0,0 +1,149 @@ +--- +include_toc: true +--- + +# M2 native init validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Boot and usage](Developer-Init) · [Roadmap](Developer-Roadmap) + +Validated on **2026-09-21 (Asia/Shanghai)** on the x86_64 Arch workstation. +M2's acceptance criteria passed in a complete ARM virtual machine: native s6 +owns PID 1, and `s6-rc change` controls an actual test service. This is a generic +ARM boot result, not a Raspberry Pi boot or performance measurement. + +## Recorded environment and outputs + +| Item | Result | +| --- | --- | +| Workstation | Arch Linux x86_64, Linux 7.2.6-arch2-1, Python 3.14.7 | +| Void source pin | `02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1` | +| Init packages | s6-linux-init 1.2.0.1_1, s6 2.15.0.0_1, s6-rc 0.6.1.1_1 | +| Emulation | QEMU system/user ARM 11.1.1_1; PRoot 5.2.0_1 for build-time child execution | +| VM | ARM `virt`, Cortex-A72 software emulation, 2 CPUs, 1 GiB RAM, AHCI disk, serial console | +| Generic test kernel | Signed Void `linux6.12-6.12.110_1.aarch64.xbps`; SHA-256 pinned in `config/vm-test.conf` | +| Image creator | mke2fs 1.47.4, libarchive tar import; no privileged mount | +| Final rootfs build | `out/rootfs-build.gSQUvr/`, built `2026-09-20T18:14:46Z` | +| Rootfs archive | 346,808,320 bytes; 109 configured packages; 1,167 ARM ELF paths | +| Successful boot test | `out/m2-vm.wYgNtQ/`; stable link `out/m2-vm-latest/` | + +Final rootfs SHA-256: + +```text +81113e027b5e65c7885aac9d4b88541de92470aaffb1fefd9508de30b97f0cad +``` + +VM disk SHA-256, verified unchanged after the automated boot: + +```text +00b48913a157de7a1107bf1bfff2c96db83dab48599fbe76be2ab433142d27d6 +``` + +The test directory retains `serial.log`, `command-line`, `test-environment.txt`, +`test-inputs.sha256`, `disk-before.sha256`, and `artifacts.sha256`. The rootfs build +retains its package inventory, exact package archives, build inputs, and hashes. +Convenience links may change after future builds; this report names the retained +successful directories. Rolling Void repositories are not a complete release +snapshot, and local FDS packages remain unsigned development artifacts. + +## What the ARM guest proved + +The VM booted the exported filesystem with its generated native init launcher. +The test overlay preserves all 54 entries in the actual compiled service database +and the original stage-2 script. Its hook runs the original stage 2 to completion, +then runs the checker as a child. It neither substitutes a shell PID 1 nor compiles +a different service graph. The checker is absent from the normal rootfs archive. + +The serial log records: + +```text +FDS_M2_PID1: /usr/bin/s6-svscan +FDS_M2_FILESYSTEM: read-only root, writable tmpfs, root ownership and capabilities +FDS_M2_BOOT: getty, eudev and Dasung supervised without a monitor +FDS_M2_SERVICE: s6-rc start, echo, supervised restart, stop and start passed +FDS_M2_PASS +``` + +The guest verified a read-only ext4 root, rejected a write under `/etc`, found +writable tmpfs at `/run`, `/tmp`, `/var/tmp`, and `/var/log`, and checked hostname, +root ownership, protected shadow permissions, and ping's restored capability. +Eudev answered its control-channel readiness request; getty and Dasung were up. + +`test-echo` started down. The checker brought it up through s6-rc, waited for +actual listener readiness, received an echo over its private Unix socket, restarted +it under supervision and observed a new PID, stopped it, started it again, and +stopped it before shutdown. Native s6 then completed power-off. The VM had no +network, host filesystem share, physical USB/display access, or host disk passthrough. +QEMU's disposable overlay preserved the backing disk. + +The interactive `make vm` workflow was also exercised: native PID 1, read-only +mounts, active-service listing, Dasung status, manual echo-service control, and +exit-to-power-off. Its transcript is `out/logs/m2-vm-shell.log`. Dasung returned +valid status JSON with `connected: false`, as expected without a monitor. The +explicit VM shell stops getty to take over the serial console; normal base +accounts remain locked. + +## Checks run + +These commands completed successfully during M2 validation: + +```sh +make bootstrap +make smoke-test +make check +make rootfs PROFILE=cli +make rootfs-test +make init-test +make dasung-test +make vm +``` + +Rootfs acceptance checks cover package integrity, ARM glibc execution, static +Dasung ELF properties, ownership/capability restoration, and rejection of wrong +ABIs, forbidden packages, incomplete configuration, host executables, missing +boot members, a shell init replacement, and unsupported profiles. + +Dasung regressions passed all six Rust unit tests, the serial simulator, package +inspection, and its independent s6 logger/socket/restart/stop test on a read-only +root. The final device-trigger guard changed only the init service definition; +the rootfs and full ARM boot checks were repeated afterward. + +Shell/Python syntax, local documentation links, English project-owned text, +recorded build/test source hashes, and unchanged upstream Void tracked files +were checked. The source pin was preserved. + +## Issues caught and fixed + +- The test kernel already mounts devtmpfs. Asking native init to mount `/dev` + again caused a genuine kernel panic. The generated launcher now consumes the + existing mount and leaves `/run` creation to s6-linux-init. +- The generated init template contains FIFOs. Auditing/export now permits them + only within its `run-image` subtree, and avoids reading them as ordinary files. +- The upstream CA installer hides updater failures. One build lacked the bundle + despite package configuration reporting success. Finalization now runs the + updater explicitly, checks its exit status, verifies the resulting cache, and + reports the failing command if any required finalization step fails. +- VM image creation uses a UTF-8 locale for libarchive's Unicode paths. AHCI uses + a QEMU snapshot overlay, with temporary files in the retained test directory. +- Initial device triggering now propagates a failed subsystem trigger instead + of continuing and reporting only the final command's result. + +The two essential internal s6-rc helpers remain until native shutdown's final +cleanup; warnings that the ordinary service-stop phase leaves them running are +expected. Known rootless `wall`, `write`, and `xbps-uchroot` ownership adjustments +remain handled and verified by archive export, as documented for M1. + +## Explicit skips and limits + +The M0 smoke script still skips its optional host-PATH `qemu-aarch64` execution +because QEMU is installed inside the project-local container. The separate ARM +userspace and complete ARM VM tests both passed. + +No Pi firmware, NVMe-to-cartridge root transition, production kernel/initramfs, +physical Dasung picture or cold-power recovery, bay mapping, DATA safety, +networking, or graphical profile was tested. There is no installable Pi image. +The VM kernel is only a test fixture. Its log timestamps and 180-second test +deadline are not boot or shutdown benchmarks. M3 and later implementation remain +outside this change. diff --git a/Developer-M3-Validation.md b/Developer-M3-Validation.md new file mode 100644 index 0000000..5a58547 --- /dev/null +++ b/Developer-M3-Validation.md @@ -0,0 +1,47 @@ +--- +include_toc: true +--- + +# M3 static Rust tooling validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +Validated on 2026-09-21 (Asia/Shanghai), x86_64 Arch Linux with pinned Rust 1.98.0. +This records the M3 diagnostic foundation before M4 adds the real boot handoff. +See [tooling usage](Developer-Tooling) and the [through-M12 ledger](Developer-Implementation-Status). + +Commands completed successfully: + +```sh +make tooling +make tooling-test +make bootstrap +make smoke-test +make check +``` + +Both exported binaries were AArch64 static ELF executables without a dynamic +interpreter, shared-library requirements or GLIBC symbol requirements. Supplemental +`ldd` printed `not a dynamic executable`. At this milestone: + +| Artifact | Bytes | SHA-256 | +| --- | --- | --- | +| `out/fds` | 537424 | `04052554eef3c592230aa912d6dbcaec9bc37c168c0e240b924a4fb215a68815` | +| `out/fds-stage0` | 375824 | `930afd537aaf8e90715e5ec73d70480b44fbe9dfe3c0615014ed78b36239bd7c` | + +Six shared-library tests passed. Actual ARM execution then verified tool identity, +JSON output, the master-plan manifest example, strict/bounded manifest rejection, +missing/unique/ambiguous SYSTEM selection, device-path rejection, explicit recovery +selection, and duplicate boot-option rejection. All filesystem fixtures were +temporary; no physical block device or monitor was accessed. + +Build/source hashes are in `out/manifests/fds-tools*.sha256`; logs are in +`out/logs/m3-tooling.log`, `m3-tooling-test.log`, `m3-bootstrap.log`, +`m3-smoke-test.log`, and `m3-check.log`. Later milestones will replace these tool +artifacts; the hashes above describe the validated M3 version. + +The M0 smoke script's optional host-PATH QEMU execution was skipped, as before; +the M3 integration script explicitly ran both ARM tools using container QEMU. +No physical Pi result or boot-speed claim is made. M3's stage0 refused normal +boot invocation rather than claiming its diagnostic skeleton could switch roots. diff --git a/Developer-M4-Validation.md b/Developer-M4-Validation.md new file mode 100644 index 0000000..9fdc944 --- /dev/null +++ b/Developer-M4-Validation.md @@ -0,0 +1,100 @@ +--- +include_toc: true +--- + +# M4 software boot validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +Validated on the x86_64 Arch workstation on 2026-09-21 (Asia/Shanghai). +Physical Pi testing is deferred by the user. This report establishes the +software boot machinery, not the physical NVMe/RP1/display path or performance +targets. M5 supplies the ordinary-user FDS console and timing instrumentation. + +## Verified path + +The cross-built Pi 5 kernel loaded the actual Rust-only initramfs, discovered +GPT partition identity, mounted EROFS read-only, transferred proc/sys/dev mounts, +stopped and reaped the early Dasung controller, and executed native s6. Guest +checks confirmed 16 KiB pages, s6-svscan as PID 1, the static packaged FDS CLI, +matching module release, loading/unloading the `dummy` kernel module, and one +base Dasung process. Ownership, ping capabilities, volatile filesystems and +start/restart/stop of the test service also passed. + +The final VM matrix, `out/m4-vm.flkV6j/`, passed nine cases: + +1. Uncompressed initramfs. +2. Gzip initramfs. +3. Legacy-LZ4 initramfs. +4. Zstandard initramfs. +5. No SYSTEM at startup, followed by virtual USB insertion and automatic boot. +6. Two SYSTEMs refused, followed by explicit console recovery. +7. Recovery selected by kernel option while SYSTEM is also present. +8. Invalid SYSTEM filesystem refused, with recovery still usable. +9. Foreign OS root refused, with recovery still usable. + +Each case reached the guest checks and orderly power-off. Input image hashes +were unchanged. The test uses QEMU 11.1.1, `virt`, `max` CPU, TCG, 1 GiB RAM and +two virtual CPUs, without networking or host-device passthrough. Its recovery +disk is a selection fixture, not the independent M12 recovery product. + +The FAT32 boot builder produced a 512 MiB `FDS_BOOT` partition, checked it with +`fsck.fat`, extracted it with mtools and compared every file hash. Two consecutive +builds matched byte-for-byte. Two full kernel builds also produced identical +kernel Image, Pi 5 DTB and XBPS package hashes. These narrow reproducibility +results do not establish a complete offline release rebuild. + +## Commands and artifacts + +The completed sequence included `make kernel`, `make tooling`, `make tooling-test`, +`make rootfs PROFILE=cli`, initramfs construction, and these final regression gates: + +```text +make bootstrap +make smoke-test +make check +make rootfs-test +make init-test +make system-card +make boot-volume +make boot-test +make dasung-test +``` + +Final logs are `out/logs/m4-final-*.log`; rootfs construction is recorded in +`out/logs/m4-rootfs.log`. The configured rootfs is `out/rootfs-build.p0HSQv/`: +111 packages, 1,168 inspected ARM ELF files, and a 407,009,280-byte tar. + +| Artifact | SHA-256 | +| --- | --- | +| Rootfs tar | `ba06bdcb8f4fc537c8cb4ce9d31404d3cd1d14f01b41cb597a53ec48f0ba6710` | +| SYSTEM GPT image | `d15af6feccbaeba77920e0b5ee13076dbdc80a076c973068a1f68e86bb599bf1` | +| Pi kernel Image | `78f027ad3124da7b06d4559f49ea7f63c312cd24e653d0f8e29ee2391f520bc4` | +| Kernel XBPS package | `6c361df5bf31ef517b6bad7e44220a8126f7c2fac341379e8c1d6b39fcbe8c20` | +| Uncompressed initramfs | `bc8905c5ca40fe827a44f7dd73f88aa1c99a3b84fa474535ccb36a212682b514` | +| Production FAT boot partition | `940b7d62ba5923a3f92db6939aca1534f0f2061376c1fec7604a50e6d3b3f0b8` | + +The kernel is 6.12.87-fds1, built from Raspberry Pi commit +`ae4d75fb36deba0fe1a986d9bfae65755e082dd0` with the FDS configuration delta. +Void remains pinned to `02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1` with no tracked +upstream changes. Firmware package/version/hash are pinned in +`config/pi-firmware.conf`. Detailed package inventories and source inputs remain +beside each versioned build. + +## Findings and limits + +The original Cortex-A72 VM CPU did not support the retained 16 KiB kernel page +configuration; `max` successfully runs the Pi kernel. A repeated VM test exposed +getty's shutdown hangup discarding pending serial output. The checker now stops +getty after verifying it and reopens the console before final service checks; +no boot delay was added. + +The M0 host-PATH QEMU execution remains an optional skip. Actual ARM userspace +and full-system VM execution passed using project-local emulators. Physical +Pi firmware boot, EEPROM changes, NVMe/RP1, monitor EDID acceptance, image quality, +cold-power display recovery, bay wiring/power and real boot/shutdown timings +remain explicitly untested. + +Use the [boot guide](Developer-Boot) to reproduce these artifacts and understand the +distinction between a boot partition, SYSTEM cartridge and complete internal disk. diff --git a/Developer-M4-Work.md b/Developer-M4-Work.md new file mode 100644 index 0000000..d1aa4fb --- /dev/null +++ b/Developer-M4-Work.md @@ -0,0 +1,64 @@ +--- +include_toc: true +--- + +# M4 boot integration development record + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +The user has authorized the remaining software through M12. This page records +M4 development decisions. Software validation is complete; see [M4 evidence](Developer-M4-Validation). Physical Pi validation remains deferred. + +The Pi kernel overlay starts from the exact source commit and archive checksum in +Void's pinned `rpi5-kernel` recipe (6.12.87). It uses `bcm2712_defconfig` plus the +small FDS delta, verifies Kconfig actually applied that delta, and exports the Pi 5 +Image, DTBs, overlays, and modules without modifying upstream Void files. +The upstream Pi 5 16 KiB page configuration is retained. + +Critical storage, EROFS, keyboard, console and VC4 display support are built in. +VC4 also requires the SND/SND_SOC core to be built in; optional sound devices can +remain modules. CH341 SPI is disabled for the Dasung companion bridge. Generic +ARM VM drivers are built into this same kernel so we can exercise software boot +without claiming to emulate Pi firmware or RP1. Debug/BTF artifacts are omitted; +module authentication is part of immutable image/release authentication rather +than random build-generated module keys. + +`make kernel` uses the existing Void build container and cross compiler. Its +additional host dependencies are Perl and Python for kernel generation, bc for +build-time arithmetic, Bison/Flex for Kconfig, kmod for module indexes, OpenSSL +headers for kernel build utilities, and archive tools. OpenSSL is a host build +dependency, not a new Rust runtime dependency. The source and checksum are recorded +in `packages/fds-kernel/template`. + +Image builders use a separate project-local prefix at `.host/image-tools` for +signed Void x86_64 `erofs-utils`, `dosfstools` and `mtools`, plus their resolved +libraries. They create filesystem images without privileged loop mounts and do +not mutate the shared kernel build container. Versions and cached inputs are +recorded under `out/manifests/image-tool-*`; full input locking is M12 work. + +Stage0 now has a Rust PID-1 implementation: direct filesystem +syscalls, authenticated kernel uevents, explicit normal/recovery partition +selection, early Dasung supervision, EROFS validation, mount transfer, and exec +of native s6. Its normal path refuses to run outside root PID 1 on ramfs/tmpfs. +The implementation reuses the locked `libc` crate for these Linux syscalls. +The full initramfs matrix passed all four formats, missing-media USB insertion, +ambiguous SYSTEM recovery, explicit recovery, invalid filesystems and foreign +root rejection. A refreshed base package/rootfs build is being completed. +QEMU uses its `max` CPU because the retained 16 KiB Pi kernel cannot execute +on the old Cortex-A72 VM model. + +The FAT32 boot builder verifies every file after reading it back with mtools. +Directory timestamps are normalized as well as file timestamps; two consecutive +builds from the same inputs produced the same complete image SHA-256. This narrow +result is not a claim of complete offline or release reproducibility. +Two complete kernel builds also produced identical kernel Image, Pi 5 DTB and +XBPS package digests. The helper now reuses a build after checking recorded +source inputs, its package and all exported files. `--rebuild` bypasses reuse. + +The [boot guide](Developer-Boot) documents usable build commands, image boundaries, +waiting-console commands, display policy, and remaining physical checks. + +References: [Raspberry Pi cross-compilation](https://www.raspberrypi.com/documentation/computers/linux_kernel.html#cross-compile-the-kernel), +[Linux initramfs](https://docs.kernel.org/filesystems/ramfs-rootfs-initramfs.html), +and [root handoff semantics](https://man7.org/linux/man-pages/man8/switch_root.8.html). diff --git a/Developer-M5-Validation.md b/Developer-M5-Validation.md new file mode 100644 index 0000000..c85e908 --- /dev/null +++ b/Developer-M5-Validation.md @@ -0,0 +1,76 @@ +--- +include_toc: true +--- + +# M5 software validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +M5's ordinary-user console and software timing tools passed on the x86_64 Arch +build host using the actual AArch64 kernel, initramfs and SYSTEM image in QEMU +11.1.1. Physical Pi boot, power-on, display and timing acceptance is deferred. +These runs completed on 2026-09-20 UTC (2026-09-21 in Asia/Shanghai). + +## Usable console and event evidence + +`make console-test` passed in `out/m5-vm.fuml66z0/`: + +- Native s6 starts the local autologin console as `fds`, UID/GID 1000. Commands + execute at `FDS>`, HOME and working directory are `/home/fds`, the temporary + home accepts writes, and SYSTEM rejects writes. +- Reports contain stage0 entry, SYSTEM discovery, root mount, root switch, + s6 stage-2 entry and first-console readiness for the current boot. Desktop + readiness is absent. No missing event is replaced with a zero timestamp. +- Exiting and restarting the console preserves the first readiness event. +- Holding eudev startup behind a test FIFO does not prevent the usable prompt. + The test releases that gate after executing a console command. +- Saved-report comparison rejects an unexplained regression greater than + 100 ms, permits an explicit explanation, and rejects incompatible platforms. + +The normal run measured 1,669.027 ms kernel-clock-to-console and 664.032 ms +s6-stage-2-to-console. The blocked-eudev run measured 1,659.990 ms and +581.113 ms. These are individual emulator observations, not physical budgets. +Nine further boots compare Rust optimization levels in +`out/rust-profiles.x_wgx1kp/`; see [Performance](Developer-Performance) for the table, +raw report locations, clock boundaries and decision to retain `opt-level=z`. +`make console-vm` was also opened interactively and verified UID 1000. + +## Build execution correction + +PRoot intermittently aborted during ARM subprocess execution on this host. One +failure also returned a misleading zero status. Runtime checks now require an +explicit final child acknowledgement, and finalization propagates failures. +The image builder now uses a private user-namespace `binfmt_misc` registration +with static QEMU, requiring Linux 6.7 or newer. It leaves the workstation's +registration table unchanged. The existing Void `util-linux` package provides +mount and capability-drop helpers; no target package was added for emulation. + +`tests/integration/rootfs-emulation.py` verified nested ARM execution, exact +nonzero exit propagation, isolation from the host registry and removal of +setup capabilities before target execution. Only namespace-local SETFCAP remains +for package configuration. A fresh complete rootfs build passed afterward. +Historical M1/M2 reports retain the older PRoot workflow as historical evidence. + +## Final artifacts + +| Artifact | SHA-256 | +| --- | --- | +| Rootfs TAR, `out/rootfs-build.9kTEAH/` | `eef5764034d315b5e012679ef7ed165c6961b69aa88f9514c77cf08ef67bf8f7` | +| SYSTEM GPT/EROFS, `out/system-build.o948uzwd/` | `d2e00f56f1ecc689de455a9066259cefa95bd0bbd271e7f2d3d4cd3235dffd91` | +| Uncompressed initramfs, `out/initramfs-build.dzggpetk/` | `2a6009800f4c73707b165a3a3142b5baff688c685b7ea9fcd196e9ab6079a4de` | +| Production FAT32 boot partition, `out/boot-build.bzakcn8s/` | `cd242813ded7c860d291c1dbf984bf69b48cc3b58da93f9bdebf83068139abf1` | +| Kernel Image, unchanged from M4 | `78f027ad3124da7b06d4559f49ea7f63c312cd24e653d0f8e29ee2391f520bc4` | + +The rootfs contains 111 packages and 1,169 AArch64 ELF files. Its TAR is +407,480,320 bytes. Per-build package/input inventories accompany the image. +The boot FAT is a partition image, not a complete internal NVMe disk. + +The final sequential regression run exited zero: `make bootstrap`, +`make smoke-test`, `make tooling-test`, `make check`, `make rootfs-test`, +`make init-test`, `make system-card`, `make boot-volume`, `make boot-test`, +`make console-test`, and `make performance-test`. Logs are +`out/logs/m5-verified-*.log`. The native-init VM is `out/m2-vm.sYk3Hu/`; +the nine-case stage0 matrix is `out/m4-vm.RHUJX5/`. Shared Rust tests passed. +M0 still reports its optional host-PATH QEMU skip; project-local ARM execution +and full-system emulation actually ran. No physical boot benchmark is claimed. diff --git a/Developer-M6-Validation.md b/Developer-M6-Validation.md new file mode 100644 index 0000000..2923756 --- /dev/null +++ b/Developer-M6-Validation.md @@ -0,0 +1,81 @@ +--- +include_toc: true +--- + +# M6 cartridge software validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +Verified on 2026-09-21 on the x86_64 Arch build host. These results use actual +AArch64 binaries, kernel USB events, s6 services and filesystem mounts in QEMU +11.1.1. Physical Pi wiring, bay calibration, simultaneous physical devices and +USB electrical behavior remain deferred. Twelve-device virtual stress follows +in M11; this milestone does not claim that test has already run. + +## Behavior verified + +`make cartridge-test` passed in `out/m6-vm.u4w9b7pl/`: + +- The unconfigured production image reports twelve UNCONFIGURED bays. A probe + discovers the virtual controller's actual topology; the fixture then maps its + USB 2/3 companion ports explicitly. No `/dev/sdX` name selects a bay. +- The SYSTEM image boots from virtual USB and is reported PROTECTED. Eject fails. +- A virtual keyboard is recognized by the hardware catalog without a filesystem. +- The ordinary FDS user can query the daemon. A different UID is rejected even + when the test temporarily relaxes socket file permissions. Slow and oversized + clients do not prevent another control request from completing. +- Inserting ENVIRONMENT media triggers a validated read-only mount. Writes fail. + Metadata is inspected privately before the mount becomes publicly accessible. +- A shell inside the mount and an additional bind mount each prevent SAFE. + After those uses end, eject unmounts successfully and reports SAFE. A service + restart preserves that ejected state until removal and reinsertion. +- Reinsertion into another port follows that bay. Wrong manifest classes, + symlink metadata, and multiple named partitions are rejected without choosing + a first candidate. Surprise removal clears inventory; restart remounts valid, + nonejected read-only media. +- Holding cartridged before its readiness notification does not prevent the + ordinary UID-1000 FDS console from accepting a command. + +Every VM disk used read-only remained byte-identical. Tests use disposable image +files and virtual USB devices, never host disks or physical USB passthrough. +VM test helpers and the temporary root test console do not ship in SYSTEM. + +Shared unit tests also cover USB bus renumbering, mapping aliases and overlapping +mappings, manifest symlinks/FIFOs, malformed contracts, and absent USB subsystems. +The older native-init VM revealed the absent-USB case; its corrected empty-inventory +behavior passed a new unit test and the complete native-init boot afterward. + +## Final artifacts and regression evidence + +| Artifact | SHA-256 | +| --- | --- | +| Rootfs TAR, `out/rootfs-build.FpSNiJ/` | `00fc851affe0f1c42d289cc05ae376788beae9a6b0dce39c6a1f3b86687d1baf` | +| SYSTEM, `out/system-build.df52fwqj/` | `908645b47bfe207470eeb82436f8a644d34cf3703156a7bdcbf2a3534e9346fe` | +| Production boot FAT, `out/boot-build.bka9vs1y/` | `fb54582a9bd3bb393299002a65a5b0035e806e6733e81a437369a0b152b297e2` | + +The rootfs contains 112 packages and 1,170 AArch64 ELF files; its TAR is +408,248,320 bytes. `fds-cartridged` is statically linked, installed by its own +mandatory base package and supervised by native s6. No external Rust dependency +or locked external version changed; the lockfile change only adds the workspace +member. Existing libc, serde, serde_json and toml provide the required interfaces. + +The final pipeline exited zero: fresh rootfs and initramfs, tooling tests, +`make check`, rootfs checks, native-init boot, SYSTEM/boot-volume construction, +the nine-case stage0 matrix, console tests and cartridge tests. Logs are +`out/logs/m6-verified-*.log`. M0 bootstrap and smoke-test also passed in +`out/logs/m6-final-bootstrap.log` and `out/logs/m6-final-smoke-test.log`. +The optional M0 host-PATH emulator skip remains explicit; project-local ARM +execution and full-system tests actually ran. + +Native-init evidence is in `out/m2-vm.S16Noo/`; stage0 cases are in +`out/m4-vm.XBTVEW/`; ordinary-console traces are in `out/m5-vm.51n612y3/`. +The normal trace measured 1,753.878 ms from the kernel clock to console, versus +1,669.027 ms in the final M5 normal trace: +84.851 ms, below the 100 ms explanation +threshold. The actual ARM comparison tool passed; see +`out/logs/m6-boot-regression.log`. These are individual emulator observations, +not proof of physical Pi latency or a statistically established timing change. + +Writable DATA, consumer shutdown and syncfs follow in M7; desktop/profile +activation follows in M8. Read [Cartridge usage](Developer-Cartridges) for exact commands, +configuration syntax, current states and safe-eject boundaries. diff --git a/Developer-M7-Validation.md b/Developer-M7-Validation.md new file mode 100644 index 0000000..36abd53 --- /dev/null +++ b/Developer-M7-Validation.md @@ -0,0 +1,82 @@ +--- +include_toc: true +--- + +# M7 writable DATA validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +Software acceptance passed on 2026-09-21 using the x86_64 Arch build workstation, +QEMU 11.1.1, the actual AArch64 Pi kernel and static FDS tools. Physical Pi, +removable-media power loss and real shutdown/eject latency remain deferred. + +## DATA acceptance workload + +`make data-test` passed in `out/m7-vm.drg2a4ml/`: + +- Without DATA, the ordinary UID-1000 console can write its temporary home. + Inserting a valid ext4 DATA cartridge activates `/data`; user writes persist + through eject and reinsertion while the temporary home remains separate. +- An unmanaged shell working directory and an extra bind mount each prevent + SAFE. The additional-mount check runs before stopping managed programs. +- A managed writer and its child run as UID/GID 1000 with no effective + capabilities and `no_new_privs`. At least 128 MiB is written while repeatedly + overwriting a 64 MiB file. Both processes intentionally ignore TERM. +- Eject escalates to kernel cgroup termination after its exit deadline, observes + that the complete group has exited, runs syncfs, closes its writeback handle, + unmounts normally and only then reports SAFE. The managed process count is zero. +- SAFE persists across a daemon restart until removal/reinsertion. Reinsertion + reads the previously saved user file, followed by another successful eject. +- Two DATA candidates in one discovery snapshot remain read-only. Explicit + selection activates the requested bay; another request cannot replace an + active session underneath its users. +- Pulling DATA during dirty writes produces actual kernel storage errors. The + daemon stops the remaining managed descendants and clears the vanished mount. + Eject afterward fails; this path never claims SAFE or a clean filesystem. + +After the VM releases its disks, the test extracts each safely ejected ext4 +partition into an ordinary file. Host e2fsprogs 1.47.4 runs `e2fsck -fn` on both; +both exit zero. `debugfs` extracts the 64 MiB stress payload, whose size and SHA-256 +match the complete expected byte pattern. No host filesystem mount is involved. +See `primary-fsck.log`, `secondary-fsck.log`, `filesystem-tools.log`, and +`surprise-removal-kernel.log` in the evidence directory. + +During development, removal exposed partially disappearing USB sysfs entries. +Discovery now skips incomplete individual entries, retains a still-present +mounted disk identity, and does not restart the daemon merely because another +USB device is being removed. Fixture checks and the final hotplug workload passed. +Kernel diagnostics sharing the test serial console are preserved in logs and +excluded only from the CLI JSON parser; errors are not treated as successful I/O. + +## Final image and regressions + +| Artifact | SHA-256 | +| --- | --- | +| Rootfs TAR, `out/rootfs-build.wFuKoS/` | `cdd7c2c8809bf144234470839cc082014f9757276e1d2f769496511c056fe127` | +| SYSTEM, `out/system-build.y7aiuhbi/` | `1675f9225ccca746045f7f999fed5b9b48d7bdd23869aa65a90f97a4219aafab` | +| Production boot FAT, `out/boot-build.abgq4799/` | `fb54582a9bd3bb393299002a65a5b0035e806e6733e81a437369a0b152b297e2` | + +The rootfs has 112 packages, 1,170 AArch64 ELF files and a 408,309,760-byte TAR. +M7 adds no external Rust dependency or target package. Its process tracking uses +already-enabled Linux cgroup v2 and PID-limit support. cgroup setup belongs to +the cartridge daemon and adds no console dependency. Test filesystem tools are +the host e2fsprogs tools already used by the M2 workflow. + +The final sequential pipeline exited zero: fresh rootfs and initramfs, tooling +checks, formatting/negative checks, rootfs checks, native-init VM, SYSTEM and boot +volume, all nine stage0 cases, ordinary console tests, cartridge tests and DATA +tests. Logs are `out/logs/m7-verified-*.log`. Native-init evidence is +`out/m2-vm.1sPKEy/`; the cartridge regression is `out/m6-vm._cgfbexe/`. +The temporary root test console and writer binary are confined to disposable +fixtures and do not ship in the production rootfs. + +The ordinary-console trace in `out/m5-vm.6_55lhge/` measures 1,769.585 ms from the +kernel clock, versus M6's 1,753.878 ms: +15.707 ms. The ARM regression comparison +passes the 100 ms explanation threshold (`out/logs/m7-boot-regression.log`). +These individual VM observations do not establish Pi performance targets. + +Attached-media writeback fault injection, broader stress and complete shutdown +ordering remain part of M10/M11. Physical flash-controller cache and power-loss +behavior require hardware. Read [DATA usage](Developer-Data) for the implemented user +workflow and [the ledger](Developer-Implementation-Status) for remaining milestones. diff --git a/Developer-M8-Validation.md b/Developer-M8-Validation.md new file mode 100644 index 0000000..be7cd98 --- /dev/null +++ b/Developer-M8-Validation.md @@ -0,0 +1,126 @@ +--- +include_toc: true +--- + +# M8 desktop and network validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Usage guide](Developer-Desktop) · [Implementation ledger](Developer-Implementation-Status) + +M8 software acceptance passed on 2026-09-21, including corrected CLI and +development images and the complete M0–M7 regression suite. These results concern +actual ARM software under QEMU, not a physical Raspberry Pi or E-Ink display. + +## Functional acceptance + +`make rootfs PROFILE=development`, `make desktop-test`, `make rootfs-test`, and +`make system-card PROFILE=development` passed. Evidence is retained in +`out/m8-vm.kf5ohhq8/` and `out/logs/m8-verified-*.log`. + +The test verified: + +- No X server, WindowMaker, or DHCP client runs by default. +- The ordinary FDS user can request and stop the graphical session. +- X11 requires its authority cookie; a client without the cookie is rejected. +- WindowMaker manages a real xterm, and injected X11 keyboard input executes a + command inside that terminal as the ordinary user. +- The screenshot shows the shipped grayscale controls, Terminus font, white + background, and FDS terminal prompt. It is `desktop.png` in the evidence folder. +- No user font cache is generated when the session starts. Image construction + normalizes directory timestamps before generating the packaged cache. +- Stopping the desktop leaves no Xvfb, WindowMaker, or xterm process; Dasung + remains running. Starting a second session succeeds without a reboot. +- Each activation gets its own request/readiness times. The boot report retains + the first desktop-ready event instead of replacing it on restart. +- A recognized ENVIRONMENT cartridge starts the desktop. Safe eject and surprise + removal stop it. Manual deactivation remains effective while media is inserted; + removal/reinsertion permits automatic activation again. Unknown profiles do not + start a session. +- A PROGRAM cartridge initially disallows execution and never autoruns. Explicit + launch runs as UID 1000 with the application/library paths set. A symlink + escaping `app` and a path traversal name are rejected. Eject stops managed + consumers before unmounting and reporting SAFE. +- A virtual USB Ethernet adapter activates DHCP and receives a real virtual + lease. DHCP-provided DNS reaches the volatile resolver file. Explicit network + off stops the client and leaves the managed interfaces administratively down; + restarting networking obtains an address again. +- With Ethernet present but no DHCP server, the console and service controls + remain usable. DHCP readiness is never a lease wait. + +The measured first activation was **1048.946380 ms** from request to WindowMaker +readiness in this software-emulated VM. This is a measurement of this run, not a +Pi performance claim. `activation.json` contains the raw CLOCK_BOOTTIME values. + +## Retained development artifacts + +| Artifact | Location | SHA-256 | +| --- | --- | --- | +| Development rootfs, 499210240 bytes | `out/rootfs-build.eTfJu2/rootfs-aarch64.tar` | `d3ee3b622c2ce9c34783a72a6b40d6a29b624768ac978b6db840347e32ea46c0` | +| Development SYSTEM GPT/EROFS | `out/system-build.p61o8p_o/system.img` | `e7c5c2e716260969eacbab3cd3b0a6bae16005826de1a90f0c325f3f60125855` | + +The rootfs audit counted 191 packages and 1402 AArch64 ELF files. Its package +archives, signatures, versions, scripts, ownership, and hashes are retained +beside the rootfs. The graphical test uses a disposable root-console fixture with +Xvfb; the normal image still selects Xorg and the ordinary-user console. + +## Regression acceptance + +The corrected CLI rootfs contains 180 packages and 1390 AArch64 ELF files. +`make tooling-test`, `make check`, `make rootfs-test`, `make init-test`, +`make boot-test`, `make console-test`, `make cartridge-test`, and `make data-test` +all passed. Bootstrap and smoke-test also passed after the namespace-wrapper +change; their logs are `out/logs/m8-final-bootstrap.log` and +`out/logs/m8-final-smoke.log`. + +| Artifact | Location | SHA-256 | +| --- | --- | --- | +| CLI rootfs, 495953920 bytes | `out/rootfs-build.0ayTj2/rootfs-aarch64.tar` | `1b82905789d00ba78f4474ad8613303c1d5684ba7f5f55a30cdde785a35cae06` | +| CLI SYSTEM | `out/system-build.jjgm8xox/system.img` | `be530d8076fffe1ca0de7951952e1bcec763bf6e811d068e40f75961ca3e46e1` | +| FAT32 boot partition | `out/boot-build.newd2icx/boot.fat` | `518deb1b8bee9f013d60c4cd22bea4380357e001db558a74026e2511da2c50a0` | + +The boot matrix is in `out/m4-vm.TPiKHf/`, console checks in +`out/m5-vm.f9612amj/`, cartridge checks in `out/m6-vm.cnvl_fxp/`, and sustained +DATA writes plus independent filesystem checks in `out/m7-vm.rko_14_q/`. +The corrected archive and booted guest both verify the terminal-accounting +helper's `root:utmp` ownership, and the guest verifies loopback is up. + +The normal kernel-to-console observation was 1854.724813 ms, compared with +1769.584786 ms in `out/m5-vm.6_55lhge/`: **+85.140027 ms**. The actual static ARM +`fds-boottrace compare` passed without a regression exception; output is in +`out/logs/m8-verified-boot-comparison.log`. This single software-emulated comparison +is below the 100 ms review threshold and does not establish Pi performance. + +## Test corrections and build findings + +The original `xprop -spy` approach could miss readiness when the EWMH atom was +created during startup. The final static helper speaks the small required part +of X11 directly: it installs its event subscription before starting WindowMaker, +then observes the supporting-window property. It adds no external Rust crate or +Xlib dependency to the FDS executable. + +Terminus needed its supplied fontconfig acceptance rule enabled. Font caches +also needed the same normalized timestamp as the exported archive; otherwise +image creation invalidated directory timestamps and caused a user cache rebuild. +The final VM test checks the resulting behavior. + +QEMU's restricted user network omitted DNS from its DHCP offer. The test now +runs QEMU in a separate network namespace and uses its normal virtual DHCP/DNS +backend there. That namespace has no host or Internet route. QEMU diagnostics +are saved separately from guest serial output, so emulator messages cannot be +mistaken for a CLI JSON response. No workstation network interface, physical +USB device, or physical block device is passed to the guest. + +## Deferred acceptance + +Pi DRM/VC4 output, virtual-terminal switching on the physical machine, monitor +resolution and E-Ink legibility/refresh quality, physical keyboard/pointer input, +and real CLI-to-desktop latency remain untested. The base Dasung daemon is still +mandatory and independent of optional services. Its physical recovery behavior +remains subject to the [Dasung hardware procedure](Developer-Dasung). + +The development profile currently adds X11 diagnostics. The broader development +toolchain, independent recovery, image signing, complete offline input lock, and +release packaging remain part of M12. Full shutdown ordering and broader fault +injection remain M10/M11 work. diff --git a/Developer-M9-Validation.md b/Developer-M9-Validation.md new file mode 100644 index 0000000..45214c9 --- /dev/null +++ b/Developer-M9-Validation.md @@ -0,0 +1,165 @@ +--- +include_toc: true +--- + +# M9 media-tool validation + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Media-tool usage](Developer-Media-Tools) · [Implementation ledger](Developer-Implementation-Status) + +M9 software acceptance passed on 2026-09-21. Both packaged profiles, ARM image +construction, confirmed virtual USB writes, SYSTEM creation/write/boot, and the +earlier-milestone regression checks passed. All devices in these tests are +disposable regular files exposed only to isolated ARM virtual machines. Physical +Pi, media-controller and display tests remain deferred. + +## Verified behavior + +`make media-image-test` created SYSTEM, DATA, PROGRAM and ENVIRONMENT images with +the actual static ARM `fds-burn`. Independent `sfdisk`, Python SHA-256, +`fsck.erofs` and `e2fsck` checked the outputs. Invalid GPT extents, extra partitions, +internal partition labels and inconsistent backups were rejected even when the +fixture recomputed valid CRCs. Existing output files and recursive source/output +layouts were refused. Evidence: `out/m9-images.zqojr8hc/`. + +`make media-test` verified these cases through the ordinary FDS user's CLI: + +- Inspection reports the bay's capacity, logical sector size, model, serial and + kernel insertion identity. The running SYSTEM and mounted media refuse writes. +- Preview and cancellation leave the target unchanged. A confirmation is bound + to its operation, bay, insertion and image hash; a wrong phrase cannot proceed. +- Changing source bytes after preview is detected before writing. Replacing the + device in the same bay invalidates the old confirmation and leaves the new + insertion untouched. +- A PROGRAM image is written, flushed, read back and marked SAFE. Its backup GPT + moves to the end of a larger device. SAFE survives a daemon restart; a new + insertion mounts normally and supports explicit unprivileged execution. +- PROGRAM, DATA and ENVIRONMENT images are created on the running ARM machine. + DATA permits ordinary-user writes, retains data across removal/reinsertion and + passes independent host `e2fsck` after safe eject. +- Cartridge status remains usable while a separate write worker is active. + Source DATA cannot be ejected while the worker holds its image, including + across the worker's private mount namespace. +- Killing the cartridge daemon during an awaiting-confirmation operation leaves + a failed record on restart. It never becomes SAFE or automatically remounts. +- A valid EROFS/GPT containing an approximately 60 KiB malformed manifest is + rejected before writing. Its diagnostic stays bounded and printable, the + daemon retains the same process ID, and subsequent status requests work. + +The final virtual USB evidence is `out/m9-vm.cdlj6_r7/`. The crash case uses native s6 +supervision controls; see the [upstream s6-svc interface](https://skarnet.org/software/s6/s6-svc.html). + +## SYSTEM creation through boot + +`tests/integration/m9-system.py` mounts a complete prepared SYSTEM after boot, +then runs the real ARM filesystem/image builder against that read-only tree. +The ordinary user previews and confirms a write to another virtual cartridge. +After device flush and readback verification, a fresh VM boots the written disk +through the actual Pi kernel, stage0 and native s6 to the ordinary FDS prompt. +The test verifies UID 1000, read-only SYSTEM behavior, Dasung, the preserved +`root:utmp` terminal-helper ownership, and packaged Terminus caches without a +new user cache. Final evidence: `out/m9-system.ej8oz_av/`. + +This demonstrates creation and boot of a usable SYSTEM on ARM, not just a valid +partition header. It does not emulate Raspberry Pi firmware, physical NVMe/RP1, +USB power behavior or Dasung video output. + +## Implementation and dependencies + +`fds-burn` is static-musl Rust. `fds-inspect` and `fds-eject` are static aliases of +`fds`; the daemon starts a bounded worker for each media operation. The worker +opens the exact disk with exclusive access, checks live identity again before +writing, validates the image's filesystem and manifest, and verifies actual +readback after flushing and invalidating the device cache. A root-owned record +prevents interrupted operations from acquiring a false SAFE status after restart. + +Protection checks cover mounts, swap, kernel holders and the three reserved +internal partition names. Existing primary and backup GPTs are also inspected +directly, so protection does not depend solely on partition uevent timing. Native +4 KiB logical-sector disks are explicitly unsupported by this writer. + +The eight new locked Rust dependency entries are explained in the +[media guide](Developer-Media-Tools#dependencies-and-current-evidence). Filesystem tools +and GNU coreutils already belong to the base image; their package dependencies +are now declared explicitly. No init replacement, background network service, +new native library dependency of an FDS executable, or boot-time cache builder +was introduced. Void's tracked source files remain unchanged. + +## Findings resolved during validation + +A private checker device node initially failed to open because `/run` is mounted +with `nodev`. The final code creates an inaccessible private directory on +`devtmpfs`, opens its read-only device inode, and immediately unlinks it. The +filesystem checker receives that handle and runs as UID 1000; global `/dev/loopN` +permissions are unchanged. Image mounts exist only in the worker's private mount +namespace and disappear when it exits. + +The namespace review also identified the need to block eject until the worker +exits: unmounting only the daemon's namespace could otherwise report a source +DATA cartridge SAFE while the worker retained its mount. The VM test now covers +that exact case. + +Malformed TOML can include most of its source text in an error message. Worker +diagnostics are now limited to 2048 UTF-8 bytes plus a truncation suffix, with +control characters replaced by spaces. This prevents cartridge contents from +overflowing the daemon's status protocol or injecting terminal controls. + +The sustained-write regression initially failed because kernel I/O diagnostics +interrupted CLI JSON in the middle of a serial line during deliberate removal. +The test now executes each command once, saves its stdout, and transfers it with +base64 and a SHA-256 checksum. A damaged transfer is retried without rerunning +the command. Kernel output remains in the serial log and a separate diagnostic +capture. The corrected test passed in `out/m7-vm.t5d5fd_4/`, including independent +ext4 checks and the exact 64 MiB write payload. + +## Packaged artifacts and regressions + +The rebuilt CLI rootfs contains 180 packages and 1393 AArch64 ELF files. All FDS +executables remain static-musl. Package archives, input hashes, ownership and +capability records are retained beside the rootfs. + +| Artifact | Location | SHA-256 | +| --- | --- | --- | +| CLI rootfs, 498616320 bytes | `out/rootfs-build.gs9ZTA/rootfs-aarch64.tar` | `4abbc4e4e1823ef78acde2645d915160aa8e1c0f8b3173e8bc73f8d6b923d502` | +| CLI SYSTEM | `out/system-build.a74aryls/system.img` | `b7a71dad45c259c92ec5a5ac72b87356132f642df5c33d74d20e2e6ca564dee6` | +| FAT32 boot partition | `out/boot-build.oapz3fuf/boot.fat` | `ac3ee864c6293dd30a640fbbc0514b08acb92cda2d25d05128d234229308ac8f` | + +The earlier-milestone regression run passed tooling, formatting/build checks, +rootfs rejection checks, native init, the stage0 boot matrix, console and +cartridge tests. Its logs are `out/logs/m9-final-*.log`; the corrected DATA run +is `out/logs/m9-data-retry3.log`. The final diagnostic change was then rebuilt +into the CLI archive and passed the complete media write/boot and archive checks +in `out/logs/m9-hardened-*.log`. + +Native init evidence is `out/m2-vm.ijzHZC/`, the boot matrix is +`out/m4-vm.sxNb8D/`, console checks are `out/m5-vm.6suv4qjj/`, and cartridge +checks are `out/m6-vm.o5sjhatc/`. Kernel-to-console was 1854.858231 ms in this +VM run, compared with M8's 1854.724813 ms: **+0.133418 ms**. The actual static ARM +comparison passed without an exception; see `out/logs/m9-boot-comparison.log`. +This is below the 100 ms review threshold and is not a Pi performance result. + +The development rootfs contains 191 packages and 1405 AArch64 ELF files. +`make rootfs-test`, `make desktop-test` and development SYSTEM construction passed, +followed by the final tooling and formatting/build checks. Evidence is +`out/m8-vm.498wsxsv/` and `out/logs/m9-hardened-*.log`. The screenshot was inspected +and retains the grayscale Terminus terminal. First desktop activation was +1111.300247 ms in this VM; physical display timing remains unmeasured. + +| Artifact | Location | SHA-256 | +| --- | --- | --- | +| Development rootfs, 501882880 bytes | `out/rootfs-build.sQCq5V/rootfs-aarch64.tar` | `87e9a35829432f07e4a4bfa4c69eabd5ba0f30a25eadfed05ea9393bd1fb2afa` | +| Development SYSTEM | `out/system-build.l5e3j1pg/system.img` | `bcb8dfa7ec6e3207080f5c68074cc0c27d1bea63a0b51561c8ec628691e03611` | + +The generic rootfs convenience links were restored to the CLI build after the +development checks. `out/rootfs-cli.tar` and `out/rootfs-development.tar` retain +explicit pointers to these two verified archives. + +## Deferred and remaining acceptance + +Physical media throughput, flash-controller write-cache behavior, USB brownouts, +power-loss durability and Pi boot remain deferred. No throughput or hardware +latency claim is made. Broader I/O failure injection and twelve-device stress +belong to M11; ordered power-off and persistent DATA error handling belong to M10; +signing, independent recovery and complete release inputs belong to M12. diff --git a/Developer-Master-Plan.md b/Developer-Master-Plan.md new file mode 100644 index 0000000..83a5597 --- /dev/null +++ b/Developer-Master-Plan.md @@ -0,0 +1,2561 @@ +--- +include_toc: true +--- + +# FDS/OS MASTER IMPLEMENTATION PLAN + +This is the complete design specification, including future components and +command interfaces. **M0–M12 software acceptance is complete, including Dasung base +integration and the signed local 0.1.0 release.** +Tests requiring physical Pi hardware are deferred; software and VM checks remain +in scope. See the [implementation ledger](Developer-Implementation-Status). A command appearing +here is not evidence that it exists in the checkout. Start with the +[project overview](Overview) and [first-build walkthrough](Developer-Getting-Started) +for current usage, or the [roadmap](Developer-Roadmap) for implementation status. +The 67 numbered sections below retain the full project requirements. + +## 1. Project objective + +Implement an independent Linux distribution for Raspberry Pi 5: + +**FDS/OS — Felis Data Systems Operating System** + +FDS/OS targets a retro portable computer with these characteristics: + +- Raspberry Pi 5 +- aarch64 +- 12 USB cartridge bays +- removable SYSTEM / DATA / PROGRAM / ENVIRONMENT cartridges +- Dasung Paperlike 13K grayscale E-Ink display +- removable battery +- Very fast boot +- Very fast shutdown +- native s6 init +- Void/XBPS package ecosystem +- glibc as the system ABI +- Rust + static musl for FDS-owned programs wherever practical + +Core design principle: + +> Preserve compatibility with modern Linux software while making FDS-owned software as independent of distribution runtime libraries as practical. + +--- + +# 2. ABI and libc strategy + +The system must have two distinct userspace domains. + +## 2.1 General userspace + +Use: + +```text +glibc +``` + +Applies to: + +```text +GNU coreutils +GNU binutils +bash +grep +sed +gawk +findutils +util-linux +procps-ng + +Xorg +WindowMaker + +XBPS packages + +Python +GCC +LLVM +third-party software +``` + +The system package architecture is: + +```text +aarch64 +``` + +Do not use: + +```text +aarch64-musl +``` + +--- + +## 2.2 FDS control plane + +FDS-maintained programs default to: + +```text +Rust ++ +aarch64-unknown-linux-musl ++ +static linking +``` + +Include at least: + +```text +fds-stage0 +fds +fds-cartridged +fds-burn +fds-inspect +fds-eject +fds-profile +fds-power +fds-boottrace +``` + +Principle: + +```text +FDS-owned runtime program + ↓ +static musl where practical + ↓ +Linux syscall ABI +``` + +These programs should not depend on the system's glibc version. + +The following checks must pass: + +```bash +file /usr/bin/fds +ldd /usr/bin/fds +``` + +Expected output resembles: + +```text +statically linked +not a dynamic executable +``` + +--- + +# 3. Do not convert the entire system to musl + +Do not force the following components to use musl: + +```text +Xorg +WindowMaker +general XBPS packages +GNU toolchain +graphics stack +plugin-heavy applications +``` + +Do not attempt to maintain two complete repositories: + +```text +glibc repository +musl repository +``` + +The purpose of musl is limited to: + +> Providing a stable, self-contained static linking environment for FDS-owned control-plane executables. + +--- + +# 4. Init architecture + +FDS/OS must be a native s6 system. + +Final architecture: + +```text +Linux + ↓ +/sbin/init + ↓ +s6-linux-init + ↓ +s6-svscan + ↓ +s6-rc +``` + +Prohibited: + +```text +systemd → s6 +runit → s6 +OpenRC → s6 +custom shell PID1 → s6 +``` + +PID 1 must ultimately belong to the s6 architecture. + +Use: + +```text +skalibs +execline +s6 +s6-linux-init +s6-rc +``` + +The s6-rc source database resides at: + +```text +/etc/s6-rc/source +``` + +compiled database: + +```text +/etc/s6-rc/compiled +``` + +live database: + +```text +/run/s6-rc +``` + +**The compiled database must be generated during image construction.** + +Normal boot must never run: + +```text +s6-rc-compile +``` + +--- + +# 5. Package architecture + +Use Void Linux only for: + +```text +upstream package ecosystem +XBPS +xbps-src +glibc packages +GNU packages +kernel packaging reference +third-party packages +``` + +FDS defines its own: + +```text +fds-base +``` + +Do not directly depend on an all-inclusive Void base metapackage. + +Initial dependencies of `fds-base`: + +```text +glibc +glibc-locales + +coreutils +binutils +findutils +diffutils +grep +sed +gawk +bash +tar +gzip +xz + +util-linux +procps-ng +shadow +kbd + +e2fsprogs +dosfstools +erofs-utils + +eudev +kmod + +iproute2 +iputils + +pciutils +usbutils + +ca-certificates +tzdata + +xbps + +skalibs +execline +s6 +s6-linux-init +s6-rc + +fds-base-files +fds-cartridged +fds-cli +fds-dasungd +``` + +FDS-owned static-musl binaries must not require: + +```text +musl runtime package +``` + +musl is already statically linked into each executable. + +--- + +# 6. Build host + +Primary build machine: + +```text +x86_64 Arch Linux +``` + +Target: + +```text +aarch64 Raspberry Pi 5 +``` + +Normal development **must not require an ARM build host**. + +The Pi 5 serves as: + +```text +deployment target +hardware validation target +performance target +``` + +It is not the primary build machine. + +Void's xbps-src supports foreign Linux hosts. Cross-build with: + +```bash +./xbps-src -a aarch64 pkg PACKAGE +``` + +This performs a cross build. + +Rust FDS components use: + +```text +aarch64-unknown-linux-musl +``` + +This Rust target explicitly supports cross-compilation from other hosts. + +--- + +# 7. Repository layout + +Create: + +```text +fds-os/ +│ +├── README.md +├── Makefile +├── Cargo.toml +│ +├── fds-os.wiki/ # Authoritative Gitea wiki submodule +│ ├── Home.md +│ ├── Overview.md +│ ├── Hardware-Setup.md +│ ├── Developer-Guide.md +│ ├── Developer-Architecture.md +│ ├── Developer-Boot.md +│ └── Developer-Performance.md +│ +├── vendor/ +│ └── void-packages/ +│ +├── packages/ +│ ├── fds-base/ +│ ├── fds-base-files/ +│ ├── fds-kernel/ +│ ├── fds-cartridged/ +│ ├── fds-cli/ +│ └── fds-eink/ +│ +├── rust/ +│ ├── fds-common/ +│ ├── fds-stage0/ +│ ├── fds-cli/ +│ ├── fds-cartridged/ +│ ├── fds-burn/ +│ └── fds-boottrace/ +│ +├── s6/ +│ └── source/ +│ +├── profiles/ +│ ├── cli.toml +│ ├── development.toml +│ ├── windowmaker.toml +│ └── recovery.toml +│ +├── image/ +│ ├── build-rootfs +│ ├── build-system-cartridge +│ ├── build-boot-volume +│ ├── build-initramfs +│ └── build-recovery +│ +├── tools/ +│ ├── bootstrap-host +│ ├── build-package +│ ├── qemu-test +│ ├── burn +│ └── configure-pi-eeprom +│ +├── tests/ +│ ├── unit/ +│ ├── integration/ +│ └── hardware/ +│ +└── out/ +``` + +--- + +# 8. Managing the Void upstream + +Do not maintain a long-lived fork with changes throughout void-packages. + +Use: + +```text +vendor/void-packages +``` + +Pin it to a specific commit. + +Record: + +```text +VOID_PACKAGES_COMMIT +``` + +Keep this submodule clean. Prepare a writable checkout at `.host/void-packages` +from the same pinned commit. Inject FDS packages from `packages/` into that +build checkout; keep generated sources, caches and build containers outside +the submodule. + +Required relationship: + +```text +same FDS commit ++ +same Void commit += +same package input +``` + +Do not drift with daily changes to the Void rolling repository. + +--- + +# 9. Rust workspace + +Create the workspace: + +```toml +[workspace] +members = [ + "rust/fds-common", + "rust/fds-stage0", + "rust/fds-cli", + "rust/fds-cartridged", + "rust/fds-burn", + "rust/fds-boottrace", +] +``` + +Default release profile: + +```toml +[profile.release] +opt-level = "z" +lto = true +codegen-units = 1 +panic = "abort" +strip = true +``` + +Benchmark performance-critical binaries with: + +```text +opt-level = z +vs +opt-level = 2 +vs +opt-level = 3 +``` + +Do not assume that the smallest executable starts fastest. + +--- + +# 10. Rust dependency policy + +Use Clap for every Rust command-line interface, including test helpers. Declare +typed commands, options, validation, and generated help; do not hand-parse the +argument vector. The workspace shares the version already used by Dasung and +pins it in `Cargo.lock`. Kernel command-line files and cartridge metadata remain +separate domain formats with their own strict parsers. + +FDS core programs should avoid: + +```text +tokio +openssl +glibc-specific crates +libudev-sys +large asynchronous runtimes +``` + +Prefer: + +```text +std +libc crate +rustix +nix +serde +toml +small synchronous event loop +netlink/sysfs directly where reasonable +``` + +Do not introduce a large runtime merely to listen for a few uevents. + +--- + +# 11. Boot architecture + +The machine's internal NVMe belongs to the: + +```text +machine firmware layer +``` + +The SYSTEM cartridge contains the: + +```text +operating system userspace +``` + +Boot sequence: + +```text +Raspberry Pi EEPROM + ↓ +internal NVMe + ↓ +kernel + ↓ +tiny FDS initramfs + ↓ +fds-stage0 + ↓ +locate SYSTEM cartridge + ↓ +mount EROFS + ↓ +switch_root + ↓ +s6 + ↓ +READY +``` + +Do not make Raspberry Pi firmware search all 12 USB cartridges for a boot device. + +--- + +# 12. Internal NVMe layout + +Recommended layout: + +```text +GPT + +p1 FDS_BOOT + FAT32 + 512 MB + +p2 FDS_RECOVERY + EROFS + 1–2 GB + +p3 FDS_INTERNAL + ext4 +``` + +## FDS_BOOT + +Contains: + +```text +Pi firmware +kernel +DTBs +config.txt +cmdline.txt +FDS initramfs +``` + +## FDS_RECOVERY + +Contains a complete, independent recovery environment. + +## FDS_INTERNAL + +Record: + +```text +hardware configuration +Bay topology +boot performance history +cached cartridge metadata +diagnostics +``` + +Do not store ordinary user files here. + +--- + +# 13. SYSTEM cartridge + +SYSTEM cartridge: + +```text +GPT ++ +partition name FDS_SYSTEM ++ +EROFS +``` + +For example: + +```text +/dev/??? +└── GPT PARTNAME=FDS_SYSTEM +``` + +Never depend on: + +```text +/dev/sda +/dev/sdb +``` + +With 12 USB devices, these names are completely unreliable identifiers. + +root: + +```text +read-only +``` + +--- + +# 14. fds-stage0 + +`fds-stage0` is a static-musl Rust executable. + +Target: + +```text +single executable initramfs control program +``` + +Avoid including the following wherever practical: + +```text +shell +BusyBox +glibc +Python +udev +``` + +Its responsibilities are limited to: + +```text +mount /proc +mount /sys +mount devtmpfs + +enumerate block devices + +find GPT PARTNAME FDS_SYSTEM + +mount EROFS root + +move /proc /sys /dev + +switch_root + +exec /sbin/init +``` + +Prefer direct use of: + +```text +mount(2) +openat +read +ioctl +netlink/sysfs +pivot_root/switch_root equivalent +``` + +Do not invoke a shell or external commands. + +--- + +# 15. Missing SYSTEM cartridge + +For the selected Dasung monitor, early userspace must provide the display's +control/keepalive and correct video mode before showing this prompt. The base +SYSTEM daemon starts later and cannot satisfy missing-SYSTEM display requirements +by itself. Implement and verify this handoff as part of Pi boot integration. + +When no SYSTEM cartridge is present: + +Display: + +```text +FELIS DATA SYSTEMS +PORTABLE COMPUTER FP-85 + +BOOT ROM 0.1 + +SYSTEM MEDIA NOT PRESENT + +INSERT SYSTEM CARTRIDGE +``` + +Then: + +```text +continue listening for block device changes +``` + +When a SYSTEM cartridge is inserted: + +**Resume boot immediately.** + +A reboot must not be required. + +--- + +# 16. Multiple SYSTEM cartridges + +When multiple partitions are found with: + +```text +FDS_SYSTEM +``` + +Do not choose one arbitrarily. + +Display: + +```text +MULTIPLE SYSTEM CARTRIDGES + +BAY 02 FDS/OS 0.1 +BAY 08 FDS/OS DEVELOPMENT + +SELECT SYSTEM: +``` + +Version 1 may enter a recovery/debug console. + +Add interactive selection in version 2. + +--- + +# 17. Kernel strategy + +Start the first version from Void's Raspberry Pi 5 kernel packaging. + +Do not maintain a large kernel fork from day one. + +Create: + +```text +fds-kernel +``` + +Maintain only: + +```text +config delta +packaging +version policy +``` + +Build boot-critical drivers into the kernel wherever practical: + +```text +BCM2712 +RP1 +PCIe +USB host +USB hub +USB mass storage +SCSI disk +NVMe +devtmpfs +EROFS +ext4 +FAT +HID keyboard +console +DRM/KMS +``` + +optional: + +```text +Wi-Fi +Bluetooth +USB Ethernet +USB Serial +audio +SDR +``` + +These optional drivers may be modules. + +--- + +# 18. Initramfs + +Target: + +```text +As close as practical to: + +/init -> fds-stage0 +``` + +Test these formats: + +```text +uncompressed cpio +lz4 +zstd +gzip +``` + +Benchmark on a real Pi 5. + +The following observation: + +```text +zstd image smaller +``` + +does not establish that boot will be faster. + +NVMe capacity is inexpensive. + +Boot milliseconds matter. + +--- + +# 19. Raspberry Pi firmware optimisation + +production EEPROM: + +Disable unnecessary network installation behavior. + +In particular: + +```text +NET_INSTALL_ENABLED=0 +NET_INSTALL_AT_POWER_ON=0 +``` + +Raspberry Pi's official guidance states that USB controller initialization and device enumeration to detect a network-install keyboard add about one second to boot, making these features worth disabling on embedded devices that need fast startup. + +Provide: + +```text +tools/configure-pi-eeprom +``` + +Together with: + +```text +development.conf +production.conf +``` + +All EEPROM changes must be: + +```text +documented +reversible +``` + +--- + +# 20. Filesystem model + +SYSTEM: + +```text +EROFS +read-only +``` + +runtime: + +```text +/run tmpfs +/tmp tmpfs +/var/tmp tmpfs +/var/log tmpfs +``` + +Do not run root filesystem fsck during boot. + +Do not write logs to SYSTEM. + +Do not update SYSTEM in place. + +--- + +# 21. User data + +DATA cartridge: + +```text +GPT PARTNAME=FDS_DATA +filesystem=ext4 +``` + +After insertion: + +```text +/data +``` + +Alternatively: + +```text +/home/fds +``` + +Mount this path from the DATA cartridge. + +When no DATA cartridge is present: + +```text +ephemeral tmpfs home +``` + +Therefore: + +> The machine itself does not own user data. + +--- + +# 22. Native s6 service graph + +Minimum boot bundle: + +```text +boot +├── runtime-fs +├── hostname +├── console +├── eudevd +├── udev-trigger +├── cartridged +├── getty +└── dasungd +``` + +Explicitly excluded from the boot bundle: + +```text +network +DHCP +NTP +sshd +D-Bus +Xorg +WindowMaker +Bluetooth +Wi-Fi +``` + +--- + +# 23. Fast boot principle + +The interactive shell should depend only on: + +```text +root mounted +console ready +/dev minimally usable +``` + +It must not depend on: + +```text +all USB devices enumerated +cartridge catalog scan +network +clock synchronization +GUI +``` + +The intended sequence is: + +```text +power on + ↓ +kernel + ↓ +root + ↓ +s6 + ↓ +FDS> +``` + +Everything else proceeds in parallel afterward. + +--- + +# 24. eudev + +The normal system uses: + +```text +eudev +``` + +However, do not use: + +```text +udevadm settle +``` + +as a global boot barrier. + +Correct sequence: + +```text +start eudevd + ↓ +trigger existing devices + ↓ +continue boot +``` + +Services that need a particular device must handle their own readiness. + +--- + +# 25. cartridged + +Implement: + +```text +fds-cartridged +``` + +static-musl Rust. + +Responsibilities: + +```text +listen for USB/block events +identify physical hub/port +map device → BAY 01..12 +inspect storage metadata +mount/unmount cartridges +activate FDS profiles +communicate state to fds CLI +``` + +Do not depend on: + +```text +/dev/sdX ordering +``` + +Use: + +```text +sysfs USB topology +``` + +to identify physical bays. + +--- + +# 26. Bay topology + +Configuration: + +```text +/etc/fds/bays.toml +``` + +Example: + +```toml +[front] +hub = "usb-topology-id" + +[front.ports] +1 = 1 +2 = 2 +3 = 3 +4 = 4 +5 = 5 +6 = 6 + +[rear] +hub = "usb-topology-id" + +[rear.ports] +1 = 7 +2 = 8 +3 = 9 +4 = 10 +5 = 11 +6 = 12 +``` + +Users see: + +```text +BAY 01 +... +BAY 12 +``` + +They must not see: + +```text +1-2.4.1 +``` + +except in debug mode. + +--- + +# 27. Cartridge metadata + +At the root of a storage cartridge: + +```text +/FDS/CARTRIDGE.TOML +``` + +Format: + +```toml +format = 1 + +[cartridge] +id = "fds.windowmaker" +name = "WINDOW SYSTEM" +class = "environment" +version = "0.1" + +[media] +writable = false + +[activation] +profile = "windowmaker" +``` + +classes: + +```text +system +data +program +environment +hardware +utility +``` + +--- + +# 28. Hardware cartridges + +USB hardware without a filesystem includes: + +For example: + +```text +USB Ethernet +USB Serial +USB Wi-Fi +USB Audio +``` + +Identify it using: + +```text +VID +PID +serial +USB class +physical bay +``` + +Use these identifiers for recognition. + +Maintain: + +```text +/etc/fds/hardware-catalog.toml +``` + +Do not require every hardware cartridge to contain storage. + +--- + +# 29. ENVIRONMENT cartridge v1 + +Do not place the entire Xorg and WindowMaker runtime on cartridges in version 1. + +Implement the activation logic first: + +The SYSTEM image already contains: + +```text +Xorg +WindowMaker +fonts +theme +``` + +The ENVIRONMENT cartridge initially provides only: + +```text +capability key ++ +configuration +``` + +On insertion: + +```text +WINDOW SYSTEM + ↓ +cartridged + ↓ +fds-profile activate windowmaker + ↓ +s6-rc change +desktop +``` + +On removal: + +```text +desktop down + ↓ +return CLI +``` + +Move the GUI runtime onto the cartridge in version 2. + +--- + +# 30. WindowMaker stack + +Display managers are prohibited. + +Do not use: + +```text +GDM +SDDM +LightDM +``` + +Use: + +```text +s6 + ↓ +Xorg + ↓ +WindowMaker +``` + +A single-user machine may use: + +```text +autologin FDS console +``` + +Activating a GUI profile starts the user's graphical session directly. + +--- + +# 31. E-Ink profile + +Use the grayscale profile as the default WindowMaker appearance on every +display. Install it globally and seed new user preferences without replacing +existing customization. The native FDS Control panel follows the same style. + +The monitor controller is base hardware support, independent of the desktop +profile. Include the existing Rust `dasungd` for the user's Paperlike 13K grayscale +in every SYSTEM image and the native s6 boot bundle. Maintain the confirmed +profile, static-musl target, reconnect handling, and SPI-driver protection. +See [Dasung integration](Developer-Dasung) for implementation, usage, and the remaining +Pi and physical recovery checks. It must not require an ENVIRONMENT cartridge. + +Create: + +```text +fds-eink +``` + +Configuration: + +```text +no compositor +no transparency +no fade +no desktop animation +no smooth window animation +outline move +outline resize +static wallpaper +high contrast +``` + +WindowMaker should prioritize: + +```text +black +white +2–4 grayscale levels +``` + +Fonts: + +```text +Terminus +other high-contrast bitmap-friendly fonts +``` + +Do not update the clock every second. + +--- + +# 32. PROGRAM cartridge + +Create software cartridges on a generic Linux workstation using `fds-cartridge`. +Each format-2 software recipe identifies a Void source package and its commands. +The tool builds with `xbps-src`, installs the package and runtime dependencies +into a private tree, and creates GPT images with metadata partition 1 plus one +or more EROFS program partitions. + +Payloads contain `programs/SOFTWARE-ID/` installed roots, including executable +files and libraries. Verify tree integrity and architecture on insertion; run +new software directly from the read-only EROFS mount without guest extraction. +Retain reading of legacy PROGRAM media and catalogue-format-1 archives. + +Publish commands through `/run/fds/bin` using a managed foreground launcher. +Keep `fds run BAY -- SOFTWARE-ID:COMMAND` for managed background execution. +Both paths run as the ordinary user and participate in safe eject and unplug +cleanup. Resolve names by lowest bay, then lexical selector; expose qualified +`bBAY:SOFTWARE-ID:COMMAND` aliases without overriding base system commands. + +Provide `fds-control`, a native static Rust X11 panel styled to match the default +grayscale WindowMaker desktop. It displays bays and software, opens commands in +a terminal, requests rescans and safe eject, and reports service errors. + +--- + +# 33. FDS CLI + +`fds` is a static-musl Rust binary. + +Implement at least: + +```text +fds info + +fds bays + +fds bay 3 + +fds cartridge 3 + +fds eject 3 + +fds profiles + +fds profile activate windowmaker + +fds poweroff + +fds reboot + +fds boot-profile +``` + +Output should be concise, fast, and suitable for E-Ink. + +For example: + +```text +FDS> fds bays + +01 SYSTEM FDS/OS 0.1 ONLINE +02 DATA FELIS ONLINE +03 ENVIRONMENT WINDOW SYSTEM ONLINE +04 PROGRAM TEXT TOOLS ONLINE +05 EMPTY +06 EMPTY + +07 ETHERNET EC-1 ONLINE +08 EMPTY +09 EMPTY +10 EMPTY +11 EMPTY +12 EMPTY +``` + +--- + +# 34. fds-burn + +The target media client remains static-musl Rust. New software creation and +writing use native Linux workstation `fds-cartridge`: build Void source packages, construct +and verify the complete metadata-first GPT image, preview the USB target, then +write and verify that full image. Never assemble software partitions piecemeal +on the destination drive. See [the workstation workflow](Workstation). + +Responsible for: + +```text +inspect target +partition media +write image +verify +write cartridge metadata +``` + +Typical usage: + +```text +fds burn system fds-system-cli.img BAY04 +``` + +Alternatively: + +```text +fds burn data BAY04 --label FELIS +``` + +Destructive operations require explicit confirmation. + +--- + +# 35. Package installation model + +A running SYSTEM cartridge must not use: + +```text +sudo xbps-install -Su +``` + +to modify itself in place. + +Use XBPS for: + +```text +building images +resolving package dependencies +resolving libraries for installed PROGRAM package trees +development profile +``` + +SYSTEM update sequence: + +```text +build new SYSTEM image + ↓ +write new cartridge + ↓ +swap cartridge +``` + +rollback: + +```text +put old cartridge back +``` + +--- + +# 36. Image builder + +Implement: + +```text +make rootfs PROFILE=cli +``` + +Internal workflow: + +```text +create empty root + ↓ +xbps-install -r ROOT ... + ↓ +install FDS packages + ↓ +apply profile + ↓ +generate locales + ↓ +generate caches + ↓ +compile s6-rc database + ↓ +remove build junk + ↓ +make EROFS +``` + +All of the following must happen at build time: + +```text +locale generation +font cache generation +s6-rc compilation +ldconfig +package configuration +``` + +Do not perform them during boot. + +--- + +# 37. Build interface + +Eventually support: + +```text +make bootstrap + +make packages + +make rootfs PROFILE=cli + +make rootfs PROFILE=development + +make system-card PROFILE=cli + +make system-card PROFILE=development + +make initramfs + +make kernel + +make boot-volume + +make recovery + +make all +``` + +output: + +```text +out/ +├── fds-boot.img +├── fds-initramfs.img +├── fds-recovery.img +├── fds-system-cli.img +├── fds-system-development.img +└── packages/ +``` + +--- + +# 38. Cross compilation policy + +All FDS Rust binaries must support building on x86_64 with: + +```bash +cargo build \ + --release \ + --target aarch64-unknown-linux-musl +``` + +All FDS XBPS packages should support the following wherever practical: + +```bash +./xbps-src -a aarch64 pkg PACKAGE +``` + +For cross-built packages, xbps-src selects the corresponding cross environment and exposes variables such as `XBPS_CROSS_BASE` and `XBPS_RUST_TARGET`. + +If a package cannot be cross-built: + +Fallback priority: + +```text +1. Use an official Void aarch64 binary +2. Fix the cross build +3. QEMU native-aarch64 build +4. Build natively on the Pi only as a last resort +``` + +The normal build pipeline must not require a Pi. + +--- + +# 39. Host-side QEMU testing + +Set up this workflow on the Framework: + +```text +x86_64 Arch + ↓ +cross build + ↓ +aarch64 image + ↓ +QEMU smoke test + ↓ +real Pi test +``` + +Use QEMU to test: + +```text +s6 graph +shell +package contents +static-musl binaries +config parsing +filesystem layout +shutdown logic +cartridge metadata +``` + +Use a real Pi to test: + +```text +firmware +RP1 +USB topology +12 bays +E-Ink +power behaviour +actual boot timing +``` + +--- + +# 40. Boot performance targets + +Define: + +```text +T0 power applied +T1 kernel entered +T2 fds-stage0 entered +T3 switch_root +T4 s6 running +T5 CLI accepts input +T6 WindowMaker accepts input +``` + +Target: + +```text +kernel → CLI: +< 1.5 seconds target +< 2.0 seconds maximum target + +s6 → CLI: +< 400 ms target + +cold power → CLI: +< 3 seconds stretch goal +< 5 seconds hard goal + +CLI → WindowMaker: +< 2 seconds +``` + +These are targets, not claimed performance results. + +Actual measurements are required. + +--- + +# 41. fds-boottrace + +Implement: + +```text +fds-boottrace +``` + +Record: + +```text +kernel timestamp +stage0 start +SYSTEM found +root mounted +switch_root +s6 start +console ready +desktop ready +``` + +Output: + +```text +FDS BOOT PROFILE + +KERNEL .............. 0.713 s +STAGE0 .............. 0.094 s +SYSTEM DISCOVERY .... 0.082 s +ROOT SWITCH ......... 0.031 s +S6 .................. 0.047 s +CONSOLE ............. 0.073 s + +KERNEL → READY ...... 1.040 s +``` + +Save benchmarks in: + +```text +fds-os.wiki/Developer-Performance.md +``` + +--- + +# 42. Performance regression rule + +Any commit that causes: + +```text +boot regression > 100ms +``` + +must: + +```text +explain it +or fix it +``` + +Do not add an artificial delay for a retro splash screen: + +```text +sleep 1 +``` + +If FDS can boot in 800 ms, let it boot in 800 ms. + +--- + +# 43. Fast boot blacklist + +Prohibited on the critical path: + +```text +DHCP +DNS +NTP +sshd +Wi-Fi setup +Bluetooth setup +package updates +package verification +font cache generation +locale generation +s6-rc compilation +global udev settle +GUI login manager +Python +filesystem fsck for read-only root +persistent log flush +``` + +These activities must run: + +```text +build time +on demand +or parallel after READY +``` + +--- + +# 44. Networking + +network bundle: + +```text +network +``` + +By default, it is not part of: + +```text +boot +``` + +When an Ethernet cartridge is present or the user explicitly requests networking: + +```text +s6-rc change +network +``` + +DHCP must never delay the appearance of: + +```text +FDS> +``` + +The prompt must remain independent of DHCP. + +--- + +# 45. Logging + +Normal logging location: + +```text +/run/log +``` + +tmpfs. + +Persist only: + +```text +important errors +explicit debug captures +boot performance +``` + +Do not persist tens of megabytes of journal data as a server distribution might. + +FDS does not use a journal service. + +This is intentional. + +--- + +# 46. Shutdown + +Provide: + +```text +fds poweroff +``` + +Sequence: + +```text +freeze new cartridge operations + ↓ +stop GUI + ↓ +stop program services + ↓ +stop network + ↓ +sync DATA filesystems + ↓ +unmount DATA + ↓ +stop remaining services + ↓ +poweroff +``` + +SYSTEM root: + +```text +read-only +``` + +It therefore does not need to wait for journal writes to SYSTEM. + +--- + +# 47. Shutdown performance + +Target: + +```text +idle poweroff: +< 1 second target + +normal writable DATA: +< 1.5 seconds target +``` + +Do not use: + +```text +sleep 5 +``` + +to wait for daemons. + +s6 must base shutdown on actual process exit and readiness. + +--- + +# 48. Eject semantics + +Users cannot safely unplug writable DATA without ejecting it first. + +Command: + +```text +fds eject 02 +``` + +Behavior: + +```text +identify dependent programs + ↓ +stop them + ↓ +syncfs() + ↓ +unmount + ↓ +mark SAFE +``` + +Output: + +```text +BAY 02 + +DATA CARTRIDGE FELIS + +SAFE TO REMOVE +``` + +A physical LED or eject mechanism may be integrated later. + +--- + +# 49. Recovery environment + +FDS_RECOVERY must not depend on a SYSTEM cartridge. + +Include at least: + +```text +bash +GNU coreutils +util-linux +filesystem tools +XBPS tools + +fds +fds-burn +fds-inspect +``` + +FDS tools in recovery remain static-musl binaries. + +Recovery supports: + +```text +inspect SYSTEM +burn SYSTEM +repair DATA +diagnose USB bays +inspect boot log +``` + +--- + +# 50. Security + +Strictly prohibited: + +```text +/FDS/autorun.sh +``` + +executed automatically as root when a cartridge is inserted. + +Cartridge manifests must be declarative. + +For example: + +```text +profile = "windowmaker" +``` + +Instead of: + +```text +run = "/media/card/haha.sh" +``` + +PROGRAM cartridges run with ordinary user privileges by default. + +--- + +# 51. Initial boot UI + +Production boot should suppress Linux kernel console noise. + +Display: + +```text +FELIS DATA SYSTEMS +PORTABLE COMPUTER FP-85 + +FDS BOOT ROM 0.1 + +MEMORY ........ READY +SYSTEM ........ FDS/OS 0.1 +CARTRIDGE BUS . READY + +READY. +_ +``` + +However: + +**Do not delay startup to display these lines one at a time.** + +A single repaint is sufficient. + +A retro appearance does not require a four-second delay. + +--- + +# 52. Development SYSTEM cartridge + +The development profile may include a large toolset: + +```text +gcc +clang +binutils +make +cmake +meson +ninja +pkg-config + +rust +cargo + +git +gdb +strace +perf + +vim/neovim +``` + +Provided that these tools: + +```text +do not start daemons +``` + +they should not significantly slow boot. + +Disk usage is not boot latency. + +--- + +# 53. Milestone M0 — Host build environment + +Implement: + +```text +repo skeleton +pin void-packages +install/use XBPS host tools +xbps-src bootstrap +aarch64 cross toolchain +Rust musl target +``` + +Acceptance: + +On the x86_64 Framework, it must be possible to: + +```text +build aarch64 package +build static-musl hello-world +``` + +Verify that the executable is static. + +--- + +# 54. M1 — FDS base rootfs + +Implement: + +```text +fds-base +fds-base-files +glibc system +GNU userland +XBPS +``` + +Acceptance: + +Generate: + +```text +out/rootfs-aarch64.tar +``` + +Verify: + +```text +glibc present +GNU coreutils present +GNU binutils present + +musl runtime package not required +BusyBox absent +systemd absent +runit absent +``` + +--- + +# 55. M2 — Native s6 + +Implement the complete stack: + +```text +s6-linux-init +s6-svscan +s6-rc +``` + +Acceptance: + +In the booted rootfs: + +```text +PID1 belongs to s6 architecture +``` + +Also verify that: + +```text +s6-rc change +``` + +can control a test service. + +--- + +# 56. M3 — static-musl FDS tooling + +Implement: + +```text +fds-common +fds CLI skeleton +fds-stage0 skeleton +``` + +All components use: + +```text +Rust +aarch64-unknown-linux-musl +static +``` + +Acceptance: + +```text +readelf +file +ldd +``` + +must demonstrate the absence of a glibc runtime dependency. + +--- + +# 57. M4 — Pi boot + +Implement: + +```text +FDS_BOOT +kernel +initramfs +fds-stage0 +SYSTEM EROFS +switch_root +s6 +``` + +Acceptance: + +Pi: + +```text +power on + ↓ +internal NVMe kernel + ↓ +USB SYSTEM cartridge + ↓ +FDS> +``` + +--- + +# 58. M5 — Fast CLI + +Optimize: + +```text +firmware +kernel +initramfs +stage0 +s6 graph +getty +shell startup +``` + +Acceptance: + +```text +kernel → FDS> <2 sec +``` + +Create the first measured version of: + +```text +fds-os.wiki/Developer-Performance.md +``` + +--- + +# 59. M6 — Cartridge daemon + +Implement: + +```text +udev/netlink events +hub discovery +12 bay topology +metadata +mounting +eject +``` + +Acceptance: + +With 12 USB devices inserted simultaneously: + +```text +every device maps to the correct physical BAY +``` + +--- + +# 60. M7 — DATA cartridge + +Implement: + +```text +mount +home/data integration +safe eject +``` + +Acceptance: + +After a sustained write test: + +```text +fds eject +``` + +must leave the filesystem clean. + +--- + +# 61. M8 — WindowMaker + +Implement: + +```text +Xorg +WindowMaker +E-Ink theme +environment profile +``` + +Acceptance: + +On insertion of an ENVIRONMENT cartridge: + +```text +CLI → WindowMaker +``` + +On cartridge removal or profile deactivation: + +```text +WindowMaker → CLI +``` + +No reboot is required. + +--- + +# 62. M9 — Cartridge image tools + +Implement: + +```text +fds burn +fds inspect +fds format +``` + +Acceptance: + +The machine itself can create: + +```text +SYSTEM +DATA +PROGRAM +ENVIRONMENT +``` + +cartridge. + +--- + +# 63. M10 — Shutdown optimisation + +Complete: + +```text +service stop graph +DATA sync +DATA unmount +fast halt +``` + +Acceptance: + +```text +idle shutdown <1s target +normal shutdown <1.5s target +``` + +Filesystem corruption is unacceptable. + +--- + +# 64. M11 — 12-device stress testing + +Hardware: + +```text +12 USB devices +``` + +Test: + +```text +cold boot +hot insert +hot remove +simultaneous enumeration +simultaneous IO +reboot +shutdown +``` + +Check: + +```text +bay mapping stability +USB resets +kernel errors +boot regression +power issues +``` + +--- + +# 65. M12 — Production hardening + +Complete: + +```text +release signing +image versioning +package pinning +reproducible build work +recovery +documentation +production EEPROM config +``` + +Release: + +```text +FDS/OS 0.1 +``` + +--- + +# 66. Codex implementation rules + +Codex must follow these rules: + +1. Implement only the current milestone at a time. +2. Do not implement later features ahead of schedule. +3. Keep the repository buildable at every milestone. +4. Default every custom FDS binary to static musl. +5. Keep general userspace on glibc. +6. Do not introduce systemd. +7. Do not introduce runit. +8. Do not use BusyBox to implement early userspace. +9. Do not use `sleep` to hide races. +10. Do not let networking block the console. +11. Do not generate caches during boot when they can be generated at build time. +12. Do not depend on `/dev/sdX` ordering. +13. Do not automatically execute arbitrary root scripts from cartridge contents. +14. Explain the necessity of each new dependency. +15. Support every boot optimization with benchmarks. +16. Support x86_64 to aarch64 cross builds for all FDS packages wherever practical. +17. Use native Pi builds only as a documented fallback. +18. Keep documentation synchronized with the implementation. + +--- + +# 67. First Codex task + +Implement only M0 now. + +Specific requirements: + +```text +A. Create the repository skeleton. + +B. Add vendor/void-packages, + pinned to a specific commit. + +C. Write tools/bootstrap-host, + supporting Arch Linux x86_64. + +D. Verify the static XBPS host tools. + +E. bootstrap xbps-src build environment. + +F. Verify: + + ./xbps-src -a aarch64 pkg + +G. Initialize the Rust workspace. + +H. Configure: + + aarch64-unknown-linux-musl + +I. Build: + + rust/fds-smoketest + +J. Verify: + + - architecture = AArch64 + - static executable + - no glibc dependency + +K. Provide: + + make bootstrap + make smoke-test + +L. Create fds-os.wiki/Developer-Build-Host.md, + accurately documenting all dependencies and commands. + +M. Add automated checks that exit nonzero on failure. + +N. Do not begin s6, kernel, GUI, or cartridge daemon implementation. + +On completion, report: + +1. changed files +2. exact build commands +3. smoke-test output +4. known limitations +5. next milestone proposal +``` + +The original M0 gate above has been satisfied. The user explicitly authorized +M1; its workflow and evidence are in [M1 rootfs](Developer-Rootfs) and +[M1 validation](Developer-M1-Validation). The user subsequently authorized M2; see [Native init](Developer-Init). +The user subsequently authorized implementation through M12, with physical Pi +testing deferred. The implementation ledger above records the current evidence +and remaining work. diff --git a/Developer-Media-Tools.md b/Developer-Media-Tools.md new file mode 100644 index 0000000..f27b8a6 --- /dev/null +++ b/Developer-Media-Tools.md @@ -0,0 +1,162 @@ +--- +include_toc: true +--- + +# Cartridge image tools + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Cartridge usage](Developer-Cartridges) + +For new software cartridges, use the [Linux workstation workflow](Developer-Workstation): +`fds-cartridge` builds xz bundles and a complete metadata-first `1+m` GPT image, +then previews and verifies a whole-drive write. Software image creation is not +an on-Pi operation. The commands below cover existing target media management +and legacy compatibility. + +Historical M9 software acceptance verified image creation for all four classes, +confirmed virtual USB writes, and boot of a newly written SYSTEM. Physical media +and Pi testing remain deferred; see [validation evidence](Developer-M9-Validation). +Images built before M9 do not contain these commands. Build a new image to use +them: + +```sh +make rootfs PROFILE=cli +make rootfs-test +make system-card PROFILE=cli +make media-image-test +make media-test +``` + +Run these build commands on the x86_64 host from the repository root, after the +[first-build prerequisites](Developer-Getting-Started). The integration checks use +disposable virtual disks. The `fds` examples below run at the ARM FDS console; +the static ARM executables do not run directly on an x86_64 shell. For an +interactive console, use `make console-vm`; its initial bay map is deliberately +unconfigured, so use the tests for the complete virtual write workflow. + +## Images and cartridge writes + +An image is a regular file containing a complete GPT and its filesystems. Legacy +classes use one partition; current software media use metadata plus payload +partitions. Creating +one does not touch a disk. Writing one selects a physical bay and requires a +confirmation tied to that insertion, image hash and operation identifier. + +Configure the physical bay map first using the [cartridge guide](Developer-Cartridges). +Unconfigured or ambiguous bays are refused. At the FDS console, inspect the target and eject any existing filesystem first: + +```sh +fds inspect BAY04 +fds eject BAY04 +fds burn system /data/fds-system-cli.img BAY04 +``` + +Do not eject a blank disk: it has no mounted filesystem. The running SYSTEM, +internal BOOT/RECOVERY/INTERNAL partitions, mounts, swap and device-mapper holders +are protected. The tools support 512-byte logical sectors. A larger target keeps +the image's filesystem size; GPT backup metadata moves to the actual disk end. +Unused space is not a secure erase of old data. + +All cartridge eject requests are refused while a media operation is active. This +keeps a source DATA filesystem attached while the worker holds its image file. +Finish or cancel the operation first, then eject the source normally. + +The preview shows the bay, model, serial when available, kernel insertion number, +capacity, image class and SHA-256. Type the complete displayed phrase to proceed. +There is no generic `--yes` switch. A wrong phrase, changed image or different +insertion is rejected. Completion requires a flush, cache invalidation, readback, +and valid final partition tables. A failed or interrupted write never reports SAFE. + +For scripts, `fds --json burn CLASS IMAGE BAY04` stops at the prepared preview. +Use its exact `id` and `confirmation` fields with: + +```sh +fds burn confirm OPERATION_ID 'EXACT PHRASE FROM PREVIEW' +fds burn status OPERATION_ID +fds burn wait OPERATION_ID +fds burn cancel OPERATION_ID +``` + +`wait` waits for a terminal result; confirm or cancel from another console if +the operation is still awaiting confirmation. Cartridge +status remains available while the worker hashes or writes. Confirmation expires +after five minutes. Removing/reinserting media invalidates its insertion identity. +A service restart records an unfinished operation as failed, leaving it unmounted. + +## Existing DATA and ENVIRONMENT creation + +Use a writable working directory with enough free space for both a filesystem +image and its GPT wrapper. `/home/fds` is temporary RAM storage; `/data` is better +for large images, provided the source DATA is not the cartridge being overwritten. + +```sh +cd /home/fds +fds format data BAY04 --label FELIS +fds format environment BAY04 --profile windowmaker --label 'WINDOW SYSTEM' +``` + +Each command first creates an image, then presents the same explicit confirmation. +Labels are the display names shown by FDS; fixed GPT partition names identify +the cartridge class. DATA uses ext4 and normally fills the target's available whole MiB. Use +`--size-mib 512` to choose a smaller filesystem. Its root belongs to the ordinary +FDS user. Create PROGRAM software bundles on the workstation using the linked +workflow; target PROGRAM creation now reports that requirement. +ENVIRONMENT stores only a declarative profile. See [Desktop usage](Developer-Desktop). + +`fds burn data BAY04 --label FELIS` is the same creation workflow. `fds-inspect` +and `fds-eject` are static aliases of the corresponding `fds` commands. + +## Build a regular image from a prepared directory + +Prepare a directory containing a valid `FDS/CARTRIDGE.TOML`, plus the class's +payload. The manifest format is described in [Cartridges](Developer-Cartridges). Then: + +```sh +fds-burn create data data-tree data.img --size-mib 128 +fds-burn create environment environment-tree environment.img +fds-burn create system prepared-root system.img +fds inspect image system.img +``` + +The output must be outside the source directory, and existing output files are +never replaced. SYSTEM requires a complete prepared FDS root, including native +init, cartridge management and Dasung; preserve its intended ownership, file +modes, capabilities and prebuilt caches. On the build host, the normal supported +SYSTEM workflow remains `make rootfs PROFILE=cli` followed by `make system-card`. +`fds format system prepared-root BAY04` combines prepared-root construction with +the confirmed write workflow. It does not install packages or change the running +read-only SYSTEM. + +SYSTEM construction must be able to read every source file, including protected +account files, while preserving root ownership and capabilities. Run that +construction step from a privileged maintenance session with a correctly +prepared tree. The normal FDS console is UID 1000 and does not gain access to +protected source files merely by invoking `fds-burn`. The ordinary user's +supported workflow is to burn an already-built SYSTEM image. The VM acceptance +test separately exercises privileged SYSTEM creation and ordinary-user confirmed +writing, then boots the result. + +## Dependencies and current evidence + +The new static Rust crate uses pinned `sha2` 0.10.9 for streaming image hashes. +Its digest/block-buffer/crypto-common/generic-array/typenum dependencies provide +the hashing primitives; cpufeatures selects CPU implementations and version_check +is a build helper. These eight additions are locked in `Cargo.lock`; no glibc, +OpenSSL, async runtime or network client is added to an FDS executable. +[Upstream SHA-2 API](https://docs.rs/sha2/0.10.9/sha2/). + +Filesystem construction uses the already-installed glibc `erofs-utils` and +`e2fsprogs`; the retained legacy preparation code uses GNU `cp` from `coreutils`. These are explicit +`fds-cli` package dependencies, and `fds-cartridged` depends on `fds-cli` for its +worker. They add no packages to the existing base selection. The static tool creates and verifies GPT itself. A read-only loop +inside a private mount namespace lets the worker validate the filesystem and +manifest before confirmation. The filesystem checker runs as UID 1000 through a +private, already-open read-only device handle; cartridge code is never executed. + +The [M9 validation report](Developer-M9-Validation) records the actual ARM construction, +virtual USB writes, independent GPT/filesystem checks and SYSTEM boot evidence. +Writer unit tests also cover source mutation, readback corruption, cancellation +and overlapping backup relocation on slightly larger targets. Physical flash +behavior and power-loss testing remain deferred. diff --git a/Developer-Packages.md b/Developer-Packages.md new file mode 100644 index 0000000..1ddceb9 --- /dev/null +++ b/Developer-Packages.md @@ -0,0 +1,158 @@ +--- +include_toc: true +--- + +# Building and inspecting packages + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Development](Developer-Development) · [Glossary](Developer-Glossary) + +FDS uses Void's XBPS package format and xbps-src build machinery for ordinary +Linux software. M0 verifies this route by cross-compiling GNU hello 2.12.3_1 for +**aarch64 glibc** on the x86_64 Arch workstation. + +This guide assumes `make bootstrap` and `make smoke-test` have succeeded. All +commands run from the repository root. They inspect a local package; they do not +install it on Arch or create a Pi root filesystem. + +## What each tool does + +| Tool or file | Role | +| --- | --- | +| `vendor/void-packages/srcpkgs/hello/template` | Pinned upstream recipe: sources, checksums, dependencies, build/install steps | +| `xbps-src` | Creates the build environment, cross-compiles, and packages files | +| `.host/xbps/usr/bin/xbps-query` | Reads package metadata and dependencies | +| `.host/xbps/usr/bin/xbps-rindex` | Creates or updates a package repository index | +| `.host/xbps/usr/bin/xbps-install` | Installs packages into a selected root; bootstrap uses it for the build container | +| `*.aarch64.xbps` | Package archive with metadata and installed file contents | +| `aarch64-repodata` | Index allowing XBPS to find packages in an output directory | + +Arch's pacman manages workstation packages. XBPS manages Void/FDS packages. An +XBPS archive cannot be passed to pacman, and an ARM package cannot be executed as +native x86_64 software just because it was built on this workstation. + +## Rebuild the verified package + +```sh +./tools/build-package hello +``` + +The helper runs `xbps-src -a aarch64 pkg hello` and exports the main package to: + +```text +out/packages/hello-2.12.3_1.aarch64.xbps +``` + +It reuses upstream's cache and build state; an already built package may not need +recompiling. The helper's log is `out/logs/xbps-hello.log`. The package version in +this guide comes from the current source pin, not a promise about a future pin. + +Standalone `tools/build-package` does not index exported output. To refresh the +local index for this package: + +```sh +XBPS_ARCH=aarch64 .host/xbps/usr/bin/xbps-rindex \ + -a out/packages/hello-2.12.3_1.aarch64.xbps +``` + +`make smoke-test` already performs that indexing step. Explicit `XBPS_ARCH` +matters: a host tool's default architecture is not the architecture of the +package you want to inspect. + +## Inspect the built package + +Query only the local exported repository, avoiding a remote prebuilt package: + +```sh +XBPS_ARCH=aarch64 .host/xbps/usr/bin/xbps-query \ + -i --repository "$PWD/out/packages" -p pkgver hello +XBPS_ARCH=aarch64 .host/xbps/usr/bin/xbps-query \ + -i --repository "$PWD/out/packages" -p architecture hello +XBPS_ARCH=aarch64 .host/xbps/usr/bin/xbps-query \ + -i --repository "$PWD/out/packages" -x hello +``` + +The first two commands should print `hello-2.12.3_1` and `aarch64`. The third lists +runtime dependencies recorded by the package. `-i` ignores configured repositories, +so the explicit output directory supplies the lookup here. + +List the archive contents and extract just the executable into a new directory: + +```sh +tar -tf out/packages/hello-2.12.3_1.aarch64.xbps +extract_dir=$(mktemp -d "$PWD/out/hello-inspect.XXXXXX") +tar -xf out/packages/hello-2.12.3_1.aarch64.xbps \ + -C "$extract_dir" ./usr/bin/hello +./tools/verify-elf "$extract_dir/usr/bin/hello" aarch64 glibc +readelf -lW "$extract_dir/usr/bin/hello" +readelf -dW "$extract_dir/usr/bin/hello" +printf 'Extracted program: %s/usr/bin/hello\n' "$extract_dir" +``` + +The check should print `PASS: aarch64 glibc ELF`. The program headers name +`/lib/ld-linux-aarch64.so.1`, and dynamic entries include `libc.so.6`. This checks +the program inside the archive, not just its filename or metadata. The scratch +folder remains under `out/` for inspection. + +Running this dynamic ARM program also needs ARM glibc and its loader. M0 verifies +its package and ELF structure, not hello execution under QEMU. The static Rust +program can be run with the simpler [QEMU example](Developer-Getting-Started#7-optionally-execute-the-arm-rust-program). + +## Host, build container, and target architectures + +```text +Host utility: x86_64 static-musl XBPS runs on Arch +Build container: x86_64 glibc Void supplies native build tools +Target package: aarch64 glibc GNU hello runs on ARM Linux +FDS Rust program: aarch64 static-musl executable runs on ARM Linux +``` + +The wrapper selects `XBPS_ARCH=x86_64` for build-container operations and +`-a aarch64` for the target cross build. Package indexing/query examples override +`XBPS_ARCH=aarch64`. Using `aarch64-musl` would select the wrong general userspace +ABI for FDS, even though FDS-owned Rust executables use musl internally. + +## FDS packages and overlays + +| Package directory | Responsibility and status | +| --- | --- | +| `packages/fds-base/` | Implemented: base system package selection and dependencies | +| `packages/fds-base-files/` | Implemented: FDS identity, filesystem layout, and base configuration | +| `packages/fds-init/` | Implemented: native init skeleton, runtime mounts, console and service graph | +| `packages/fds-kernel/` | Implemented: pinned Pi kernel, DTBs, overlays and matching modules | +| `packages/fds-cartridged/` | Cartridge daemon packaging | +| `packages/fds-cli/` | Implemented: verified static-musl FDS CLI | +| `packages/fds-eink/` | E-Ink configuration and related integration | + +All nine FDS base packages, including cartridged, E-Ink configuration and the +bounded DHCP client, are built by `make packages` or `make rootfs PROFILE=cli`. +The separate `packages/fds-dasungd/` overlay is active: +use `make dasung` to prepare its verified +static binary and build the package. It is required by `image/base-packages.list` +and belongs to the base boot bundle; see [Dasung](Developer-Dasung). + +The build deliberately ignores +directories without a `template`. [Development](Developer-Development#how-future-package-overlays-fit) +explains how active overlays are copied without changing upstream files. + +The builder assembles a configured rootfs tar from this explicit set; see [Rootfs](Developer-Rootfs). +M2 adds native init and verifies it with a full [ARM VM boot](Developer-Init). +`make system-card` now packages it as read-only EROFS inside GPT. Runtime updates +replace SYSTEM media; PROGRAM media uses self-contained images instead of +modifying the root. See [Media tools](Developer-Media-Tools) for those implemented +workflows; the small package smoke-test only checks the build foundation. + +## What the records prove + +`out/manifests/artifacts.sha256` records the output hashes from the most recent +successful smoke-test. `void-package-inputs.sha256` records cached XBPS archive +hashes, and `void-build-packages.txt` records packages left installed after the +build. Logs retain additional build dependency information. + +The Void source commit, Rust toolchain, and static XBPS archive are pinned. +These ordinary build records alone do not freeze the rolling Void repository. +M12's separate [frozen-input workflow](Developer-Reproducible-Builds) captures selected +packages and build dependencies and compares independent offline builds. +See its acceptance status before making a reproducibility claim. diff --git a/Developer-Performance.md b/Developer-Performance.md new file mode 100644 index 0000000..a5eaab2 --- /dev/null +++ b/Developer-Performance.md @@ -0,0 +1,184 @@ +--- +include_toc: true +--- + +# Performance targets and measurement + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Boot](Developer-Boot) · [Services](Developer-Services) + +**Physical Pi performance remains unmeasured.** The target budgets below are +requirements. M5 now records actual software boot events and provides full-system +ARM VM measurements; these are kept separate from physical targets. The numerical +example in the master plan remains illustrative, not a benchmark. + +## Target budgets + +| Transition | Target | Current measurement | +| --- | --- | --- | +| Kernel entry → usable CLI | Under 1.5 s desired; under 2.0 s maximum target | Not measured | +| s6 running → usable CLI | Under 400 ms | Not measured | +| Cold power → usable CLI | Under 3 s stretch goal; under 5 s hard goal | Not measured | +| CLI → usable WindowMaker | Under 2 s | Not measured | +| Idle power-off | Under 1 s | Not measured | +| Normal power-off with writable DATA | Under 1.5 s | Not measured | + +“Usable” means accepting input, not merely drawing a banner or starting a process. +A quick screen followed by a blocked console does not meet the console target. +Shutdown must finish required data operations; losing writes is not an optimization. + +## Timestamp boundaries + +| Marker | Event | +| --- | --- | +| T0 | Power applied | +| T1 | Kernel entered | +| T2 | `fds-stage0` entered | +| T3 | Root transition (`switch_root` or equivalent) | +| T4 | s6 running | +| T5 | CLI accepts input | +| T6 | WindowMaker accepts input | + +Kernel-to-console is T5 − T1; s6-to-console is T5 − T4; cold boot is T5 − T0. +For desktop activation after the console is already usable, record a separate +activation-request timestamp and measure through T6. Do not include arbitrary +user delay at the prompt as desktop startup time. + +The implemented `fds-boottrace` records stage0 entry, SYSTEM discovery, root mount, +root switch, s6 stage-2 entry and first-console readiness. Desktop readiness is +recorded by the M8 desktop session after its startup checks and remains absent +when no desktop has started. Events use Linux +`CLOCK_BOOTTIME`; its origin is the kernel boot clock, not external power-on. +The s6 marker is the start of stage 2, after native PID 1 exists. Console readiness +is recorded immediately before Bash presents its first prompt; the VM test then +sends real commands and verifies their results. These instrumentation boundaries +are explicit approximations to T1/T4/T5, not external electrical/display measurements. + +## Collect and compare a report + +On FDS, as the ordinary console user: + +```sh +fds boot-profile +fds --json boot-profile > ~/boot-profile.json +``` + +The report lists measured intervals and missing events. A direct-root M2 VM has +no stage0 events; they remain absent rather than becoming zero-duration samples. +The default home is temporary until DATA integration, so export any report you +want to retain before powering off. Host tests save their reports under `out/`. + +To compare two saved reports on FDS: + +```sh +fds-boottrace compare baseline.json current.json +``` + +The command requires the same platform and clock and returns nonzero for a +kernel-to-console regression greater than 100 ms. A justified regression can be +recorded with `--explain "Reason for the measured regression"`; keep the explanation +and raw reports with the change. This does not make unlike scenarios comparable. + +On the workstation, run the actual ARM tool through the project emulator: + +```sh +./tools/in-void qemu-aarch64 "$PWD/out/fds-boottrace" compare \ + "$PWD/baseline.json" "$PWD/current.json" +``` + +`make console-test` boots the ordinary-user console, verifies temporary home and +read-only SYSTEM, collects a trace, and tests the regression threshold. A second +boot holds eudev behind a FIFO gate: the test uses the prompt successfully before +releasing the gate. There is no sleep or artificial readiness delay in the target. +The gate exists only in a disposable test image. + +## Physical measurement procedure + +Each benchmark record should contain: + +1. Hardware identity: Pi revision, firmware/EEPROM version, NVMe and SYSTEM media, + USB hubs/devices, power supply/battery conditions, and display setup. +2. Software identity: FDS revision, kernel configuration/version, image digest, + package/input records, active profile, and relevant boot configuration. +3. Scenario: cold power-on versus reboot, number and classes of inserted devices, + DATA presence, network state, and whether media or caches were already warm. +4. Instrumentation: where each marker was captured, timestamp resolution, clock + source, and how external power-on timing aligns with kernel timestamps. +5. Repeated raw samples, sample count, a typical value and observed range, with + failed or incomplete runs recorded rather than silently discarded. + +Use a monotonic time source for intervals within the running OS. T0 occurs before +the kernel can timestamp anything, so cold-power timing needs external capture +or another explicitly justified measurement method. Do not subtract unrelated +clock readings and present the result as a precise interval. + +Boot with one SYSTEM, with DATA, and with all 12 bays populated should be separate +scenarios. Missing SYSTEM followed by insertion is also a separate readiness test. +A single favorable run cannot establish behavior under the hardware stress cases. + +## Regression and optimization rules + +The master plan requires any boot regression greater than 100 ms to be explained +or fixed. Compare like-for-like hardware, images, scenarios, and measurement +boundaries; keep the raw measurements with the change report. + +Optimization should remove work from the console's dependency path or make that +work faster. Prepare service databases and caches during image construction. +Start optional networking and desktop services when needed. Handle device readiness +through events and explicit dependencies. Do not add sleeps for a retro effect, +wait for all USB devices before the console, or hide unfinished work behind READY. + +Rust's current `opt-level=z` is a starting configuration, not evidence that it is +the fastest choice for future control programs. Real programs and hardware must +guide later comparisons. + +## Recorded VM comparison + +The final M5 comparison is stored in `out/rust-profiles.x_wgx1kp/`, with nine +serial logs, nine raw reports, executable digests and `comparison.json`. +It uses the real Pi kernel and SYSTEM EROFS, QEMU 11.1.1 TCG, `virt`, `max` CPU, +1 GiB RAM and two virtual CPUs. Only stage0's Rust optimization level changes; +the kernel and SYSTEM image are held constant. Each level has three measured boots. + +| Stage0 optimization | Static executable size | Median kernel-clock → console | Median s6 stage 2 → console | +| --- | ---: | ---: | ---: | +| `z` | 474,128 bytes | 1,788.575 ms | 714.436 ms | +| `2` | 517,688 bytes | 1,692.750 ms | 665.673 ms | +| `3` | 533,224 bytes | 1,714.568 ms | 702.154 ms | + +These are **emulator measurements on a development workstation**, including its +scheduling noise. Three samples do not establish a hardware winner. Keep `z` +as the current default and repeat on the assembled Pi before selecting an +optimization level for physical boot speed. In particular, these numbers do not +establish either the physical 2-second console target or the 400 ms s6 target. + +Reproduce the comparison after building the current SYSTEM and boot inputs: + +```sh +make performance-test +``` + +The helper creates separate build directories and initramfs files for each level; +it does not overwrite the normal tool or initramfs outputs. Read +`out/rust-profiles-latest/comparison.json` for the measured artifact identities. +The uncompressed initramfs remains the default. All four formats boot in the M4 +matrix, but compression speed must still be compared on physical NVMe/Pi hardware. + +M10 records shutdown events and VM observations in [its validation report](Developer-M10-Validation). +M11 records repeated empty/twelve-device VM comparisons, including the +reviewed greater-than-100 ms samples, in [its validation report](Developer-M11-Validation). +No physical firmware, cold-power, desktop or shutdown timing is claimed here. + +M12's [repeated console checkpoint](Developer-M12-Validation#repeated-console-timing-checkpoint) +compares five alternating pairs with fixed kernel/initramfs inputs after the +startup clock floor and release-tool packaging changes. The median increased +29.435310 ms; the initial greater-than-100 ms single-sample difference was +investigated and its raw result retained. + +The subsequent [Clap migration checkpoint](Developer-CLAP-Validation#executable-size-and-startup-measurements) +passed five alternating before/after boot pairs with a −37.276280 ms median +change and repeated the three-sample `z`/`2`/`3` comparison on the updated +stage0. These are image-level VM observations; the default remains `z` and +physical Pi measurements are still deferred. diff --git a/Developer-Power.md b/Developer-Power.md new file mode 100644 index 0000000..8a248aa --- /dev/null +++ b/Developer-Power.md @@ -0,0 +1,139 @@ +--- +include_toc: true +--- + +# Shutdown and reboot + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [DATA usage](Developer-Data) · [Media tools](Developer-Media-Tools) + +M10 software acceptance passed; see the [validation report](Developer-M10-Validation). +Physical timing and durability targets remain unmeasured. Rebuild the rootfs to +obtain these commands; images built before M10 do not contain them. + +## Normal use + +At the ordinary FDS console: + +```sh +fds poweroff +fds reboot +``` + +Use one command for the action you want. `fds-power poweroff` and `fds-power +reboot` are equivalent static aliases. No root password or setuid executable is +needed: the root cartridge daemon authenticates the local FDS user and performs +the controlled operation. + +Preparation freezes new cartridge operations, stops the desktop, stops managed +PROGRAM/DATA jobs and their descendants, then stops networking and all DHCP +children. It flushes DATA through the filesystem descriptor retained since +activation, changes the filesystem to read-only, checks writeback again, and +unmounts it normally. Other cartridges are unmounted next. Only after those +steps succeed does native s6 stop remaining services and request kernel halt or +reboot. Dasung remains part of the base service set until that final service stop. + +The read-only transition prevents new writes while the error-tracking descriptor +is closed for unmount. An open writer blocks that transition. A working +directory or unexpected additional mount can block unmount. No lazy unmount is +used to turn a busy or faulty cartridge into a successful shutdown. + +## If shutdown is blocked + +The machine stays running and the command reports the cause. Inspect it with: + +```sh +fds power status +fds --json power status +``` + +The JSON response includes the current phase, requested action, error, whether +native shutdown has begun, and actual `CLOCK_BOOTTIME` timestamps for preparation +steps. It is a software diagnostic, not a physical power-off measurement. + +Common cases: + +| Reported problem | What to do | +| --- | --- | +| Active media operation | Let an existing write finish, or use `fds burn cancel OPERATION_ID`. A preview awaiting confirmation must be cancelled. New confirmations are refused while shutdown is frozen. Then retry shutdown. | +| Busy DATA filesystem | Close files and programs that were started outside FDS management. Leave `/data` in every shell with `cd /`. Remove any extra mounts you deliberately created. Then retry shutdown. | +| DATA I/O error or interrupted session | Do not interpret this as SAFE. Inspect the read-only data and recover/check the cartridge in a maintenance environment before writable use. Restarting the daemon does not clear this fault. | +| Service transition failure | Inspect the corresponding volatile service logs under `/run/log`, resolve the failure, and retry. | + +If native shutdown has **not** begun, you can abandon preparation: + +```sh +fds power resume +``` + +This enables new cartridge operations again. It does not restart stopped jobs. +Start the desktop or network explicitly when needed, using `fds profile activate +windowmaker` and `fds network on`. DATA faults remain quarantined. + +If native shutdown has already begun, it cannot be cancelled. Its guard remains +alive while preparation is blocked. Resolve the cause, leave busy directories, +then retry `fds poweroff` to complete preparation; the original native action +continues. This also protects direct administrative use of the native `poweroff` +or `reboot` commands. The final native hook never treats an unsuccessful +preparation command as permission to kill processes and power down. + +Flushing real storage may take time. A client timeout is an error, never proof +that a filesystem was unmounted or a request completed. There is no automatic +force-poweroff path after a timeout. + +## DATA recovery records + +Before enabling writable DATA, FDS creates a root-owned session record under +`/run/fds/data-sessions`. Successful sync and unmount clear it. A daemon crash +can lose the original descriptor's writeback error history; a remaining record +therefore quarantines the same kernel insertion even if a later flush appears +successful. A known I/O fault is also recorded there. + +The record survives daemon restarts during this boot. It is tied to the kernel +insertion identity and does not pretend to survive complete power loss or a new +insertion. Filesystem recovery and physical durability still require separate +acceptance. Shutdown preparation itself is recorded under `/run/fds/power`, so +restarting the daemon does not silently unfreeze a blocked shutdown. + +## Build and virtual tests + +After the [boot-image prerequisites](Developer-Boot), run on the build host: + +```sh +make rootfs PROFILE=development +make rootfs-test +make media-image-test +make power-test +make data-test +make desktop-test +``` + +The development profile supplies Xvfb for the graphical shutdown case. The +tests use private ARM virtual machines and disposable image files, with no +physical disks or USB devices passed through. They exercise native halt/reboot, +busy refusal, explicit resume, active-write protection, daemon crashes, and +independent filesystem/payload checks, plus an attached-device writeback EIO +that must survive daemon restarts. M11 extends media-writer fault injection and +twelve-device stress. Physical Pi timing, power rails, battery and +flash-controller behavior remain deferred. + +## Dependencies and native init behavior + +No new package or external Rust crate is introduced. `fds-init` explicitly +depends on `fds-cartridged` because native shutdown requires its DATA checks. The CLI now declares the +already-locked `libc` crate directly for its native shutdown guard's identity, +inotify and signal-wait syscalls. DHCP children use the existing cgroup v2 +controller and native FDS helpers. `fds-power` is another static copy of `fds`, +with behavior selected by its executable name. + +Native s6 still owns shutdown and PID 1. Its configurable final grace period is +set to zero after FDS has verified DATA and the service manager has waited for +managed service exits. Process groups use real exit events and bounded TERM/KILL +deadlines, rather than sleeping before assuming a process has stopped. + +References: [s6 shutdown](https://skarnet.org/software/s6-linux-init/s6-linux-init-shutdown.html), +[native init configuration](https://skarnet.org/software/s6-linux-init/s6-linux-init-maker.html), +[Linux remount behavior](https://man7.org/linux/man-pages/man2/mount.2.html), and +[filesystem error reporting](https://man7.org/linux/man-pages/man2/syncfs.2.html). diff --git a/Developer-Recovery.md b/Developer-Recovery.md new file mode 100644 index 0000000..201213a --- /dev/null +++ b/Developer-Recovery.md @@ -0,0 +1,214 @@ +--- +include_toc: true +--- + +# Recovery and rollback + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Boot images](Developer-Boot) · [Media tools](Developer-Media-Tools) · [EEPROM](Developer-EEPROM) + +Recovery is a separate, read-only FDS image. It contains native s6, Bash, GNU +utilities, XBPS, filesystem tools, the static FDS tools, and the base Dasung +controller. It does not need files or programs from a SYSTEM cartridge. +The software workflow is covered by `make recovery-test`; physical Pi recovery remains deferred. +The [complete internal disk](Developer-Internal-Storage) includes this recovery image; +its guide separates verified image construction from deferred physical +installation. The build commands below create ordinary files and do not write a +host disk. + +## Build the recovery image + +Use the prepared x86_64 build host described in [Your first build](Developer-Getting-Started). +From the repository root, run these commands sequentially: + +```sh +make recovery +make rootfs-test +make recovery-test +``` + +The full VM test also needs the Pi kernel/initramfs and a known-good CLI SYSTEM +image. On a fresh checkout, build those first with `make rootfs PROFILE=cli`, +`make rootfs-test`, `make initramfs`, and `make system-card PROFILE=cli`, then +run the recovery sequence above. Run builds and VM tests sequentially because +they share the same project-local Void environment. + +`make recovery` builds the `recovery` rootfs profile independently, configures all +packages and caches at image construction time, then creates EROFS. Its outputs +are: + +| File | Meaning | +| --- | --- | +| `out/rootfs-recovery.tar` | Complete recovery rootfs, including package database and compiled s6 configuration | +| `out/fds-recovery.img` | Raw EROFS payload for the internal GPT partition named `FDS_RECOVERY` | +| `out/recovery-build.*/manifest.json` | Rootfs hash, filesystem hash, profile and size | +| `out/manifests/rootfs-latest/` | Exact selected package archives and build input records | + +The latest-rootfs pointers also select this recovery build. Profile-specific +rootfs pointers preserve earlier CLI/development builds. A recovery EROFS payload +is not a whole-disk image and is not a SYSTEM cartridge. The SYSTEM builder +rejects a recovery rootfs to prevent confusing those roles. + +## Enter recovery + +Stage0 can enter recovery in two ways: + +1. At its missing-SYSTEM or ambiguous-SYSTEM console, type `recovery` and Enter. +2. For a deliberate maintenance boot, use `fds.boot=recovery` in the boot + partition's single-line `cmdline.txt`, replacing `fds.boot=normal` if present. + Restore normal mode when maintenance is complete. + +Stage0 requires exactly one readable `FDS_RECOVERY` partition and mounts it +read-only. It does not silently choose between duplicate partitions. The ARM VM +suite supplies disposable virtual partitions; physical firmware/NVMe/display +behavior must still be checked on the Pi. + +The local prompt is: + +```text +FDS RECOVERY — LOCAL MAINTENANCE CONSOLE +RECOVERY# +``` + +This is an explicit **root maintenance console**. It reads startup files from the +immutable recovery image and uses a temporary home under `/run/fds`. The root +password remains locked; no SSH or network login service is started. Normal CLI +and development images continue to use the ordinary `fds` account and `FDS>`. + +Recovery inspects cartridges read-only. It does not automatically mount DATA +writable, activate an ENVIRONMENT desktop, or enable Ethernet. Those actions +require explicit commands. The Dasung controller remains in the base boot bundle. + +## Inspect a failed system + +```sh +fds info +fds bays +fds bay 2 +fds inspect BAY02 +fds topology +fds boot-profile +lsusb -t +lsblk -o NAME,MAJ:MIN,MODEL,SERIAL,SIZE,RO,TYPE,FSTYPE,PARTLABEL,MOUNTPOINTS +dmesg +``` + +Use the calibrated bay number and the displayed model/serial to identify a +cartridge. Do not infer a bay from `/dev/sda` or from discovery order. A valid +inactive SYSTEM mounts read-only under `/run/fds/media/NN`; `fds bay N` reports +the actual path. A bad filesystem or manifest produces an error instead of +running anything from the cartridge. + +An empty bay configuration produces `UNCONFIGURED`, not guessed bay numbers. +See [Cartridges](Developer-Cartridges) for calibration. Permanent machine configuration +on `FDS_INTERNAL` is part of the remaining M12 work. + +## Check and repair DATA + +Start with a read-only check. For example, for DATA in bay 2: + +```sh +fds recovery check BAY02 +``` + +This command is available only to root in the recovery image. It identifies one +USB disk and one `FDS_DATA` partition, unmounts its read-only view, reserves the +disk exclusively, verifies GPT and the kernel partition identity, and invokes +`e2fsck -f -n`. It does not repair the filesystem. A clean result leaves DATA +unmounted and reports `SAFE TO REMOVE`. + +If DATA was explicitly activated writable, eject that session first. Close any +shell whose current directory is on DATA and any other reader before checking; +an ordinary busy-unmount failure is reported, never bypassed with lazy unmount. + +If the check reports problems, review its log and preserve a backup where +possible. Preview the repair with: + +```sh +fds recovery repair BAY02 +``` + +The preview displays the model, serial, capacity and a confirmation command. +Copy that exact command only after checking the intended cartridge. Its token +binds the bay, kernel insertion number and current boot. A token from another +insertion or boot is rejected. The preview makes no filesystem changes. + +Confirmed repair runs `e2fsck -f -p`, which performs conservative automatic +repairs and stops when manual judgement is required. It then flushes the device, +invalidates its block cache, and runs a second `e2fsck -f -n`. Only a successful +verification produces `DATA REPAIRED AND VERIFIED` and `SAFE TO REMOVE`. + +The checker uses a temporary kernel loop device backed by the already verified +partition descriptor. This lets `e2fsck` take its own exclusive device claim +while FDS retains the physical whole-disk reservation. The loop is removed +automatically when its last descriptor closes. This uses the existing kernel +loop driver, `libc` crate and base `e2fsprogs`; no new package or Rust dependency +is introduced. + +Failed or interrupted checks/repairs retain a quarantine record across cartridge +daemon restarts for the same insertion. They do not inherit an earlier SAFE +status. The checker holds the disk reservation and is killed if its supervising +daemon dies. The command runs synchronously: bay operations and orderly shutdown +wait for it to finish; the local shell remains usable. A client timeout is not +success and does not authorize removing media. Inspect the log and wait for the +service to finish before retrying. Logs are in `/run/fds/recovery/` and are +limited to 16 MiB per invocation. + +This narrow recovery command requires a readable GPT and primary ext filesystem +signature. It deliberately does not guess partition boundaries, recreate a +broken GPT, or force answers to destructive `e2fsck` questions. Such cases require +an offline backup and expert use of the included filesystem tools. It cannot +recover data that was never written to storage. + +For structured results, prefix the command with `fds --json`. After a successful +check or repair, remove/reinsert DATA to inspect it again. Recovery still keeps +it read-only until `fds data use N` is explicitly requested. + +## Prepare replacement SYSTEM media + +A known-good SYSTEM image must be supplied on separate source media. Do not use +a damaged image as an update source. For example, source DATA in bay 2 and an +unmounted replacement cartridge in bay 4: + +```sh +fds bay 2 +fds inspect image /run/fds/media/02/fds-system-cli.img +fds inspect BAY04 +fds burn system /run/fds/media/02/fds-system-cli.img BAY04 +``` + +Use the actual source mount shown by `fds bay 2`. If the destination already +contains a mounted cartridge, run `fds eject 4` first. The burn preview identifies +the source checksum, destination capacity/model/serial and confirmation command. +Follow [Media creation and writing](Developer-Media-Tools) for confirmation and status commands. +Completion requires device flush, readback and GPT verification. Recovery uses +the same protected writer as the main system: it refuses mounted destinations, +active root storage and disks containing internal FDS partition names. + +Keep the previous known-good SYSTEM cartridge. After a successful replacement +write, use `fds poweroff`, swap SYSTEM while powered off, and boot normally. +Rollback means restoring the previous SYSTEM cartridge; it does not undo DATA +file changes, application migrations, internal boot updates or EEPROM changes. + +## Capture diagnostics and shut down + +```sh +bash /usr/share/fds/capture-hardware /tmp/recovery-capture +cat /tmp/recovery-capture/status.tsv +fds power status +fds poweroff +``` + +The collector records identities, USB topology, mounts, kernel messages, packages +and boot events with checksums. Its output is temporary unless copied to a +healthy DATA cartridge after explicitly activating that DATA with `fds data use N`. +Do not activate the damaged cartridge merely to save a report. Eject writable +DATA after copying, or let `fds poweroff` perform the normal verified shutdown. +See [Shutdown](Developer-Power) when shutdown reports a blocking DATA error. + +Recovery is independent of SYSTEM, but it still depends on working internal +boot storage, the kernel and firmware. An external rescue medium is required +when that layer fails. EEPROM preparation and rollback are described separately +in [EEPROM configuration](Developer-EEPROM); no EEPROM is changed by these recovery tools. diff --git a/Developer-Releases.md b/Developer-Releases.md new file mode 100644 index 0000000..2de8299 --- /dev/null +++ b/Developer-Releases.md @@ -0,0 +1,174 @@ +--- +include_toc: true +--- + +# Release signatures and verification + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [M12 evidence](Developer-M12-Validation) + +`fds-release` signs an artifact manifest and verifies its signature and every +listed file. The static ARM executable is included in the base CLI package, so +CLI, development and recovery images can all verify downloads. The workstation +build supplies an x86_64 signer with the same format. + +The signing tool has passed host, static ARM and independent OpenSSL acceptance. +The complete frozen-input, versioned local 0.1.0 release passed acceptance; +see [M12 validation](Developer-M12-Validation#final-local-release-acceptance) for its identity +and [Offline rebuilds](Developer-Reproducible-Builds) for the workflow. +The examples below describe the working signing interface; they do not identify +a published or hardware-qualified FDS release. + +## Build and test the tools + +After [bootstrap](Developer-Getting-Started), run: + +```sh +make signing +make signing-test +``` + +The workstation executable is +`target/x86_64-unknown-linux-gnu/release/fds-release`. The ARM executable is +`target/aarch64-unknown-linux-musl/release/fds-release`; `make tooling` also +exports it to `out/fds-release`. An ARM file cannot execute directly on the +x86_64 workstation. `make signing-test` runs that file through the project-local +QEMU interpreter as well as testing the host executable. + +OpenSSL is a **host test dependency** used as an independent Ed25519 +implementation. It is not linked into the FDS verifier. The test creates private +disposable keys under its `out/m12-signing.*` directory, checks exact signatures, +then exercises wrong keys, altered manifests, damaged or missing artifacts, +symlinks, FIFOs, malformed fields and unsafe paths. Test keys are not release +keys. + +## Create and retain a signing key + +For a local signing identity, choose an existing private directory outside the +checkout and its build output. This example creates a new directory; choose a +different name if it already exists: + +```sh +mkdir -m 700 "$HOME/fds-signing" +target/x86_64-unknown-linux-gnu/release/fds-release keygen \ + "$HOME/fds-signing/release-01" +``` + +The command creates `release-01.key` (32 secret bytes, mode 0600) and +`release-01.pub` (the hexadecimal public key). Existing files are never +overwritten. It prints SHA-256 of the decoded 32-byte public key, never the +secret. This fingerprint differs from `sha256sum release-01.pub`, which hashes +the hexadecimal text and its newline. Retain the private key separately from release downloads and source +archives. Anyone who has that key can sign a release under that identity. + +If public-key export failed after private-key creation, retain the private file +and export it to a new path: + +```sh +target/x86_64-unknown-linux-gnu/release/fds-release public-key \ + "$HOME/fds-signing/release-01.key" "$HOME/fds-signing/recovered.pub" +``` + +Private keys must be regular files owned by the invoking user with no group or +other permissions. The tool rejects symlinks and invalid key lengths. Key +generation uses Linux `getrandom`; temporary seed storage and the signing key +are zeroized when dropped. + +## Sign an assembled directory + +An assembled release directory must already contain its artifacts and a +`manifest.json` matching the format below. Signing verifies every artifact's +length and hash before creating `manifest.sig`: + +```sh +target/x86_64-unknown-linux-gnu/release/fds-release sign \ + /path/to/release-directory --key "$HOME/fds-signing/release-01.key" +``` + +If a signature already exists, use a new release directory. The tool deliberately +does not replace signatures or silently rewrite the manifest. A failed operation +may leave its new output file for inspection; it never declares it verified. + +## Verify before using images + +Obtain the signer's public key through a channel you already trust, or compare +its fingerprint against a separately authenticated value. A public key included +beside an untrusted download does not establish its identity by itself. + +On FDS, including recovery: + +```sh +fds-release verify /data/downloads/fds-release --key /data/keys/fds-release.pub +``` + +On the workstation, use the host executable instead: + +```sh +target/x86_64-unknown-linux-gnu/release/fds-release verify \ + /path/to/release-directory --key /path/to/trusted-fds-release.pub +``` + +A successful command reports `VERIFIED FDS/OS 0.1.0`, the number of checked +artifacts, and `Hardware validation: deferred`. Failure returns exit status 2 +and an explanation. Keep downloaded images unchanged between verification and +use. Verification covers only files listed in the manifest; unrelated extra +files are not endorsed. The [media writer](Developer-Media-Tools) performs its own +image/layout validation and readback checks when writing a cartridge. + +These are distribution signatures. The Pi firmware and stage0 currently do not +enforce them at boot, so this is not a secure-boot implementation or a claim +that the hardware has been tested. + +## Manifest and signature format + +The JSON document has exactly these fields: + +```json +{ + "format": 1, + "version": "0.1.0", + "source_epoch": 1789909701, + "source_sha256": "<64 lowercase hexadecimal characters>", + "void_commit": "02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1", + "hardware_validation": "deferred", + "files": [ + { + "name": "fds-system-cli-0.1.0.img", + "bytes": 123456, + "sha256": "<64 lowercase hexadecimal characters>" + } + ] +} +``` + +This is a schema illustration, not a usable manifest: the size and hash +placeholders must be replaced with values from the actual artifact. Format 1 +accepts a bounded numeric-leading release/development version, a positive source epoch, a 40-character Void +commit, and 1–64 nonempty artifacts. Filenames must be flat safe ASCII names; +paths, duplicates, hidden names, reserved manifest filenames and unknown fields +are rejected. The manifest is bounded to 1 MiB. Files must be regular files, +opened without following symlinks; hashing streams their bytes and checks for +changes during the read. + +The signature is Ed25519 over the exact bytes +`FDS/OS release manifest v1` followed by a NUL byte and the raw `manifest.json` +bytes. JSON whitespace is therefore authenticated too. `manifest.sig` contains +128 lowercase hexadecimal characters and a newline. The public-key file contains +64 lowercase hexadecimal characters and a newline. Verification rejects weak +public keys and uses strict signature verification. + +The implementation uses locked `ed25519-dalek` 3.0.0 for Ed25519, +`zeroize` 1.9.0 for secret buffers, and the existing SHA-256/JSON crates. +Its locked transitive dependencies provide curve arithmetic, digest and +signature types. Default dalek features are disabled; no asynchronous runtime, +OpenSSL binding or shared target library is added. The dependencies and their +license metadata are recorded in `Cargo.lock` and the cached crate sources. +The build collects their license texts, including build dependencies and +vendored libusb notices, into `/usr/share/licenses/fds-cli/RUST-NOTICES.txt`. +The usage guide is installed at `/usr/share/doc/fds/releases.md` in every profile. +Algorithm interoperability is checked against the +[RFC 8032 test vector](https://www.rfc-editor.org/rfc/rfc8032.html#section-7.1) +and OpenSSL; strict verification follows the +[dalek API](https://docs.rs/ed25519-dalek/3.0.0/ed25519_dalek/struct.VerifyingKey.html). diff --git a/Developer-Reproducible-Builds.md b/Developer-Reproducible-Builds.md new file mode 100644 index 0000000..5d16ea4 --- /dev/null +++ b/Developer-Reproducible-Builds.md @@ -0,0 +1,197 @@ +--- +include_toc: true +--- + +# Frozen inputs and offline rebuilds + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Build host](Developer-Build-Host) · [Release signatures](Developer-Releases) + +M12 adds a local input snapshot, fresh restore, network-isolated build, and +complete artifact comparison. **Two fresh network-isolated builds passed with +all 27 artifacts byte-identical.** The [M12 evidence page](Developer-M12-Validation#final-local-release-acceptance) +records the source/input identities, signed local release and actual archive +round-trip. This result applies to the recorded inputs and documented build +host; it does not claim equivalence across arbitrary host environments. + +Normal builds still use Void's signed rolling repositories. A frozen build uses +the selected package files and build environment from one explicit snapshot. +It does not silently consult a newer repository if an input is missing. + +Current builds keep the upstream reference in `vendor/void-packages/` and use +`.host/void-packages/` for overlays and cached build state. New snapshots and +restores use this separate layout. Restoring an older +snapshot retains its historical layout and saved build scripts; its files and +acceptance are not rewritten. See [the workspace change](Developer-Void-Workspace) for +fresh validation of this migration. + +## What is frozen + +`tools/frozen-inputs create NEW_DIRECTORY` records: + +- Project-owned source, documentation, configuration and package overlays, with + a content identity independent of an FDS Git commit. This works before the + repository's first commit too; it does not invent a commit identifier. + Documentation is copied from the populated `fds-os.wiki` submodule, including + its pages and images but excluding Git metadata. Offline restores therefore + need no connection to Gitea. Older frozen releases retain their original layout. +- The exact pinned Void checkout in a Git bundle, preserving upstream files and + the shallow-history boundary needed to read its commit timestamp after restore. +- Cargo's locked vendored sources, the selected Rust toolchain and its static + ARM target. Cargo credentials and user configuration are not copied. +- The project-local Void binary build environment, static XBPS host tools and + image-tool prefix. Temporary build trees, personal home directories and + mount points are excluded. +- Cached build dependencies, exact third-party packages selected by each of + the three rootfs profiles, and kernel/DHCP/hello source archives. Local FDS + packages are rebuilt from source instead of seeding the output repository. +- Pinned Pi firmware, the VM kernel fixture, and the official EEPROM preview + inputs. The preview is not a backup of a physical Pi. + +`lock.json` records paths, kinds, modes, lengths, hashes and symlink targets. +Added, missing or changed entries fail verification. The snapshot is several +gigabytes. The completed offline diagnostic build occupied about 24 GB and its +snapshot occupied 6.9 GB. Two restored builds need separate space; retained VM +test images add further usage. Check available space before starting and keep +headroom beyond these observations, which are not upper bounds. The workstation still supplies the documented +Arch/Linux prerequisites, including Python, bubblewrap, Git, Make and compression +tools. Their environment is recorded; this is not a replacement for a host OS. + +## Prepare the snapshot + +Complete the [first-build setup](Developer-Getting-Started), then run the complete +artifact build from the repository root: + +```sh +make all +``` + +`make all` runs bootstrap and package construction, builds both normal SYSTEM +profiles and independent recovery, then creates the four initramfs formats, +production Pi boot volume, complete internal disk, signing tools and production +EEPROM preview. It does not write physical media or change EEPROM settings. +Use `make packages` to build only the FDS base packages. Builders perform their +normal format, ABI and rootfs checks; the complete runtime acceptance suite +remains separate. + +Runtime test targets select their required profile explicitly: init, boot, +console, cartridge, DATA, media and stress checks use CLI; desktop and power +checks use development; recovery checks use recovery. They no longer depend on +which rootfs happened to be built last. `make rootfs-test` intentionally checks +the most recently exported archive, so use it immediately after a profile build +when validating that specific export. + +Fetch all locked crates, including platform-specific sources that Cargo's vendor +operation requires even when they are not built for ARM: + +```sh +cargo fetch --locked +./tools/frozen-inputs create "$PWD/out/inputs-0.1.0" +./tools/frozen-inputs verify "$PWD/out/inputs-0.1.0" +``` + +Use a new output directory. The command will not overwrite a previous snapshot. +The snapshot operation itself is offline; missing selected packages or locked +crate sources fail with an error. The subsequent fresh build establishes whether +the complete dependency set was captured. Keep failed outputs for diagnosis and +retry into a new directory. +Run every build, snapshot and VM workflow sequentially: the normal checkout +shares one Void masterdir. Snapshot creation locks rootfs/base-package assembly, +but a separate standalone kernel or VM command must not be started concurrently. + +## Restore into two independent trees + +If starting from the signed release's input archive, extract it as the ordinary +build user with `tar --extract --zstd --same-permissions --no-same-owner` and +the desired `--file` and new `--directory` paths. Mode preservation matters: +default extraction can mask writable bits or remove sticky bits and cause the +input lock to reject an otherwise intact archive. Verify the extracted snapshot +before restoring it. The release's README provides the complete commands. + +The destination parent directory must already exist, and the destination itself +must not exist. Use paths without whitespace, as required by xbps-src: + +```sh +./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-a" +./tools/frozen-inputs restore "$PWD/out/inputs-0.1.0" "$PWD/out/rebuild-b" +``` + +Restoration verifies every locked entry first. Each new tree gets its own Void +masterdir, source/download caches, Cargo directory and Rustup directory. It starts +without an FDS target directory or generated FDS packages/images. + +Run the builds sequentially and retain their full logs: + +```sh +out/rebuild-a/tools/in-frozen-build make all >out/rebuild-a.log 2>&1 +out/rebuild-b/tools/in-frozen-build make all >out/rebuild-b.log 2>&1 +./tools/compare-builds out/rebuild-a out/rebuild-b \ + --report out/reproducibility-0.1.0.json +``` + +The wrapper changes to the restored project directory, uses its private Rust +caches, makes the rest of the workstation read-only, and removes network access +for the build and all child processes. Missing input therefore cannot be hidden +by a download. FDS's xbps-src wrapper keeps dependency resolution enabled and +excludes remote repositories; the custom repository filename includes `local` +so the pinned xbps-src also retains it in its cross sysroot. Rootfs assembly uses the frozen per-profile +repository. Setting `FDS_OFFLINE=1` alone selects offline inputs but does not +provide the wrapper's network isolation. + +The comparison checks both rootfs tars, recovery rootfs, all installable images, +all four initramfs formats, Pi kernel/DTB, every FDS package and the EEPROM +configuration/rollback outputs. It reports each mismatch and exits nonzero if +anything differs. Matching files alone do not attest how they were built; retain +the isolated build logs alongside the report. + +## Assemble a local signed release + +After both complete builds match, create or choose a signing key as described in +[Release signatures](Developer-Releases). Use the assembler in the first restored tree: + +```sh +out/rebuild-a/tools/assemble-release \ + --inputs "$PWD/out/inputs-0.1.0" \ + --comparison-build "$PWD/out/rebuild-b" \ + --output-directory "$PWD/out/fds-os-0.1.0" \ + --key "$HOME/fds-signing/release-01.key" +``` + +The assembler verifies the snapshot again, checks that the first checkout still +matches its source identity, compares the complete pair, and verifies recorded +rootfs build inputs. It creates versioned image names, source and complete input +archives, package inventories, comparison evidence and usage guides. It signs +the manifest and immediately verifies every listed file. The private key is +never copied into the release, and existing output directories are refused. + +The result is a **local** signed release with physical validation marked +`deferred`; it is not published or installed automatically. The supplied key +defines its signing identity. Distribute its fingerprint through an independently +trusted channel before someone relies on the bundled public key. + +## Determinism and troubleshooting + +FDS uses the pinned Void commit's timestamp for package and image timestamps. +Owned Rust binaries are stripped. `make` uses `tools/cargo-build` to map checkout, +Rustup and Cargo registry source paths to stable `/usr/src/fds` paths; otherwise +panic messages can embed the builder's absolute directory even in stripped +binaries. The helper preserves the target's static linking flags, and its flags +participate in Cargo's build cache. Direct `cargo build` remains useful for local +development, but release construction uses the helper. This uses Rust's +[source-path remapping](https://doc.rust-lang.org/rustc/command-line-arguments.html#--remap-path-prefix-remap-source-names-in-output) +and Cargo's [merged configuration arrays](https://doc.rust-lang.org/cargo/reference/config.html). +The kernel has explicit build user/host/time, +and filesystem UUIDs derive from content/configuration identities. Rootfs export +restores target ownership and capabilities instead of preserving the builder's +UID. It removes only the build-specific auxiliary linker cache and fetched XBPS +repository indexes; the installed package database, keys and runtime linker +cache remain available. + +If verification reports changed snapshot entries, restore the original snapshot +or create a new one from the intended inputs. Do not edit `lock.json` to hide a +mismatch. If the two builds differ, use the comparison's named artifacts and +each build's rootfs/package manifests to identify the first differing stage. +Archive compression is not a substitute for comparing the uncompressed images. +Physical performance and durability still require the actual Pi and storage. diff --git a/Developer-Roadmap.md b/Developer-Roadmap.md new file mode 100644 index 0000000..595415c --- /dev/null +++ b/Developer-Roadmap.md @@ -0,0 +1,84 @@ +--- +include_toc: true +--- + +# Roadmap and current status + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Master plan](Developer-Master-Plan) · [M0 evidence](Developer-M0-Validation) + +**M0–M12 software acceptance is complete. Physical Pi testing is deferred.** +The user has additionally authorized [Dasung base integration](Developer-Dasung): an +imported daemon, ARM package, mandatory base selection, and s6 service sources. +M2 starts it under native s6 in a booted ARM virtual machine. +This is a sequence of acceptance milestones, not a release-date schedule. +The user authorized continued implementation through M12, deferring physical Pi +testing. Work proceeds in order with software/VM evidence recorded in the +[implementation ledger](Developer-Implementation-Status). + +Current development additionally implements [workstation software cartridges +and QEMU hotplug tooling](Developer-Workstation). Its [acceptance record](Developer-Workstation-Validation) +is separate from the frozen 0.1.0 artifacts below. + +## Milestones and acceptance boundaries + +| Milestone | Status | Deliverable and acceptance | +| --- | --- | --- | +| **M0 — Build host** | **Verified** | On x86_64 Arch, bootstrap the pinned Void build environment, cross-build an aarch64 glibc package, and build/inspect a static-musl Rust executable. See the recorded skips below. | +| **M1 — Base rootfs** | **Implemented** | `fds-base`, `fds-base-files`, and `out/rootfs-aarch64.tar` with glibc, GNU tools, XBPS and Dasung; no BusyBox, systemd, runit or musl runtime package. [Usage](Developer-Rootfs) · [Evidence](Developer-M1-Validation). | +| **M2 — Native s6** | **Verified in ARM VM** | Native s6 PID 1, read-only root, eudev/getty/Dasung supervision, test-service control, and orderly power-off. [Usage](Developer-Init) · [Evidence](Developer-M2-Validation). | +| M3 — FDS tooling foundations | Verified | Add shared Rust code and CLI/stage0 skeletons; verify static-musl AArch64 artifacts without a glibc runtime dependency. | +| M4 — Pi boot | Software verified; hardware deferred | Kernel/initramfs/SYSTEM/boot-volume builds and nine VM boot cases passed. The ordinary-user prompt follows in M5; physical NVMe/RP1/display acceptance is deferred. [Evidence](Developer-M4-Validation). | +| M5 — Fast console | Software verified; physical timing deferred | Ordinary-user console, boot tracing and regression checks passed. [Evidence](Developer-M5-Validation). Physical performance targets remain unmeasured. | +| M6 — Cartridge daemon | Software verified; physical calibration deferred | Handle events, configured 12-bay topology, metadata, mounting, and eject; virtual hotplug checks passed. Twelve-device stress follows in M11 and physical calibration remains deferred. | +| M7 — DATA integration | Software verified; physical media behavior deferred | Integrate data/home mounts and safe eject; leave the filesystem clean after sustained writes and ejection. | +| M8 — WindowMaker | Software verified; physical display deferred | Xorg/WindowMaker, grayscale theme, ENVIRONMENT activation, managed PROGRAMs and Ethernet passed virtual display/network tests. [Usage](Developer-Desktop) · [Evidence](Developer-M8-Validation). | +| M9 — Media tools | Software verified; physical media behavior deferred | ARM image creation, protected and confirmed writes, readback verification, and boot of a newly written SYSTEM passed. [Usage](Developer-Media-Tools) · [Evidence](Developer-M9-Validation). | +| M10 — Shutdown | Software verified; physical timing/durability deferred | Ordered native halt/reboot, busy refusal, persistent DATA faults and independent filesystem checks passed. [Usage](Developer-Power) · [Evidence](Developer-M10-Validation). | +| M11 — Hardware stress | Software verified; physical tests deferred | Twelve-device cold/hotplug and I/O stress, native reboot/halt, readiness independence and six media failure cases passed. [Usage](Developer-Stress-Testing) · [Evidence](Developer-M11-Validation). | +| M12 — Production hardening | Software verified; hardware deferred | Signed local FDS/OS 0.1.0, exact frozen inputs, two matching offline builds, recovery/internal settings, documentation and reversible EEPROM previews. [Final evidence](Developer-M12-Validation#final-local-release-acceptance). | + +Detailed requirements remain in sections 53–65 of the [master plan](Developer-Master-Plan). +The ledger distinguishes implemented software, remaining work, and tests that +still require physical hardware. + +## What M0 completion means + +The initial validation successfully ran `make bootstrap`, `make smoke-test`, and +`make check`. It produced and inspected an actual ARM glibc GNU hello package and +an actual static-musl ARM Rust executable. It also checked failure cases for ELF +validation, source pin drift, modified upstream files, and overlay conflicts. + +It did **not** run the ARM program in QEMU, boot a Pi, test a physical bay map, or +measure boot/shutdown performance. QEMU was absent and was explicitly reported +as an optional skip. The later snapshot and offline-build work belongs to M12. +See [M0 validation](Developer-M0-Validation) for the exact environment, outputs, and hashes. + +## From native init to cartridge boot + +M1 changes the output from isolated test artifacts to a base operating-system +filesystem. Its first result is a rootfs archive with an explicit package set +and exclusions. That archive alone will not establish native init or Pi boot; +those are M2 and M4 acceptance boundaries. + +M2 now boots that filesystem with a generic ARM kernel and exercises native +s6 supervision. `make init-test` automates the acceptance test; `make vm` opens +a temporary development shell. Neither supplies a Pi boot image or measures +physical boot performance. M3 adds verified FDS Rust foundations; +[tooling usage](Developer-Tooling) explains its commands. M4's [boot guide](Developer-Boot) +covers the Pi kernel, initramfs, SYSTEM image, boot volume, and full stage0 tests. + +## Find the current command interface + +The master plan includes a design directory sketch as well as runtime commands. +The checkout now implements the complete build interface, cartridge inspection +and media management, and native power-off/reboot. A few script locations and +configuration formats differ from the original sketch; the +[implementation ledger](Developer-Implementation-Status#interface-and-policy-decisions) +maps those entries to the actual interfaces. Use [Development](Developer-Development) +for targeted rebuilds and the project overview for the complete build/VM path. + +Use `make help` to inspect the current build entry points. Follow +[Your first build](Developer-Getting-Started) for the complete usable workflow today. diff --git a/Developer-Rootfs.md b/Developer-Rootfs.md new file mode 100644 index 0000000..9f542c6 --- /dev/null +++ b/Developer-Rootfs.md @@ -0,0 +1,244 @@ +--- +include_toc: true +--- + +# Building and using the ARM root filesystem + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Validation](Developer-M1-Validation) + +The assembler produces a configured **aarch64 glibc Linux userspace** at +`out/rootfs-aarch64.tar`. It contains GNU tools, XBPS, the s6 tools, the cartridge manager, optional desktop stack, and the Dasung +Paperlike 13K controller. You can inspect it and run its ARM programs on the build +workstation. M2 adds native s6 init, runtime mounts and getty. The tar itself has +no bootable partition table or Pi boot volume; do not flash the tar to a disk. +Follow [Native init and ARM VM](Developer-Init) to boot it and open a development shell. +M12 also adds `make rootfs PROFILE=recovery` and `make recovery`; see +[Recovery](Developer-Recovery) for its distinct local maintenance console and read-only +cartridge defaults. Each build preserves a profile-specific +`out/rootfs-PROFILE.tar` pointer in addition to the latest-rootfs pointer. + +## 1. Prepare the workstation + +Use the x86_64 Arch setup in [Your first build](Developer-Getting-Started). M0 should pass +before you build M1: + +```sh +make bootstrap +make smoke-test +make check +``` + +M1 also needs **Python 3.14 or newer** on Arch. It inspects XBPS metadata and zstd +package archives and produces the tar using only the standard library: + +```sh +sudo pacman -S --needed python +python3 --version +``` + +The rootfs builder installs `qemu-user-aarch64` and `util-linux` into the existing +project-local Void build container. Static QEMU executes ARM programs; util-linux +supplies namespace-local mount and capability setup. Linux 6.7+ provides a private +`binfmt_misc` registry inside the new user namespace, allowing ARM child programs +to execute without PRoot's syscall translation. Mount/setup capabilities are +dropped before package scripts run; only namespace-local file-capability support +remains. These are build tools, not additional target requirements. No host-wide +binfmt registration, sudo chroot, or host boot configuration is needed. +The Dasung musl compiler dependencies remain documented +in [Dasung integration](Developer-Dasung#dependencies-and-source-provenance). + +Allow several additional GB for package downloads, staging trees, the archive, +and retained inputs. Build commands share one Void container; run them sequentially. + +## 2. Build the CLI base + +From the checkout root: + +```sh +make rootfs PROFILE=cli +``` + +The first run downloads signed Void ARM packages and may take several minutes. +Later runs reuse the download cache, rebuild the local FDS packages, and assemble +a fresh root. Success ends with: + +```text +PASS: FDS rootfs built and verified: out/rootfs-aarch64.tar +``` + +`cli` boots to the ordinary-user FDS console and includes optional desktop and +Ethernet services. They start only when requested by the user or eligible media. +`PROFILE=development` adds native C/C++/Rust compilers, build systems, Git, +debuggers and Vim, plus virtual display diagnostics; see +[Development](Developer-Development) and the [desktop guide](Developer-Desktop). +Each archive embeds its profile identity. Unknown +profiles and mismatched SYSTEM image labels fail explicitly. + +| Output | Purpose | +| --- | --- | +| `out/rootfs-aarch64.tar` | Exported root filesystem, with target ownership, permissions and file capabilities | +| `out/rootfs-aarch64/` | Staging tree for inspection and emulated commands; owned by your host user | +| `out/manifests/rootfs-latest/` | Exact package list, checksums, metadata, scripts and build inputs | +| `out/manifests/rootfs-latest/packages/` | Copies of every selected XBPS package and available upstream signatures | +| `out/packages/fds-base-0.1.0_1.aarch64.xbps` | Explicit base dependency metapackage | +| `out/packages/fds-base-files-0.1.0_1.aarch64.xbps` | FDS identity, accounts, defaults and layout specification | +| `out/logs/rootfs-build.log` | Package installation, configuration and acceptance output | + +The first three paths are links into one retained `out/rootfs-build.*` directory. +A failed build keeps its workspace for diagnosis and does not publish a partial +archive over the last successful one. Copy the tar itself when moving the result +to another machine; copying only the link does not copy the artifact. + +## 3. Inspect the result + +```sh +tar -tf out/rootfs-aarch64.tar | head -40 +cat out/rootfs-aarch64/etc/os-release +cat out/manifests/rootfs-latest/packages.json +(cd out/manifests/rootfs-latest && sha256sum -c rootfs.sha256 packages.sha256) +``` + +The default hostname is `fds`, the timezone is UTC, and the default language is +English (`en_US.UTF-8`). Account passwords are locked and no SSH server starts. +CLI and development boot to the local `fds` account (UID 1000) through autologin; +recovery has a local root maintenance console. The build shell below runs as +emulated root so it can inspect and configure the staging tree. It is separate +from the normal booted user session. + +## 4. Run ARM programs on the workstation + +```sh +./tools/in-rootfs out/rootfs-aarch64 /usr/bin/ls --version +./tools/in-rootfs out/rootfs-aarch64 /usr/bin/readelf --version +./tools/in-rootfs out/rootfs-aarch64 /usr/bin/xbps-query -l +./tools/in-rootfs out/rootfs-aarch64 /usr/bin/dasungd check +./tools/in-rootfs out/rootfs-aarch64 /bin/bash --login +``` + +Inside the shell, try `getconf GNU_LIBC_VERSION`, `locale charmap`, `ls /usr/bin`, +or `xbps-query fds-base`. Type `exit` to return to the workstation. + +This runs ARM userspace on the host kernel through QEMU. It does not boot an ARM +kernel or validate Pi drivers. Bubblewrap makes the target the actual process +root, supplies minimal devices, and mounts temporary `/run` and `/tmp`. Native +emulation tools are mounted read-only under `/tmp/fds-build-host`. Physical USB +and DRM are absent, so this helper cannot control your monitor. Use it for trusted +image-build commands. Do not use +this shell to test disk formatting, mounts, power-off, or real device access. + +Commands can modify the staging tree. Those changes do **not** change the exported +tar. Rebuild to get a clean staging tree and archive. Built SYSTEM images remain +read-only: change packages at image build time, not by upgrading a live SYSTEM. +For the booted target's on-demand Ethernet and DNS policy, see +[Desktop and Ethernet](Developer-Desktop). + +## 5. Run the acceptance checks + +```sh +make rootfs-test +``` + +This verifies the archive and saved package checksums, restores a disposable +tree, and runs ARM glibc, GNU tools, a shell pipeline, locale, XBPS integrity, +Dasung configuration, and s6 database queries under QEMU. It also checks every +ELF file for the target architecture, validates archive permissions and ping +capabilities, and tests rejection of forbidden packages, unfinished configuration, +wrong ABIs, a host executable, missing boot entries, a shell init replacement, +and unsupported profiles. + +The image has no BusyBox, systemd, runit, or musl runtime package. Dasung carries +its own statically linked musl and libusb. Its service database is compiled at +build time and includes the base monitor bundle. M2 boots the current graph under native s6 PID 1 in a full ARM VM. Physical +monitor recovery still requires a Pi test. + +## What goes into the base + +`packages/fds-base/template` owns the explicit dependency set. The assembler also +consumes `image/base-packages.list`, so the required hardware package cannot be +lost when selecting a profile. + +| Package group | Reason | +| --- | --- | +| glibc, glibc-locales | GNU C runtime and compiled English UTF-8 locale | +| coreutils, binutils, findutils, diffutils, grep, sed, gawk, bash, tar, gzip, xz | GNU command-line tools, ELF inspection, scripting and archives | +| util-linux, procps-ng, shadow, kbd | Standard system, process, account and console utilities | +| e2fsprogs, dosfstools, erofs-utils | Tools for the planned ext4, FAT and EROFS media | +| eudev, kmod | Device discovery tools and module management | +| iproute2, iputils | Network inspection and configuration tools | +| pciutils, usbutils | Hardware inspection | +| ca-certificates, tzdata, xbps | TLS trust, timezones, and the package database/tools | +| skalibs, execline, s6, s6-linux-init, s6-rc | Native init, process supervision, and service dependency tools | +| fds-base-files, fds-init, fds-dasungd | FDS defaults, native init/service graph, and the mandatory monitor controller | + +XBPS resolves shared libraries and other transitive dependencies. The exact +resolved set is in `packages.json`; do not mistake the direct dependency list for +the entire installed set. Planned `fds-cli` and `fds-cartridged` packages are +deliberately absent until their implementation milestones. + +## Assembly details + +The FDS base-files package supplies `usr/share/fds/layout.json`. The assembler +applies that manifest before XBPS extraction to create merged `/usr` links and +empty mountpoints. Upstream xbps-src reserves these links for Void's own +`base-files`; this approach keeps its tracked files and lint rules unchanged. + +Remote packages are signature-checked using Void keys from the pinned checkout. +Local FDS packages are built here and are unsigned development artifacts. A +foreign-architecture XBPS transaction unpacks them; the target's own XBPS then +configures every package under QEMU. Locales, CA certificates, the udev hardware +database, the dynamic linker cache, native init environment, and the complete +s6 service database are ready before export. Certificate generation also runs as +an explicit checked step: the upstream CA package wrapper suppresses updater +errors. There is no first-boot package configuration step. + +Unused upstream runit service files are excluded by the shipped XBPS `noextract` +rules. This does not replace upstream package templates or add runit to the image. + +The staging directory cannot preserve arbitrary target UID/GID values as a normal +host user. Export restores ownership from the exact XBPS archive headers, applies +the util-linux `wall`/`write` tty-group and XBPS `xbps-uchroot` xbuilder-group adjustments, and gives generated +files root ownership. Namespaced Linux capabilities are converted to the portable +root-owned encoding. The archive, not the host-owned staging tree, is the authority +for installation metadata. When eventually restoring as root, use GNU tar with +`--numeric-owner --same-permissions --xattrs --xattrs-include=security.capability`. +For complete disk images and the deferred physical installation procedure, see +[Internal storage](Developer-Internal-Storage). Do not write this rootfs tar to a disk. + +The rootless configuration log therefore reports group-change failures for +`wall`, `write`, and `xbps-uchroot`. These three known adjustments are applied +and checked in the exported archive. Other installation errors require investigation. +The builder also reapplies and verifies ping's `CAP_NET_RAW` with direct QEMU +inside a user namespace and checks the actual stored xattr. + +Package versions, input hashes and selected archives are retained for inspection. +The Void binary repository is rolling; a fresh build later may resolve different +versions. Those records alone do not establish reproducibility. Use the +separate [frozen-input and offline rebuild workflow](Developer-Reproducible-Builds) for +release construction and consult its recorded acceptance status. + +## If a build fails + +Read the last error in `out/logs/rootfs-build.log`; keep the printed build-workspace +path. A package setup error is a build failure, not a reason to publish an +unconfigured filesystem. Correct the issue and run the build again; downloads +already in `out/cache/rootfs` are reused. + +If you changed an overlay and receive `Stale overlay`, compare your edited +`packages/fds-*` directory with the named generated copy under +`vendor/void-packages/srcpkgs/`. Move only that generated copy out of the way and +rerun. Never discard tracked upstream changes to silence the pin check. See +[Troubleshooting](Developer-Troubleshooting) for the existing build-container checks. + +References: [Void chroot installation](https://docs.voidlinux.org/installation/guides/chroot.html), +[XBPS configuration](https://man.voidlinux.org/xbps.d.5), +[Linux binary-format handlers](https://docs.kernel.org/admin-guide/binfmt-misc.html), and +[QEMU userspace emulation](https://www.qemu.org/docs/master/user/main.html). + +The rootless exporter restores ownership changes that package install scripts +cannot represent in a single-UID user namespace. This includes tty ownership for +`wall`/`write`, the xbuilder group for `xbps-uchroot`, and the utmp group for the +terminal-accounting helper. Archive checks validate these identities explicitly; +the ARM boot check also inspects the installed helper's actual group. diff --git a/Developer-Services.md b/Developer-Services.md new file mode 100644 index 0000000..b5a66ca --- /dev/null +++ b/Developer-Services.md @@ -0,0 +1,149 @@ +--- +include_toc: true +--- + +# Services, readiness, and shutdown + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Native init and ARM VM](Developer-Init) · [Cartridges](Developer-Cartridges) + +**Implemented in M2:** native s6 PID 1, a compiled base service graph, console +login, eudev, the Dasung controller, and a controllable test service. The +[init walkthrough](Developer-Init) explains how to boot the ARM VM and try the commands +below. M6 adds the base cartridge service; M8 adds optional desktop/network profiles as described in the [desktop guide](Developer-Desktop). + +## Startup and service state + +```text +Linux → /sbin/init → s6-linux-init → s6-svscan (PID 1) + └── rc.init → s6-rc-init → s6-rc change boot +``` + +The generated execline launcher immediately executes native init. Stage 1 prepares +writable `/run`, starts the catch-all logger, and replaces itself with s6-svscan. +Bash stage-2 scripts run as ordinary children. They initialize the live database +and activate the `boot` bundle. No service compiler runs during boot. + +| Location in FDS | Purpose | +| --- | --- | +| `/etc/s6-rc/source` | Packaged, readable service definitions | +| `/etc/s6-rc/compiled` | Dependency database compiled under ARM emulation during the build | +| `/run/s6-rc` | Live service state on tmpfs | +| `/run/service` | Supervision directories created at runtime | +| `/run/uncaught-logs` | Native init catch-all log, also copied to the console | +| `/run/log/dasungd` | Rotating monitor-controller log | + +These are guest paths, not instructions to start services on the Arch host. +The Void build container never boots the FDS service graph. + +## The base boot graph + +| Service | Function | Dependencies | +| --- | --- | --- | +| `runtime-fs` | Mount proc, sysfs, devpts and volatile temporary/log paths | Kernel or stage 0 supplies `/dev`; native stage 1 supplies `/run` | +| `hostname` | Set the image's hostname | None | +| `console` | Create volatile login accounting files | `runtime-fs` | +| `getty` | Provide a console login | `console`, `hostname` | +| `eudevd` | Process kernel device events | `runtime-fs`; readiness checked through its control channel | +| `udev-trigger` | Request initial subsystem and device events | Ready `eudevd` | +| `machine-config` | Validate internal NVMe settings and publish a fixed boot snapshot, or image defaults | `runtime-fs`; no console dependency | +| `cartridged-runtime` | Prepare bounded cartridge log storage | `runtime-fs` | +| `cartridged-log`, `cartridged` | USB events, bay inventory, validated mounts and IPC | Runtime paths and `machine-config`; no getty or eudev readiness dependency | +| `dasungd-runtime` | Prepare private controller state and log paths | `runtime-fs` | +| `dasungd-log`, `dasungd` | Supervise the monitor controller and its logger | Runtime paths and s6's logging pipeline | + +The bundle selects these services through their dependencies; it does not impose +one serial list. Console startup does not wait for device enumeration, monitor +reconnection, networking, or cartridge discovery. Dasung remains part of the base, +independent of a GUI or an inserted ENVIRONMENT cartridge. See [Dasung](Developer-Dasung). + +M6 includes the real static `cartridged` service and its logger. Network, DHCP, +NTP, sshd, D-Bus, Xorg, WindowMaker, +Bluetooth, and Wi-Fi are absent from the current boot bundle. + +## Inspect and control services + +Inside the shell opened by `make vm`: + +```sh +s6-rc -l /run/s6-rc -a list +s6-svstat /run/service/getty +s6-svstat /run/service/eudevd +s6-svstat /run/service/dasungd +cat /run/log/dasungd/current +s6-rc -l /run/s6-rc -u change test-echo +s6-svstat -o up,ready,pid /run/service/test-echo +printf 'hello\n' | s6-ipcclient /run/fds-test/echo.sock s6-ioconnect -t 3000 +s6-rc -l /run/s6-rc -d change test-echo +``` + +`test-echo` is a root-only Unix-socket echo server. Its database entry ships in +this development rootfs, but it is excluded from the normal boot bundle. Its +`test-runtime` dependency creates `/run/fds-test` with mode 0700. The ARM boot +test verifies replies, a supervised restart with a new PID, and stop/start state +transitions. It also verifies that getty and Dasung are running without hardware. + +## Readiness and failures + +A process existing does not always mean its service is usable. The echo server +notifies s6 after it starts listening. Eudev has no native s6 notification, so +`s6-notifyoncheck` checks its control channel within a bounded deadline. The +initial device trigger does not wait for every unrelated device to settle. + +There is no global `udevadm settle` or sleep added to hide a boot race. A missing +monitor leaves Dasung supervised and disconnected; it does not hold up getty. +Service errors go to the runtime logs and console. Investigate the failed +service and its dependencies instead of extending an arbitrary startup delay. + +## Shutdown + +`poweroff`, `reboot`, `halt`, and `shutdown` are the native s6-linux-init wrappers. +M10's `fds poweroff` and `fds reboot` first freeze new operations, stop optional +sessions and managed jobs, then verify DATA and cartridge unmounts. A native +shutdown hook also enforces this preparation for administrative use of the +wrappers. Only successful preparation proceeds to the remaining s6 service stop +and final kernel action. The two internal +s6-rc helper services are marked essential and remain until native final cleanup; +its diagnostic warnings about not stopping them are expected. + +The [shutdown guide](Developer-Power) explains commands, failure recovery and the current +M10 acceptance status. No physical Pi shutdown measurement is claimed. See +[Performance](Developer-Performance) for the hardware targets and measurement boundaries. + +M8 ENVIRONMENT profiles select OS-defined service bundles after console readiness. +Profile metadata is declarative; a cartridge cannot inject a root shell +script into startup. See [Cartridges](Developer-Cartridges) for that trust boundary. + +## Cartridge service + +M6 adds `cartridged` and its bounded volatile logger to the base boot bundle. +It depends on runtime filesystem setup and M12 machine settings loading, +independently of getty and eudev. See [Internal storage](Developer-Internal-Storage) for +the read-only loading policy and explicit fallback diagnostics. Its +readiness notification means the control socket exists, not that all cartridges +have finished enumeration. Getty has no cartridge dependency. + +The service listens before scanning and reconciles kernel USB/block events with +current sysfs state. See [Cartridges](Developer-Cartridges) for configuration, controls, +restart behavior and diagnostics. Dasung stays a separate mandatory base service. + +## Optional desktop and network graph + +The `desktop` bundle selects `xserver` and `desktop-session`. Xorg's display-fd +notification establishes server readiness; the static session helper subscribes +to X11 property events before launching WindowMaker, then observes its EWMH +supporting-window property. This avoids missing an event when the atom does not +yet exist. Session processes run as fds in a root-owned cgroup. Their descendants +are stopped before Xorg. Native service supervision handles unexpected exits. + +The `network` bundle selects `dhcp`. Its process readiness does not wait for a +lease; DHCP remains asynchronous. Both bundles have bounded volatile loggers, +and neither belongs to `boot`. Their runtime directories are prepared by +separate oneshots. See [Desktop and Ethernet](Developer-Desktop) for user commands. + +M10 places the DHCP client and its privilege-separated children in a root-owned +cgroup. Network stop checks actual group exit as well as the supervised main +process, then brings the managed interfaces down and clears volatile resolver +state. It introduces no additional daemon or boot dependency. diff --git a/Developer-Software-Format.md b/Developer-Software-Format.md new file mode 100644 index 0000000..0d1cd02 --- /dev/null +++ b/Developer-Software-Format.md @@ -0,0 +1,124 @@ +--- +include_toc: true +--- + +# Software cartridge format 1 + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Build and use a cartridge](Developer-Workstation) · [Original cartridge classes](Developer-Cartridges) + +A software cartridge is a complete GPT disk image with **1 + m** partitions, +where `m` is 1–32. Partition 1 describes every software package in all payload +partitions. All partitions use read-only EROFS. This format is the current +PROGRAM creation format; existing single-partition PROGRAM media remain readable. +SYSTEM, DATA and ENVIRONMENT layouts retain their existing meanings. + +## Disk and filesystem layout + +```text +GPT protective MBR + primary GPT + 1 FDS_METADATA + FDS/CARTRIDGE.TOML + FDS/SOFTWARE.TOML + 2 FDS_PAYLOAD02 + bundles/demo.hello.tar.xz + bundles/demo.editor.tar.xz + 3 FDS_PAYLOAD03 + bundles/demo.report.tar.xz + ... +backup GPT +``` + +Partition entries are consecutive, use Linux filesystem type GUIDs, unique +nonzero partition UUIDs, no GPT attributes, and nonoverlapping MiB-aligned +extents. Partition 1 starts at LBA 2048. Both GPT copies, their headers and CRCs +must agree. A full-drive write to a larger USB device relocates the backup table +and header. The existing on-target image parser and the workstation writer +share these checks. + +`FDS/CARTRIDGE.TOML` uses the existing format-1 identity with `class = "program"` +and `media.writable = false`. It contains no startup command. A sample catalogue: + +```toml +format = 1 + +[[software]] +id = "demo.hello" +name = "AArch64 hello" +version = "1.0" +architecture = "aarch64" +partition = 2 +archive_bytes = 2308 +unpacked_bytes = 70504 +entries = 2 +sha256 = "REPLACE_WITH_THE_64_CHARACTER_LOWERCASE_SHA256" +[software.commands] +hello = "bin/hello" +``` + +Those lengths and the digest are illustrative. The builder generates actual +values. IDs and command names use the existing restricted FDS identifier syntax. +IDs are unique per cartridge. Every declared payload partition must contain +exactly the catalogue's `bundles/.tar.xz` archive names, and every payload +partition must be represented. Multiple software entries may share a partition. +The catalogue contains 1–128 software entries and at most 64 KiB of TOML. + +## Archive contract + +Bundles are deterministic USTAR archives compressed with single-threaded xz. +Entries are sorted, timestamps/UID/GID are zero, and permissions normalize to +0755 for directories/executables or 0644 for ordinary data. Host source symlinks +to regular files inside the root are flattened into regular files. Directory or +escaping symlinks are rejected; the archive itself never contains links. + +The reader rejects: + +- Absolute, parent-traversing, repeated-separator, non-UTF-8 or control-character paths. +- Duplicate entries, file-as-parent conflicts, links, devices, sockets, FIFOs, + privileged permissions, and GNU/PAX extension entries. +- Incorrect SHA-256, compressed length, unpacked length, entry count, or command + paths that do not name executable regular files. +- ELF files that are not little-endian 64-bit AArch64, or any ELF in a bundle + declared `architecture = "any"`. +- Invalid xz data, nonzero material after the tar end marker, and resource-limit + violations. Each archive is at most 512 MiB compressed, 1 GiB unpacked and + 65,536 entries. Xz decompression has a 256 MiB memory limit. + +A source path must fit USTAR's path fields. Packaging reports paths that cannot +be represented rather than emitting an unsupported extension header. + +## Guest lifecycle + +On insertion the daemon validates the complete GPT against kernel partition +geometry and the current USB disk identity. It reads metadata without following +symlinks, mounts every payload read-only/noexec/nosuid/nodev, and checks the +archive inventory and lengths. `fds bay BAY` reports the full catalogue; +`fds bays` stays compact. No archive command runs automatically on insertion. + +`fds run BAY -- SOFTWARE-ID:COMMAND [ARGUMENT...]` verifies and extracts that +package on first use. It creates a private temporary filesystem, applies the +archive contract, then makes the finished tree read-only and accessible for +execution. The runtime cache has a stricter **256 MiB per software tree** limit, +including reserved inode overhead; a host-valid larger bundle may therefore +need splitting before running. Caches live under `/run/fds/software//` and +vanish after eject, unplug, restart cleanup, or shutdown. + +Managed consumers run as UID/GID 1000 with no new privileges in the bay's cgroup. +`FDS_APP`, `PATH`, `LD_LIBRARY_PATH` and `XDG_DATA_DIRS` point to that software +root. DISPLAY/XAUTHORITY retain the established optional desktop integration. +The xz utility is already an explicit dependency of the mandatory `fds-base` +package; runtime extraction adds no target package. + +Build recipes, compilers, installation hooks and privileged archive scripts are +never part of this runtime path. + +Safe eject stops consumers, rejects unexpected mount aliases, unmounts caches, +payloads and metadata, then reports SAFE. Confirmed surprise removal permits +lazy detachment after stopping consumers. Service restart removes stale mounts +before rescanning. SYSTEM stays immutable throughout. + +SHA-256 detects corruption and binds an archive to its catalogue; it does not +establish a publisher's identity. Only run software you trust. Release signature +verification remains the separate [release workflow](Developer-Releases). diff --git a/Developer-Stress-Testing.md b/Developer-Stress-Testing.md new file mode 100644 index 0000000..1b3e9fe --- /dev/null +++ b/Developer-Stress-Testing.md @@ -0,0 +1,158 @@ +--- +include_toc: true +--- + +# Twelve-bay stress tests + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Cartridge configuration](Developer-Cartridges) · [Shutdown](Developer-Power) + +M11 exercises twelve devices together and failures during media writing. The +software tests run on the build workstation. The separate physical procedure +below is for the assembled Pi; no virtual result certifies its wiring, power +budget, USB hubs, storage controllers or display. + +## Run the virtual tests + +Complete the [boot-image prerequisites](Developer-Boot), then run sequentially: + +```sh +make rootfs PROFILE=cli +make rootfs-test +make system-card PROFILE=cli +make media-image-test +make stress-test +``` + +`make stress-test` compiles two test-only static ARM workloads, boots the actual +packaged CLI image and Pi kernel, then runs the media failure suite. To repeat +only the latter after those prerequisites: + +```sh +make media-fault-test +``` + +No host disk or physical USB device is passed through. All media are disposable +regular files inside a new `out/m11-vm.*` or `out/m11-faults.*` directory. QEMU runs +inside the existing private network namespace. The SYSTEM root is a protected +read-only virtual disk; the twelve additional USB disks stress all twelve bay +slots. Earlier boot tests cover a removable USB SYSTEM itself. + +The virtual controller explicitly has twelve USB 2 and USB 3 ports. Its live +controller identity is discovered before generating the test map. This is not a +map to install on the Pi. Physical hub chains and USB 2/3 aliases must be calibrated +on the actual machine using [the cartridge guide](Developer-Cartridges). + +The test records: + +- Three cold-start pairs with empty and fully populated bays, using the actual + ordinary-user console prompt and identical image/controller configuration. +- Twelve correct cartridge IDs and serials, followed by three simultaneous + removal/insertion cycles with reversed enumeration and rotated bay positions. +- Eleven direct-I/O readers checking known bytes while DATA writes and flushes + 64 MiB. Per-thread intervals must overlap, and QEMU's real block counters must + show reads/writes for every disk. +- Console readiness while the cartridge daemon is deliberately held at an + event gate with all twelve disks present. No sleep releases that gate. +- Native reboot during sustained DATA writes, independent `e2fsck`, exact payload + verification, another boot with a different device declaration order, and halt. +- Kernel logs checked for unexpected I/O failures, USB resets, filesystem errors + and controller failures. Intentional fault tests have separate logs. + +Boot samples, shutdown events, bay inventories, I/O counters and kernel logs stay +beside the images. A greater-than-100 ms device-load change must receive the +recorded review explanation; it must not be hidden by selecting a faster sample. +Three emulator samples do not establish a Pi performance claim. + +The fault suite injects actual EIO during writing, cache flush and readback. Other +cases interrupt a confirmed transfer after at least 64 MiB by cancellation, +worker termination or daemon termination. Failed operations must remain failed +across restart, the daemon must remain usable, and none may acquire SAFE status. +A test-only QEMU bandwidth limit keeps the large transfer observable; production +code has no corresponding delay. See [QEMU fault injection](https://www.qemu.org/docs/master/devel/testing/blkdebug.html). + +## Prepare the physical session + +Use twelve labelled test cartridges and a written bay inventory. Include the +intended SYSTEM, a backed-up or disposable DATA cartridge, and the PROGRAM, +ENVIRONMENT and hardware cartridges you plan to use. Keep a known-working SYSTEM +available for rollback. Record the Pi revision, EEPROM version, hub/controller +models, cable arrangement, power source, storage serials and image hashes. + +Label physical bays `01` through `12`. Confirm one device at a time with +`fds topology` and `fds bay N`, then create the permanent machine map following +the cartridge guide. Check both USB 2 and USB 3 modes where supported. A serial +identifies the cartridge, while the stable controller/port path identifies its +bay; the `/dev/sdX` name is not a bay label. Do not copy the VM map. + +No destructive operation is necessary merely to collect diagnostics. For write +stress, use test media whose contents can be replaced. Any `fds burn` or +`fds format` operation still requires its exact per-operation confirmation. + +## Capture evidence on the target + +Copy [capture-hardware](http://gitea.home.arpa/felis/fds-os/src/branch/main/tools/capture-hardware) to the DATA cartridge. At the +FDS console, run it explicitly through Bash; DATA intentionally does not permit +direct execution: + +```sh +bash /data/capture-hardware /data/capture-before +``` + +The destination must be a new absolute directory. It is created private to the +current user. The script collects FDS status, topology, boot events, kernel logs, +USB/block inventories, mounts, network links and package versions. It writes a +status table and checksums. It does not change services, mounts, firmware, disks +or networking. A command denied to the ordinary user is recorded with its exit +status; use an administrative maintenance session for the missing diagnostic if +needed. A successful capture is not a hardware PASS. + +Choose a new directory for every capture. If DATA itself is under investigation, +write to `/tmp/capture-before` instead and copy the directory to trusted storage +before shutdown; `/tmp` is volatile. Review captured device serials and network +identifiers before sharing the files. + +## Physical acceptance sequence + +Copy the [physical session template](http://gitea.home.arpa/felis/fds-os/src/branch/main/tests/hardware/session-template.json) +and fill it with observed values. Null means unknown, not zero. +Record every run, including failures. The following is an initial repeatable +matrix, not evidence that it has already passed: + +| Scenario | Procedure and required evidence | +| --- | --- | +| Cold boot baseline | Run five full cold boots with the intended SYSTEM and no optional media. Capture boot reports and external power-on/display timing. | +| Fully populated cold boot | Repeat five times with all twelve bays occupied. Compare prompt timing, every physical bay/serial pair and kernel errors against baseline. | +| Hot insertion | Insert each cartridge individually, then insert several together. Verify all twelve positions and expected activation. Capture before/after inventories. | +| Safe removal | Stop explicit jobs and use `fds eject N` for storage. Remove only after SAFE. Verify that reusing the same bay does not retain the old cartridge's identity or SAFE state. | +| Rotated placement | Move the test cartridges to different bays and repeat. Each bay must follow its physical position regardless of Linux disk names. | +| Simultaneous I/O | Read known files from every readable cartridge while a managed DATA job writes a known payload. Record hashes, throughput and logs; USB hardware without storage needs its own functional workload. | +| Reboot under load | Request `fds reboot`. Verify the ordered shutdown record, boot again, and compare DATA hashes. Run a filesystem check only after that filesystem is unmounted in maintenance/recovery. | +| Poweroff under load | Request `fds poweroff`; record software phases and external power/display behavior. A blocked command is not permission to unplug DATA. | +| Power and USB behavior | Observe the intended supply/battery configuration under simultaneous device load, within each component's ratings. Record undervoltage indications, resets, disconnects and any display recovery failure. | + +Use external instrumentation for power applied, visible E-Ink readiness and actual +power-off. Kernel timestamps cannot measure time before the kernel starts or prove +that a flash controller honored a flush. Do not convert a VM measurement into a +physical result. Intentional power-loss or surprise-removal durability tests need +a separate disposable-data session and an explicit test plan; ordinary SAFE +acceptance does not establish power-loss durability. + +## Pass, fail and follow-up + +A physical run fails if a cartridge appears in the wrong bay, a confirmed write +or DATA eject reports success after an actual I/O error, a busy DATA shutdown +forces poweroff, a kernel/controller error is unexplained, or verified payloads +change. Capture logs before restarting a service or rebooting so its evidence is +not lost. Preserve both the observed failure and any later successful rerun. + +Compare the same image, power source and device population when evaluating a code +change. Explain or fix a boot regression over 100 ms. Keep the idle <1 second and +normal DATA <1.5 second shutdown targets separate from measured outcomes. The +physical report should explicitly list untested devices, profiles and power modes. + +No new target dependency is introduced by M11. Test workloads use Rust's standard +library; the capture script uses Bash and tools already included in the base +image. QEMU, filesystem tools and Python were existing host test dependencies. diff --git a/Developer-Tooling.md b/Developer-Tooling.md new file mode 100644 index 0000000..49dd425 --- /dev/null +++ b/Developer-Tooling.md @@ -0,0 +1,151 @@ +--- +include_toc: true +--- + +# FDS Rust tools + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [Implementation ledger](Developer-Implementation-Status) + +M3 introduces `fds-common`, the `fds` command, and the `fds-stage0` diagnostic +entry point. Both executables cross-build as static-musl AArch64 programs. The +normal OS continues to use glibc; no target musl runtime package is added. + +For native Linux software packaging and QEMU hotplug, use `make workstation` +on Arch, or `make workstation-binaries` on other Linux distributions, and follow +the [current workstation guide](Workstation). Those Clap tools are host binaries; +`make tooling` below builds the static ARM runtime tools. + +## Build and test + +After the [workstation setup](Developer-Getting-Started): + +```sh +make tooling +make tooling-test +``` + +The outputs are `out/fds`, `out/fds-stage0`, `out/fds-boottrace`, `out/fds-cartridged`, `out/fds-profile`, `out/fds-burn`, `out/fds-inspect`, `out/fds-eject`, `out/fds-power` and `out/fds-release`, with hashes and source-input +records under `out/manifests/fds-tools*.sha256`. The tests run shared Rust unit +tests on x86_64, inspect the ARM ELF files, then execute those actual ARM binaries +with the project-local QEMU userspace emulator. No physical monitor or disk is +opened by this test. + +The M3 workspace members reuse the already-locked `serde`, `serde_json`, +and `toml` dependencies: typed configuration, machine-readable output, and the +specified cartridge format. No asynchronous runtime, OpenSSL library, libudev +binding, C compiler dependency, or additional native library is introduced. + +All Rust command-line interfaces use **Clap**, with typed commands, generated +help, and argument validation before execution. The workspace shares the Clap +dependency already used by Dasung; `Cargo.lock` pins version 4.6.7 and its derive +support. It adds no shared runtime library. The dependency-free smoketest has no +command-line interface. Test helpers also use Clap and build through Cargo. +The [migration validation](Developer-CLAP-Validation) records parser, full-image, VM +and startup comparison evidence. + +M9 adds media creation and confirmed writes; see [Media tools](Developer-Media-Tools) +for its current validation state and the new SHA-256 dependency. +M12 adds the base `fds-release` signature verifier and a workstation signer; +see [Release signatures](Developer-Releases) for key handling, verification and the +Ed25519 dependency rationale. + +## Find commands and options + +Run these inside FDS (or prefix them with the emulator invocation below on the +workstation): + +```sh +fds --help +fds help format +fds format data --help +fds recovery repair --help +fds machine --help +fds-burn create --help +fds-release verify --help +``` + +Help is specific to the selected command. `-h` and `--help` print help without +performing an operation; `--version` reports the executable version. Missing, +unknown, duplicated, and conflicting arguments produce a usage error with exit +status 2. Operational failures still come from the existing FDS safety checks. + +`fds --json bays` and `fds bays --json` are equivalent. The installed aliases +`fds-inspect`, `fds-eject`, and `fds-power` have their own generated help and +accept the same relevant arguments as `fds inspect`, `fds eject`, and `fds power`. +Bay selectors accept `1` through `12`, including the `BAY1` through `BAY12` form. + +Use `--` to separate a managed program from FDS options: + +```sh +fds run 2 -- editor --help +``` + +Here `--help` belongs to `editor`. Every argument after the separator is passed +to the managed program unchanged, including `--json` and another `--`. +Media writes still require the exact confirmation associated with the current +cartridge insertion and image; accepting CLI syntax never authorizes a write. + +## Inspect identity and cartridge metadata + +Run the ARM programs through the existing container: + +```sh +./tools/in-void qemu-aarch64 "$PWD/out/fds" info +./tools/in-void qemu-aarch64 "$PWD/out/fds" --json info +./tools/in-void qemu-aarch64 "$PWD/out/fds" inspect \ + "$PWD/tests/fixtures/manifests/windowmaker.toml" +``` + +`info` reports tool version, build target, kernel and the visible PID 1. In +userspace emulation those kernel/process values describe the execution environment, +not a booted FDS machine. Full-system boot remains a separate test. + +Manifest inspection reads at most 64 KiB and rejects unknown keys, unsupported +format versions, unsafe identifiers, terminal control characters, writable +SYSTEM/PROGRAM/ENVIRONMENT media, and activation actions outside ENVIRONMENT. +A profile name is declarative; validation does not execute or activate it. +DATA requires writable media. `--json` gives parsed, validated metadata. + +Unknown commands and invalid data return nonzero. Bay state and eject now come from the base cartridge daemon; see +[Cartridge usage](Developer-Cartridges) for available operations and remaining milestones. + +## Stage-0 diagnostics + +```sh +./tools/in-void qemu-aarch64 "$PWD/out/fds-stage0" --help +./tools/in-void qemu-aarch64 "$PWD/out/fds-stage0" --check-cmdline /path/to/cmdline +``` + +Supported FDS kernel options are `fds.boot=normal|recovery` and `fds.debug=0|1`. +Normal boot selects GPT `PARTNAME=FDS_SYSTEM`; recovery selects `FDS_RECOVERY`. +Duplicate/unknown FDS options are rejected, while unrelated kernel options are +left to the kernel. Read-only `--probe SYSFS_ROOT [FDS_SYSTEM|FDS_RECOVERY]` +reports missing, unique, or ambiguous partitions from kernel-reported identities. +It never chooses the first of multiple SYSTEMs and does not infer identity from +`sdX` order. + +With no arguments, stage0 now performs the [M4 boot path](Developer-Boot) as root PID 1 +in the initramfs. It refuses that operation in an ordinary shell. The full-system +boot matrix exercises root handoff; diagnostics alone do not establish a boot. +M4 reuses the locked `libc` crate for direct Linux syscalls and adds no shared +runtime dependency. The [M2 VM](Developer-Init) remains a separate direct-root fixture. + +## Boot-event reporting + +M5 adds `fds boot-profile` (or `fds --json boot-profile`) for the current boot. +`fds-boottrace` adopts stage0 events into volatile runtime storage and records +the first console readiness; a restarted shell does not replace that timestamp. +Missing events remain absent. See [Performance](Developer-Performance) for event +boundaries, saved-report comparison, and the greater-than-100 ms regression rule. + +## Desktop, applications, and Ethernet + +M8 adds `fds profiles`, `fds profile activate windowmaker`, `fds profile deactivate`, +`fds network on|off`, and PROGRAM launches with `fds run N -- executable-name`. +The standalone `fds-profile` is also a static ARM executable. Its root-only s6 +helpers create the authenticated X server and supervise the unprivileged desktop +session; ordinary callers use the existing peer-checked control socket. +See [Desktop and programs](Developer-Desktop) for a complete usage walkthrough. diff --git a/Developer-Troubleshooting.md b/Developer-Troubleshooting.md new file mode 100644 index 0000000..424ac80 --- /dev/null +++ b/Developer-Troubleshooting.md @@ -0,0 +1,250 @@ +--- +include_toc: true +--- + +# Troubleshooting the build + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +[Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Build reference](Developer-Build-Host) + +Find the first failing command or `ERROR:` in the log, fix that problem, then +repeat the same step. Do not continue past a failed bootstrap and interpret later +missing files as independent problems. Run all commands below from the repository +root, as your regular user, unless explicitly shown otherwise. + +## Find the right log + +| Failing action | Where to look | +| --- | --- | +| `make bootstrap` | `out/logs/bootstrap.log` | +| Void build-container creation | `out/logs/xbps-bootstrap.log` | +| `make smoke-test` | `out/logs/smoke-test.log` | +| GNU hello build | `out/logs/xbps-hello.log` | +| `make check` | Terminal output; this target does not save a log automatically | + +For example, after a failed smoke-test: + +```sh +tail -n 80 out/logs/smoke-test.log +rg -n 'ERROR:|FAILED|error:|No space left|Permission denied' out/logs/ +df -h . +``` + +A search with no matches exits 1. Read the surrounding log too: upstream tools +may report failures using other wording. Save logs before re-running if you need +the previous output; see [Development](Developer-Development#preserve-logs-and-identify-inputs). + +## Unsupported host, root execution, or missing commands + +- `M0 requires an x86_64 Linux build host` or `supports Arch Linux`: use the + supported host described in [Your first build](Developer-Getting-Started). Changing + the check alone does not port the build system to another platform. +- `Run as a normal user, not root`: leave the root shell and run the build from + your normal account. Only host package installation uses sudo. +- `Missing host command`: install the prerequisites from the first-build guide, + then repeat bootstrap. `readelf` comes from binutils, `bwrap` from bubblewrap, + and `flock` from util-linux. +- `checkout path without whitespace`: move or obtain the checkout at a path + with no spaces, tabs, or other whitespace, then use that directory consistently. + +If a previous root-run left files that your account cannot write, inspect their +ownership before correcting it. Do not respond by running all later builds as root. + +## Bubblewrap or user namespace failure + +Reproduce the same capability check bootstrap uses: + +```sh +bwrap --ro-bind / / --unshare-user --uid 0 --gid 0 true +``` + +If it reports `Operation not permitted` or a namespace error, the host kernel, +container policy, or security configuration prevents the unprivileged container. +Use an Arch host that permits this operation, or have the host administrator +resolve the restriction. Repeated downloads, sudo builds, and a different Pi +will not resolve this preflight failure. Success is a zero exit status with no +output; run `make bootstrap` again once that works. + +## Void submodule missing, wrong, or modified + +Inspect before changing anything: + +```sh +cat VOID_PACKAGES_COMMIT +git submodule status vendor/void-packages +git -C vendor/void-packages status --short +git -C vendor/void-packages diff HEAD -- +``` + +If the submodule is missing, `make bootstrap` initializes it. The directory must +be part of a Git checkout; an unpacked source archive is insufficient. If a clone +was interrupted by a network error, restore connectivity and repeat bootstrap. + +`Void checkout mismatch` means the checked-out commit differs from the explicit +pin. A `+` in `git submodule status` means it differs from the parent repository's +gitlink. These are two separate checks. Do not update the pin merely to silence +the error; first determine why the commits differ. + +`Tracked Void files were changed` means upstream source has local edits. Preserve +and review the diff. Move intentional FDS changes into the overlay design or a +separate patch for review; restore upstream only after saving work you need. +Generated ignored files such as `etc/conf`, `hostdir`, and the masterdir are +normal. Never use a broad reset or clean command without checking what it removes. + +## Configuration or overlay conflict + +For `etc/conf differs from config/xbps-src.conf`, compare the two: + +```sh +diff -u .host/void-packages/etc/conf config/xbps-src.conf +``` + +Put the desired settings in `config/xbps-src.conf`, then follow +[the explicit synchronization procedure](Developer-Development#change-build-configuration-deliberately). +A `diff` exit status of 1 means differences were found, not that the comparison failed. + +For `Stale overlay`, compare the named `.host/void-packages/srcpkgs/fds-*` +directory with its source under `packages/`. Preserve any unique edits in the +source first. Move that specific generated copy into a backup directory under +`out/`, outside `srcpkgs/`, then run `./tools/prepare-void` again. Do not move or +remove unrelated upstream packages. The Dasung package is now an active overlay. After changing it, reconcile that +generated copy by this same procedure before rebuilding. + +`Overlay would replace an upstream package` is an intentional protection. Choose +an FDS-owned package name and design an overlay; do not disable the check or edit +the tracked upstream template in place. + +## Download failure, checksum mismatch, or apparent stall + +Bootstrap and package builds need network access. The first run can spend much +of its time downloading compiler packages. Read the latest log and check disk +space before assuming that a quiet download is a deadlock: + +```sh +tail -n 40 out/logs/bootstrap.log +tail -n 40 out/logs/xbps-hello.log +du -sh out/downloads .host/void-packages/hostdir +df -h . +``` + +A log or directory may not exist if its step has not started. Bootstrap's static +XBPS download has retries and a timeout; an incomplete transfer uses a `.part` +filename. A retry re-downloads that partial file. Completed cached archives are +checksum-checked on every bootstrap. + +For a checksum failure, retain the error and compare the named file against +`config/host-tools.conf`. If the cached completed archive is corrupt, move that +specific archive to a backup location and repeat bootstrap so it downloads again. +Do not change the expected hash to match an unexplained download. If a clean +re-download still disagrees, stop and investigate the source. + +A Void mirror timeout is separate from the source pin: the templates are pinned, +but bootstrap and missing build dependencies still use rolling binary repositories. +An unavailable historical dependency may need a reviewed input/mirror solution; +blindly changing the source commit does not establish an equivalent build. + +## XBPS command not found or package index seems empty + +The XBPS tools are intentionally not installed in the host's `/usr/bin`. +`make bootstrap` creates `.host/xbps/usr/bin/`. FDS build helpers set PATH for you. +For manual queries, use the explicit paths in the [package guide](Developer-Packages). + +When querying or indexing ARM output, set `XBPS_ARCH=aarch64`. The host's static +musl executable can otherwise default to its own architecture and ignore the ARM +package. `make smoke-test` uses the correct override. `tools/build-package` exports +packages but does not update `out/packages/aarch64-repodata`; use the indexing +command in the package guide after a standalone package build. + +## Rust target, linker, or architecture error + +Check the selected toolchain and local configuration: + +```sh +rustup show active-toolchain +rustup target list --installed +cat rust-toolchain.toml +cat .cargo/config.toml +printenv RUSTFLAGS CARGO_BUILD_TARGET CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER +``` + +`printenv` exits nonzero when a listed variable is unset; that is normal here. +Bootstrap installs the required target and rustfmt for the pinned toolchain. +Repeat it if those components are missing. + +Build from the repository root with the exact Cargo command in +[Development](Developer-Development#edit-and-rebuild-the-rust-program). Environment +variables, command-line `+toolchain` overrides, or outside Cargo configuration +can change the target, flags, or linker. Reconcile such overrides with the +checked-in `rust-lld` and static CRT configuration. Adding an arbitrary system +linker or glibc library is not a repair for a static-musl build. + +## Exec format error, ldd output, and QEMU skips + +`./out/fds-smoketest` normally gives `Exec format error` on x86_64: it is an ARM +program. Run it explicitly with optional `qemu-aarch64`, as described in +[the first-build guide](Developer-Getting-Started#7-optionally-execute-the-arm-rust-program). +Do not change the target to x86_64 merely to make direct execution work. + +`ldd: not a dynamic executable` with exit 1 is expected for the static program. +Use `./tools/verify-elf out/fds-smoketest aarch64 static` for the actual check. +Host `ldd` cannot establish that a foreign ARM program will run. + +`SKIP: ARM execution` means QEMU is absent. Compilation and ELF checks still run. +If QEMU is present but fails, the smoke-test must fail; inspect its diagnostic. +The dynamic GNU hello program also requires an ARM glibc loader and libraries, +so the simple static-program QEMU command is not a complete hello runtime setup. + +## Missing artifacts or check fails before any build + +`make check` expects the Cargo release artifact in `target/`. Run the Rust build +or `make smoke-test` first. A bare Cargo build updates `target/`, while +`make smoke-test` also refreshes `out/fds-smoketest` and the output manifests. +If checksums fail after editing or replacing outputs, inspect what changed and +rebuild through smoke-test; do not edit the manifest to conceal a mismatch. + +For a Rust formatting failure, run `cargo fmt --all`, inspect the source diff, +then repeat `make check`. + +## Busy /tmp message during Void bootstrap + +The initial verified bootstrap emitted +`rm: cannot remove '//tmp': Device or resource busy` during upstream cleanup. +The bubblewrap bind mount explained that instance; upstream exited successfully, +and container GCC, the cross build, and the public M0 commands all passed. + +Do not treat every later `/tmp` error as harmless. Confirm a zero command exit +status and the final bootstrap PASS, then run smoke-test. If the command exits +nonzero, report it as a failure with the surrounding log. + +## Rootfs, images, and runtime commands + +M1 implements `make rootfs PROFILE=cli` and `make rootfs-test`. If the target is +missing, check that you are in the current checkout. Read [M1 rootfs](Developer-Rootfs) +for prerequisites, shell access, logs, and retained failed-build directories. +Python 3.14+ is required for XBPS archive inspection. Supported profiles are +`cli`, `development`, and `recovery`; unknown names fail explicitly. Package +configuration must pass before an archive is published. + +`tools/in-rootfs` supports ARM child execution through a private user-namespace +binary-format handler. It requires Linux 6.7+ and namespaced `binfmt_misc` support. +The workstation's global registry is unchanged. For a single ELF executable, +`--direct` invokes QEMU without installing even a private child-execution handler. +For example: `./tools/in-rootfs out/rootfs-aarch64 --direct /usr/bin/xbps-pkgdb -a`. + +The tar is configured userspace, not a flashable image. M2 adds `make init-test` +for a full ARM boot and `make vm` for a temporary development shell. Follow +[Native init and ARM VM](Developer-Init) for its dependencies, expected output, and +failure logs. For the full kernel/stage0/SYSTEM path, use `make all`, +`make boot-test`, then `make console-vm`. Run `fds bays` and `fds poweroff` +inside the booted FDS console, not on the Arch host. An exported ARM binary will +not execute directly on x86_64. Physical Pi acceptance remains deferred. + +## What to include in a bug report + +Include the exact failing command, its exit status if available, the relevant log +excerpt, host distribution and architecture, current Void pin, and whether this +is the first build or a previously working checkout. Mention local configuration +changes and toolchain overrides. Preserve the initial error; a later missing-file +message often only describes its consequence. diff --git a/Developer-Versioning.md b/Developer-Versioning.md new file mode 100644 index 0000000..563433f --- /dev/null +++ b/Developer-Versioning.md @@ -0,0 +1,82 @@ +--- +include_toc: true +--- + +# Build versions and the workstation package + +`tools/version` is the authoritative FDS build-version resolver. It reads this +repository's Git history, never the Void submodule's history. All FDS Rust +commands, base XBPS packages, login/boot banners and `/etc/os-release` use it. +SYSTEM cartridge metadata copies the version already embedded in its rootfs, +so wrapping an older rootfs does not mislabel it as the current source version. + +Release tags are numeric `MAJOR.MINOR` or `MAJOR.MINOR.PATCH`, optionally prefixed +with `v`. Unrelated tags are ignored. The nearest reachable release tag is used: + +| Source state | Example build version | +| --- | --- | +| Exactly release tag `1.0`, clean checkout | `1.0` | +| Two commits after `1.0` | `1.0.r2.g123456789abc` | +| No reachable release tag | `0.dev.g123456789abc` | +| Uncommitted changes to tracked files | Append `.dirty` | + +Untracked files do not affect the version, matching Git’s usual dirty-checkout +convention. Add new release sources to Git before tagging. + +Versions are bounded to 32 letters, digits and periods, starting with a digit, +for the cartridge metadata, XBPS and Arch package formats. A shallow clone +without a reachable release tag gets a development version; fetch the intended +release tag/history before building a release. No value is silently invented +when both Git identity and an exported identity are missing. + +Each Rust crate uses the shared build script, including direct Cargo builds. +Cargo notices tag/ref changes, index changes and source edits, avoiding an old +version string in a cached binary. Cargo's workspace metadata uses the SemVer +form `1.0.0`; the public build version comes from Git. Protocol/schema versions, +application recipe versions, the Linux kernel version and third-party package +versions are independent and remain unchanged. + +`tools/prepare-void` writes the resolved value to the writable build checkout's +`fds-version` file. FDS-owned XBPS templates read it; base-files substitutes it +into `/usr/lib/os-release` and `/etc/issue`. `/etc/os-release` links to that +installed identity file. Runtime checks compare installed command versions +against `VERSION_ID`. Superseded FDS packages are archived under +`out/package-history/` so the active repository cannot accidentally select an +older development package whose version sorts above a release. + +Frozen source exports save a validated `FDS_VERSION` file before inventorying +and hashing their contents. Their builds use that saved value without Git or +network access. Legacy 0.1.0 snapshots retain their original scripts and layout; +the historical release stays unchanged. Release verification accepts supported +version strings independently of the verifier's own version, so newer tools can +still authenticate an older signed release using the supplied trust key. + +## Build and install on Arch + +`packaging/arch/PKGBUILD` packages the local checkout. `make workstation` invokes +makepkg, building both binaries in the existing Cargo cache and recording the +result in `out/workstation/package-path.txt`. It builds without implicit package +installation; required build commands are checked directly. Runtime feature +packages are declared as optional because project-local runners are supported. +`make workstation-install` explicitly invokes pacman to install the exact +recorded package. Generic Linux users use `make workstation-binaries`. + +The package includes binaries, the user/developer manuals, source examples, +MIT licensing and dependency notices. It has no service, hook or installation +script. The workflow follows the official [PKGBUILD specification](https://man.archlinux.org/man/PKGBUILD.5.en) +and [makepkg interface](https://man.archlinux.org/man/makepkg.8.en). + +## Verification + +`make check` includes release/no-tag/dirty/export and real Cargo cache-invalidation +checks. `make workstation-test` exercises the installed-tree cartridge creator; +`make emulator-test` covers stopping and starting the same directory, preserved +logs/overlays/mappings, and rejecting a restart of a live VM. +`make workstation-package-test` installs the real Arch package with pacman in a +disposable root and executes both installed commands. After building matching +images, `python3 tests/integration/version-images.py` reads the identity and +metadata directly from the GPT/EROFS image and executes its twelve FDS commands +under ARM emulation to compare their versions. Image acceptance +requires a new CLI rootfs and SYSTEM image: inspect their identity, execute the +ARM tools and run the VM checks. A VM result does not establish physical Pi or +Dasung power-cycle behavior. diff --git a/Developer-Void-Workspace.md b/Developer-Void-Workspace.md new file mode 100644 index 0000000..1bd6d79 --- /dev/null +++ b/Developer-Void-Workspace.md @@ -0,0 +1,81 @@ +--- +include_toc: true +--- + +# Clean upstream and writable Void workspace + +FDS keeps upstream source separate from its build state: + +| Path | Ownership and purpose | +| --- | --- | +| `vendor/void-packages/` | Clean upstream Git submodule at `VOID_PACKAGES_COMMIT` | +| `packages/` | Authoritative FDS base-package sources, including Dasung | +| `examples/software/*/void/` | Authoritative example PROGRAM package sources | +| `.host/void-packages/` | Generated writable checkout, package copies, local configuration, caches and build container | + +`make bootstrap` creates the build checkout locally at the pinned commit. It +copies Git objects independently; there is no network fetch for this step, no +Git alternates dependency and no hardlinks into the upstream source. Normal +package bootstrap still uses its configured repositories. + +`tools/prepare-void` copies FDS package overlays into the writable checkout and +retains the existing checks against replacing upstream packages or overwriting +changed generated copies. Workstation builds default to the same checkout. +Explicitly selecting a Git submodule with `--void-packages` is rejected before +source files or build state can be written there. + +The workspace helper migrates old, matching generated files and build caches by +renaming them. It compares every file, mode and symlink in each known source +copy first. Unknown upstream files, staged changes, independent edits and +conflicting destinations stop migration. Run builds and migration sequentially. +The workspace lock prevents two preparations from changing the layout together. +Python is already a build-host prerequisite; this introduces no new dependency. + +For this checkout, the 58 staged additions matched the main-repository sources. +Their local commit `47580322e801ef195c9bf279fd1fe4d8a1f723e9` and both staged +patches were saved under `.host/backups/void-overlays-20260922-134611/` before +unstaging those copies. The parent gitlink was restored to upstream commit +`02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1`, matching the declared pin. All eleven +package directories, `hostdir`, `masterdir-x86_64` and `etc/conf` were moved into +the build checkout. No upstream tracked source was changed. + +An explicit future pin update advances the writable checkout without replacing +its caches. Git refuses updates that would overwrite local files. Continue to +record the parent gitlink and `VOID_PACKAGES_COMMIT` together. + +Frozen-input creation uses the separate build checkout. New restores recreate +that layout. Older saved projects retain their +original layout and scripts; the frozen 0.1.0 release is untouched. Restore +compatibility tests do not imply a new reproducibility or physical-hardware claim. + +## Validation + +Fresh checks for this change passed: + +- `make bootstrap`, `make smoke-test`, and `make check`: migrated container use, + ARM/glibc package compilation, static-musl compilation, clean upstream state, + exact overlay/cache preservation, shallow pin updates, rejection of independent + edits and source conflicts, and cleanup protections. +- `make workstation-test`: `out/workstation-images.ckkodex6`; real Void source + builds from `.host/void-packages`, image validation and confirmed file write/readback. + Selecting the upstream submodule as a build target is rejected without writes. +- `make emulator-test`: `out/emu-test.op5azhlm`; full twelve-bay PROGRAM/DATA, + foreground terminal, eject/unplug, legacy reading and service-restart checks. +- `make release-contract-test`: `out/m12-release-contracts.wmsfrcl0`; both separate + and legacy frozen-layout restores, retained source caches, local offline + repositories and existing tamper/release-comparison checks. +- Documentation links, formatting, Python syntax and a final empty upstream Git + status, including ignored files, were checked. + +Logs are in `out/logs/void-workspace/`; source and log hashes, pins, native binary +hashes and fixture results are recorded in `out/manifests/void-workspace-acceptance.json`. +The optional host-native ARM execution shortcut was skipped in smoke-test. ARM +execution was exercised by the full emulator suite. Physical hardware was not used. + +The first emulator attempt (`out/emu-test.463emq8u`) failed on an empty cartridge +IPC response during initial discovery in its admin VM. The complete rerun passed +without target-system changes. SYSTEM, kernel and initramfs hashes matched the +previous accepted inputs. The cause of that first response failure was not +established; its command/serial logs are retained. Future failures now collect +live service diagnostics before VM shutdown, without retrying failed commands +or relaxing their acceptance conditions. diff --git a/Developer-Workstation-Tooling-Plan.md b/Developer-Workstation-Tooling-Plan.md new file mode 100644 index 0000000..09ab0cf --- /dev/null +++ b/Developer-Workstation-Tooling-Plan.md @@ -0,0 +1,33 @@ +--- +include_toc: true +--- + +# Workstation tooling plan + +The current source of requirements is [the active revision](Developer-Current-Revision). +The public tools remain native Linux Clap applications: `fds-cartridge` and +`fds-emulator`. Full OS cross builds keep their separate pinned Void environment. + +New software creation uses Void source templates and `xbps-src`. The image builder +installs packages and dependencies on the workstation, verifies installed trees, +and writes them directly into EROFS program partitions with format-2 metadata. +Creation no longer generates xz software bundles. Legacy readers stay available. + +The emulator still uses actual FDS ARM kernel/userspace, twelve virtual USB bays, +read-only SYSTEM/PROGRAM disks, private writable DATA overlays, the public serial +console, safe eject, forced removal and native shutdown. Hardware timing and +physical behavior require the procedures in the hardware notes. + +Acceptance must cover real source-package builds, image integrity and write/readback, +foreground PATH commands and terminal behavior, background `fds run`, eject/unplug, +service restart and the rendered X11 control panel. Keep the frozen 0.1.0 release +unchanged. Build new matching artifacts and record their hashes and source inputs. + +See [user instructions](Workstation) and [format reference](Software-Format) +for the supported interface. Historical xz acceptance remains in +[the earlier workstation record](Developer-Workstation-Validation). + +The Arch distribution package is `fds-tools`, built by `make workstation` and +installed by `make workstation-install`. `make workstation-binaries` preserves +the generic Linux build. Stopped emulator directories support a fresh boot while +retaining logs, DATA overlays and archived cartridge mappings. diff --git a/Developer-Workstation-Validation.md b/Developer-Workstation-Validation.md new file mode 100644 index 0000000..4e147fd --- /dev/null +++ b/Developer-Workstation-Validation.md @@ -0,0 +1,105 @@ +--- +include_toc: true +--- + +# Workstation extension acceptance + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +**Software acceptance passed on 2026-09-21.** This record covers the new native +Linux software/cartridge builder, whole-image writer, public QEMU emulator and +guest software runtime. It is separate from the immutable local 0.1.0 release. + +The host was Arch Linux x86_64, with QEMU 11.1.1 and erofs-utils 1.9.4. The public +workstation tools use ordinary Linux interfaces and distribution tools; other +Linux distributions and ARM workstations have not been exercised in this run. +The guest tests used the actual FDS Pi kernel, current initramfs and current CLI +SYSTEM on QEMU `virt`. The separate native-init test used its documented generic +ARM kernel fixture. Neither environment substitutes for physical Pi testing. + +## Workstation and emulator evidence + +| Check | Result and evidence | +| --- | --- | +| Typed Rust interfaces and archive contracts | **62 tests passed** across all host-testable workspace crates; `out/logs/workstation-all-rust-tests.log`. The architecture-restricted smoketest is validated separately by the ARM build. | +| Workstation builds and images | `out/workstation-images.5jczgwr7/acceptance.json` and `commands.log`; the recorded CLI SHA-256 matches the delivered `fds-cartridge` binary. | +| Public emulator lifecycle | `out/emu-test.36iqvx0j/acceptance.json`, `inputs.sha256`, `commands.log` and serial logs. The input hash matches the delivered `fds-emulator` binary. | +| Exact documentation examples | `out/logs/workstation-documented-example.log`; the compiled C program and report script were packaged into `out/demo-tools.img`, booted, run as UID 1000, safely ejected and shut down. | +| Incompatible initramfs | `out/emu-old-initramfs-final/console.log`; start promptly reports the complete unsupported boot-option diagnostic and stops the failed VM. | +| Current build inventory | `out/manifests/workstation-acceptance.json`, `workstation-artifacts.sha256` and `workstation-sources.sha256`. These are local evidence records, not a new signed release. | + +The host suite executed a real AArch64 build recipe and packaged a portable +script. It built both separate-payload three-partition images and shared-payload +two-partition images; unchanged recipes produced identical bytes. Independent +`sfdisk`, `xz` and `tar` checks passed. Whole-image writes to exact-size and larger +disposable files passed flush/readback and backup-GPT checks. + +Negative checks covered wrong confirmations, changed source images, stale target +previews, corrupt GPT, corrupt archives, invalid catalogue mappings, ELF files in +an architecture-independent bundle, escaping source symlinks, unsafe archive +paths/types, privileged modes, incorrect lengths/counts, trailing content and a +file replacing an implied parent directory. Existing writer failure tests remain +passing. The final host suite also uses the relative image-tool runner shown in +the guide, including extraction inside the inspection sandbox. + +The emulator suite exercised **all twelve virtual USB bays**, catalogue reporting, +execution from both payload partitions and from shared partitions, read-only +runtime caches, safe eject, forced removal during execution, and reinsertion. +It verified that managed consumers and mounts disappear and that QEMU retains +only the SYSTEM block nodes after removal. It also checked literal guest +arguments, guest exit statuses, interactive console detach and terminal-state +restoration, DATA overlays preserving their original image, and native shutdown +with active software. + +Normal-image workflows used the ordinary FDS user. A **separate test-only admin +console image** exercised service interruption/restart cleanup, unexpected +payload mount refusal and on-target PROGRAM creation rejection. That admin +console is not part of the delivered base SYSTEM. + +## Regression commands completed + +| Command | Evidence log under `out/logs/` | +| --- | --- | +| `make bootstrap` | `workstation-bootstrap.log` | +| `make smoke-test` | `workstation-smoke-test.log` | +| `make check` | `workstation-check.log`; final Rust formatting also passed | +| `make rootfs PROFILE=cli` | `workstation-rootfs-cli-accepted.log` | +| `make system-card PROFILE=cli` | `workstation-system-cli-accepted.log` | +| `make initramfs` | `workstation-initramfs.log` | +| `make rootfs-test` | `workstation-rootfs-test-accepted.log` | +| `make init-test` | `workstation-init-test-accepted.log` | +| `make tooling-test` | `workstation-tooling-test.log` | +| `make boot-test` | `workstation-boot-test.log` — all nine boot cases passed | +| `make cartridge-test` | `workstation-cartridge-test.log` | +| `make data-test` | `workstation-data-test.log` | +| `make media-image-test` | `workstation-media-image-test.log` | +| `make media-test` | `workstation-media-test-accepted.log`, including actual SYSTEM write/readback and boot | +| `make power-test` | `workstation-power-test.log` | +| `make desktop-test` | `workstation-desktop-test.log` | +| `make workstation-test` | `workstation-release-final.log` | +| `make emulator-test` | `workstation-emulator-final.log` | + +The legacy media tests now require PROGRAM creation to fail inside FDS while +retaining prepared-image writing and legacy application execution. New software +creation is exercised through the native workstation tool. The ARM CLI test also +checks the explicit emulator boot option alongside strict invalid-option rejection. + +The final CLI rootfs is `out/rootfs-build.KfWKvx/rootfs-aarch64.tar`; its SYSTEM +is `out/system-build.u1050ckh/system.img`. The matching emulator initramfs is +`out/initramfs-build.y6x9022y/initramfs.cpio`. Public convenience paths point to +these accepted artifacts. Upstream Void tracked files remain unchanged at +`02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1`. + +## Skips and hardware boundary + +M0's optional host `qemu-aarch64` execution was skipped because that command is +not installed in the host PATH; ARM static/glibc compilation and ELF validation +passed. The required guest tests ran separately through the project-local QEMU. + +These checks do not measure Pi boot/shutdown speed, RP1 USB behavior, USB power +cycling, real drive writeback, physical bay wiring, or Dasung display recovery. +Use the [new software-cartridge hardware procedure](Developer-Workstation#physical-pi-acceptance-procedure-deferred), +[stress procedure](Developer-Stress-Testing), and [Dasung guide](Developer-Dasung) for those +checks. No physical USB disk or EEPROM was written. The frozen 0.1.0 release and +its source/build-input archives remain unchanged; it does not include this extension. diff --git a/Developer-Workstation.md b/Developer-Workstation.md new file mode 100644 index 0000000..28e999e --- /dev/null +++ b/Developer-Workstation.md @@ -0,0 +1,357 @@ +--- +include_toc: true +--- + +# Build software cartridges and run FDS in QEMU + +Development reference and historical context. For current operating instructions, use the [user manual](Home). Acceptance applies only to the source and artifacts identified in each record. + + +This page records the earlier archive-based software prototype. For installed +Void software trees, the Arch `fds-tools` package, and reusable emulator sessions, +follow the [current walkthrough](Workstation). + +This historical guide uses a **Linux workstation**, not the Raspberry Pi. It takes you from +source software to a cartridge disk image, runs that image in FDS, and explains +how to write the same complete image to USB. All commands run from the checkout +root unless marked as guest commands. + +The two native host tools are `fds-cartridge` and `fds-emulator`. They use typed +Clap interfaces; append `--help` to any command or subcommand. The target's `fds` +command remains a static AArch64 executable. The new host tools are never part +of the Pi base image. Dasung support remains in the base boot bundle. + +The accepted local `out/fds-os-0.1.0/` release predates this extension. Leave that +release intact and build current images for the emulator. See +[workstation acceptance](Developer-Workstation-Validation) for current evidence. + +For accumulated build and test images, use `make clean-preview` and `make clean`. +Current images, native tools, latest test fixtures and personal emulator sessions +are kept; obsolete automated test sessions are disposable. See +[Cleanup](Developer-Cleanup) for the exact scope and how to preserve an older test run. + +## Install and build the workstation tools + +Use a normal Linux account with Rust/Cargo, a C linker, Bash and Make. The +repository's `rust-toolchain.toml` selects the Rust version. Runtime prerequisites +are: + +| Tool | Purpose | +| --- | --- | +| `qemu-system-aarch64` | Runs the FDS ARM kernel and userspace using portable TCG | +| `qemu-img` | Creates a separate writable overlay for DATA; source images stay unchanged | +| `mkfs.erofs`, `fsck.erofs` from erofs-utils | Build and verify cartridge partitions; tested with 1.9.4 | +| `xz` | Compress/decompress software tarballs with a decompression memory limit | +| `bwrap` from bubblewrap | Restricts filesystem inspection to a private writable staging directory | +| A software-specific cross compiler | Builds AArch64 applications; unnecessary for portable scripts | +| `sfdisk`, `tar`, `mke2fs`, Python 3 | Independent integration tests; not needed to boot an existing image | + +Install these with your distribution's package manager. Unprivileged user +namespaces must work for bubblewrap. Building **these host tools** does not need +Void, XBPS, an Arch host, a Pi, root, or a running FDS guest: + +```sh +make workstation +export PATH="$PWD/out/workstation:$PATH" +fds-cartridge doctor +fds-emulator doctor +``` + +Current acceptance ran on Arch Linux x86_64. The tools use native Linux +interfaces and distribution-provided utilities; other distributions and ARM +workstations have not yet been exercised by this acceptance run. + +The outputs are native binaries in `out/workstation/`. The build explicitly +selects the workstation architecture instead of the workspace's default ARM +target. Rust dependencies are pinned in `Cargo.lock`. `tar` 0.4.46 and its +`filetime` dependency provide archive decoding/creation; FDS additionally checks +entry types, paths, counts, sizes, executable architecture and hashes. + +For this repository's **existing Arch/Void setup**, reuse its local tools: + +```sh +make workstation +./tools/prepare-image-tools +./tools/in-void xbps-install -y qemu-img +out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools doctor +out/workstation/fds-emulator doctor --qemu-runner tools/in-void +``` + +The install command above changes only the project-local build container. The +optional runners accept a program name followed by its arguments; omitting them +uses programs in the workstation's `PATH`. Set the image runner on every +cartridge create, inspect or preview invocation that needs it. A QEMU runner is +saved in the session and reused for later DATA insertions. + +## Build two software bundles + +Each software recipe declares an id, display name, version, target architecture, +root directory, and named executable commands. A trusted `[build]` section may +run a workstation compiler. It is executed only by `software build`, never by +`software pack`, inspection, or a cartridge insertion. + +The [hello recipe](http://gitea.home.arpa/felis/fds-os/src/branch/main/examples/software/hello/software.toml) builds an AArch64 +C executable. Install an AArch64 glibc cross compiler as `aarch64-linux-gnu-gcc`, +then run: + +```sh +fds-cartridge software build examples/software/hello/software.toml out/demo-hello +fds-cartridge software pack examples/software/report/software.toml out/demo-report +fds-cartridge software inspect out/demo-hello +``` + +If your cross compiler has a different name, edit the recipe's `build.command`. +With this checkout's existing Void cross toolchain, the equivalent is: + +```sh +mkdir -p examples/software/hello/root/bin +./tools/in-void aarch64-linux-gnu-gcc -O2 examples/software/hello/hello.c \ + -o examples/software/hello/root/bin/hello +out/workstation/fds-cartridge software pack \ + examples/software/hello/software.toml out/demo-hello +out/workstation/fds-cartridge software pack \ + examples/software/report/software.toml out/demo-report +``` + +Use **one** of those hello workflows. Outputs must not already exist: choose +new output names when rebuilding, then update your cartridge recipe. The example +bundles each contain `software.toml` and `.tar.xz`. The archive contains the +contents of the software root, such as `bin/hello`; it has no leading `root/`. + +For your own application, copy a recipe and populate its root with `bin/`, +`lib/` and `share/` as needed. C/C++ glibc builds target AArch64; static Rust +programs should use `aarch64-unknown-linux-musl`. Set `architecture = "any"` only +for scripts/data without ELF files. An x86_64 executable is rejected. Shared +libraries must be compatible with the target SYSTEM's glibc, or bundled as +appropriate. There is no cross-cartridge dependency resolver. + +## Assemble the cartridge image + +The example cartridge recipe groups the two bundles into two payload partitions: + +```sh +fds-cartridge create examples/software/cartridge.toml out/demo-tools.img +fds-cartridge inspect out/demo-tools.img +``` + +For the project-local EROFS tools, use: + +```sh +out/workstation/fds-cartridge --image-tool-runner tools/in-image-tools \ + create examples/software/cartridge.toml out/demo-tools.img +``` + +The resulting complete disk image has **three GPT partitions**: + +| Partition | GPT name | Contents | +| --- | --- | --- | +| 1 | `FDS_METADATA` | Cartridge identity and the software catalogue | +| 2 | `FDS_PAYLOAD02` | `bundles/demo.hello.tar.xz` | +| 3 | `FDS_PAYLOAD03` | `bundles/demo.report.tar.xz` | + +To put both programs in one payload partition, use one `[[payload]]` entry with +`bundles = ["../../out/demo-hello", "../../out/demo-report"]`. This makes a +two-partition image. Each payload group creates exactly one partition. The first +partition identifies every software bundle and its payload partition. + +Creation finishes only after verifying both GPT tables, all EROFS partitions, +every archive and the catalogue. Inspection prints JSON, including offsets, +lengths, SHA-256 and available commands. Repeating an unchanged recipe produces +the same image bytes. See [the format](Developer-Software-Format) for limits and rules. + +## Boot current FDS in the emulator + +You need three matching current artifacts: the FDS Pi kernel, its initramfs, and +a SYSTEM image containing the guest software runtime. Build these on the OS +build workstation using [the build guide](Developer-Getting-Started). For an already +bootstrapped checkout: + +```sh +make rootfs PROFILE=cli +make system-card PROFILE=cli +make initramfs +make workstation +out/workstation/fds-emulator --session out/my-emulator start \ + --qemu-runner tools/in-void +``` + +If QEMU is installed directly on your Linux workstation, omit `--qemu-runner`. +To use images copied from another build machine, supply their paths: + +```sh +fds-emulator --session out/my-emulator start \ + --kernel /path/to/kernel_2712.img \ + --initramfs /path/to/fds-initramfs.img \ + --system /path/to/fds-system-cli.img +``` + +Create `out/` first if using a fresh directory. The session directory must be +**new**, private, and short enough for Unix sockets (under 90 bytes including its +absolute parent path). Start waits for the FDS prompt and verifies the emulator +bay configuration. It uses two emulated CPUs and 1024 MiB RAM by default; +`--memory-mib 2048` increases RAM. `--timeout` adjusts the boot deadline. + +The session records inputs, virtual devices, serial output and QEMU diagnostics. +QEMU stays running after the command exits. SYSTEM and software images are +read-only. No physical disks, host network interface, or Pi monitor are attached. + +## Insert, use and remove cartridges + +From the workstation: + +```sh +fds-emulator --session out/my-emulator insert 01 out/demo-tools.img +fds-emulator --session out/my-emulator guest -- fds bay 01 +fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.hello:hello +fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.report:report +fds-emulator --session out/my-emulator guest -- tail -20 /run/log/cartridged/current +fds-emulator --session out/my-emulator eject 01 +``` + +Insertion is asynchronous: `fds bay 01` may briefly show `EMPTY` before USB +storage discovery completes. Retry the status command until it reports +`MOUNTED READ ONLY`, or inspect its error. The catalogue lists exact commands. +`fds run` starts a managed process and prints its PID; its output goes to the +cartridge service log. Packages are verified and extracted into temporary, +read-only RAM filesystems on first run. They are not compiled on the guest. + +All twelve bays, numbered `01`–`12`, are available. Simultaneously mounted +PROGRAM cartridges need distinct cartridge IDs; inserting duplicate IDs is +reported as a guest error. `eject` asks FDS to stop +managed consumers, release mounts and declare `SAFE`, then removes the virtual +USB device. If FDS refuses, QEMU leaves the cartridge attached. To deliberately +simulate an accidental pull, use: + +```sh +fds-emulator --session out/my-emulator unplug 01 +``` + +A forced unplug can lose DATA writes. The guest must then detect the removal +and clean up its processes and mounts. `status` reports the actual QEMU devices +and block nodes as well as the recorded image paths. + +Use the interactive shell if preferred: + +```sh +fds-emulator --session out/my-emulator console +``` + +At the `FDS>` prompt, type `fds bays`, `fds bay 01`, or `fds run 01 -- +demo.hello:hello`. **Ctrl-] detaches**; it does not stop the VM. Use a second +workstation terminal for insertion/unplug. Detach the console before `guest`, +`eject`, or ordinary `stop`, because they also need exclusive serial access. + +Finish with: + +```sh +fds-emulator --session out/my-emulator stop +``` + +This invokes native FDS shutdown. `stop --force` cuts virtual power without that +sequence. Logs and DATA overlays remain in the session directory. Choose a new +session directory to boot again; stopped sessions are retained for inspection, +not resumed from RAM snapshots. + +## Writable DATA images + +Insert an existing FDS DATA disk image with the same `insert` command. The +emulator creates a unique `data--.qcow2` overlay in the session directory; +all guest writes go there. The original image remains unchanged. Safe eject +retains the overlay, while inserting the original image again starts a fresh +one. Software cartridges remain read-only. + +After eject or shutdown, export an overlay if you need the changed DATA contents: + +```sh +qemu-img convert -f qcow2 -O raw /path/to/session/data-02-IDENTIFIER.qcow2 out/saved-data.img +``` + +Keep the original backing image at its recorded path until conversion completes. +Never convert or edit an image while it is attached to a running VM. + +## Write the completed image to USB + +The burn flow always starts with the complete image created above. It does not +construct partitions directly on a drive. Identify the intended **whole USB +drive**, unmount it, then make a preview as your normal user: + +```sh +fds-cartridge preview out/demo-tools.img /dev/sdX out/usb-preview.json +``` + +Replace `/dev/sdX` with the actual whole USB drive. The JSON records its model, +size, insertion identity, exact image hash and an exact `confirmation` string. +Review these before copying the full phrase into the write command: + +```sh +sudo /absolute/path/to/fds-cartridge write out/usb-preview.json \ + --confirm 'COPY THE EXACT confirmation VALUE FROM THE PREVIEW' +``` + +The writer rechecks source and target identity, requires a USB whole disk, +rejects mounted/protected storage, writes the **full image**, flushes and verifies +readback. On larger drives it relocates the backup GPT to the end. A replaced +USB drive or changed image requires a fresh preview. This destroys the selected +drive's existing contents. No physical drive has been written by the automated +acceptance tests. + +To rehearse on a disposable file without root or USB hardware: + +```sh +truncate -s 64M out/disposable-usb.img +fds-cartridge preview out/demo-tools.img out/disposable-usb.img \ + out/file-preview.json --file-target +fds-cartridge write out/file-preview.json --confirm 'COPY THE EXACT confirmation VALUE' +fds-cartridge inspect out/disposable-usb.img +``` + +The file must be at least as large as the source image. Use `--image-tool-runner` +for preview/inspection if EROFS tools are provided by the local wrapper. + +## Troubleshooting + +| Symptom | What to do | +| --- | --- | +| Required executable not found | Run the corresponding `doctor`; install the named host utility or select an explicit runner | +| Bubblewrap namespace failure | Enable unprivileged user namespaces according to your workstation policy; do not run creation/inspection as root | +| Output already exists | Use a new bundle/image/preview/session name; creation does not overwrite outputs | +| Unknown `fds.emulator` option or missing emulator settings | Rebuild both the current initramfs and SYSTEM; the frozen 0.1.0 images predate this feature | +| Console in use | Detach with Ctrl-] before guest commands, safe eject, or shutdown | +| Guest command timed out | Inspect `console.log`; the command may still be running, so do not blindly repeat a write | +| Incomplete insertion | Run `unplug BAY` to reconcile the recorded intent with actual QEMU devices, then reinsert | +| Software digest/path/architecture error | Rebuild the bundle and cartridge on the workstation; the guest will not run an invalid archive | +| Cache limit exceeded | Reduce the software bundle; one runtime tree including inode overhead is limited to 256 MiB | +| Safe eject blocked | Close unmanaged processes or extra mounts using the media, then retry; `unplug` is only for deliberate failure simulation | +| VM will not boot | Read session `console.log` and `qemu.log`; verify all three supplied boot artifacts belong to the current build | + +QEMU exercises the actual Linux/FDS software path. It does not emulate Pi +firmware, RP1, USB power sequencing, the physical twelve-bay wiring, or Dasung +power recovery. Those acceptance checks remain hardware procedures. + +## Physical Pi acceptance procedure (deferred) + +After assembling and calibrating the physical bay map, use a disposable USB +cartridge and the current matching SYSTEM/initramfs. Perform these checks on the +Pi; the VM results do not replace them: + +1. Build the hello/report cartridge on the workstation, record its image hash, + and use preview/confirmation to write that entire image to the chosen USB drive. +2. Insert it in a calibrated bay. Record `fds --json bay BAY`; confirm both + software entries and their declared payload partitions appear. +3. Run `demo.hello:hello` and `demo.report:report`. Save their output from the + cartridge log and confirm UID 1000. Check `/proc/self/mountinfo` for read-only + payload and software-cache mounts. +4. Run `fds eject BAY`, verify SAFE, then remove and reinsert the cartridge in + another calibrated bay. Repeat both commands. Record any USB enumeration or + I/O errors. +5. With a disposable cartridge and a deliberately long-running test program, + test surprise removal separately. Verify its managed processes terminate and + all of that bay's software mounts disappear. Do not use valuable DATA for + this failure test. +6. Exercise native shutdown with active software, cold boot, and a real USB + power cycle. Save actual observations and kernel/service logs. Follow the + separate [Dasung hardware procedure](Developer-Dasung) for monitor recovery. + +Record the Pi, hub, USB drive and kernel versions with the result. Hardware +latency, electrical behavior and data durability remain unverified until these +measurements are performed on the assembled machine. diff --git a/EEPROM.md b/EEPROM.md new file mode 100644 index 0000000..dbe8694 --- /dev/null +++ b/EEPROM.md @@ -0,0 +1,125 @@ +--- +include_toc: true +--- + +# Pi 5 EEPROM configuration + +[Boot images](Developer-Boot) · [Recovery](Recovery) · [User manual](Home) + +The profile tool creates reviewable firmware/configuration files on the +workstation. Applying them to a Pi is a separate maintenance operation. Preserve +the machine's original firmware and configuration before changing them. + +## Preview a profile on the build workstation + +From the repository root: + +```sh +./tools/configure-pi-eeprom --profile production +./tools/configure-pi-eeprom --profile development +``` + +The first invocation downloads three checksum-pinned inputs into +`.host/eeprom/`: the official configuration script, the Pi 5 preview firmware and +its upstream license. Later calls verify and reuse them. `FDS_OFFLINE=1` requires +all inputs to exist and refuses a network download. + +The command prints its new `out/eeprom-PROFILE.*` directory. It contains: + +| File | Purpose | +| --- | --- | +| `base.bin` | Exact input firmware retained unchanged | +| `original.conf` | Configuration to restore | +| `configured.conf` | Reviewed profile merged with the supplied settings | +| `configured.bin` | Real EEPROM image with the new configuration | +| `rollback.bin` | Same base firmware with `original.conf` restored | +| `review.diff` | Human-readable configuration changes | +| `manifest.json` | Input identity, selected settings and output SHA-256 hashes | +| `LICENSE` | Complete upstream tool and firmware notices | + +Default inputs are **preview defaults, not a backup of your Pi**. A rollback image +restores the configuration supplied to this command on the supplied base firmware. +It cannot reconstruct a different firmware version or settings that were never +provided. Retain the actual machine's configuration and matching firmware input +before applying anything to that machine. + +## Profile behavior + +| Setting | Production | Development | +| --- | --- | --- | +| `BOOT_ORDER` | `0xf6`: NVMe, repeat | `0xf16`: NVMe, SD, repeat | +| `BOOT_UART` | `0` | `1` | +| `NET_INSTALL_ENABLED` | `0` | `0` | +| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | +| `POWER_OFF_ON_HALT` | `1` | `1` | +| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | + +Boot order is read from the right. Both profiles avoid scanning the twelve USB +cartridges for firmware boot. Development retains an SD rescue path. Both disable +network-install keyboard detection; Raspberry Pi documents that this detection +adds USB initialization and enumeration work. [Official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). + +On Pi 5, the power-off setting requests PMIC standby on halt; the dedicated power +button remains the wake mechanism. The wait setting leaves cold power-on boot +enabled. After shutdown, check that attached power hardware follows the selected policy. [Official power settings](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#POWER_OFF_ON_HALT). + +The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf) and +[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf). Review these files before choosing a profile. + +## Preserve the machine's settings + +In a Raspberry Pi maintenance environment with the official EEPROM utilities, +collect its configuration and version information before changing firmware: + +```sh +rpi-eeprom-config > pi-current.conf +rpi-eeprom-update > pi-eeprom-status.txt +``` + +Retain the matching original firmware image separately. The status report helps +identify it; it is not a binary backup. Copy the saved inputs to the workstation, +then prepare a new directory: + +```sh +./tools/configure-pi-eeprom \ + --profile production \ + --base-image /absolute/path/to/original-pi5-firmware.bin \ + --current-config /absolute/path/to/pi-current.conf \ + --output-directory /absolute/path/to/new-eeprom-review +``` + +The base must be a regular 2 MiB Pi 5 image, not a device node. The output directory +must not already exist. The helper preserves unrelated configuration lines and +conditional sections. It replaces all occurrences of the six managed settings, +including conditional overrides, with the selected profile's final `[all]` +settings. Review that change in `review.diff`; the exact original conditional +configuration remains in `original.conf` and `rollback.bin`. + +The upstream parser reads the generated images back before success is reported. +This checks file construction and configuration roundtrip, not hardware +compatibility. No secure-boot key, fuse, customer signature or OTP setting is +modified by this helper. + +## Apply and roll back + +Apply the reviewed configuration from a Raspberry Pi maintenance environment +with the official EEPROM utilities, then follow the updater's restart instructions. + +```sh +sudo rpi-eeprom-config --apply ./configured.conf ./base.bin +``` + +The updater may program EEPROM immediately or schedule an update depending on +its platform and update method. Follow its output and verify the resulting +configuration/version after the required restart. To restore the supplied +configuration on the same supplied firmware: + +```sh +sudo rpi-eeprom-config --apply ./original.conf ./base.bin +``` + +If the firmware version itself changed, use the saved matching original firmware +for the rollback operation. Keep the maintenance SD and an external EEPROM rescue +route available; internal recovery cannot repair an EEPROM that prevents internal +boot. The official [EEPROM update and recovery guide](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom) +describes that physical workflow. diff --git a/Getting-Started.md b/Getting-Started.md new file mode 100644 index 0000000..a0c66df --- /dev/null +++ b/Getting-Started.md @@ -0,0 +1,117 @@ +--- +include_toc: true +--- + +# Build and start FDS + +This guide creates a bootable FDS SYSTEM image and starts it in QEMU. Use an +x86_64 Arch Linux workstation for the complete OS build. To build application +cartridges on another Linux distribution, use the [workstation guide](Workstation). +A prepared set of kernel, initramfs and SYSTEM images can be run on a Linux +workstation without building the OS locally. + +For the physical computer, follow [Run FDS on a Raspberry Pi 5](Hardware-Setup). +It covers HDMI-plus-USB monitor wiring, both disks, EEPROM, first boot and recovery. + +## Prepare the workstation + +Clone both repositories together, or initialize the wiki in an existing checkout: + +```sh +git clone --recurse-submodules gitea@192.168.1.193:felis/fds-os.git +cd fds-os +# For an existing checkout, run this from its root instead: +git submodule update --init fds-os.wiki +``` + +The manual is stored directly in `fds-os.wiki/`. OS builds and workstation +packages read the same pages. The remote above uses the configured SSH server; +the browser address is [gitea.home.arpa](http://gitea.home.arpa/felis/fds-os). + +Use a normal user account, a Git checkout with its submodule metadata, and a +checkout path without spaces. Initial preparation needs Internet access. The +host must support unprivileged user namespaces and namespaced binfmt_misc +(Linux 6.7 or newer). Full builds and VM tests use tens of GiB per build tree; +check free space with `df -h .` and use [cleanup](Cleanup) between build batches. + +From the checkout directory, install host prerequisites: + +```sh +sudo pacman -S --needed bash coreutils binutils git curl make file tar xz gzip zstd \ + bubblewrap rustup ca-certificates findutils diffutils grep sed gawk util-linux \ + python e2fsprogs libarchive lz4 +bwrap --ro-bind / / --unshare-user --uid 0 --gid 0 true +``` + +Python must be 3.14 or newer. The namespace check exits successfully without +output. Build commands below run as your normal user, sequentially; they share +one project-local Void build container. + +```sh +make bootstrap +make smoke-test +make check +``` + +Bootstrap prepares pinned Rust and XBPS tools and the Void build container. +The upstream reference stays clean in `vendor/void-packages/`. Generated package +copies, build configuration and caches live in `.host/void-packages/`; edit +FDS package sources in `packages/`, not in either generated checkout. +The smoke test checks the AArch64 build toolchains; `make check` validates the +build guardrails. Stop and resolve any failed command before continuing. Logs +are in `out/logs/`. Repeated preparation reuses verified downloads and caches. + +## Build the images + +```sh +make rootfs PROFILE=cli +make system-card PROFILE=cli +make initramfs +make workstation +``` + +The rootfs build includes the kernel and base packages. These are the three +inputs needed by the emulator: + +| File | Purpose | +| --- | --- | +| `out/kernel/boot/kernel_2712.img` | ARM kernel | +| `out/fds-initramfs.img` | Early startup and SYSTEM discovery | +| `out/fds-system-cli.img` | Read-only operating system cartridge | + +`make rootfs PROFILE=development` followed by `make system-card PROFILE=development` +builds a SYSTEM with compilers, Git, Vim, debuggers and display diagnostics. +Both profiles include WindowMaker and the Dasung daemon. The desktop starts on +request. `make all` also assembles recovery, boot and internal-storage images; +see [real-hardware setup](Hardware-Setup) when preparing a physical machine. + +## Start the emulator + +With the project-local QEMU tools: + +```sh +./tools/in-void xbps-install -y qemu-img +out/workstation/fds-emulator --session out/my-emulator start --qemu-runner tools/in-void +out/workstation/fds-emulator --session out/my-emulator console +``` + +If QEMU is installed directly on the workstation, omit `--qemu-runner`. +Use a new session directory for the first run; after `stop`, you can `start` the +same session again. Logs and DATA overlays are retained, and bays start empty. +At the `FDS>` prompt: + +```sh +fds info +fds bays +fds --help +``` + +The shell runs as the ordinary `fds` user. Press **Ctrl-]** to detach from the +console; the VM remains running. Stop it from the workstation: + +```sh +out/workstation/fds-emulator --session out/my-emulator stop +``` + +Continue with [building and inserting software cartridges](Workstation). +That guide also covers using copied boot images, DATA overlays and USB writing. diff --git a/Gitea-Wiki.md b/Gitea-Wiki.md new file mode 100644 index 0000000..38ecdb9 --- /dev/null +++ b/Gitea-Wiki.md @@ -0,0 +1,155 @@ +--- +include_toc: true +--- + +# Editing the Gitea wiki + +[Home](Home) · [All pages](Page-Index) · [Developer guide](Developer-Guide) + +The wiki is the **single source of truth for FDS documentation**. Its Git +repository is checked out at `fds-os.wiki/` inside the OS repository. Edit the +pages here or through Gitea's wiki editor. Both methods change the same wiki +repository; there is no export command, generated wiki, or second manual to +keep in sync. + +Read it at [the FDS/OS wiki](http://gitea.home.arpa/felis/fds-os/wiki/Home). +The main repository's README is only an entry point. Build scripts read the +wiki files directly when including documentation in OS, Dasung, workstation +and release packages. + +## Get the documentation locally + +A fresh checkout can include both submodules: + +```sh +git clone --recurse-submodules gitea@192.168.1.193:felis/fds-os.git +cd fds-os +``` + +For an existing OS checkout: + +```sh +git submodule update --init fds-os.wiki +``` + +This checks out the documentation revision recorded by the OS repository. +`make bootstrap` also initializes missing documentation. It leaves an already +populated wiki checkout in place so local edits remain available. + +For documentation work without the OS source, clone the wiki directly: + +```sh +git clone gitea@192.168.1.193:felis/fds-os.wiki.git +``` + +The configured SSH host is `192.168.1.193`; the browser host is +`http://gitea.home.arpa`. These addresses belong to the local Gitea server. + +## Edit and publish + +The simplest workflow is Gitea's **Edit** button on a wiki page. Saving commits +the change directly to the wiki repository. Use the Git workflow for changes +across several pages or for editing alongside code. + +From the OS checkout, with local wiki changes committed before switching branches: + +```sh +git -C fds-os.wiki switch main +git -C fds-os.wiki pull --ff-only +# Edit fds-os.wiki/*.md and any referenced images. +make wiki-check +git -C fds-os.wiki diff --check +git -C fds-os.wiki diff +``` + +Stage the pages you changed and any new images. For example: + +```sh +git -C fds-os.wiki add Hardware-Setup.md +git -C fds-os.wiki commit -m "Clarify hardware setup" +git -C fds-os.wiki push origin main +``` + +The push updates the published wiki directly. No copy, export or import step is +required. If someone edited through Gitea, pull those changes before editing +locally. Resolve divergent edits with normal Git merge or rebase workflows; +`pull --ff-only` deliberately stops instead of choosing for you. + +When the OS checkout should use that documentation revision for builds or a +release, record the submodule commit in the parent repository: + +```sh +git add fds-os.wiki +git commit -m "Update documentation revision" +git push +``` + +Publish the wiki commit before publishing a parent commit that references it. +The parent stores a commit pointer, not another copy of the pages. Web edits can +be published independently; update the pointer when selecting the documentation +snapshot to accompany the OS source. Do not run `submodule update --remote` +across all submodules: the Void source must remain pinned. + +## Page names and links + +All pages and images live at the wiki root. Use English filenames with hyphenated +words and consistent capitalization. Gitea page links omit the `.md` extension: + +| File | Link written inside another wiki page | +| --- | --- | +| `Home.md` | `[Home](Home)` | +| `Hardware-Setup.md` | `[Hardware setup](Hardware-Setup)` | +| `Developer-Build-Host.md` | `[Build host](Developer-Build-Host)` | +| Section within a page | `[First boot](Hardware-Setup#6-power-on-and-reach-the-console)` | +| `Asset-images-fds-control.png` | `![FDS Control](Asset-images-fds-control.png)` | + +These are wiki routes, not ordinary Markdown file paths. When reading the raw +files offline, open the named `.md` file in this directory. Use the browser wiki +for rendered navigation. Images are committed beside the pages; no external +image host is required. + +Start a regular page with the following frontmatter to enable Gitea's section +navigation, then add its title: + +```markdown +--- +include_toc: true +--- + +# Page title +``` + +Use standard Markdown headings, fenced command blocks and tables. For source +files outside the wiki, use a complete browser URL, for example +[the machine bay configuration](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/machine/bays.toml). +A path such as `../config/machine/bays.toml` would resolve inside the wiki. +Generated `out/`, `.host/` and `target/` artifacts are local evidence, not published +wiki attachments. Describe their paths as code and identify the relevant build; +do not link to files that Gitea cannot serve. + +## Navigation and checks + +- `Home.md` is the landing page; capitalization matters. +- `Overview.md` explains the project and the main workflows. +- `Page-Index.md` lists every content page. Add new pages here. +- `_Sidebar.md` provides task-oriented navigation. Add commonly used guides here. +- `_Footer.md` provides common navigation and the historical-evidence note. +- `Developer-*` pages contain design references and acceptance history. + +From the parent checkout, `make wiki-check` checks page names, the page index, +local links, section anchors, images and links into the source repository. +`make check` includes this check. It does not modify or publish documentation. +No additional dependencies are needed beyond the build host's Python 3. + +Installed documentation and release archives are build outputs copied from this +wiki. Edit the wiki source, then rebuild the package to refresh those copies. +Frozen historical releases keep their original documentation and acceptance +records. Their measurements do not validate changed sources. + +## Gitea format references + +The format follows the server's Gitea 1.27 wiki behavior: + +- [Gitea Markdown features](https://docs.gitea.com/usage/repository/markdown/) +- [Wiki page paths and special filenames](https://github.com/go-gitea/gitea/blob/v1.27.0/services/wiki/wiki_path.go) +- [Wiki rendering and raw files](https://github.com/go-gitea/gitea/blob/v1.27.0/routers/web/repo/wiki.go) diff --git a/Hardware-Setup.md b/Hardware-Setup.md new file mode 100644 index 0000000..7204287 --- /dev/null +++ b/Hardware-Setup.md @@ -0,0 +1,396 @@ +--- +include_toc: true +--- + +# Run FDS on a Raspberry Pi 5 + +[User manual](Home) · [Dasung wiring and diagnostics](Dasung) · [Recovery](Recovery) + +This guide takes a partly assembled Pi from prepared disks to its first FDS +console, then adds monitor control and physical bay numbering. Start with one +SYSTEM cartridge; the twelve-bay assembly and a desktop are not prerequisites. +These are instructions for physical testing, not a record of a successful Pi +boot. Existing QEMU results do not establish HDMI compatibility, USB power +stability, monitor recovery or physical boot times. + +Follow steps 1–6 to reach the first console. Steps 7–9 check the monitor, install +bay mappings and save results. If the screen stays blank, jump to +[first-boot troubleshooting](#when-the-first-boot-fails). + +Commands marked **workstation** run on the Linux build computer, **maintenance +OS** means a separate Raspberry Pi OS rescue installation, and **FDS** means the +console on the Pi. FDS does not start SSH or networking automatically. + +## 1. Prepare the minimum hardware + +You need: + +- A Raspberry Pi 5 with cooling and a suitable USB-C power supply. +- A PCIe NVMe adapter/HAT and NVMe drive for internal boot and recovery storage. +- One USB storage device for SYSTEM, a USB keyboard, and the Paperlike 13K. +- HDMI video cabling, a USB-A-to-USB-C **data** cable for the monitor controller, + and the monitor's supplied power adapter. +- A Linux workstation and an NVMe enclosure/adapter for writing the disks. + A separate maintenance microSD card is useful for EEPROM and display diagnosis. + +The current FDS installation uses **two disks**: + +| Disk | Contents | When it is needed | +| --- | --- | --- | +| Internal PCIe NVMe | `FDS_BOOT`, `FDS_RECOVERY`, `FDS_INTERNAL` | Every FDS boot; also recovery without SYSTEM | +| USB SYSTEM cartridge | `FDS_SYSTEM` | Normal FDS operation | + +PROGRAM, DATA and ENVIRONMENT cartridges can wait until the console works. +If only microSD is connected today, use it for Raspberry Pi OS maintenance and +cable checks. This guide's complete FDS installation requires NVMe: persistent +machine settings deliberately accept PCIe NVMe only. Writing the internal image +to microSD or leaving NVMe in a USB enclosure does not provide that supported +layout. A SYSTEM image alone contains no Pi firmware boot partition. + +Connect or reseat the PCIe ribbon and HAT with power disconnected, following the +adapter's instructions. For the first boot, attach SYSTEM directly to a Pi USB +port if the cartridge hubs are not ready. Changing the hub wiring later requires +a fresh bay calibration. + +## 2. Wire HDMI video, USB control and power separately + +The Pi's micro-HDMI ports are **video outputs**; the monitor receives that video. +The Pi 5's USB-C socket supplies power to the Pi and does not provide DisplayPort +Alt Mode video. A USB-C display cable alone cannot replace HDMI on this board. +See [Raspberry Pi's display connection guide](https://www.raspberrypi.com/documentation/computers/getting-started.html#display). + +| Connection | Purpose | +| --- | --- | +| Pi micro-HDMI output → Paperlike HDMI video input | Picture; use the correct HDMI connector sizes for both ends | +| Pi USB-A host port → Paperlike USB-C data port | Candidate USB control path for `dasungd`; verify enumeration in step 7 | +| Supplied Dasung power adapter → monitor power input | Monitor power, separate from USB control | +| Pi power supply → Pi USB-C power input | Pi power | + +Use the monitor's HDMI input selection, if its controls offer one. A charging-only +USB cable will not expose the controller. This HDMI-plus-USB arrangement is the +route to test; whether this unit exposes both controller interfaces while HDMI +is active still needs observation. HDMI can show a picture even when USB control +is missing, and USB replies do not prove the picture is correct. + +Use the supplied Dasung DC power connection: the manufacturer specifically warns +that ordinary USB power is insufficient for reliable 13K operation. +[DASUNG power guidance](https://shop.dasung.com/pages/q-a). +For Pi 5, a supply supporting 5 V/5 A permits a larger USB power budget; a 3 A +supply limits downstream peripherals to 600 mA. Use powered hubs for the cartridge +assembly rather than budgeting the monitor and twelve cartridges from the Pi. +[Raspberry Pi power guidance](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#power-supply). + +The shipped FDS boot configuration forces this monitor's recorded 3200 × 2400 +timing at about 37 Hz. That timing was established on the original workstation. +DASUNG documents that some HDMI hosts instead use 27 Hz. FDS's forced EDID means +you should not assume automatic fallback. If HDMI is blank or unstable, use the +[native-EDID diagnostic boot](Dasung#try-the-monitors-own-hdmi-edid) before +changing monitor identity or USB matching rules. +[DASUNG HDMI refresh-rate guidance](https://shop.dasung.com/pages/dasung-paperlike-13k-the-worlds-first-13-3-inch-37hz-3k-e-ink-monitor). + +## 3. Build a matching image set on the workstation + +Complete [Prepare the workstation](Getting-Started#prepare-the-workstation), +including bootstrap, smoke test and build checks. From the repository root, run +the following commands **sequentially**, stopping if any fails: + +```sh +make rootfs PROFILE=cli +make system-card PROFILE=cli +make recovery +make initramfs +make boot-volume BOOT_MODE=production +make internal-image +``` + +This builds one normal SYSTEM and independent recovery, then combines the current +kernel, initramfs and recovery into the internal disk. The CLI rootfs includes +the matching kernel modules, native s6 and the base Dasung controller. Rebuild +the complete sequence after a source change; existing convenience links in +`out/` can otherwise point to components from different builds. + +`BOOT_MODE=production` puts the interactive console on the HDMI display with +USB keyboard input. It is suitable for first bring-up without a serial cable. +`PROFILE=development` is a separate choice that adds compilers and debugging +tools to SYSTEM; it does not select the console transport. The smaller CLI +profile is sufficient here, and includes the optional desktop too. + +| Output | Write it to | +| --- | --- | +| `out/fds-internal.img` | Whole NVMe disk | +| `out/fds-system-cli.img` | Whole USB SYSTEM disk | +| `out/fds-boot.img` | Raw FAT partition payload; already included in the internal image | +| `out/fds-recovery.img` | Raw EROFS partition payload; already included in the internal image | + +Do not write the last two files as whole disks. `make all` builds additional +profiles and release tooling; it is unnecessary for this first-boot image set. +Use [cleanup](Cleanup) to review retained build trees when space is tight. + +Record the inputs before flashing. On the **workstation**, choose a new session +directory and retain the printed build logs/manifests with it: + +```sh +mkdir out/hardware-first +cp tests/hardware/session-template.json out/hardware-first/session.json +git rev-parse HEAD > out/hardware-first/source-commit.txt +git diff --binary HEAD > out/hardware-first/source-changes.patch +./tools/version > out/hardware-first/build-version.txt +sha256sum out/fds-internal.img out/fds-system-cli.img \ + out/fds-boot.img out/fds-recovery.img out/fds-initramfs.img \ + out/kernel/boot/kernel_2712.img > out/hardware-first/images.sha256 +cp out/boot-volume/files/cmdline.txt out/hardware-first/cmdline-built.txt +``` + +Keep any untracked build inputs separately too. These checksums identify your +local build; they are not a release signature. For a downloaded release, follow +[signature verification](Releases) and use its matching complete image set. +The historical `0.1.0` release and its acceptance records remain separate from +new local builds. + +## 4. Check NVMe boot configuration + +If the Pi already boots from this NVMe adapter, first record its EEPROM settings; +changing firmware is not a prerequisite just to try FDS. Otherwise, use a +separate Raspberry Pi OS maintenance microSD with the official EEPROM utilities. +Keep that rescue medium available throughout bring-up. + +In the **maintenance OS**, before editing: + +```sh +rpi-eeprom-config > pi-current.conf +rpi-eeprom-update > pi-eeprom-status.txt +``` + +Copy these files to the workstation and preserve the matching original firmware +as described in [EEPROM preparation and rollback](EEPROM). To review the +installed boot configuration, the official editor is: + +```sh +sudo rpi-eeprom-config --edit +``` + +For initial FDS bring-up, `BOOT_ORDER=0xf16` means NVMe first, SD fallback, then +repeat (read from the right). A non-HAT+ NVMe adapter may also require +`PCIE_PROBE=1`; follow the adapter's instructions and the +[official PCIe boot procedure](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#boot-from-pcie). +Preserve unrelated board settings, follow the updater's restart instructions, +then read the configuration back. FDS already supplies `dtparam=pciex1` in +`config.txt`; EEPROM discovery and the kernel's PCIe enablement are separate. + +The [FDS EEPROM development profile](EEPROM#profile-behavior) also keeps SD +fallback and enables firmware UART logging. Its offline preview is not a backup +of your board and does not apply itself. Do not switch to NVMe-only production +boot order until the rescue and normal boot paths have been checked. + +## 5. Write the two disks + +On the **workstation**, follow the complete +[guarded disk-writing procedure](Internal-Storage#install-the-internal-disk). +It includes disk identification by model/serial, an explicit erase confirmation, +readback, and backup-GPT relocation. Run it separately for these assignments: + +| Pass | `fds_image` | `fds_disk` | +| --- | --- | --- | +| Internal disk | `$PWD/out/fds-internal.img` | The verified NVMe enclosure's whole-disk `/dev/disk/by-id/…` path | +| SYSTEM | `$PWD/out/fds-system-cli.img` | A different, verified USB cartridge's whole-disk `/dev/disk/by-id/…` path | + +Both writes erase the selected disk. Follow all checks in that procedure; a +`-part1` path is not a whole-disk destination. The disks must use 512-byte logical +sectors and be at least as large as their respective images. + +Confirm the three internal partition labels and the USB's single `FDS_SYSTEM` +label after writing. Safely disconnect the disks, install NVMe on the powered-off +Pi, and connect exactly one SYSTEM. Do not leave the NVMe enclosure connected as +a duplicate source of FDS partitions. + +## 6. Power on and reach the console + +Power the monitor and any required hub, attach video, USB control, keyboard and +SYSTEM, then power the Pi. Remove the maintenance SD for this attempt so an SD +fallback cannot be mistaken for an FDS boot. + +The expected sequence is Pi firmware → kernel and stage0 from NVMe → read-only +SYSTEM from USB → native s6 → `FDS>`. The production image is quiet, so a missing +splash or scrolling kernel log is not by itself a failure. At **FDS**, run: + +```sh +fds info +cat /usr/share/fds/image-profile +cat /proc/cmdline +findmnt -no SOURCE,FSTYPE,OPTIONS / +ps -p 1 -o pid,comm,args +fds machine status +fds bays +fds topology +fds boot-profile +``` + +Expect the intended version/profile, an EROFS root mounted read-only, and native +s6 as PID 1. `fds machine status` should report `internal_nvme` for valid internal +settings; `image_defaults` means settings loading fell back and needs diagnosis. +An empty initial map still legitimately reports `UNCONFIGURED` bays. Stage0 +discovers SYSTEM by its GPT partition label before bay mapping, so that does not +prevent reaching the console. Do not unplug the running SYSTEM even if it has +not yet acquired a bay number. + +If no SYSTEM is found, stage0 waits for media. Its prompt accepts `list`, +`rescan`, `recovery`, `reboot` and `poweroff`; inserting one valid SYSTEM should +resume boot. Two matching SYSTEMs are ambiguous: remove the unintended one. +This prompt is an early-boot command interface, not Bash. + +The normal console is the ordinary `fds` user (UID 1000), with a temporary home +and locked account passwords. For root diagnostics, shut down with `fds poweroff`, +remove SYSTEM after shutdown, boot again and type `recovery` at stage0's +missing-SYSTEM prompt. The independent recovery image opens `RECOVERY#`. +Use that root console for monitor diagnostics and initial machine configuration; +there is no default root password to enter at `FDS>`. + +## 7. Verify HDMI and the Dasung controller + +At **`RECOVERY#`**, with both HDMI and USB data connected: + +```sh +lsusb +lsusb -t +dasungd status +dasungd query +tail -80 /run/log/dasungd/current +``` + +The existing profile expects UART `1a86:7523` and companion bridge `1a86:5512` +under the same USB hub, plus a connected DRM output matching the configured +monitor serial. Query again and check status after replies arrive: `connected` +alone is not the same as `responsive`. See [Dasung diagnostics](Dasung) for +DRM capture, blank HDMI, missing USB interfaces and the 37 Hz override. + +Confirm visible text entry, a refresh, and parameter readback separately. Once +basic operation works, test a **monitor-only** off/on cycle while retaining Pi +power, SYSTEM and hub power; record the picture, USB re-enumeration and daemon +reconnection. This is not permission to unplug writable storage. A passing VM +controller test cannot establish physical monitor recovery. + +## 8. Install the first bay map without a DATA cartridge + +Stay at **`RECOVERY#`**. Keep the hubs in their intended final ports. Insert one +identifiable test USB device at a time and record `fds topology` for each physical +slot. Test USB 2 and USB 3 connections separately where supported. Map only the +slots currently assembled; unmapped slots can remain unconfigured. + +Create editable settings in RAM: + +```sh +fds machine export /tmp/my-machine +cat /tmp/my-machine/machine.toml +fds topology +``` + +Recovery includes Bash but no full-screen editor. Use a here-document to enter +the measured map. **The path below is a syntax example, not a Pi port identity**: +replace it and the port number with your actual observation before installing. +If the observed device path ends in `:usb2/1/3`, the parent hub ends in +`:usb2/1` and downstream port `3` maps to your chosen bay number: + +```sh +cat > /tmp/my-machine/bays.toml <<'EOF' +[first_usb2] +hub = "platform/REPLACE_WITH_OBSERVED_CONTROLLER:usb2/1" +[first_usb2.ports] +3 = 1 +EOF +``` + +Add separate groups for the measured USB 3 aliases and other hubs following +[the mapping reference](Developer-Cartridges#calibrate-the-physical-bay-map). +Do not map keyboard/monitor connections as cartridge bays or copy the emulator's +map. Validation checks syntax and conflicts; only your insertion tests establish +that a mapping matches the physical labels. + +```sh +cat /tmp/my-machine/bays.toml +fds machine validate /tmp/my-machine +fds machine install /tmp/my-machine +fds reboot +``` + +Installation writes the complete settings bundle to NVMe and retains the previous +one; it does not change the active mapping during this boot. With SYSTEM still +absent, select `recovery` again after reboot. Verify `fds machine status`, then +`fds bay 01` as you move the test device through the configured slots. Use +`fds eject N` and wait for SAFE before removing storage managed in a mapped bay. + +Later, [export a backup and update settings from DATA](Internal-Storage#update-settings-from-recovery). +Do not reflash the whole NVMe merely to change bay numbering: that would replace +saved settings and diagnostics. For normal boot, shut down, insert SYSTEM, +and power on again. + +## 9. Save evidence and shut down + +At **`RECOVERY#`**, collect a fresh capture and explicitly save it to NVMe before +losing the RAM filesystem: + +```sh +bash /usr/share/fds/capture-hardware /tmp/hardware-first +cat /tmp/hardware-first/status.tsv +dasungd status > /tmp/hardware-first/dasung-status.json +cp /run/log/dasungd/current /tmp/hardware-first/dasung.log +tar -C /tmp -czf /tmp/hardware-first.tar.gz hardware-first +fds machine store hardware-first.tar.gz /tmp/hardware-first.tar.gz +fds power status +fds poweroff +``` + +Stored names must be new and files must be at most 16 MiB. The collector's +`SHA256SUMS` covers its own original outputs; the two manually added Dasung files +are additional evidence. Retrieve the archive later with +`fds machine fetch hardware-first.tar.gz /tmp/retrieved-hardware-first.tar.gz` +from recovery, then copy it to healthy, explicitly activated DATA and safely eject. +This capture describes the recovery boot; collect another capture in normal FDS +on DATA when that path is ready. `/tmp`, `/run`, logs and the normal home are +otherwise lost at shutdown. + +Fill the workstation session record with observed results, wiring, EEPROM version, +image hashes and remaining failures. Leave unperformed cases `not_run`. Log +events measure software stages; physical power-on-to-picture and power-off timing +need an external observation. Follow [physical acceptance testing](Developer-Stress-Testing#physical-acceptance-sequence) +when the complete assembly is ready. + +Use `fds poweroff` and wait for shutdown before disconnecting power or swapping +SYSTEM. If shutdown is blocked, inspect `fds power status` and resolve the +reported storage problem. An E-Ink picture can remain visible after power is +removed, so the retained picture is not proof that the Pi is still running. +After basic console/display checks, try the [optional desktop](Desktop) and +[cartridge workflow](Cartridges). + +## When the first boot fails + +| Observation | Next check | +| --- | --- | +| Firmware cannot find NVMe | Adapter/ribbon seating, supply, EEPROM boot order, and conditional `PCIE_PROBE=1`; return to the maintenance SD | +| Stage0 says SYSTEM is missing | Whole GPT SYSTEM image, exactly one `FDS_SYSTEM`, USB power and `list`; bay calibration is not required at this stage | +| HDMI is blank or unstable | Monitor input/power, cable connector sizes, then [native HDMI EDID](Dasung#try-the-monitors-own-hdmi-edid) or the serial console below | +| Kernel text appears but keyboard cannot reach a prompt | Check whether `BOOT_MODE=development` made UART the primary console | +| Picture works, `dasungd` does not | USB data cable, both USB IDs, DRM serial, permissions and daemon log; see [Dasung diagnostics](Dasung) | +| `fds bays` is unconfigured | Complete step 8 using physical topology observations | +| `fds machine status` uses image defaults | Check the reported reason, NVMe transport, partition layout and clean internal ext4; do not run DATA repair against internal NVMe | +| USB resets or disappears under load | Reduce attached devices and check the Pi, monitor and hub supplies separately | + +### Use a serial console when HDMI is unavailable + +Use the Pi 5's dedicated debug UART connector and a compatible 3.3 V UART/debug +probe, following the [official UART documentation](https://www.raspberrypi.com/documentation/computers/configuration.html#configuring-uarts). +Connect the probe's host USB to the workstation, then open its serial terminal at +**115200 baud, 8 data bits, no parity, 1 stop bit, no flow control**. This is a +separate connection from the monitor's USB cable. Save the terminal log. + +For an initial, still-disposable disk build, replace step 3's boot mode with +`make boot-volume BOOT_MODE=development` before `make internal-image`. This +enables verbose kernel output and selects `ttyAMA10` as the primary userspace +console. The development EEPROM profile separately enables firmware UART output. +Boot mode does not grant root; select recovery for a root shell. + +For an already provisioned NVMe, use the +[offline BOOT edit procedure](Dasung#edit-the-boot-command-line-offline) and +copy the single line from [cmdline-development.txt](http://gitea.home.arpa/felis/fds-os/src/branch/main/image/pi5/cmdline-development.txt) +into the disk's `cmdline.txt`; preserve any deliberate native-EDID diagnostic +change. This avoids overwriting settings to switch consoles. Restore the saved +command line to return to display/keyboard operation after debugging. diff --git a/Home.md b/Home.md new file mode 100644 index 0000000..33303fe --- /dev/null +++ b/Home.md @@ -0,0 +1,35 @@ +--- +include_toc: true +--- + +# FDS/OS user manual + +FDS/OS runs a cartridge-based Linux computer. Start with the overview, then choose +the guide for building software, trying the emulator, or operating the machine. +Commands in workstation guides run on your Linux computer; commands in the +operating guides run at the FDS console unless stated otherwise. + +This wiki is the authoritative manual. Edit these pages directly in Gitea or +in the `fds-os.wiki` checkout; see [Editing the wiki](Gitea-Wiki). +Use the sidebar or [all pages](Page-Index) to find a guide. + +| Guide | What you will learn | +| --- | --- | +| [Overview](Overview) | How the computer, internal storage and cartridges fit together | +| [Run on a Raspberry Pi 5](Hardware-Setup) | Wire HDMI/USB/power, prepare NVMe and SYSTEM, reach the first console, calibrate bays and diagnose hardware | +| [Build and start FDS](Getting-Started) | Prepare the OS build workstation, create images and boot the emulator | +| [Software and emulator walkthrough](Workstation) | Build Void source packages, assemble cartridges, insert them and write USB media | +| [Using cartridges](Cartridges) | Inspect bays, run applications, resolve command names and eject safely | +| [DATA and files](Data) | Use persistent storage and manage active DATA | +| [Desktop and control panel](Desktop) | Open WindowMaker, manage cartridges graphically and customize the theme | +| [Software format](Software-Format) | Write source recipes and understand the installed-program image layout | +| [Internal storage](Internal-Storage) | Install boot/recovery storage and manage machine settings | +| [Recovery](Recovery) | Inspect failed media and check or repair DATA | +| [Release verification](Releases) | Verify downloaded images before use | +| [Dasung display](Dasung) | Inspect and control the built-in Paperlike daemon | +| [Cleanup](Cleanup) | Reclaim obsolete build output without losing current images or personal data | +| [Troubleshooting](Troubleshooting) | Diagnose build, cartridge, desktop and emulator problems | +| [Gitea wiki](Gitea-Wiki) | Edit documentation directly in the wiki | + +Implementation plans, dependency rationale, measurements, hardware procedures and +historical test results are maintained separately in [developer notes](Developer-Guide). diff --git a/Internal-Storage.md b/Internal-Storage.md new file mode 100644 index 0000000..5662295 --- /dev/null +++ b/Internal-Storage.md @@ -0,0 +1,247 @@ +--- +include_toc: true +--- + +# Internal storage and machine settings + +[Documentation index](Home) · [Boot images](Developer-Boot) · [Recovery](Recovery) + +Internal storage contains boot files, an independent recovery system and +persistent machine settings. The operating system itself lives on a removable +SYSTEM cartridge. This guide covers preparing the internal disk and updating +its settings from recovery. + +For the complete first-install sequence, including cabling, EEPROM and a first +bay map without DATA, use [the real-hardware setup guide](Hardware-Setup). + +## Build the complete disk image + +Build the component images first, in this order: + +```sh +make kernel +make initramfs +make boot-volume BOOT_MODE=production +make recovery +make internal-image +``` + +The result is `out/fds-internal.img`, a **complete GPT disk image**. Its versioned +build directory contains `layout.json`, source payload checksums, and readback +verification. This differs from `out/fds-boot.img` (a raw FAT partition) and +`out/fds-recovery.img` (a raw EROFS partition). + +| Partition | Format | Default allocation | Purpose | +| --- | --- | --- | --- | +| 1: FDS_BOOT | FAT32, EFI system type | 512 MiB | Pi firmware, kernel, DTBs, configuration and initramfs | +| 2: FDS_RECOVERY | EROFS | 1 GiB | Complete independent maintenance system | +| 3: FDS_INTERNAL | ext4 | 256 MiB | Machine settings and explicitly saved diagnostics | + +The builder pads the recovery partition without changing its EROFS contents, +aligns partitions to 1 MiB, writes both GPT copies, verifies each payload, and +checks the table independently with `sfdisk`. Internal ext4 is fully initialized +before deployment. It contains root-owned `config/` and private `diagnostics/`. +Unused capacity beyond this initial layout is not automatically expanded. + +For a 2 GiB recovery allocation or larger settings partition: + +```sh +./image/build-internal --recovery-mib 2048 --internal-mib 512 +``` + +## Install the internal disk + +Use an NVMe enclosure or another Linux machine that can access the target drive +while the Pi is off. First verify a downloaded release using a separately trusted +public key as described in [Release signatures](Releases). Its complete disk +is named `fds-internal-0.1.0.img`; a local build uses `out/fds-internal.img`. +Use the complete disk image, not the separate BOOT or RECOVERY payload. + +List disks before and after connecting the intended drive: + +```sh +lsblk -d -o NAME,PATH,MODEL,SERIAL,SIZE,TRAN,LOG-SEC +ls -l /dev/disk/by-id/ +``` + +Match the physical model, serial and capacity. Choose a persistent **whole-disk** +`/dev/disk/by-id/` path without a `-partN` suffix; never guess a `/dev/sdX` name. +Unmount every target partition and disable any swap on it. Do not select the +workstation's system disk. The current images require 512-byte logical sectors +and a disk at least as large as the image; 4 KiB logical-sector media is not +supported by this layout. + +Open Bash (`bash`), replace both paths below, then run the block. It checks the +selected disk again, requires a typed confirmation, writes and reads back the +complete image, and relocates the backup GPT when the disk is larger than the +image. It does not expand any partition or filesystem. + +```bash +( + set -euo pipefail + fds_image="$PWD/out/fds-internal.img" + fds_disk=/dev/disk/by-id/REPLACE_WITH_THE_TARGET_DISK + [[ -f "$fds_image" && -s "$fds_image" && -b "$fds_disk" ]] + [[ $(lsblk -dnro TYPE "$fds_disk") == disk ]] + [[ $(sudo blockdev --getss "$fds_disk") == 512 ]] + fds_bytes=$(stat -Lc %s "$fds_image") + (( $(sudo blockdev --getsize64 "$fds_disk") >= fds_bytes )) + lsblk -p -o NAME,TYPE,MODEL,SERIAL,SIZE,MOUNTPOINTS "$fds_disk" + if lsblk -nrpo MOUNTPOINTS "$fds_disk" | grep '[^[:space:]]' >/dev/null; then + echo 'Target has mounted filesystems or swap; stop and release them first.' >&2 + exit 1 + fi + if lsblk -nrpo TYPE "$fds_disk" | grep -Ev '^(disk|part)$' >/dev/null; then + echo 'Target has device-mapper or other active descendants; stop.' >&2 + exit 1 + fi + read -r -p "Type ERASE $fds_disk to erase this disk: " fds_confirmation + [[ "$fds_confirmation" == "ERASE $fds_disk" ]] + sudo dd if="$fds_image" of="$fds_disk" bs=4M conv=fsync status=progress + sudo blockdev --flushbufs "$fds_disk" + sudo cmp -n "$fds_bytes" "$fds_image" "$fds_disk" + sudo sfdisk --lock=yes --relocate gpt-bak-std "$fds_disk" + sudo sfdisk --verify "$fds_disk" + sudo blockdev --flushbufs "$fds_disk" +) +``` + +Every command must succeed. The byte comparison occurs before relocating GPT, +because relocation intentionally changes disk-table headers. GNU `dd`'s `fsync` +flushes output before it returns; `sfdisk`'s `gpt-bak-std` moves the backup header +to the end of the target. See the [GNU dd manual](https://www.gnu.org/s/coreutils/manual/html_node/dd-invocation.html) +and [sfdisk manual](https://man7.org/linux/man-pages/man8/sfdisk.8.html). +These commands use the already documented coreutils, diffutils and util-linux +host tools; they do not add a target daemon. + +Confirm that the disk shows `FDS_BOOT`, `FDS_RECOVERY` and `FDS_INTERNAL` in that +order using `lsblk -o NAME,PARTLABEL,FSTYPE,MOUNTPOINTS`. Unmount anything the +desktop automatically mounted, then safely disconnect the enclosure and install +the NVMe in the powered-off Pi. Follow [EEPROM preparation](EEPROM) for the +separately reviewed NVMe boot settings. + +Prepare the **first SYSTEM cartridge on the workstation** using the same guarded +write block, with its two path assignments changed to the chosen SYSTEM image +and a different, empty USB cartridge disk. For a local build select +`out/fds-system-cli.img` or `out/fds-system-development.img`; release files are +`fds-system-cli-0.1.0.img` and `fds-system-development-0.1.0.img`. They are +alternative complete GPT images, each containing one `FDS_SYSTEM` partition. +Readback and backup-GPT relocation apply to this disk too. Check its partition +label, safely disconnect it, and insert exactly one SYSTEM cartridge in the Pi +before normal boot. Subsequent cartridge creation and updates can use FDS's +confirmed [media workflow](Developer-Media-Tools) after bay calibration. + +## Configure the machine before building + +Copy `config/machine/` to your own directory. It contains three files: + +- `machine.toml`: `format = 1` and a short human-readable `name`. +- `bays.toml`: the measured controller/port map described in + [Cartridges and bay calibration](Cartridges). +- `hardware-catalog.toml`: optional USB identification names using the same schema + as the base catalog. Entries are data and cannot run commands. + +The supplied bay map is empty. Record actual controller/port identities for +each physical slot before installing settings. USB 2 and USB 3 companion ports +need explicit aliases for the same bay. + +```sh +cp -a config/machine out/my-machine +# Edit the three files in out/my-machine using your editor. +make internal-image MACHINE_CONFIG=out/my-machine +``` + +The builder compiles a native host copy of `fds` and uses the same strict parser +as the target system. You can also validate or pack settings yourself: + +```sh +cargo build --locked --offline --release --target x86_64-unknown-linux-gnu -p fds-cli +./target/x86_64-unknown-linux-gnu/release/fds machine validate out/my-machine +./target/x86_64-unknown-linux-gnu/release/fds machine pack out/my-machine out/my-machine.json +``` + +The three source files become one atomic `config/machine.json` document on +FDS_INTERNAL. Names, lengths, bay aliases, catalog fields and unknown keys are +validated before use. Do not put passwords or private keys in this configuration: +its active snapshot is readable by the local FDS user. + +## What happens at boot + +The `machine-config` native s6 oneshot precedes `cartridged`. It does **not** +precede the console or Dasung controller. It accepts exactly one non-removable +NVMe disk with the three named partitions in the order above. USB lookalikes are +ignored; multiple eligible NVMe disks are rejected instead of choosing by name. + +The settings partition must be clean ext4 with the expected label. Loading uses +`ro,noload,nosuid,nodev,noexec` in a private mount namespace, validates the entire +settings bundle, copies it into `/run/fds/machine/`, and unmounts. No filesystem +repair, journal replay, cache compilation or persistent write occurs during +normal boot. The cartridge service uses that snapshot throughout this boot, +including after a service restart. + +If internal storage is missing, unclean, invalid or ambiguous, the service records +an explanation and uses the immutable image's `/etc/fds/` defaults. The console +still opens. Check the actual source before treating bays as calibrated: + +```sh +fds machine status +fds --json machine status +fds machine export /tmp/current-machine +``` + +Export creates a new directory with the three editable source files. It never +overwrites an existing directory. The status source is `internal_nvme` or +`image_defaults`. A temporarily unavailable NVMe is not adopted later in the same +boot; resolve the issue and reboot to load its settings. This prevents changing +bay identities underneath active cartridge operations. + +## Update settings from recovery + +Bring the edited source directory on a DATA cartridge. At the local root +`RECOVERY#` console, identify its bay with `fds bays`; healthy DATA is mounted +read-only under `/run/fds/media/NN`. For example, if it is in BAY 02: + +```sh +fds machine validate /run/fds/media/02/my-machine +fds machine install /run/fds/media/02/my-machine +fds reboot +``` + +Installation is restricted to root in the recovery image. It writes the complete +validated bundle atomically, saves the old bytes as `config/previous.json`, +flushes and unmounts the internal filesystem, and reports success only after +those steps. A reboot activates the new settings. The currently running bay map +is unchanged, so active media does not move to a different bay mid-operation. +Keep a copy of your previous source directory on DATA or the build host to +reinstall it if the new calibration is wrong. + +An invalid existing JSON document can be replaced this way. Wrong ownership, +symlinks, an unclean filesystem or a damaged directory require offline filesystem +maintenance first. Recovery does not automatically repair internal NVMe; its +`fds recovery repair` command is deliberately limited to DATA cartridges. + +## Save and retrieve diagnostics + +Logs and boot records stay in RAM by default. Root may explicitly save a file +of up to 16 MiB. Saved names cannot contain paths, and existing names are refused. +These operations mount internal ext4 only for the operation and then unmount it. +For example, from recovery: + +```sh +fds --json boot-profile >/tmp/boot.json +fds machine store boot-first.json /tmp/boot.json +fds --json bays >/tmp/cartridge-inventory.json +fds machine store cartridges-first.json /tmp/cartridge-inventory.json +bash /usr/share/fds/capture-hardware /tmp/hardware-capture +tar -C /tmp -czf /tmp/hardware-capture.tar.gz hardware-capture +fds machine store hardware-first.tar.gz /tmp/hardware-capture.tar.gz +fds machine fetch boot-first.json /tmp/retrieved-boot.json +``` + +Use distinct names for subsequent sessions. A failed flush or unmount is an +error, not a successful save. These are machine diagnostics; do not use this +facility as ordinary user storage. The saved cartridge inventory is a persistent +diagnostic snapshot, including metadata already inspected during this boot. +Retrieve it with `fds machine fetch cartridges-first.json /tmp/saved-inventory.json`. +The daemon's live metadata cache remains volatile and is rebuilt from currently +attached devices; saved snapshots are never used to authorize media actions. diff --git a/Overview.md b/Overview.md new file mode 100644 index 0000000..e97ee82 --- /dev/null +++ b/Overview.md @@ -0,0 +1,84 @@ +--- +include_toc: true +--- + +# FDS/OS + +FDS/OS is a Linux operating system for the Felis FP-85, a Raspberry Pi 5 computer +with twelve USB cartridge bays and a Dasung Paperlike 13K display. It combines +Void Linux packages, native s6 services and static Rust system tools with a +simple cartridge workflow: insert software, run it, and eject it when finished. + +## How the system fits together + +| Component | Purpose | +| --- | --- | +| Internal NVMe | Holds boot files, recovery and persistent machine settings | +| SYSTEM cartridge | Supplies the read-only operating system; swapping it changes the OS | +| PROGRAM cartridge | Supplies applications and their dependencies, ready to execute | +| DATA cartridge | Stores writable files at `/data` | +| ENVIRONMENT cartridge | Selects an installed environment, such as WindowMaker | + +The default interface is a console. WindowMaker provides an optional grayscale +retro desktop and an FDS Control panel for inspecting bays, launching programs +and ejecting cartridges. The same theme is used on ordinary screens and E-Ink. +The Dasung controller is part of the base system and starts independently of +the desktop or any ENVIRONMENT cartridge. + +## Choose a starting point + +- **Browse the manual:** start at [Home](Home), use the sidebar, or browse [all pages](Page-Index). +- **Run on your Raspberry Pi 5:** follow [the real-hardware setup guide](Hardware-Setup), including HDMI-plus-USB Dasung wiring, disk installation and first boot. +- **Build and try the emulator:** follow [Build and start FDS](Getting-Started). +- **Build an application cartridge:** use the [software and emulator walkthrough](Workstation). +- **Use the computer:** read [Using cartridges](Cartridges), [DATA](Data), and [the desktop guide](Desktop). +- **Install or maintain a machine:** see [internal storage](Internal-Storage), [release verification](Releases), and [recovery](Recovery). +- **Free build space:** run `make clean-preview`, then `make clean`; see [cleanup](Cleanup). + +The [user manual](Home) is organized around tasks. Build internals, +design decisions, hardware procedures and validation records live separately in +[developer notes](Developer-Guide). + +## A typical software session + +These commands run on FDS after inserting the example PROGRAM cartridge in bay 1: + +```sh +fds bays +fds bay 01 +hello 'Hello from FDS' +fds run 01 -- demo.report:report +fds eject 01 +``` + +`hello` runs in your current terminal. `fds run` starts a managed background +command. Both forms are tracked for eject. Wait for **SAFE** before removing +media; use `fds poweroff` to shut down the whole computer. + +## Build tools + +Run these from the repository root: + +| Command | Result | +| --- | --- | +| `make wiki-check` | Check the documentation in `fds-os.wiki` | +| `make workstation` | Arch `fds-tools` package containing both workstation commands | +| `make workstation-install` | Install that package using pacman | +| `make workstation-binaries` | Native commands for other Linux distributions | +| `make bootstrap` | Prepares the complete OS build environment on x86_64 Arch Linux | +| `make rootfs PROFILE=cli` | Builds the ARM base filesystem and its packages | +| `make system-card PROFILE=cli` | Creates a whole-disk SYSTEM cartridge image | +| `make initramfs` | Creates early boot userspace | +| `make all` | Builds the complete CLI/development, recovery and internal image set | +| `make clean` | Removes obsolete generated workspaces and Rust build output | + +New software cartridges are built from metadata and Void source packages using +`xbps-src` on a Linux workstation. Their EROFS partitions contain installed +programs and dependencies directly. FDS does not extract those programs at launch. + +The workstation walkthrough includes prerequisites, complete example commands, +expected outputs, emulator lifecycle and the confirmed USB-writing workflow. +Append `--help` to either native tool for its command reference; run `make help` +for the complete build target list. + +Build identities come from Git release tags. See [versioning](Developer-Versioning) for release, development and source-export behavior. diff --git a/Page-Index.md b/Page-Index.md new file mode 100644 index 0000000..0ede5ea --- /dev/null +++ b/Page-Index.md @@ -0,0 +1,84 @@ +--- +include_toc: true +--- + +# All documentation pages + +## User manual + +- [Build and start FDS](Getting-Started) +- [Build software cartridges and use the emulator](Workstation) +- [Dasung Paperlike display](Dasung) +- [DATA and persistent files](Data) +- [FDS/OS user manual](Home) +- [FDS/OS](Overview) +- [Internal storage and machine settings](Internal-Storage) +- [Pi 5 EEPROM configuration](EEPROM) +- [Editing the Gitea wiki](Gitea-Wiki) +- [Reclaim build space](Cleanup) +- [Recovery and rollback](Recovery) +- [Run FDS on a Raspberry Pi 5](Hardware-Setup) +- [Software cartridge format](Software-Format) +- [Troubleshooting](Troubleshooting) +- [Use cartridges and run programs](Cartridges) +- [Verify a release](Releases) +- [WindowMaker and FDS Control](Desktop) + +## Developer references + +- [Boot images and stage0](Developer-Boot) +- [Build software cartridges and run FDS in QEMU](Developer-Workstation) +- [Build versions and the workstation package](Developer-Versioning) +- [Building and inspecting packages](Developer-Packages) +- [Building and using the ARM root filesystem](Developer-Rootfs) +- [Cartridge and desktop revision](Developer-Current-Revision) +- [Cartridge image tools](Developer-Media-Tools) +- [Clean upstream and writable Void workspace](Developer-Void-Workspace) +- [Dasung Paperlike 13K base-system integration](Developer-Dasung) +- [Desktop, programs, and networking](Developer-Desktop) +- [Developer notes](Developer-Guide) +- [FDS Rust tools](Developer-Tooling) +- [FDS/OS MASTER IMPLEMENTATION PLAN](Developer-Master-Plan) +- [Frozen inputs and offline rebuilds](Developer-Reproducible-Builds) +- [Glossary](Developer-Glossary) +- [Implementation through M12](Developer-Implementation-Status) +- [Internal storage and machine settings](Developer-Internal-Storage) +- [M0: Arch Linux x86_64 build host](Developer-Build-Host) +- [M2: native s6 init and the ARM development VM](Developer-Init) +- [M4 boot integration development record](Developer-M4-Work) +- [Performance targets and measurement](Developer-Performance) +- [Pi 5 EEPROM configuration](Developer-EEPROM) +- [Reclaim build space](Developer-Cleanup) +- [Recovery and rollback](Developer-Recovery) +- [Release signatures and verification](Developer-Releases) +- [Roadmap and current status](Developer-Roadmap) +- [Services, readiness, and shutdown](Developer-Services) +- [Shutdown and reboot](Developer-Power) +- [Software cartridge format 1](Developer-Software-Format) +- [Troubleshooting the build](Developer-Troubleshooting) +- [Twelve-bay stress tests](Developer-Stress-Testing) +- [Understanding FDS/OS](Developer-Architecture) +- [Using cartridges and configuring bays](Developer-Cartridges) +- [Working on FDS/OS](Developer-Development) +- [Workstation tooling plan](Developer-Workstation-Tooling-Plan) +- [Writable DATA and managed programs](Developer-Data) +- [Your first build](Developer-Getting-Started) + +## Acceptance history + +- [Clap command-line migration validation](Developer-CLAP-Validation) +- [Dasung integration validation](Developer-Dasung-Validation) +- [M0 validation report](Developer-M0-Validation) +- [M1 root filesystem validation](Developer-M1-Validation) +- [M2 native init validation](Developer-M2-Validation) +- [M3 static Rust tooling validation](Developer-M3-Validation) +- [M4 software boot validation](Developer-M4-Validation) +- [M5 software validation](Developer-M5-Validation) +- [M6 cartridge software validation](Developer-M6-Validation) +- [M7 writable DATA validation](Developer-M7-Validation) +- [M8 desktop and network validation](Developer-M8-Validation) +- [M9 media-tool validation](Developer-M9-Validation) +- [M10 shutdown validation](Developer-M10-Validation) +- [M11 stress validation](Developer-M11-Validation) +- [M12 production preparation: software evidence](Developer-M12-Validation) +- [Workstation extension acceptance](Developer-Workstation-Validation) diff --git a/Recovery.md b/Recovery.md new file mode 100644 index 0000000..00a4f76 --- /dev/null +++ b/Recovery.md @@ -0,0 +1,184 @@ +--- +include_toc: true +--- + +# Recovery and rollback + +[Documentation index](Home) · [Boot images](Developer-Boot) · [Write cartridges](Workstation#write-a-cartridge-to-usb) · [EEPROM](EEPROM) + +FDS recovery is an independent maintenance system on internal storage. It can +inspect a failed SYSTEM, check DATA and help prepare replacement cartridges. + +For a machine whose bays have never been configured, follow +[initial bay calibration](Hardware-Setup#8-install-the-first-bay-map-without-a-data-cartridge) +at the root recovery console before relying on a DATA cartridge for settings. + +## Build the recovery image + +Use the prepared x86_64 build host described in [Your first build](Getting-Started). +From the repository root, run these commands sequentially: + +```sh +make recovery +``` + +`make recovery` builds the `recovery` rootfs profile independently, configures all +packages and caches at image construction time, then creates EROFS. Its outputs +are: + +| File | Meaning | +| --- | --- | +| `out/rootfs-recovery.tar` | Complete recovery rootfs, including package database and compiled s6 configuration | +| `out/fds-recovery.img` | Raw EROFS payload for the internal GPT partition named `FDS_RECOVERY` | +| `out/recovery-build.*/manifest.json` | Rootfs hash, filesystem hash, profile and size | +| `out/manifests/rootfs-latest/` | Exact selected package archives and build input records | + +The latest-rootfs pointers also select this recovery build. Profile-specific +rootfs pointers preserve earlier CLI/development builds. A recovery EROFS payload +is not a whole-disk image and is not a SYSTEM cartridge. The SYSTEM builder +rejects a recovery rootfs to prevent confusing those roles. + +## Enter recovery + +Stage0 can enter recovery in two ways: + +1. At its missing-SYSTEM or ambiguous-SYSTEM console, type `recovery` and Enter. +2. For a deliberate maintenance boot, use `fds.boot=recovery` in the boot + partition's single-line `cmdline.txt`, replacing `fds.boot=normal` if present. + Restore normal mode when maintenance is complete. + +Stage0 requires exactly one readable `FDS_RECOVERY` partition and mounts it +read-only. It does not silently choose between duplicate partitions. + +The local prompt is: + +```text +FDS RECOVERY — LOCAL MAINTENANCE CONSOLE +RECOVERY# +``` + +This is an explicit **root maintenance console**. It reads startup files from the +immutable recovery image and uses a temporary home under `/run/fds`. The root +password remains locked; no SSH or network login service is started. Normal CLI +and development images continue to use the ordinary `fds` account and `FDS>`. + +Recovery inspects cartridges read-only. It does not automatically mount DATA +writable, activate an ENVIRONMENT desktop, or enable Ethernet. Those actions +require explicit commands. The Dasung controller remains in the base boot bundle. + +## Inspect a failed system + +```sh +fds info +fds bays +fds bay 2 +fds inspect BAY02 +fds topology +fds boot-profile +lsusb -t +lsblk -o NAME,MAJ:MIN,MODEL,SERIAL,SIZE,RO,TYPE,FSTYPE,PARTLABEL,MOUNTPOINTS +dmesg +``` + +Use the calibrated bay number and the displayed model/serial to identify a +cartridge. Do not infer a bay from `/dev/sda` or from discovery order. A valid +inactive SYSTEM mounts read-only under `/run/fds/media/NN`; `fds bay N` reports +the actual path. A bad filesystem or manifest produces an error instead of +running anything from the cartridge. + +## Check and repair DATA + +Start with a read-only check. For example, for DATA in bay 2: + +```sh +fds recovery check BAY02 +``` + +This command is available only to root in the recovery image. It identifies one +USB disk and one `FDS_DATA` partition, unmounts its read-only view, reserves the +disk exclusively, verifies GPT and the kernel partition identity, and invokes +`e2fsck -f -n`. It does not repair the filesystem. A clean result leaves DATA +unmounted and reports `SAFE TO REMOVE`. + +If the check reports problems, review its log and preserve a backup where +possible. Preview the repair with: + +```sh +fds recovery repair BAY02 +``` + +The preview displays the model, serial, capacity and a confirmation command. +Copy that exact command only after checking the intended cartridge. Its token +binds the bay, kernel insertion number and current boot. A token from another +insertion or boot is rejected. The preview makes no filesystem changes. + +Confirmed repair runs `e2fsck -f -p`, which performs conservative automatic +repairs and stops when manual judgement is required. It then flushes the device, +invalidates its block cache, and runs a second `e2fsck -f -n`. Only a successful +verification produces `DATA REPAIRED AND VERIFIED` and `SAFE TO REMOVE`. + +Failed or interrupted checks/repairs retain a quarantine record across cartridge +daemon restarts for the same insertion. They do not inherit an earlier SAFE +status. The checker holds the disk reservation and is killed if its supervising +daemon dies. The command runs synchronously: bay operations and orderly shutdown +wait for it to finish; the local shell remains usable. A client timeout is not +success and does not authorize removing media. Inspect the log and wait for the +service to finish before retrying. Logs are in `/run/fds/recovery/` and are +limited to 16 MiB per invocation. + +This narrow recovery command requires a readable GPT and primary ext filesystem +signature. It deliberately does not guess partition boundaries, recreate a +broken GPT, or force answers to destructive `e2fsck` questions. Such cases require +an offline backup and expert use of the included filesystem tools. It cannot +recover data that was never written to storage. + +For structured results, prefix the command with `fds --json`. After a successful +check or repair, remove/reinsert DATA to inspect it again. Recovery still keeps +it read-only until `fds data use N` is explicitly requested. + +## Prepare replacement SYSTEM media + +A known-good SYSTEM image must be supplied on separate source media. Do not use +a damaged image as an update source. For example, source DATA in bay 2 and an +unmounted replacement cartridge in bay 4: + +```sh +fds bay 2 +fds inspect image /run/fds/media/02/fds-system-cli.img +fds inspect BAY04 +fds burn system /run/fds/media/02/fds-system-cli.img BAY04 +``` + +Use the actual source mount shown by `fds bay 2`. If the destination already +contains a mounted cartridge, run `fds eject 4` first. The burn preview identifies +the source checksum, destination capacity/model/serial and confirmation command. +Follow [Media creation and writing](Developer-Media-Tools) for confirmation and status commands. +Completion requires device flush, readback and GPT verification. Recovery uses +the same protected writer as the main system: it refuses mounted destinations, +active root storage and disks containing internal FDS partition names. + +Keep the previous known-good SYSTEM cartridge. After a successful replacement +write, use `fds poweroff`, swap SYSTEM while powered off, and boot normally. +Rollback means restoring the previous SYSTEM cartridge; it does not undo DATA +file changes, application migrations, internal boot updates or EEPROM changes. + +## Capture diagnostics and shut down + +```sh +bash /usr/share/fds/capture-hardware /tmp/recovery-capture +cat /tmp/recovery-capture/status.tsv +fds power status +fds poweroff +``` + +The collector records identities, USB topology, mounts, kernel messages, packages +and boot events with checksums. Its output is temporary unless copied to a +healthy DATA cartridge after explicitly activating that DATA with `fds data use N`. +Do not activate the damaged cartridge merely to save a report. Eject writable +DATA after copying, or let `fds poweroff` perform the normal verified shutdown. +See [Shutdown](Developer-Power) when shutdown reports a blocking DATA error. + +Recovery is independent of SYSTEM, but it still depends on working internal +boot storage, the kernel and firmware. An external rescue medium is required +when that layer fails. EEPROM preparation and rollback are described separately +in [EEPROM configuration](EEPROM); no EEPROM is changed by these recovery tools. diff --git a/Releases.md b/Releases.md new file mode 100644 index 0000000..a32156a --- /dev/null +++ b/Releases.md @@ -0,0 +1,47 @@ +--- +include_toc: true +--- + +# Verify a release + +Verify a downloaded release before writing its images to a cartridge or internal +disk. Verification checks the release manifest signature and the length and +SHA-256 of every listed artifact. Obtain the signer's public key through a +trusted channel independent of the download. + +On a workstation, build the verifier with `make signing` after bootstrap. On FDS, +`fds-release` is included in the base system and recovery. + +## Verify before using images + +Obtain the signer's public key through a channel you already trust, or compare +its fingerprint against a separately authenticated value. A public key included +beside an untrusted download does not establish its identity by itself. + +On FDS, including recovery: + +```sh +fds-release verify /data/downloads/fds-release --key /data/keys/fds-release.pub +``` + +On the workstation, use the host executable instead: + +```sh +target/x86_64-unknown-linux-gnu/release/fds-release verify \ + /path/to/release-directory --key /path/to/trusted-fds-release.pub +``` + +Signature verification is an explicit pre-installation step. It is not enforced +by Pi firmware or stage0 during boot. + +Success reports `VERIFIED` and identifies the release. Failure returns a nonzero +exit code with an explanation. Keep files unchanged between verification and +writing. Files absent from the signed manifest are outside its scope. + +The cartridge writer separately checks image layout, destination identity and +readback. Release verification does not install images; follow the +[USB-writing workflow](Workstation#write-a-cartridge-to-usb) or +[internal installation](Internal-Storage). + +Release maintainers can find key creation, signing and format details in the +[developer reference](Developer-Releases). diff --git a/Software-Format.md b/Software-Format.md new file mode 100644 index 0000000..5be25a4 --- /dev/null +++ b/Software-Format.md @@ -0,0 +1,114 @@ +--- +include_toc: true +--- + +# Software cartridge format + +A PROGRAM cartridge is a complete GPT disk image with one EROFS metadata partition +and one or more EROFS payload partitions. New media uses catalogue format 2 and +stores installed Void package trees. Source recipes and build tools remain on +the workstation; FDS executes the installed files directly. + +## Software source recipe + +```toml +format = 2 +id = "example.hello" +name = "Hello" +version = "1.0" + +[commands] +hello = "usr/bin/hello" + +[source] +package = "hello" +``` + +`package` names a source package in the selected `void-packages` checkout. For a +custom package, also set `template = "void"`: that directory must contain a +normal Void `template` and any `files/` or `patches/` directories it uses. Paths +are relative to this recipe. The [included hello package](http://gitea.home.arpa/felis/fds-os/src/branch/main/examples/software/hello/void/template) +is a complete example. + +Custom sources are copied into `srcpkgs/PACKAGE` inside the selected writable +build checkout. FDS uses `.host/void-packages/`; its `vendor/void-packages/` +submodule remains unchanged. The builder rejects a Git submodule as its build +checkout. Existing +tracked Void sources are never overwritten. If a generated source copy differs, +inspect it, preserve any independent edits, then remove that generated copy and +retry. An unchanged copy can be reused. + +`fds-cartridge` builds with `xbps-src -a aarch64`, installs the resulting package +and runtime dependencies into a fresh tree, and records installed XBPS versions. +Use package dependency declarations to request runtime libraries and utilities. +Each software tree is self-contained with respect to its installed dependencies; +there is no cross-cartridge package resolver. + +Software IDs and public command names use lowercase ASCII letters, digits, +periods, hyphens and underscores, with a maximum length of 64. Command paths are +relative to the installed root and must resolve to executable files within it. + +## Cartridge recipe + +```toml +format = 2 +id = "example.tools" +name = "Example tools" +version = "1.0" + +[[payload]] +sources = ["hello/software.toml", "report/software.toml"] +``` + +Each payload section creates one partition. Sources may be recipe paths or +previously built directories containing `software.toml` and `root/`. The metadata +partition records which software belongs to which partition. Software IDs and +cartridge identity must satisfy the metadata validation rules. + +## On-disk contents + +```text +Partition 1: FDS_METADATA + FDS/CARTRIDGE.TOML + FDS/SOFTWARE.TOML +Partition 2: FDS_PAYLOAD02 + programs/example.hello/usr/bin/hello + programs/example.hello/usr/lib/... + programs/example.hello/var/db/xbps/... + programs/example.report/... +``` + +The software catalogue records identity, architecture, commands, partition, +installed package versions, entry count, byte count and SHA-256. `installed=true` +selects the direct-tree contract. The retained `unpacked_bytes` field records +installed file bytes; it does not imply an archive or guest extraction. + +The tree digest covers sorted paths, entry types, modes, contents and symlink +targets. Root-relative package symlinks are relocated to relative in-tree links +when building. Escaping links, special files, privileged/writeable executable +trees, wrong-architecture ELF files and inconsistent metadata are rejected. +Current per-tree bounds are 1 GiB of file data and 65,536 entries. + +## Runtime behavior + +FDS verifies the cartridge's geometry, identity, inventory and tree integrity +on insertion. Payloads are mounted read-only with `nosuid,nodev`. No software +command runs automatically. Published command aliases appear in `/run/fds/bin`. + +Programs run as UID/GID 1000 in the bay's managed process group. `FDS_APP` names +the installed tree; PATH, library paths and data search paths include its +`usr/bin`, `usr/lib` and `usr/share`. Dynamically linked ELF entry points use the +package tree's own loader. Programs that hard-code unrelated system paths still +need to be compatible with FDS; the tree is not a container or a writable root. + +Safe eject stops consumers and releases every payload before reporting SAFE. +A digest detects corruption, not publisher identity: choose software sources +you trust. Distribution signatures are covered by [release verification](Releases). + +## Older media + +The reader retains support for catalogue format 1 with xz payloads and legacy +single-partition PROGRAM cartridges containing `app/`. Old xz payloads retain +their temporary extraction behavior and limits. Creation commands produce only +the installed-tree format. Rebuild old software from its Void source recipe to +use direct execution. diff --git a/Troubleshooting.md b/Troubleshooting.md new file mode 100644 index 0000000..22c090c --- /dev/null +++ b/Troubleshooting.md @@ -0,0 +1,46 @@ +--- +include_toc: true +--- + +# Troubleshooting + +Read the complete command error and its log before retrying. Build logs are in +`out/logs/`; emulator `console.log` and `qemu.log` are in the selected session. +On FDS, `fds bay N`, `fds profiles` and `fds power status` report the relevant +operation state. Append `--help` to a command for accepted options. + +For physical Pi startup, use [first-boot troubleshooting](Hardware-Setup#when-the-first-boot-fails). +For the Paperlike on HDMI plus USB, use [Dasung diagnostics](Dasung#diagnose-a-connection-problem). + +| Symptom | Resolution | +| --- | --- | +| Build disk is full | Stop builds and VMs; run `make clean-preview`, then `make clean`. See [retention rules](Cleanup). | +| Missing host utility | Install the named utility or select the documented image/XBPS/QEMU runner. `doctor` checks image and emulator prerequisites. | +| Bubblewrap/user namespace failure | Check that the host permits unprivileged namespaces. Run the check in [setup](Getting-Started); do not switch software builds to root. | +| Void checkout is unprepared | Bootstrap the checkout using upstream xbps-src instructions, then select it with `--void-packages`. | +| Stale generated overlay/source package | Compare the source and generated copy. Preserve independent edits, remove only that generated copy, and retry. Never overwrite tracked upstream templates. | +| Output already exists | Use a new output directory, cartridge image or preview. A stopped emulator session can be restarted; see below. | +| Executable architecture or tree integrity error | Rebuild the source package and cartridge. FDS cannot run an invalid payload. | +| A command is unavailable | Wait for `MOUNTED READ ONLY`, inspect `fds --json bay N`, and use its qualified command alias. Run `hash -r` after removal if Bash retained an old path. | +| Empty or unconfigured physical bays | Run `fds topology` and calibrate the machine's bay map. Do not infer slots from disk names. | +| Duplicate cartridge ID | Give the second cartridge a distinct identity and rebuild it. | +| Console already in use | Detach the emulator console with Ctrl-] before `guest`, safe eject or shutdown. | +| Guest command timed out | Inspect the console log. It may still be running; do not repeat a write until its state is known. | +| Desktop activation fails | Inspect `fds profiles` and `/run/log/xserver/current`, `/run/log/desktop/current`. Check the display configuration. | +| Control panel cannot connect | Check `/run/log/cartridged/current` and that the cartridge service is running. The panel reconnects automatically. | +| Eject is blocked | Close programs, shells and extra mounts using the cartridge, then retry. Wait for SAFE before removal. | +| Shutdown is blocked | Read `fds power status`; resolve the reported storage problem and retry poweroff. `fds power resume` cancels a recoverable pending shutdown. | +| DATA needs checking | Use the [recovery workflow](Recovery); preserve a backup before repair where possible. | + +To inspect cartridge service output on FDS: + +```sh +tail -80 /run/log/cartridged/current +fds --json bay 01 +``` + +To inspect a stopped emulator, read its saved logs directly on the workstation. +Run `start` again with the same session directory and boot options after `stop`. +Logs and DATA overlays are retained; bays start empty. Saved DATA can be +[exported](Workstation#save-emulator-data). If restart reports a live process, +check `status` and stop the existing VM before retrying. diff --git a/Workstation.md b/Workstation.md new file mode 100644 index 0000000..15094fe --- /dev/null +++ b/Workstation.md @@ -0,0 +1,278 @@ +--- +include_toc: true +--- + +# Build software cartridges and use the emulator + +This walkthrough runs on a Linux workstation. It builds applications from Void +source packages, creates a complete cartridge image, runs it in FDS, and explains +how to write it to USB. Commands run from the FDS checkout root unless marked +as guest commands. + +## Prepare the tools + +The native tools are `fds-cartridge` and `fds-emulator`. On Arch Linux, build +and install their `fds-tools` package from the checkout: + +```sh +sudo pacman -S --needed base-devel rustup python git +make workstation +make workstation-install +``` + +`make workstation` produces `out/workstation/fds-tools-VERSION-1-ARCH.pkg.tar.zst` +with both commands, documentation, examples and license notices. It does not +install packages. `make workstation-install` builds that package and invokes +`sudo pacman -U` to install it into `/usr/bin`. Pacman tracks upgrades and removal; +remove it later with `sudo pacman -R fds-tools`. The exact latest package path is +recorded in `out/workstation/package-path.txt`. + +The package requires glibc, GCC runtime libraries and xz. EROFS, bubblewrap and +QEMU are optional package dependencies because the tools can use the project-local +Void runners instead. For all features using native Arch utilities, install: + +```sh +sudo pacman -S --needed bubblewrap erofs-utils qemu-system-aarch64 qemu-img +``` + +On other Linux distributions, install Rust, a C linker, Make, Python 3 and Git, +then build the same native commands without Arch packaging: + +```sh +make workstation-binaries +export PATH="$PWD/out/workstation:$PATH" +``` + +Python and Git resolve the build version; neither is needed to run the installed +Rust commands. The repository selects its pinned Rust toolchain. XBPS and a +prepared Void checkout are still separate source-build prerequisites. + +| Prerequisite | Used for | +| --- | --- | +| Prepared writable Void `void-packages` checkout and native XBPS utilities | Build source packages and install their AArch64 runtime dependencies | +| `mkfs.erofs`, `fsck.erofs` | Create and inspect read-only payload filesystems | +| `bwrap` | Inspect filesystems and install package trees in a user namespace | +| `xz` | Read older software cartridges and package/source archives | +| `qemu-system-aarch64`, `qemu-img` | Run the emulator and create writable DATA overlays | + +Install tools through your Linux distribution's package manager. Enable +unprivileged user namespaces for bubblewrap. Software builds run as an ordinary +user and execute trusted Void build templates on the workstation. Run source builds sequentially when they +share the same Void checkout. + +Prepare a Void source checkout using its [upstream instructions](https://github.com/void-linux/void-packages/blob/02a3cbc132c3c4a3a9d59e9b98f517af5dd11cd1/README.md). +Complete `xbps-src binary-bootstrap` for the workstation architecture before +using it. Keep this writable checkout outside Git submodules. The workstation itself need not run Void. Select that checkout with +`--void-packages /path/to/void-packages`; native `xbps-*` tools must be in PATH, +or selected with `--xbps-bin /path/to/bin`. Builds target AArch64/glibc. + +For the complete FDS build workstation, [bootstrap](Getting-Started) already +keeps `vendor/void-packages` as a clean upstream reference and prepares the +writable `.host/void-packages` checkout plus `.host/xbps/usr/bin`. The native +builder defaults to `.host/void-packages`. Reuse its local tools: + +```sh +./tools/prepare-image-tools +./tools/in-void xbps-install -y qemu-img +fds-cartridge --image-tool-runner tools/in-image-tools doctor +fds-emulator doctor --qemu-runner tools/in-void +``` + +An optional runner receives a utility name followed by its arguments. Image +runners select EROFS utilities; XBPS runners provide the package-installation +namespace. Without runners, the tools use native utilities and bubblewrap. + +## Build the example cartridge + +The repository includes hello and system-report Void source packages. Each +software recipe names a package, its source-template directory and its public +commands. A cartridge recipe groups those software recipes into payloads. + +With native tools and a prepared checkout: + +```sh +fds-cartridge --void-packages /path/to/void-packages \ + create examples/software/cartridge.toml out/demo-tools.img +fds-cartridge inspect out/demo-tools.img +``` + +With the project-local Void and image tools, use this equivalent command: + +```sh +fds-cartridge --image-tool-runner tools/in-image-tools \ + --xbps-tool-runner tools/in-void --xbps-bin "$PWD/.host/xbps/usr/bin" \ + create examples/software/cartridge.toml out/demo-tools.img +fds-cartridge --image-tool-runner tools/in-image-tools inspect out/demo-tools.img +``` + +Choose one route. Output paths must be new. The command builds each package +with `xbps-src`, installs the package and runtime dependencies into private +staging trees, verifies their architecture and integrity, and writes the complete +image. Source build logs go to standard error; the final result is JSON. + +The example image contains: + +| Partition | Contents | +| --- | --- | +| 1: `FDS_METADATA` | Cartridge identity and software catalogue | +| 2: `FDS_PAYLOAD02` | `programs/demo.hello/`, including `usr/bin/hello` and libraries | +| 3: `FDS_PAYLOAD03` | `programs/demo.report/`, including `usr/bin/report` and dependencies | + +There are no new xz software bundles. Programs are ready to run directly from +EROFS. Inspection validates both GPT tables, filesystem contents and software +hashes, then prints partition and command details as JSON. + +## Build a reusable installed-software directory + +To separate package compilation from cartridge assembly: + +```sh +fds-cartridge --xbps-tool-runner tools/in-void \ + --xbps-bin "$PWD/.host/xbps/usr/bin" \ + software build examples/software/hello/software.toml out/demo-hello +fds-cartridge software inspect out/demo-hello +``` + +The output contains `software.toml` and `root/`. A cartridge's `sources` list can +name this directory instead of a source recipe. For example: + +```toml +format = 2 +id = "my.tools" +name = "My tools" +version = "1.0" + +[[payload]] +sources = ["../out/demo-hello"] +``` + +Paths are relative to the cartridge recipe. Put several sources in one list to +share a partition, or add `[[payload]]` sections for separate partitions. See +[the format reference](Software-Format) for complete source metadata and limits. + +## Start FDS + +Build matching boot inputs using [Build and start FDS](Getting-Started), or +copy a kernel, initramfs and SYSTEM image from an FDS build machine. + +```sh +fds-emulator --session out/my-emulator start --qemu-runner tools/in-void +``` + +Omit `--qemu-runner` when QEMU is installed directly on the workstation. To use +copied boot images: + +```sh +fds-emulator --session out/my-emulator start \ + --kernel /path/to/kernel_2712.img \ + --initramfs /path/to/fds-initramfs.img \ + --system /path/to/fds-system-cli.img +``` + +The session directory is created private (0700). After stopping, reuse the same +directory with the same `start` command; an existing directory must contain a +valid stopped FDS session owned by you. Keep its absolute path short +(under 90 bytes) for Unix sockets. Start waits for the FDS prompt and uses two +emulated CPUs and 1024 MiB RAM; `--memory-mib 2048` increases memory. SYSTEM and +PROGRAM images are read-only. The session holds logs and any DATA overlays. + +## Insert and run software + +From the workstation: + +```sh +fds-emulator --session out/my-emulator insert 01 out/demo-tools.img +fds-emulator --session out/my-emulator guest -- fds bay 01 +fds-emulator --session out/my-emulator guest -- hello 'Hello from the emulator' +fds-emulator --session out/my-emulator guest -- fds run 01 -- demo.report:report +``` + +Discovery is asynchronous. Wait until the bay reports `MOUNTED READ ONLY` before +running commands. Direct commands return their output and exit status to the +caller. `fds run` starts a background job; read its output with: + +```sh +fds-emulator --session out/my-emulator guest -- tail -20 /run/log/cartridged/current +``` + +Open an interactive shell with `fds-emulator --session out/my-emulator console`. +At `FDS>`, use `hello`, `report`, `fds bays` or other guest commands. **Ctrl-]** +detaches without stopping the VM. Detach before using `guest`, `eject` or ordinary +`stop`, since they also need the serial console. + +All twelve virtual bays accept images. Simultaneously inserted PROGRAM cartridges +must have distinct cartridge IDs. Duplicate identities produce a guest error. +Command-name collisions are resolved as described in [Using cartridges](Cartridges). + +## Eject and stop + +```sh +fds-emulator --session out/my-emulator eject 01 +fds-emulator --session out/my-emulator status +fds-emulator --session out/my-emulator stop +``` + +Eject asks FDS to stop programs and release storage before removing the virtual +USB device. If FDS refuses, the cartridge stays attached. `stop` performs native +FDS shutdown and waits for QEMU to exit. Start the same session again: + +```sh +fds-emulator --session out/my-emulator start --qemu-runner tools/in-void +``` + +Pass any custom boot-image or memory options again. This is a fresh boot with +empty cartridge bays. Console and QEMU logs are appended, DATA overlay files +remain in place, and `previous-*.json` preserves the previous session's image +and overlay mapping. Insert cartridges again as needed. A running session, +unknown directory, insecure permissions, or an outstanding control operation +prevents restart; the tool never clears a live session to make room. + +`unplug BAY` simulates a physical pull, and `stop --force` cuts virtual power. +Use these only for deliberate failure simulation; DATA writes can be lost. + +## Save emulator DATA + +Inserting a DATA image creates a writable `data--.qcow2` overlay in the +session directory. Eject retains it. Reinserting the original image creates a +fresh overlay. To export the changed contents after eject or shutdown: + +```sh +qemu-img convert -f qcow2 -O raw /path/to/session/data-02-ID.qcow2 out/saved-data.img +``` + +Keep its backing image at the recorded path until export completes. Never convert +an overlay while a running VM is using it. Personal session directories are +preserved by `make clean`; keep them under your own names, such as `my-emulator`. + +## Write a cartridge to USB + +Identify the intended whole USB drive, unmount its partitions, then preview: + +```sh +fds-cartridge preview out/demo-tools.img /dev/sdX out/usb-preview.json +``` + +Replace `/dev/sdX` with the actual destination. The JSON identifies the drive, +capacity, image hash and exact confirmation phrase. Check those values before +writing: this replaces the selected drive's contents. + +```sh +sudo /absolute/path/to/fds-cartridge write out/usb-preview.json \ + --confirm 'COPY THE EXACT confirmation VALUE FROM THE PREVIEW' +``` + +The writer rechecks identity, writes the entire disk image, flushes it and verifies +readback. On a larger drive it moves the backup GPT to the end. A changed image +or reinserted drive requires a fresh preview. Use the image-tool runner for +preview and inspection if your EROFS utilities are in the local container. + +To rehearse with a disposable regular file: + +```sh +truncate -s "$(stat -c %s out/demo-tools.img)" out/disposable-usb.img +fds-cartridge preview out/demo-tools.img out/disposable-usb.img out/file-preview.json --file-target +fds-cartridge write out/file-preview.json --confirm 'COPY THE EXACT confirmation VALUE' +``` + +See [troubleshooting](Troubleshooting) for failed builds, namespace errors, +busy consoles, invalid media and blocked ejects. diff --git a/_Footer.md b/_Footer.md new file mode 100644 index 0000000..cc4ada0 --- /dev/null +++ b/_Footer.md @@ -0,0 +1,3 @@ +[Home](Home) · [All pages](Page-Index) · [Edit the wiki](Gitea-Wiki) · [Source repository](http://gitea.home.arpa/felis/fds-os) + +Maintained directly in the `fds-os.wiki` repository. Historical acceptance applies only to the source and artifacts identified in each record. diff --git a/_Sidebar.md b/_Sidebar.md new file mode 100644 index 0000000..fefadda --- /dev/null +++ b/_Sidebar.md @@ -0,0 +1,33 @@ +**FDS/OS** + +[Home](Home) · [Overview](Overview) · [All pages](Page-Index) + +**Start here** + +- [Run FDS on a Raspberry Pi 5](Hardware-Setup) +- [Build and start FDS](Getting-Started) +- [Build software cartridges and use the emulator](Workstation) + +**Use the computer** + +- [Use cartridges and run programs](Cartridges) +- [DATA and persistent files](Data) +- [WindowMaker and FDS Control](Desktop) +- [Dasung Paperlike display](Dasung) + +**Install and maintain** + +- [Internal storage and machine settings](Internal-Storage) +- [Pi 5 EEPROM configuration](EEPROM) +- [Recovery and rollback](Recovery) +- [Verify a release](Releases) +- [Reclaim build space](Cleanup) +- [Troubleshooting](Troubleshooting) + +**Build and contribute** + +- [Software cartridge format](Software-Format) +- [Editing the Gitea wiki](Gitea-Wiki) +- [Developer notes](Developer-Guide) +- [Cartridge and desktop revision](Developer-Current-Revision) +- [Implementation through M12](Developer-Implementation-Status) diff --git a/home.md b/home.md deleted file mode 100644 index b6cbc2d..0000000 --- a/home.md +++ /dev/null @@ -1 +0,0 @@ -welcome \ No newline at end of file