From 51edcf8714a5c0166b14e975dcb34fa4201841e8 Mon Sep 17 00:00:00 2001 From: Felis Date: Sun, 27 Sep 2026 00:29:40 +0800 Subject: [PATCH] Document SD boot with USB SYSTEM and maintenance recovery Describe the 32 GB native microSD setup for boot, recovery and machine settings, with SYSTEM retained on USB. Cover updating both FDS images, preparing a separate Raspberry Pi OS maintenance USB, and selecting the appropriate EEPROM boot order. Update architecture, storage, flashing and Dasung guidance to match the new defaults while retaining legacy NVMe support. Record rebuilt artifact identities, passing software checks, the unavailable standalone init fixture and the unexplained intermittent VM shutdown stall. Physical Pi and maintenance USB acceptance remain pending. --- Dasung.md | 10 +- Developer-Architecture.md | 6 +- Developer-Boot.md | 25 ++++- Developer-Development.md | 2 +- Developer-EEPROM.md | 41 +++++--- Developer-Glossary.md | 2 +- Developer-Guide.md | 5 +- Developer-Implementation-Status.md | 13 ++- Developer-Internal-Storage.md | 36 +++++-- Developer-Master-Plan.md | 25 +++-- Developer-Performance.md | 4 +- Developer-Recovery.md | 2 +- Developer-SD-Validation.md | 105 ++++++++++++++++++ Developer-Services.md | 2 +- EEPROM.md | 42 +++++--- Flashing.md | 34 +++--- Hardware-Setup.md | 164 +++++++++++++++++++---------- Home.md | 4 +- Internal-Storage.md | 36 ++++--- Overview.md | 5 +- Page-Index.md | 1 + Workstation.md | 2 +- _Sidebar.md | 1 + 23 files changed, 409 insertions(+), 158 deletions(-) create mode 100644 Developer-SD-Validation.md diff --git a/Dasung.md b/Dasung.md index a7156e1..6bbc1c5 100644 --- a/Dasung.md +++ b/Dasung.md @@ -160,8 +160,8 @@ HDMI profile change can be based on observed timings and identity. ### Edit the BOOT command line offline -Shut FDS down and disconnect Pi power. Connect NVMe to the Linux workstation, or -boot the separate maintenance SD and access the inactive NVMe there. Identify +Shut FDS down and disconnect Pi power. Connect the FDS SD through a reader to the Linux workstation, or +boot the separate maintenance USB and access the inactive SD in its native slot. Identify the same disk by model/serial used when flashing it. Mount only its first partition, labeled `FDS_BOOT`; FDS does not normally mount it at `/boot/firmware` as Raspberry Pi OS does. @@ -172,7 +172,7 @@ On that **workstation or maintenance OS**, replacing the partition path below: lsblk -p -o NAME,MODEL,SERIAL,SIZE,TYPE,PARTLABEL,MOUNTPOINTS sudo mkdir -p /mnt/fds-boot sudo mount -o nosuid,nodev,noexec \ - /dev/disk/by-id/REPLACE_WITH_VERIFIED_NVME-part1 /mnt/fds-boot + /dev/disk/by-id/REPLACE_WITH_VERIFIED_SD-part1 /mnt/fds-boot sudo cp -n /mnt/fds-boot/cmdline.txt /mnt/fds-boot/cmdline.before-hardware.txt sudoedit /mnt/fds-boot/cmdline.txt cat /mnt/fds-boot/cmdline.txt @@ -185,7 +185,9 @@ this operation. The no-clobber copy retains an existing backup; if repeating the experiment, inspect that backup and skip the copy step if it already exists. Do not assume an older backup contains your latest settings. Keep `cmdline.txt` on one line and record its final contents with the -test session. Safely disconnect the enclosure before moving NVMe back to the Pi. +test session. Safely disconnect the reader before returning SD to the Pi. In the +maintenance OS, a native SD is usually `/dev/mmcblk0` and its BOOT partition +`/dev/mmcblk0p1`; verify that identity with `lsblk` before substituting it above. To undo the edit, mount the same verified partition again, restore `cmdline.before-hardware.txt` over `cmdline.txt`, and unmount before rebooting. diff --git a/Developer-Architecture.md b/Developer-Architecture.md index 43149cd..d3983a1 100644 --- a/Developer-Architecture.md +++ b/Developer-Architecture.md @@ -28,7 +28,7 @@ and wiring remain deferred. Use 1. **The build workstation** is an x86_64 Arch Linux computer. It downloads tools, compiles ARM software, checks artifacts, assembles the rootfs, and boots the ARM VM. -2. **The machine firmware layer** lives on the Pi's internal NVMe. It supplies the +2. **The machine firmware layer** lives on the Pi's native microSD. It supplies the kernel, early boot program, recovery environment, and hardware configuration. 3. **The removable operating environment** comes from cartridges. SYSTEM supplies the OS root filesystem; DATA supplies persistent user files; other cartridges @@ -43,7 +43,7 @@ Build workstation └── Pi kernel + initramfs + SYSTEM → ARM VM checks Pi 5 machine (planned) - Internal NVMe Removable bays + Native microSD Removable bays ├── FDS_BOOT ───────────────→ SYSTEM: read-only OS ├── FDS_RECOVERY DATA: user files └── FDS_INTERNAL PROGRAM / ENVIRONMENT / hardware @@ -129,7 +129,7 @@ applying this layout to an existing workstation disk. ## Boot and service responsibilities -The Pi boots its kernel and small initramfs from internal NVMe. A static Rust +The Pi boots its kernel and small initramfs from native microSD. A static Rust `fds-stage0` locates a partition named `FDS_SYSTEM`, mounts its EROFS filesystem, and transfers control to native s6 in the new root. diff --git a/Developer-Boot.md b/Developer-Boot.md index fac2e3b..b9d7c4f 100644 --- a/Developer-Boot.md +++ b/Developer-Boot.md @@ -9,14 +9,14 @@ Development reference and historical context. For current operating instructions [Documentation index](Developer-Guide) · [First build](Developer-Getting-Started) · [Implementation status](Developer-Implementation-Status) -FDS keeps the machine's boot files on internal NVMe and its operating system on a +FDS keeps the machine's boot files on native microSD and its operating system on a removable SYSTEM cartridge. The Pi firmware loads the kernel and a small initial filesystem. Its Rust `fds-stage0` program finds the partition named `FDS_SYSTEM`, mounts it read-only, and hands control to native s6. Device names such as `sda` are never used as cartridge identities. The software boot path has passed on an ARM virtual machine using the actual -FDS Pi kernel. That does not verify Pi firmware, NVMe, RP1, display output, USB +FDS Pi kernel. That does not verify Pi firmware, SD/NVMe, RP1, display output, USB power or physical boot speed. Those checks still require physical testing; follow the [real-hardware setup guide](Hardware-Setup). M5 now provides an ordinary-user `FDS>` console with a temporary home @@ -65,7 +65,7 @@ project-local Void environment before `make boot-test` uses it. | `out/initramfs/initramfs.cpio*` | Uncompressed, gzip, legacy-LZ4 and Zstandard variants | | `out/rootfs-aarch64.tar` | Configured userspace archive with target ownership and capabilities | | `out/fds-system-cli.img` | Whole GPT cartridge image, containing an EROFS `FDS_SYSTEM` partition | -| `out/fds-boot.img` | **512 MiB FAT32 partition image**, labeled `FDS_BOOT`; not a whole NVMe disk | +| `out/fds-boot.img` | **512 MiB FAT32 partition image**, labeled `FDS_BOOT`; not a whole SD or NVMe disk | | `out/boot-volume/manifest.json` | Boot image hash, firmware input and every boot file hash | | `out/m4-vm-latest/` | Logs and exact inputs from the last passing full boot matrix | @@ -184,3 +184,22 @@ Run `make console-test` for automated console/readiness checks and `make performance-test` for the software optimization comparison. Read [M5 validation](Developer-M5-Validation) and [measurement boundaries](Developer-Performance) before interpreting VM timing as hardware evidence. + +## Native SD and legacy NVMe + +The current boot configuration targets native microSD and no longer forces the +external PCIe connector on. MMC block and Pi SD host drivers are built in, since +recovery/settings may be needed before SYSTEM can provide any modules. Generic +PCI SDHCI is built in solely to exercise native MMC cards in the ARM VM. +The package exporter checks the complete boot-driver configuration and rejects +stale kernels. No new userspace dependency is introduced. + +Legacy NVMe installations need their corresponding EEPROM boot order and may +need `PCIE_PROBE=1` on non-HAT+ adapters. Add `dtparam=pciex1` (alias `nvme`) under +`[pi5]` in that installation's `config.txt` when required. Those options do not +belong to the SD setup; see [Raspberry Pi PCIe boot guidance](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#boot-from-pcie). + +For current SD setup, matching SYSTEM installation and a Raspberry Pi OS USB +rescue drive, use [Hardware setup](Hardware-Setup). Current software evidence +is in [SD migration validation](Developer-SD-Validation); historical M4 records +remain unchanged. diff --git a/Developer-Development.md b/Developer-Development.md index 900ec31..d46c240 100644 --- a/Developer-Development.md +++ b/Developer-Development.md @@ -170,7 +170,7 @@ submodule. An overlay becomes active only when its directory contains a `fds-cartridged`, `fds-eink`, and `fds-dhcpcd` are active. For active overlays and future templates, `tools/prepare-void` copies each -active directory into `vendor/void-packages/srcpkgs/`. It allows an identical +active directory into `.host/void-packages/srcpkgs/`. It allows an identical repeat copy, rejects a name that collides with a tracked upstream package, and stops if a previous generated copy differs. Edit the source under `packages/`, then reconcile the specific stale generated copy. Never put the only copy of an diff --git a/Developer-EEPROM.md b/Developer-EEPROM.md index 43ddc77..dc98171 100644 --- a/Developer-EEPROM.md +++ b/Developer-EEPROM.md @@ -22,6 +22,7 @@ From the repository root: ```sh ./tools/configure-pi-eeprom --profile production ./tools/configure-pi-eeprom --profile development +./tools/configure-pi-eeprom --profile maintenance make eeprom-test ``` @@ -51,28 +52,36 @@ before applying anything to that machine. ## Profile behavior -| Setting | Production | Development | -| --- | --- | --- | -| `BOOT_ORDER` | `0xf6`: NVMe, repeat | `0xf16`: NVMe, SD, repeat | -| `BOOT_UART` | `0` | `1` | -| `NET_INSTALL_ENABLED` | `0` | `0` | -| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | -| `POWER_OFF_ON_HALT` | `1` | `1` | -| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | +| Setting | Production | Development | Maintenance | +| --- | --- | --- | --- | +| `BOOT_ORDER` | `0xf1`: SD, repeat | `0xf41`: SD, USB, repeat | `0xf14`: USB, SD, repeat | +| `BOOT_UART` | `0` | `1` | `1` | +| `NET_INSTALL_ENABLED` | `0` | `0` | `0` | +| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | `0` | +| `POWER_OFF_ON_HALT` | `1` | `1` | `1` | +| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | `0` | -Boot order is read from the right. Both profiles avoid scanning the twelve USB -cartridges for firmware boot. Development retains an SD rescue path. Both disable -network-install keyboard detection; Raspberry Pi documents that this detection -adds USB initialization and enumeration work. These settings do not establish a -measured FDS boot improvement. [Official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). +Boot order is read from the right. Production avoids firmware USB scanning. +Development tries SD first; remove a bootable SD to force USB rescue. Maintenance +lets a Raspberry Pi OS USB drive override FDS while the SD remains inserted for +inspection or repair. Leave that USB unplugged for normal FDS operation. The +FDS SYSTEM cartridge supplies userspace after stage0, not firmware boot files. + +Firmware fallback cannot recover a hang after Linux has started. USB-first +scans also add startup work; none of these profiles establishes a measured boot +time. All disable network-install keyboard detection and omit NVMe from boot +order. The Pi may need acknowledgement of its USB power warning with a 3 A +supply. Use the [hardware setup procedure](Hardware-Setup#4-prepare-usb-maintenance-and-select-sd-boot) +and [official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). On Pi 5, the power-off setting requests PMIC standby on halt; the dedicated power button remains the wake mechanism. The wait setting leaves cold power-on boot enabled. Whether the assembled computer and attached hardware behave as intended still needs a physical test. [Official power settings](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#POWER_OFF_ON_HALT). -The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf) and -[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf). No GPIO wake option or +The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf), +[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf) and +[maintenance.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/maintenance.conf). No GPIO wake option or unmeasured HDMI tuning is added. ## Preserve the machine's settings @@ -130,7 +139,7 @@ sudo rpi-eeprom-config --apply ./original.conf ./base.bin ``` If the firmware version itself changed, use the saved matching original firmware -for the rollback operation. Keep the maintenance SD and an external EEPROM rescue +for the rollback operation. Keep the maintenance USB and, for EEPROM recovery, a spare SD and an external EEPROM rescue route available; internal recovery cannot repair an EEPROM that prevents internal boot. The official [EEPROM update and recovery guide](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom) describes that physical workflow. FDS has not yet tested it on your Pi. diff --git a/Developer-Glossary.md b/Developer-Glossary.md index cc10a41..70e1ea2 100644 --- a/Developer-Glossary.md +++ b/Developer-Glossary.md @@ -46,7 +46,7 @@ Development reference and historical context. For current operating instructions | **EROFS** | The read-only filesystem format used for SYSTEM and recovery images. | | **tmpfs** | A memory-backed filesystem for temporary state. Its contents do not persist through power loss. | | **GPT / PARTNAME** | Disk partition metadata and a partition's name. Stage0 discovers SYSTEM using `FDS_SYSTEM`, rather than a changing `/dev/sdX` name. | -| **NVMe** | The internal solid-state storage interface/device. The FDS internal image holds boot, recovery, and machine configuration. | +| **NVMe** | An optional legacy internal storage interface/device. The current primary medium is native microSD; both use the BOOT/RECOVERY/INTERNAL disk layout. | | **SYSTEM / DATA** | Cartridge roles: operating-system image and persistent user files, respectively. | | **PROGRAM / ENVIRONMENT** | Cartridge roles: application payload and activation/configuration of a known environment. | | **Bay topology** | Mapping a physical USB hub/port path to the user-facing BAY 01–12 labels. | diff --git a/Developer-Guide.md b/Developer-Guide.md index b1875bb..5efe212 100644 --- a/Developer-Guide.md +++ b/Developer-Guide.md @@ -8,8 +8,9 @@ This is the engineering side of FDS: how the pieces are built, why choices were made, and what we actually measured. The [user manual](Home) describes how to use the current interfaces without the milestone history. -Start with [the active cartridge/desktop revision](Developer-Current-Revision). Its -acceptance is separate from the frozen 0.1.0 release. Old test reports describe +Current storage work is recorded in [native SD migration validation](Developer-SD-Validation). +The earlier [cartridge/desktop revision](Developer-Current-Revision) has separate +acceptance, as does the frozen 0.1.0 release. Old test reports describe the inputs named in those reports; a changed source tree needs fresh checks. ## Design and build references diff --git a/Developer-Implementation-Status.md b/Developer-Implementation-Status.md index 875933d..95b0f15 100644 --- a/Developer-Implementation-Status.md +++ b/Developer-Implementation-Status.md @@ -37,6 +37,11 @@ remain available. See [usage](Workstation), [format](Software-Format), Historical M0–M12 and Clap reports describe their recorded artifacts; they do not by themselves validate these new sources. +The subsequent native SD migration keeps SYSTEM on USB and adds a Raspberry Pi +OS maintenance USB procedure. Rebuilt images, current software checks and two +open validation limitations are recorded in [SD validation](Developer-SD-Validation). +Physical SD and maintenance USB acceptance remains pending. + ## Requirement coverage “Verified” below means software, artifact or isolated ARM VM evidence. It does @@ -55,15 +60,15 @@ the commands, artifacts, failures that led to fixes, and measured results. | 8. Void upstream | Pinned commit, unchanged tracked upstream files, protected overlays, exact frozen package inputs; [offline builds](Developer-Reproducible-Builds) | Normal online bootstrap still uses rolling repositories | | 9. Rust workspace | All required tools and aliases built; release/static settings and shared dependencies; [tooling](Developer-Tooling) | None in software scope | | 10. Dependencies | Locked synchronous dependencies, Clap policy and all 68 third-party crate notices; z/2/3 builds and repeated boot comparison | Physical optimization winner deferred | -| 11. Boot architecture | Pi kernel → static stage0 → read-only SYSTEM → native s6 and ordinary-user console; [boot evidence](Developer-M4-Validation) | Firmware, RP1 and physical NVMe/USB handoff deferred | -| 12. Internal NVMe | BOOT/RECOVERY/INTERNAL GPT layout; root-private atomic settings and bounded diagnostic snapshots persist across VM reboot; [internal storage](Developer-Internal-Storage) | Actual NVMe and power-loss durability deferred | +| 11. Boot architecture | Pi kernel → static stage0 → read-only SYSTEM → native s6 and ordinary-user console; [boot evidence](Developer-M4-Validation) | Firmware, RP1 and physical SD/USB handoff deferred | +| 12. Internal storage | Native SD and legacy NVMe BOOT/RECOVERY/INTERNAL GPT layout; root-private atomic settings and bounded diagnostic snapshots persist across VM reboot; [current SD evidence](Developer-SD-Validation), [internal storage](Developer-Internal-Storage) | Actual SD/NVMe and power-loss durability deferred; intermittent VM shutdown stall remains unexplained | | 13. SYSTEM | Named GPT partition and EROFS, strict manifest/architecture checks, immutable update by replacement; [media evidence](Developer-M9-Validation) | Physical cartridge write/boot deferred | | 14. Stage0 | Direct Linux/sysfs/netlink discovery, mount and root handoff; mandatory early Dasung process; nine-case real-kernel VM matrix | Physical display keepalive/recovery deferred | | 15. Missing SYSTEM | Explicit wait state, event-driven insertion/resume and independent recovery verified | Physical insertion latency deferred | | 16. Multiple SYSTEMs | Ambiguity enters recovery instead of choosing enumeration order; explicit recovery verified | Interactive selector is a later-version option | | 17. Kernel | Pinned Pi kernel package, built-in critical driver delta verified after Kconfig resolution, modules/DTBs packaged | Actual RP1, PCIe, graphics and USB support deferred | | 18. Initramfs | Minimal static early userspace; raw/gzip/lz4/zstd construction, extraction and boot checks | Pi decompression/timing comparison deferred | -| 19. Firmware | Production/development EEPROM previews, exact readback and rollback preparation; [EEPROM](Developer-EEPROM) | No EEPROM was read from or applied to a physical Pi | +| 19. Firmware | Production/development/maintenance EEPROM previews, exact readback and rollback preparation; [EEPROM](Developer-EEPROM) | No EEPROM was read from or applied to a physical Pi | | 20. Filesystems | SYSTEM stays read-only; volatile logs, temporary directories and state; root ownership is explicitly root:root 0755 | Physical persistence/durability deferred | | 21. User data | Persistent DATA at `/data`, temporary `/home/fds`, unprivileged sessions; [DATA evidence](Developer-M7-Validation) | Home persistence is not claimed | | 22. Service graph | Native boot bundle includes console, devices, cartridge and mandatory Dasung services; optional bundles excluded | Hardware service behavior deferred | @@ -133,7 +138,7 @@ and optional eject LEDs remain later-version options in the specification. ## Hardware work intentionally deferred The next acceptance phase needs the assembled computer: EEPROM backup/application -and rollback, NVMe-to-USB boot, RP1/PCIe/USB behavior, actual bay calibration and +and rollback, SD-to-USB boot, RP1/PCIe/USB behavior, actual bay calibration and all twelve devices, display picture/refresh/cold-power recovery, removable battery and power behavior, physical storage durability, cold boot and shutdown timing. Use [stress testing](Developer-Stress-Testing), [Dasung](Developer-Dasung), [EEPROM](Developer-EEPROM), diff --git a/Developer-Internal-Storage.md b/Developer-Internal-Storage.md index 1a37fc5..9417f5e 100644 --- a/Developer-Internal-Storage.md +++ b/Developer-Internal-Storage.md @@ -9,10 +9,10 @@ Development reference and historical context. For current operating instructions [Documentation index](Developer-Guide) · [Boot images](Developer-Boot) · [Recovery](Developer-Recovery) -The internal NVMe supplies the Pi's firmware boot files, independent recovery, +The native microSD supplies the Pi's firmware boot files, independent recovery, and machine settings. SYSTEM remains a separate removable cartridge. Ordinary user files belong on DATA, not internal storage. No physical disk is written by -any build command below; Pi/NVMe boot and power-loss tests remain deferred. +any build command below; physical Pi SD/NVMe boot and power-loss tests remain deferred. ## Build the complete disk image @@ -41,7 +41,10 @@ The builder pads the recovery partition without changing its EROFS contents, aligns partitions to 1 MiB, writes both GPT copies, verifies each payload, and checks the table independently with `sfdisk`. Internal ext4 is fully initialized before deployment. It contains root-owned `config/` and private `diagnostics/`. +The default whole image is about 1.75 GiB, so a 32 GB card has ample capacity. Unused capacity beyond this initial layout is not automatically expanded. +SYSTEM stays on its separate USB cartridge. For a Raspberry Pi OS rescue USB, +follow [the maintenance setup](Hardware-Setup#4-prepare-usb-maintenance-and-select-sd-boot). For a 2 GiB recovery allocation or larger settings partition: @@ -99,9 +102,13 @@ its active snapshot is readable by the local FDS user. ## What happens at boot The `machine-config` native s6 oneshot precedes `cartridged`. It does **not** -precede the console or Dasung controller. It accepts exactly one non-removable -NVMe disk with the three named partitions in the order above. USB lookalikes are -ignored; multiple eligible NVMe disks are rejected instead of choosing by name. +precede the console or Dasung controller. It accepts exactly one native SD card +or non-removable PCIe NVMe disk with the three named partitions in the order +above. Native SD is identified through the MMC subsystem and card type `SD`, +regardless of the block device's removable flag. USB ancestry is always rejected; +USB readers, eMMC and SDIO cannot supply machine settings. Device names such as +`mmcblk0` are not identities. Multiple eligible disks, including an old NVMe +alongside SD, are rejected instead of choosing by name. The settings partition must be clean ext4 with the expected label. Loading uses `ro,noload,nosuid,nodev,noexec` in a private mount namespace, validates the entire @@ -121,8 +128,8 @@ fds machine export /tmp/current-machine ``` Export creates a new directory with the three editable source files. It never -overwrites an existing directory. The status source is `internal_nvme` or -`image_defaults`. A temporarily unavailable NVMe is not adopted later in the same +overwrites an existing directory. The status source is `internal_sd`, `internal_nvme` or +`image_defaults`. Temporarily unavailable internal storage is not adopted later in the same boot; resolve the issue and reboot to load its settings. This prevents changing bay identities underneath active cartridge operations. @@ -148,7 +155,7 @@ reinstall it if the new calibration is wrong. An invalid existing JSON document can be replaced this way. Wrong ownership, symlinks, an unclean filesystem or a damaged directory require offline filesystem -maintenance first. Recovery does not automatically repair internal NVMe; its +maintenance first. Recovery does not automatically repair internal SD/NVMe; its `fds recovery repair` command is deliberately limited to DATA cartridges. ## Save and retrieve diagnostics @@ -188,8 +195,17 @@ and EROFS tools. A private unprivileged user namespace gives created files root ownership without requiring a root build session. `make internal-test` exercises the actual packaged runtime in ARM VMs with -virtual NVMe. Acceptance evidence belongs in [M12 validation](Developer-M12-Validation); -a passing VM does not verify the Pi EEPROM, PCIe path or physical flash durability. +virtual native SD (a generic PCI SDHCI controller) and NVMe. The SD controller +exercises the MMC card identity, recovery filesystem and settings persistence; +it is not an emulation of the Pi's SD host or EEPROM. SD tests also boot normal +SYSTEM from USB, reject SD/NVMe ambiguity and preserve an unclean card unchanged. +No new userspace dependency is needed. Boot-critical MMC drivers are built into +the kernel; PCI SDHCI is included for this generic ARM VM fixture. + +Historical acceptance remains in [M12 validation](Developer-M12-Validation). +Current SD results are recorded in [SD migration validation](Developer-SD-Validation). +A passing VM does not verify Pi firmware, physical SD/NVMe boot, maintenance USB +selection, electrical behavior or flash durability. The Linux [ext4 mount documentation](https://www.kernel.org/doc/html/latest/admin-guide/ext4.html) explains why read-only loading also disables journal replay. Filesystem creation diff --git a/Developer-Master-Plan.md b/Developer-Master-Plan.md index 83a5597..3bd0d7b 100644 --- a/Developer-Master-Plan.md +++ b/Developer-Master-Plan.md @@ -554,7 +554,7 @@ Do not introduce a large runtime merely to listen for a few uevents. # 11. Boot architecture -The machine's internal NVMe belongs to the: +The machine's native microSD (32 GB target) belongs to the: ```text machine firmware layer @@ -571,7 +571,7 @@ Boot sequence: ```text Raspberry Pi EEPROM ↓ -internal NVMe +native microSD ↓ kernel ↓ @@ -590,11 +590,19 @@ s6 READY ``` -Do not make Raspberry Pi firmware search all 12 USB cartridges for a boot device. +Production uses SD-only EEPROM boot order. Development can fall back to a +Raspberry Pi OS USB drive; the optional maintenance profile tries USB first so +the FDS SD can remain inserted for repair. This deliberately adds firmware USB +scanning only when requested. SYSTEM remains a separate USB cartridge. + +This SD arrangement supersedes the original NVMe requirement at the user's +request on 2026-09-26. Legacy PCIe NVMe settings remain supported, but native SD +and NVMe must not both provide complete internal layouts during one boot. +A USB reader is a flashing path, not accepted internal settings storage. --- -# 12. Internal NVMe layout +# 12. Internal storage layout Recommended layout: @@ -830,7 +838,8 @@ USB host USB hub USB mass storage SCSI disk -NVMe +MMC / native SD +NVMe (legacy alternative) devtmpfs EROFS ext4 @@ -884,7 +893,8 @@ zstd image smaller does not establish that boot will be faster. -NVMe capacity is inexpensive. +The 32 GB SD has ample capacity for the roughly 1.75 GiB internal image. +Unused card capacity stays unallocated; SYSTEM remains on USB. Boot milliseconds matter. @@ -916,6 +926,7 @@ Together with: ```text development.conf production.conf +maintenance.conf ``` All EEPROM changes must be: @@ -2256,7 +2267,7 @@ Pi: ```text power on ↓ -internal NVMe kernel +native microSD kernel ↓ USB SYSTEM cartridge ↓ diff --git a/Developer-Performance.md b/Developer-Performance.md index a5eaab2..d6f7414 100644 --- a/Developer-Performance.md +++ b/Developer-Performance.md @@ -98,7 +98,7 @@ The gate exists only in a disposable test image. Each benchmark record should contain: -1. Hardware identity: Pi revision, firmware/EEPROM version, NVMe and SYSTEM media, +1. Hardware identity: Pi revision, firmware/EEPROM version, SD/NVMe and SYSTEM media, USB hubs/devices, power supply/battery conditions, and display setup. 2. Software identity: FDS revision, kernel configuration/version, image digest, package/input records, active profile, and relevant boot configuration. @@ -164,7 +164,7 @@ The helper creates separate build directories and initramfs files for each level it does not overwrite the normal tool or initramfs outputs. Read `out/rust-profiles-latest/comparison.json` for the measured artifact identities. The uncompressed initramfs remains the default. All four formats boot in the M4 -matrix, but compression speed must still be compared on physical NVMe/Pi hardware. +matrix, but compression speed must still be compared on physical SD/Pi hardware. M10 records shutdown events and VM observations in [its validation report](Developer-M10-Validation). M11 records repeated empty/twelve-device VM comparisons, including the diff --git a/Developer-Recovery.md b/Developer-Recovery.md index 201213a..f807a03 100644 --- a/Developer-Recovery.md +++ b/Developer-Recovery.md @@ -62,7 +62,7 @@ Stage0 can enter recovery in two ways: Stage0 requires exactly one readable `FDS_RECOVERY` partition and mounts it read-only. It does not silently choose between duplicate partitions. The ARM VM -suite supplies disposable virtual partitions; physical firmware/NVMe/display +suite supplies disposable virtual partitions; physical firmware/SD/NVMe/display behavior must still be checked on the Pi. The local prompt is: diff --git a/Developer-SD-Validation.md b/Developer-SD-Validation.md new file mode 100644 index 0000000..5e3dac3 --- /dev/null +++ b/Developer-SD-Validation.md @@ -0,0 +1,105 @@ +--- +include_toc: true +--- + +# Native SD migration validation + +[Hardware setup](Hardware-Setup) · [Internal storage](Developer-Internal-Storage) · [EEPROM](EEPROM) + +The 2026-09-26 revision replaces the primary NVMe boot medium with native microSD +and retains SYSTEM on USB. A separate Raspberry Pi OS USB drive provides +maintenance. Frozen release `0.1.0` and its acceptance records remain unchanged. + +## Scope + +The storage loader accepts native MMC cards of type `SD`, keeps existing NVMe +support, rejects USB ancestry and refuses multiple eligible internal disks. +Normal boot remains read-only; settings installation and bounded diagnostic +storage remain explicit recovery operations. EEPROM previews now offer SD-only, +SD-then-USB and USB-then-SD profiles. They never apply firmware to hardware. + +Pi MMC drivers are explicitly built in. Generic PCI SDHCI is added to exercise +a real Linux MMC card in QEMU's ARM `virt` machine, not to emulate the Pi SD +controller. No new Rust crate or userspace package is required. + +## Software evidence + +These results apply to the local working-tree revision built as +`1.0.r3.ga14ea7714598.dirty`, not the frozen release. The manifest +`out/manifests/sd-migration.json` records the source snapshot, artifact identities, +logs and limitations. Logs are retained under `out/logs/sd-migration/`. + +| Check | Result and evidence | +| --- | --- | +| Native CLI tests | All seven CLI/program tests pass, including native SD identity, removable-flag independence, preserved NVMe detection, and rejection of USB ancestry, eMMC and unknown transports | +| `make check` and `make wiki-check` | Pass; repository policy, formatting, cleanup and wiki checks complete | +| Kernel build | Pass; required FDS and Dasung configuration is checked against the resolved kernel config; final cached build inputs verify | +| CLI and development rootfs | Both rebuilt with the current loader; `make rootfs-test` passes for each profile | +| Recovery, SYSTEM, initramfs and boot images | Rebuilt and audited; raw/gzip/lz4/zstd initramfs variants produced, both SYSTEM profiles available, complete internal GPT image validated | +| `make boot-test` | All nine cases pass with the rebuilt Pi kernel on ARM `virt`; `out/m4-vm.KLp2AG` includes real stage0, SYSTEM handoff, native s6 PID 1/service checks, recovery and immutable-input checks | +| `make internal-test` | Full rerun passes under `out/m12-internal.k16rdxnn`; see the initial shutdown failure below | +| `make eeprom-test` | Pass under `out/m12-eeprom.ixlnzj5p`; all three boot orders, immutable firmware payloads, exact rollback and overwrite refusal checked | +| `make flash-test` | Disposable-file tests pass under `out/workstation-flash.gzimnkb2`; inspection also accepts the three completed disk images below | +| `make workstation-test` | Pass under `out/workstation-images.z14xle7n`, including another flash regression run under `out/workstation-flash.qikhs4at` | +| `make emulator-test` | Pass under `out/emu-test.6lzniavt`; public emulator, guest software, twelve virtual bays, legacy reading and cartridge lifecycle checks complete | +| `make init-test` | Blocked before guest launch by the unavailable pinned upstream kernel package; not a pass | + +The internal-storage suite boots the packaged recovery and SYSTEM with QEMU's +native MMC SD device. It verifies read-only settings loading, explicit atomic +installation, bounded diagnostics, persistence on the next boot, immutable +per-boot settings, normal USB SYSTEM operation and independent ext4 checks. +The persisted status reports `source: internal_sd`. Existing NVMe tests pass, +USB copies are rejected, simultaneous eligible SD/NVMe disks are refused, and +an unclean SD filesystem falls back without writes. The VM SD backing file is +extended to 2 GiB and its backup GPT relocated for QEMU's power-of-two card-size +requirement; the distributed image geometry is unchanged. + +### Open validation limitations + +The first internal-storage run, `out/m12-internal.k1jrziss`, stalled during +shutdown after successfully loading persisted SD settings and diagnostics. +Shutdown reached the native s6 TERM/KILL phase but did not report power-down +within 120 seconds. A diagnostic reboot and five further diagnostic repetitions +powered off successfully, and a full rerun of the unchanged test suite passed. +No runtime fix, added retry or relaxed assertion was made. The cause remains +unexplained; the passing rerun does not establish that the intermittent stall +is fixed. The original log and diagnostic repetitions are retained as +`internal-first-failure.log`, `sd-shutdown-diagnose.log` and +`first-storage-run/` beneath the log directory. + +The standalone init suite still pins Void package +`linux6.12-6.12.110_1.aarch64.xbps`. Its configured upstream URL returned HTTP 404 +and no matching cached artifact was available, so `make init-test` stopped before +launching a guest. The pin was not changed as part of the SD migration. The +separate Pi-kernel boot matrix did execute the M2 guest PID 1 and service-control +checks successfully, but this does not turn the blocked standalone suite into +a pass. See `init-checks.log`. + +## Built disk images + +Both the SD image and SYSTEM must be updated: the settings loader is included +in SYSTEM as well as recovery. These images were rebuilt from the same loader. +The 1.75 GiB internal image fits on a 32 GB card; unused capacity remains +unallocated. + +| Output | Bytes | SHA-256 | +| --- | ---: | --- | +| `out/fds-internal.img` | 1,881,145,344 | `b7ae58292972d54c81dbb887281ad39cf810199c174c3985a6e44b20c59fcc62` | +| `out/fds-system-cli.img` | 495,976,448 | `3e8813f416c1e91fa827b99db104a3e9b6575884ff1fb5f99f2cda78927ff2d6` | +| `out/fds-system-development.img` | 1,991,245,824 | `fcde95d7008981b44a1a2220d82c9a999180ce39a214abae25a22970127b525a` | + +`out/manifests/sd-migration-artifacts.json` also records the exact resolved build +directories, kernel, boot volume, recovery, initramfs and three rootfs archives. +`out/manifests/sd-migration-images.sha256` provides their checksums for local +verification. These generated files are local evidence, not published wiki +attachments. Final validation notes were written after image construction; +the checkout wiki contains the latest acceptance record. + +## Physical acceptance + +No Pi EEPROM update, physical SD boot, maintenance USB boot, power-loss test or +Dasung power-cycle test was performed by this software run. Follow +[the four hardware checks](Hardware-Setup#verify-the-sd-and-maintenance-paths-on-hardware) +and record the card identity, boot order, image hashes and observations in +`tests/hardware/session-template.json`. The added cases remain `not_run` until +measured. An ARM VM pass cannot substitute for any of these observations. diff --git a/Developer-Services.md b/Developer-Services.md index b5a66ca..7e3a133 100644 --- a/Developer-Services.md +++ b/Developer-Services.md @@ -48,7 +48,7 @@ The Void build container never boots the FDS service graph. | `getty` | Provide a console login | `console`, `hostname` | | `eudevd` | Process kernel device events | `runtime-fs`; readiness checked through its control channel | | `udev-trigger` | Request initial subsystem and device events | Ready `eudevd` | -| `machine-config` | Validate internal NVMe settings and publish a fixed boot snapshot, or image defaults | `runtime-fs`; no console dependency | +| `machine-config` | Validate internal SD/NVMe settings and publish a fixed boot snapshot, or image defaults | `runtime-fs`; no console dependency | | `cartridged-runtime` | Prepare bounded cartridge log storage | `runtime-fs` | | `cartridged-log`, `cartridged` | USB events, bay inventory, validated mounts and IPC | Runtime paths and `machine-config`; no getty or eudev readiness dependency | | `dasungd-runtime` | Prepare private controller state and log paths | `runtime-fs` | diff --git a/EEPROM.md b/EEPROM.md index dbe8694..1c11d4c 100644 --- a/EEPROM.md +++ b/EEPROM.md @@ -17,6 +17,7 @@ From the repository root: ```sh ./tools/configure-pi-eeprom --profile production ./tools/configure-pi-eeprom --profile development +./tools/configure-pi-eeprom --profile maintenance ``` The first invocation downloads three checksum-pinned inputs into @@ -45,26 +46,35 @@ before applying anything to that machine. ## Profile behavior -| Setting | Production | Development | -| --- | --- | --- | -| `BOOT_ORDER` | `0xf6`: NVMe, repeat | `0xf16`: NVMe, SD, repeat | -| `BOOT_UART` | `0` | `1` | -| `NET_INSTALL_ENABLED` | `0` | `0` | -| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | -| `POWER_OFF_ON_HALT` | `1` | `1` | -| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | +| Setting | Production | Development | Maintenance | +| --- | --- | --- | --- | +| `BOOT_ORDER` | `0xf1`: SD, repeat | `0xf41`: SD, USB, repeat | `0xf14`: USB, SD, repeat | +| `BOOT_UART` | `0` | `1` | `1` | +| `NET_INSTALL_ENABLED` | `0` | `0` | `0` | +| `NET_INSTALL_AT_POWER_ON` | `0` | `0` | `0` | +| `POWER_OFF_ON_HALT` | `1` | `1` | `1` | +| `WAIT_FOR_POWER_BUTTON` | `0` | `0` | `0` | -Boot order is read from the right. Both profiles avoid scanning the twelve USB -cartridges for firmware boot. Development retains an SD rescue path. Both disable -network-install keyboard detection; Raspberry Pi documents that this detection -adds USB initialization and enumeration work. [Official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). +Boot order is read from the right. Production avoids firmware USB scanning. +Development tries SD first; remove a bootable SD to force USB rescue. Maintenance +lets a Raspberry Pi OS USB drive override FDS while the SD remains inserted for +inspection or repair. Leave that USB unplugged for normal FDS operation. The +FDS SYSTEM cartridge supplies userspace after stage0, not firmware boot files. + +Firmware fallback cannot recover a hang after Linux has started. USB-first +scans also add startup work; none of these profiles establishes a measured boot +time. All disable network-install keyboard detection and omit NVMe from boot +order. The Pi may need acknowledgement of its USB power warning with a 3 A +supply. Use the [hardware setup procedure](Hardware-Setup#4-prepare-usb-maintenance-and-select-sd-boot) +and [official bootloader configuration](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). On Pi 5, the power-off setting requests PMIC standby on halt; the dedicated power button remains the wake mechanism. The wait setting leaves cold power-on boot enabled. After shutdown, check that attached power hardware follows the selected policy. [Official power settings](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#POWER_OFF_ON_HALT). -The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf) and -[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf). Review these files before choosing a profile. +The source profiles are [production.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/production.conf), +[development.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/development.conf) and +[maintenance.conf](http://gitea.home.arpa/felis/fds-os/src/branch/main/config/eeprom/maintenance.conf). Review these files before choosing a profile. ## Preserve the machine's settings @@ -119,7 +129,7 @@ sudo rpi-eeprom-config --apply ./original.conf ./base.bin ``` If the firmware version itself changed, use the saved matching original firmware -for the rollback operation. Keep the maintenance SD and an external EEPROM rescue -route available; internal recovery cannot repair an EEPROM that prevents internal +for the rollback operation. Keep maintenance USB and a spare SD for the external EEPROM rescue +procedure available; internal recovery cannot repair an EEPROM that prevents internal boot. The official [EEPROM update and recovery guide](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom) describes that physical workflow. diff --git a/Flashing.md b/Flashing.md index 8c64029..ef1295c 100644 --- a/Flashing.md +++ b/Flashing.md @@ -58,7 +58,7 @@ privileges to read protected files and preserve its intended ownership. | Image | Destination | | --- | --- | -| `out/fds-internal.img` | Whole NVMe disk, normally connected through an enclosure | +| `out/fds-internal.img` | Whole microSD card through a reader; also usable for legacy NVMe | | `out/fds-system-cli.img` | Whole USB SYSTEM cartridge | | `out/fds-system-development.img` | Alternative SYSTEM cartridge with development tools | | A complete PROGRAM, DATA or ENVIRONMENT `.img` | Its separate cartridge disk | @@ -72,7 +72,15 @@ software metadata and installed program trees before deployment. Use an uncompressed, complete FDS GPT disk image. The raw `fds-boot.img` and `fds-recovery.img` partition payloads, compressed archives, arbitrary operating system images and 4 KiB logical-sector destinations are rejected. A disk must -have 512-byte logical sectors and enough space for the complete image. +have 512-byte logical sectors and enough space for the complete image. The +internal image is about 1.75 GiB and fits on a 32 GB SD card. The remaining +capacity stays unallocated. A reader may expose it as `/dev/sdX`; a native host +slot may use `/dev/mmcblkN`. Select the whole device, never a partition. + +Flash a newly rebuilt SYSTEM too when migrating from NVMe: the settings loader +lives in SYSTEM as well as recovery. The older loader accepts NVMe only. +Use Raspberry Pi Imager, rather than `fds-flash`, for a separate Raspberry Pi OS +[maintenance USB](Hardware-Setup#4-prepare-usb-maintenance-and-select-sd-boot). ## Create a cartridge image @@ -127,7 +135,7 @@ diagnostics go to stderr; creation failures return a nonzero exit status. ## Interactive mode -Keep the Pi powered off while preparing its NVMe. Connect only the intended +Keep the Pi powered off while preparing its boot SD. Connect only the intended installation media, then launch the guide: ```sh @@ -147,7 +155,7 @@ shows the plan and requires terminal confirmation: ```sh sudo ./out/workstation/fds-flash \ --image out/fds-internal.img \ - --device /dev/disk/by-id/REPLACE_WITH_THE_NVME_DISK + --device /dev/disk/by-id/REPLACE_WITH_THE_SD_DISK ``` Replace the destination with the verified **whole-disk** path, without a @@ -191,8 +199,8 @@ To inspect an image and destination together, without opening the disk for writi ```sh out/workstation/fds-flash \ --image out/fds-internal.img \ - --device /dev/disk/by-id/REPLACE_WITH_THE_NVME_DISK \ - --dry-run --json >out/nvme-plan.json + --device /dev/disk/by-id/REPLACE_WITH_THE_SD_DISK \ + --dry-run --json >out/sd-plan.json ``` Read the saved plan. `plan.target` contains model, serial, capacity, logical-sector @@ -211,11 +219,11 @@ provisioning inventory, then pass them explicitly: ```sh sudo -n ./out/workstation/fds-flash \ --image out/fds-internal.img \ - --device /dev/disk/by-id/REPLACE_WITH_THE_NVME_DISK \ + --device /dev/disk/by-id/REPLACE_WITH_THE_SD_DISK \ --unattended \ --expect-target TARGET_ID_FROM_THE_REVIEWED_PLAN \ --sha256 EXPECTED_IMAGE_SHA256 \ - --json >out/nvme-flash.json + --json >out/sd-flash.json ``` Replace both hash placeholders with their full 64 hexadecimal characters. @@ -223,17 +231,17 @@ Replace both hash placeholders with their full 64 hexadecimal characters. A script already running as root can invoke the binary directly. The flasher never prompts or invokes sudo in unattended mode. -For an already reviewed `out/nvme-plan.json`, this complete shell example loads +For an already reviewed `out/sd-plan.json`, this complete shell example loads the bindings without adding a JSON utility dependency: ```sh -fds_target_id=$(python3 -c 'import json; print(json.load(open("out/nvme-plan.json"))["plan"]["target_id"])') -fds_sha256=$(python3 -c 'import json; print(json.load(open("out/nvme-plan.json"))["plan"]["sha256"])') +fds_target_id=$(python3 -c 'import json; print(json.load(open("out/sd-plan.json"))["plan"]["target_id"])') +fds_sha256=$(python3 -c 'import json; print(json.load(open("out/sd-plan.json"))["plan"]["sha256"])') sudo -n ./out/workstation/fds-flash \ --image out/fds-internal.img \ - --device /dev/disk/by-id/REPLACE_WITH_THE_NVME_DISK \ + --device /dev/disk/by-id/REPLACE_WITH_THE_SD_DISK \ --unattended --expect-target "$fds_target_id" --sha256 "$fds_sha256" \ - --json >out/nvme-flash.json + --json >out/sd-flash.json ``` A successful JSON response has `status: "verified"`; a dry run has diff --git a/Hardware-Setup.md b/Hardware-Setup.md index 4c77824..2626b6a 100644 --- a/Hardware-Setup.md +++ b/Hardware-Setup.md @@ -26,31 +26,32 @@ console on the Pi. FDS does not start SSH or networking automatically. You need: - A Raspberry Pi 5 with cooling and a suitable USB-C power supply. -- A PCIe NVMe adapter/HAT and NVMe drive for internal boot and recovery storage. +- A 32 GB microSD card for boot files, recovery and persistent machine settings. - One USB storage device for SYSTEM, a USB keyboard, and the Paperlike 13K. - HDMI video cabling, a USB-A-to-USB-C **data** cable for the monitor controller, and the monitor's supplied power adapter. -- A Linux workstation and an NVMe enclosure/adapter for writing the disks. - A separate maintenance microSD card is useful for EEPROM and display diagnosis. +- A Linux workstation and an SD card reader for flashing. +- A separate USB drive for Raspberry Pi OS maintenance; it is not the SYSTEM cartridge. The current FDS installation uses **two disks**: | Disk | Contents | When it is needed | | --- | --- | --- | -| Internal PCIe NVMe | `FDS_BOOT`, `FDS_RECOVERY`, `FDS_INTERNAL` | Every FDS boot; also recovery without SYSTEM | +| Native microSD slot | `FDS_BOOT`, `FDS_RECOVERY`, `FDS_INTERNAL` | Every FDS boot; also recovery without SYSTEM | | USB SYSTEM cartridge | `FDS_SYSTEM` | Normal FDS operation | -PROGRAM, DATA and ENVIRONMENT cartridges can wait until the console works. -If only microSD is connected today, use it for Raspberry Pi OS maintenance and -cable checks. This guide's complete FDS installation requires NVMe: persistent -machine settings deliberately accept PCIe NVMe only. Writing the internal image -to microSD or leaving NVMe in a USB enclosure does not provide that supported -layout. A SYSTEM image alone contains no Pi firmware boot partition. +The roughly **1.75 GiB** internal image fits comfortably on a 32 GB card. The +remaining card capacity stays unallocated; FDS does not expand it or put SYSTEM +there. PROGRAM, DATA and ENVIRONMENT cartridges can wait until the console works. +A SYSTEM image alone contains no Pi firmware boot partition. -Connect or reseat the PCIe ribbon and HAT with power disconnected, following the -adapter's instructions. For the first boot, attach SYSTEM directly to a Pi USB -port if the cartridge hubs are not ready. Changing the hub wiring later requires -a fresh bay calibration. +Use the Pi's native microSD slot during FDS operation. A USB card reader is fine +for flashing on the workstation, but FDS deliberately rejects USB copies as +persistent machine settings storage. Disconnect the old NVMe/HAT with Pi power +removed; leaving another complete FDS internal disk attached causes ambiguous +settings and recovery discovery. For the first boot, connect SYSTEM directly to +a Pi USB port if the cartridge hubs are not ready. Changing hub wiring later +requires fresh bay calibration. ## 2. Wire HDMI video, USB control and power separately @@ -91,7 +92,10 @@ changing monitor identity or USB matching rules. ## 3. Build a matching image set on the workstation Complete [Prepare the workstation](Getting-Started#prepare-the-workstation), -including bootstrap, smoke test and build checks. From the repository root, run +including bootstrap, smoke test and build checks. When upgrading an existing +NVMe-era build, refresh any [stale generated kernel overlay](Developer-Troubleshooting#configuration-or-overlay-conflict) +and run `./tools/build-kernel --rebuild` once so the package includes the current +SD driver requirements. From the repository root, run the following commands **sequentially**, stopping if any fails: ```sh @@ -117,7 +121,7 @@ profile is sufficient here, and includes the optional desktop too. | Output | Write it to | | --- | --- | -| `out/fds-internal.img` | Whole NVMe disk | +| `out/fds-internal.img` | Whole microSD card | | `out/fds-system-cli.img` | Whole USB SYSTEM disk | | `out/fds-boot.img` | Raw FAT partition payload; already included in the internal image | | `out/fds-recovery.img` | Raw EROFS partition payload; already included in the internal image | @@ -126,6 +130,11 @@ Do not write the last two files as whole disks. `make all` builds additional profiles and release tooling; it is unnecessary for this first-boot image set. Use [cleanup](Cleanup) to review retained build trees when space is tight. +**When moving from the old NVMe setup, rebuild and flash SYSTEM as well as the +internal SD image.** The earlier machine-settings loader accepts NVMe only; +changing just the boot card cannot update the loader on an old SYSTEM cartridge. +The frozen `0.1.0` images retain that earlier behavior. + Record the inputs before flashing. On the **workstation**, choose a new session directory and retain the printed build logs/manifests with it: @@ -147,40 +156,63 @@ local build; they are not a release signature. For a downloaded release, follow The historical `0.1.0` release and its acceptance records remain separate from new local builds. -## 4. Check NVMe boot configuration +## 4. Prepare USB maintenance and select SD boot -If the Pi already boots from this NVMe adapter, first record its EEPROM settings; -changing firmware is not a prerequisite just to try FDS. Otherwise, use a -separate Raspberry Pi OS maintenance microSD with the official EEPROM utilities. -Keep that rescue medium available throughout bring-up. +On the workstation, use [Raspberry Pi Imager](https://www.raspberrypi.com/software/) +to write **Raspberry Pi OS (64-bit)** for **Raspberry Pi 5** to the separate +maintenance USB drive. Select that drive carefully: Imager erases it. Set your +own username/password and, if wanted, SSH access in Imager. `fds-flash` accepts +FDS images; use Imager for Raspberry Pi OS. -In the **maintenance OS**, before editing: +Before overwriting your working rescue SD, boot that existing Raspberry Pi OS +installation and record its EEPROM settings: ```sh rpi-eeprom-config > pi-current.conf rpi-eeprom-update > pi-eeprom-status.txt -``` - -Copy these files to the workstation and preserve the matching original firmware -as described in [EEPROM preparation and rollback](EEPROM). To review the -installed boot configuration, the official editor is: - -```sh sudo rpi-eeprom-config --edit ``` -For initial FDS bring-up, `BOOT_ORDER=0xf16` means NVMe first, SD fallback, then -repeat (read from the right). A non-HAT+ NVMe adapter may also require -`PCIE_PROBE=1`; follow the adapter's instructions and the -[official PCIe boot procedure](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#boot-from-pcie). -Preserve unrelated board settings, follow the updater's restart instructions, -then read the configuration back. FDS already supplies `dtparam=pciex1` in -`config.txt`; EEPROM discovery and the kernel's PCIe enablement are separate. +Save the reports outside the card being replaced. Change `BOOT_ORDER` in the +editor, preserving unrelated settings. Boot order is read **right to left**: -The [FDS EEPROM development profile](EEPROM#profile-behavior) also keeps SD -fallback and enables firmware UART logging. Its offline preview is not a backup -of your board and does not apply itself. Do not switch to NVMe-only production -boot order until the rescue and normal boot paths have been checked. +| Choice | Value | Behavior | +| --- | --- | --- | +| Convenient maintenance | `0xf14` | USB first, then SD, repeat; plug in Raspberry Pi OS to override the FDS SD | +| SD first | `0xf41` | SD first, then USB, repeat; remove the SD to force a maintenance boot | +| FDS production | `0xf1` | SD only, repeat; no firmware USB scan and no USB rescue fallback | + +This EEPROM choice is independent of `BOOT_MODE=production` (HDMI console) +and the CLI/development SYSTEM profile. + +For this SD-plus-rescue-USB setup, **`0xf14` is the convenient starting choice**. +With maintenance USB unplugged, firmware scans USB then boots FDS from SD. +FDS SYSTEM is not firmware-bootable, so it cannot replace the maintenance OS in +that sequence. Firmware USB scanning adds startup work; switch to SD-only only +if that tradeoff and the external rescue procedure are acceptable. + +Follow the editor/updater's restart instructions and read `rpi-eeprom-config` +back after restart. The updater can also change firmware version; preserve the +original configuration and matching firmware before applying changes, following +[EEPROM preparation and rollback](EEPROM). There is no requirement to blindly +flash a new EEPROM release just to use SD. The offline FDS profile previews do +not apply themselves to a board. No NVMe entry (`6`) or `PCIE_PROBE=1` is required +for the new storage arrangement, and the shipped SD configuration does not force +external PCIe enablement. + +Test the maintenance USB **before erasing the working rescue SD**: power off, +remove that SD, unplug SYSTEM for this test, insert maintenance USB and power on. +In Raspberry Pi OS, confirm `findmnt -no SOURCE /` and `lsblk -p` identify the USB +root disk. Shut down cleanly before reconnecting the FDS media. A USB-first +configuration also lets you later boot maintenance with the FDS SD still inserted, +so its partitions can be inspected or repaired while inactive. + +A USB fallback only applies while firmware is looking for boot files. It does +not rescue a Linux boot that has already started and then hangs. The Pi 5 may +also refuse USB boot with a 3 A supply until its power warning is acknowledged; +Raspberry Pi documents a one-boot power-button override. A suitable 5 V/5 A +supply provides a larger USB power budget for the attached devices. See +[official boot order and USB power guidance](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#bootloader-configuration). ## 5. Write the two disks @@ -192,8 +224,8 @@ sudo ./out/workstation/fds-flash --image out/fds-internal.img sudo ./out/workstation/fds-flash --image out/fds-system-cli.img ``` -Each invocation lists disks by model, serial and capacity. Select the NVMe -enclosure for the internal image and a different USB disk for SYSTEM, review the +Each invocation lists disks by model, serial and capacity. Select the SD card +reader for the internal image and a different USB disk for SYSTEM, review the plan, and enter the displayed erase phrase. Both writes replace the selected media. The tool refuses mounted/occupied disks and non-512-byte logical sectors; it writes, flushes, verifies readback and relocates the backup GPT automatically. @@ -205,17 +237,17 @@ unattended mode with explicit image checksum and target identity. Confirm the three internal partition labels and the USB's single `FDS_SYSTEM` label with `lsblk -o NAME,PARTLABEL,FSTYPE,MOUNTPOINTS`. Unmount any automatic -desktop mounts after flashing, safely disconnect the disks, install NVMe on the -powered-off Pi, and connect exactly one SYSTEM. Do not leave the NVMe enclosure -connected as a duplicate source of FDS partitions. +desktop mounts after flashing, safely disconnect the disks, insert microSD into +the native slot of the powered-off Pi, and connect exactly one SYSTEM. Remove the card reader and +maintenance USB before the first FDS boot. ## 6. Power on and reach the console Power the monitor and any required hub, attach video, USB control, keyboard and -SYSTEM, then power the Pi. Remove the maintenance SD for this attempt so an SD -fallback cannot be mistaken for an FDS boot. +SYSTEM, then power the Pi. Keep the FDS SD inserted and maintenance USB unplugged +so Raspberry Pi OS cannot be mistaken for an FDS boot. -The expected sequence is Pi firmware → kernel and stage0 from NVMe → read-only +The expected sequence is Pi firmware → kernel and stage0 from SD → read-only SYSTEM from USB → native s6 → `FDS>`. The production image is quiet, so a missing splash or scrolling kernel log is not by itself a failure. At **FDS**, run: @@ -232,7 +264,7 @@ fds boot-profile ``` Expect the intended version/profile, an EROFS root mounted read-only, and native -s6 as PID 1. `fds machine status` should report `internal_nvme` for valid internal +s6 as PID 1. `fds machine status` should report `internal_sd` for valid internal settings; `image_defaults` means settings loading fell back and needs diagnosis. An empty initial map still legitimately reports `UNCONFIGURED` bays. Stage0 discovers SYSTEM by its GPT partition label before bay mapping, so that does not @@ -318,20 +350,20 @@ fds machine install /tmp/my-machine fds reboot ``` -Installation writes the complete settings bundle to NVMe and retains the previous +Installation writes the complete settings bundle to SD and retains the previous one; it does not change the active mapping during this boot. With SYSTEM still absent, select `recovery` again after reboot. Verify `fds machine status`, then `fds bay 01` as you move the test device through the configured slots. Use `fds eject N` and wait for SAFE before removing storage managed in a mapped bay. Later, [export a backup and update settings from DATA](Internal-Storage#update-settings-from-recovery). -Do not reflash the whole NVMe merely to change bay numbering: that would replace +Do not reflash the whole SD merely to change bay numbering: that would replace saved settings and diagnostics. For normal boot, shut down, insert SYSTEM, and power on again. ## 9. Save evidence and shut down -At **`RECOVERY#`**, collect a fresh capture and explicitly save it to NVMe before +At **`RECOVERY#`**, collect a fresh capture and explicitly save it to SD before losing the RAM filesystem: ```sh @@ -371,13 +403,15 @@ After basic console/display checks, try the [optional desktop](Desktop) and | Observation | Next check | | --- | --- | -| Firmware cannot find NVMe | Adapter/ribbon seating, supply, EEPROM boot order, and conditional `PCIE_PROBE=1`; return to the maintenance SD | +| Firmware cannot boot SD | Native slot seating, complete `fds-internal.img` rather than a partition payload, `FDS_BOOT` contents and SD-enabled EEPROM order; use maintenance USB | +| Firmware still reports NVMe timeouts | Its active boot order still contains `6`; read back the EEPROM configuration and remove NVMe from the selected order | +| Maintenance USB will not boot | Check `0xf14`/`0xf41`, Raspberry Pi OS image, USB power warning and whether a bootable SD took priority | | Stage0 says SYSTEM is missing | Whole GPT SYSTEM image, exactly one `FDS_SYSTEM`, USB power and `list`; bay calibration is not required at this stage | | HDMI is blank or unstable | Monitor input/power, cable connector sizes, then [native HDMI EDID](Dasung#try-the-monitors-own-hdmi-edid) or the serial console below | | Kernel text appears but keyboard cannot reach a prompt | Check whether `BOOT_MODE=development` made UART the primary console | | Picture works, `dasungd` does not | USB data cable, both USB IDs, DRM serial, permissions and daemon log; see [Dasung diagnostics](Dasung) | | `fds bays` is unconfigured | Complete step 8 using physical topology observations | -| `fds machine status` uses image defaults | Check the reported reason, NVMe transport, partition layout and clean internal ext4; do not run DATA repair against internal NVMe | +| `fds machine status` uses image defaults | Check the reported reason, native SD placement, matching rebuilt SYSTEM, partition layout and clean internal ext4; do not run DATA repair against the SD | | USB resets or disappears under load | Reduce attached devices and check the Pi, monitor and hub supplies separately | ### Use a serial console when HDMI is unavailable @@ -394,9 +428,29 @@ enables verbose kernel output and selects `ttyAMA10` as the primary userspace console. The development EEPROM profile separately enables firmware UART output. Boot mode does not grant root; select recovery for a root shell. -For an already provisioned NVMe, use the +For an already provisioned SD, use the [offline BOOT edit procedure](Dasung#edit-the-boot-command-line-offline) and copy the single line from [cmdline-development.txt](http://gitea.home.arpa/felis/fds-os/src/branch/main/image/pi5/cmdline-development.txt) into the disk's `cmdline.txt`; preserve any deliberate native-EDID diagnostic change. This avoids overwriting settings to switch consoles. Restore the saved command line to return to display/keyboard operation after debugging. + +## Verify the SD and maintenance paths on hardware + +Record the card model/capacity, image hashes and observed results in the hardware +session. These checks are still pending on the physical Pi: + +1. With maintenance USB absent, cold-boot SD plus SYSTEM and confirm EROFS `/`, + native s6, `fds machine status` reporting `internal_sd`, and working input. +2. Shut down, remove SYSTEM, then boot SD and enter `recovery`. Confirm the same + storage source, install a measured settings bundle, save a diagnostic, and + reboot to verify both persist. Normal loading must not require ext4 repair. +3. Shut down and attach maintenance USB. With `0xf14`, keep FDS SD inserted and + confirm Raspberry Pi OS runs from USB while the SD remains a separate disk. + Use the [offline BOOT edit procedure](Dasung#edit-the-boot-command-line-offline) + if display diagnosis is needed. Unmount any SD filesystems before shutdown. +4. Remove maintenance USB after shutdown, restore SYSTEM and confirm normal FDS + boot. Recheck the settings and diagnostic saved in step 2. + +Do not remove the SD while FDS is running: it is also the recovery and settings +medium. An SD card does not remove the need for backups or controlled power-off. diff --git a/Home.md b/Home.md index cc5ce40..f604add 100644 --- a/Home.md +++ b/Home.md @@ -16,8 +16,8 @@ Use the sidebar or [all pages](Page-Index) to find a guide. | Guide | What you will learn | | --- | --- | | [Overview](Overview) | How the computer, internal storage and cartridges fit together | -| [Run on a Raspberry Pi 5](Hardware-Setup) | Wire HDMI/USB/power, prepare NVMe and SYSTEM, reach the first console, calibrate bays and diagnose hardware | -| [Flash disk images](Flashing) | Install NVMe and cartridges interactively or unattended with verification | +| [Run on a Raspberry Pi 5](Hardware-Setup) | Wire HDMI/USB/power, prepare SD and SYSTEM, reach the first console, calibrate bays and diagnose hardware | +| [Flash disk images](Flashing) | Install SD and cartridges interactively or unattended with verification | | [Build and start FDS](Getting-Started) | Prepare the OS build workstation, create images and boot the emulator | | [Software and emulator walkthrough](Workstation) | Build Void source packages, assemble cartridges, insert them and write USB media | | [Using cartridges](Cartridges) | Inspect bays, run applications, resolve command names and eject safely | diff --git a/Internal-Storage.md b/Internal-Storage.md index 3bdb340..a2f6871 100644 --- a/Internal-Storage.md +++ b/Internal-Storage.md @@ -41,7 +41,10 @@ The builder pads the recovery partition without changing its EROFS contents, aligns partitions to 1 MiB, writes both GPT copies, verifies each payload, and checks the table independently with `sfdisk`. Internal ext4 is fully initialized before deployment. It contains root-owned `config/` and private `diagnostics/`. +The default whole image is about 1.75 GiB, so a 32 GB card has ample capacity. Unused capacity beyond this initial layout is not automatically expanded. +SYSTEM stays on its separate USB cartridge. For a Raspberry Pi OS rescue USB, +follow [the maintenance setup](Hardware-Setup#4-prepare-usb-maintenance-and-select-sd-boot). For a 2 GiB recovery allocation or larger settings partition: @@ -51,10 +54,11 @@ For a 2 GiB recovery allocation or larger settings partition: ## Install the internal disk -Use an NVMe enclosure or another Linux machine that can access the target drive -while the Pi is off. First verify a downloaded release using a separately trusted -public key as described in [Release signatures](Releases). Its complete disk -is named `fds-internal-0.1.0.img`; a local build uses `out/fds-internal.img`. +Use an SD reader on the workstation while the Pi is off. The same disk layout +remains available for legacy PCIe NVMe installations. First verify a downloaded release using a separately trusted +public key as described in [Release signatures](Releases). A local build uses `out/fds-internal.img`. The frozen `0.1.0` release predates +native SD settings support; build a matching new internal image and SYSTEM +when moving to SD. Use the complete disk image, not the separate BOOT or RECOVERY payload. Install the workstation `fds-tools` package or run `make workstation-binaries`, @@ -63,7 +67,7 @@ then use [fds-flash](Flashing) for interactive or unattended installation: ```sh sudo ./out/workstation/fds-flash \ --image out/fds-internal.img \ - --device /dev/disk/by-id/REPLACE_WITH_THE_NVME_DISK + --device /dev/disk/by-id/REPLACE_WITH_THE_SD_DISK ``` Replace the destination with the verified whole-disk path. The tool shows model, @@ -86,10 +90,10 @@ media's contents; the internal image also replaces stored machine settings. The alternative development image is `out/fds-system-development.img`. Versioned release images use the same commands. -Confirm `FDS_BOOT`, `FDS_RECOVERY`, `FDS_INTERNAL` on NVMe and `FDS_SYSTEM` on +Confirm `FDS_BOOT`, `FDS_RECOVERY`, `FDS_INTERNAL` on SD and `FDS_SYSTEM` on SYSTEM with `lsblk -o NAME,PARTLABEL,FSTYPE,MOUNTPOINTS`. Unmount any automatic -desktop mounts after flashing, safely disconnect the media, and install NVMe in -the powered-off Pi. Insert exactly one SYSTEM cartridge before normal boot. +desktop mounts after flashing, safely disconnect the media, and insert SD in +the powered-off Pi's native slot. Insert exactly one SYSTEM cartridge before normal boot. Continue with [real-hardware setup](Hardware-Setup) and the separately reviewed [EEPROM configuration](EEPROM). @@ -130,9 +134,13 @@ its active snapshot is readable by the local FDS user. ## What happens at boot The `machine-config` native s6 oneshot precedes `cartridged`. It does **not** -precede the console or Dasung controller. It accepts exactly one non-removable -NVMe disk with the three named partitions in the order above. USB lookalikes are -ignored; multiple eligible NVMe disks are rejected instead of choosing by name. +precede the console or Dasung controller. It accepts exactly one native SD card +or non-removable PCIe NVMe disk with the three named partitions in the order +above. Native SD is identified through the MMC subsystem and card type `SD`, +regardless of the block device's removable flag. USB ancestry is always rejected; +USB readers, eMMC and SDIO cannot supply machine settings. Device names such as +`mmcblk0` are not identities. Multiple eligible disks, including an old NVMe +alongside SD, are rejected instead of choosing by name. The settings partition must be clean ext4 with the expected label. Loading uses `ro,noload,nosuid,nodev,noexec` in a private mount namespace, validates the entire @@ -152,8 +160,8 @@ fds machine export /tmp/current-machine ``` Export creates a new directory with the three editable source files. It never -overwrites an existing directory. The status source is `internal_nvme` or -`image_defaults`. A temporarily unavailable NVMe is not adopted later in the same +overwrites an existing directory. The status source is `internal_sd`, `internal_nvme` or +`image_defaults`. Temporarily unavailable internal storage is not adopted later in the same boot; resolve the issue and reboot to load its settings. This prevents changing bay identities underneath active cartridge operations. @@ -179,7 +187,7 @@ reinstall it if the new calibration is wrong. An invalid existing JSON document can be replaced this way. Wrong ownership, symlinks, an unclean filesystem or a damaged directory require offline filesystem -maintenance first. Recovery does not automatically repair internal NVMe; its +maintenance first. Recovery does not automatically repair internal SD/NVMe; its `fds recovery repair` command is deliberately limited to DATA cartridges. ## Save and retrieve diagnostics diff --git a/Overview.md b/Overview.md index 73e725e..72900e1 100644 --- a/Overview.md +++ b/Overview.md @@ -13,8 +13,9 @@ simple cartridge workflow: insert software, run it, and eject it when finished. | Component | Purpose | | --- | --- | -| Internal NVMe | Holds boot files, recovery and persistent machine settings | -| SYSTEM cartridge | Supplies the read-only operating system; swapping it changes the OS | +| Native microSD | Holds boot files, recovery and persistent machine settings | +| USB SYSTEM cartridge | Supplies the read-only operating system; swapping it changes the OS | +| Maintenance USB | Separate Raspberry Pi OS installation for offline repair; unplug during normal FDS boot | | PROGRAM cartridge | Supplies applications and their dependencies, ready to execute | | DATA cartridge | Stores writable files at `/data` | | ENVIRONMENT cartridge | Selects an installed environment, such as WindowMaker | diff --git a/Page-Index.md b/Page-Index.md index bbf1069..0e95872 100644 --- a/Page-Index.md +++ b/Page-Index.md @@ -83,3 +83,4 @@ include_toc: true - [M11 stress validation](Developer-M11-Validation) - [M12 production preparation: software evidence](Developer-M12-Validation) - [Workstation extension acceptance](Developer-Workstation-Validation) +- [Native SD migration validation](Developer-SD-Validation) diff --git a/Workstation.md b/Workstation.md index 9cb3793..2489abf 100644 --- a/Workstation.md +++ b/Workstation.md @@ -268,7 +268,7 @@ use `sudo ./out/workstation/fds-flash` instead of the installed command. The [flashing guide](Flashing) covers explicit device paths, dry-run JSON, unattended writes and disposable-file testing. It also covers the internal -NVMe and first SYSTEM disk; `fds-flash` accepts those complete disk images too. +SD and first SYSTEM disk; `fds-flash` accepts those complete disk images too. The older `fds-cartridge preview` / `write --confirm` interface remains supported for USB cartridge workflows. It performs deeper cartridge content inspection and diff --git a/_Sidebar.md b/_Sidebar.md index f8cf902..b048f3d 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -30,5 +30,6 @@ - [Software cartridge format](Software-Format) - [Editing the Gitea wiki](Gitea-Wiki) - [Developer notes](Developer-Guide) +- [Native SD migration](Developer-SD-Validation) - [Cartridge and desktop revision](Developer-Current-Revision) - [Implementation through M12](Developer-Implementation-Status)