update docs
This commit is contained in:
@@ -0,0 +1,210 @@
|
||||
# Recovery and rollback
|
||||
|
||||
Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.
|
||||
|
||||
|
||||
[Documentation index](README.md) · [Boot images](boot.md) · [Media tools](media-tools.md) · [EEPROM](eeprom.md)
|
||||
|
||||
Recovery is a separate, read-only FDS image. It contains native s6, Bash, GNU
|
||||
utilities, XBPS, filesystem tools, the static FDS tools, and the base Dasung
|
||||
controller. It does not need files or programs from a SYSTEM cartridge.
|
||||
The software workflow is covered by `make recovery-test`; physical Pi recovery remains deferred.
|
||||
The [complete internal disk](internal-storage.md) includes this recovery image;
|
||||
its guide separates verified image construction from deferred physical
|
||||
installation. The build commands below create ordinary files and do not write a
|
||||
host disk.
|
||||
|
||||
## Build the recovery image
|
||||
|
||||
Use the prepared x86_64 build host described in [Your first build](getting-started.md).
|
||||
From the repository root, run these commands sequentially:
|
||||
|
||||
```sh
|
||||
make recovery
|
||||
make rootfs-test
|
||||
make recovery-test
|
||||
```
|
||||
|
||||
The full VM test also needs the Pi kernel/initramfs and a known-good CLI SYSTEM
|
||||
image. On a fresh checkout, build those first with `make rootfs PROFILE=cli`,
|
||||
`make rootfs-test`, `make initramfs`, and `make system-card PROFILE=cli`, then
|
||||
run the recovery sequence above. Run builds and VM tests sequentially because
|
||||
they share the same project-local Void environment.
|
||||
|
||||
`make recovery` builds the `recovery` rootfs profile independently, configures all
|
||||
packages and caches at image construction time, then creates EROFS. Its outputs
|
||||
are:
|
||||
|
||||
| File | Meaning |
|
||||
| --- | --- |
|
||||
| `out/rootfs-recovery.tar` | Complete recovery rootfs, including package database and compiled s6 configuration |
|
||||
| `out/fds-recovery.img` | Raw EROFS payload for the internal GPT partition named `FDS_RECOVERY` |
|
||||
| `out/recovery-build.*/manifest.json` | Rootfs hash, filesystem hash, profile and size |
|
||||
| `out/manifests/rootfs-latest/` | Exact selected package archives and build input records |
|
||||
|
||||
The latest-rootfs pointers also select this recovery build. Profile-specific
|
||||
rootfs pointers preserve earlier CLI/development builds. A recovery EROFS payload
|
||||
is not a whole-disk image and is not a SYSTEM cartridge. The SYSTEM builder
|
||||
rejects a recovery rootfs to prevent confusing those roles.
|
||||
|
||||
## Enter recovery
|
||||
|
||||
Stage0 can enter recovery in two ways:
|
||||
|
||||
1. At its missing-SYSTEM or ambiguous-SYSTEM console, type `recovery` and Enter.
|
||||
2. For a deliberate maintenance boot, use `fds.boot=recovery` in the boot
|
||||
partition's single-line `cmdline.txt`, replacing `fds.boot=normal` if present.
|
||||
Restore normal mode when maintenance is complete.
|
||||
|
||||
Stage0 requires exactly one readable `FDS_RECOVERY` partition and mounts it
|
||||
read-only. It does not silently choose between duplicate partitions. The ARM VM
|
||||
suite supplies disposable virtual partitions; physical firmware/NVMe/display
|
||||
behavior must still be checked on the Pi.
|
||||
|
||||
The local prompt is:
|
||||
|
||||
```text
|
||||
FDS RECOVERY — LOCAL MAINTENANCE CONSOLE
|
||||
RECOVERY#
|
||||
```
|
||||
|
||||
This is an explicit **root maintenance console**. It reads startup files from the
|
||||
immutable recovery image and uses a temporary home under `/run/fds`. The root
|
||||
password remains locked; no SSH or network login service is started. Normal CLI
|
||||
and development images continue to use the ordinary `fds` account and `FDS>`.
|
||||
|
||||
Recovery inspects cartridges read-only. It does not automatically mount DATA
|
||||
writable, activate an ENVIRONMENT desktop, or enable Ethernet. Those actions
|
||||
require explicit commands. The Dasung controller remains in the base boot bundle.
|
||||
|
||||
## Inspect a failed system
|
||||
|
||||
```sh
|
||||
fds info
|
||||
fds bays
|
||||
fds bay 2
|
||||
fds inspect BAY02
|
||||
fds topology
|
||||
fds boot-profile
|
||||
lsusb -t
|
||||
lsblk -o NAME,MAJ:MIN,MODEL,SERIAL,SIZE,RO,TYPE,FSTYPE,PARTLABEL,MOUNTPOINTS
|
||||
dmesg
|
||||
```
|
||||
|
||||
Use the calibrated bay number and the displayed model/serial to identify a
|
||||
cartridge. Do not infer a bay from `/dev/sda` or from discovery order. A valid
|
||||
inactive SYSTEM mounts read-only under `/run/fds/media/NN`; `fds bay N` reports
|
||||
the actual path. A bad filesystem or manifest produces an error instead of
|
||||
running anything from the cartridge.
|
||||
|
||||
An empty bay configuration produces `UNCONFIGURED`, not guessed bay numbers.
|
||||
See [Cartridges](cartridges.md) for calibration. Permanent machine configuration
|
||||
on `FDS_INTERNAL` is part of the remaining M12 work.
|
||||
|
||||
## Check and repair DATA
|
||||
|
||||
Start with a read-only check. For example, for DATA in bay 2:
|
||||
|
||||
```sh
|
||||
fds recovery check BAY02
|
||||
```
|
||||
|
||||
This command is available only to root in the recovery image. It identifies one
|
||||
USB disk and one `FDS_DATA` partition, unmounts its read-only view, reserves the
|
||||
disk exclusively, verifies GPT and the kernel partition identity, and invokes
|
||||
`e2fsck -f -n`. It does not repair the filesystem. A clean result leaves DATA
|
||||
unmounted and reports `SAFE TO REMOVE`.
|
||||
|
||||
If DATA was explicitly activated writable, eject that session first. Close any
|
||||
shell whose current directory is on DATA and any other reader before checking;
|
||||
an ordinary busy-unmount failure is reported, never bypassed with lazy unmount.
|
||||
|
||||
If the check reports problems, review its log and preserve a backup where
|
||||
possible. Preview the repair with:
|
||||
|
||||
```sh
|
||||
fds recovery repair BAY02
|
||||
```
|
||||
|
||||
The preview displays the model, serial, capacity and a confirmation command.
|
||||
Copy that exact command only after checking the intended cartridge. Its token
|
||||
binds the bay, kernel insertion number and current boot. A token from another
|
||||
insertion or boot is rejected. The preview makes no filesystem changes.
|
||||
|
||||
Confirmed repair runs `e2fsck -f -p`, which performs conservative automatic
|
||||
repairs and stops when manual judgement is required. It then flushes the device,
|
||||
invalidates its block cache, and runs a second `e2fsck -f -n`. Only a successful
|
||||
verification produces `DATA REPAIRED AND VERIFIED` and `SAFE TO REMOVE`.
|
||||
|
||||
The checker uses a temporary kernel loop device backed by the already verified
|
||||
partition descriptor. This lets `e2fsck` take its own exclusive device claim
|
||||
while FDS retains the physical whole-disk reservation. The loop is removed
|
||||
automatically when its last descriptor closes. This uses the existing kernel
|
||||
loop driver, `libc` crate and base `e2fsprogs`; no new package or Rust dependency
|
||||
is introduced.
|
||||
|
||||
Failed or interrupted checks/repairs retain a quarantine record across cartridge
|
||||
daemon restarts for the same insertion. They do not inherit an earlier SAFE
|
||||
status. The checker holds the disk reservation and is killed if its supervising
|
||||
daemon dies. The command runs synchronously: bay operations and orderly shutdown
|
||||
wait for it to finish; the local shell remains usable. A client timeout is not
|
||||
success and does not authorize removing media. Inspect the log and wait for the
|
||||
service to finish before retrying. Logs are in `/run/fds/recovery/` and are
|
||||
limited to 16 MiB per invocation.
|
||||
|
||||
This narrow recovery command requires a readable GPT and primary ext filesystem
|
||||
signature. It deliberately does not guess partition boundaries, recreate a
|
||||
broken GPT, or force answers to destructive `e2fsck` questions. Such cases require
|
||||
an offline backup and expert use of the included filesystem tools. It cannot
|
||||
recover data that was never written to storage.
|
||||
|
||||
For structured results, prefix the command with `fds --json`. After a successful
|
||||
check or repair, remove/reinsert DATA to inspect it again. Recovery still keeps
|
||||
it read-only until `fds data use N` is explicitly requested.
|
||||
|
||||
## Prepare replacement SYSTEM media
|
||||
|
||||
A known-good SYSTEM image must be supplied on separate source media. Do not use
|
||||
a damaged image as an update source. For example, source DATA in bay 2 and an
|
||||
unmounted replacement cartridge in bay 4:
|
||||
|
||||
```sh
|
||||
fds bay 2
|
||||
fds inspect image /run/fds/media/02/fds-system-cli.img
|
||||
fds inspect BAY04
|
||||
fds burn system /run/fds/media/02/fds-system-cli.img BAY04
|
||||
```
|
||||
|
||||
Use the actual source mount shown by `fds bay 2`. If the destination already
|
||||
contains a mounted cartridge, run `fds eject 4` first. The burn preview identifies
|
||||
the source checksum, destination capacity/model/serial and confirmation command.
|
||||
Follow [Media creation and writing](media-tools.md) for confirmation and status commands.
|
||||
Completion requires device flush, readback and GPT verification. Recovery uses
|
||||
the same protected writer as the main system: it refuses mounted destinations,
|
||||
active root storage and disks containing internal FDS partition names.
|
||||
|
||||
Keep the previous known-good SYSTEM cartridge. After a successful replacement
|
||||
write, use `fds poweroff`, swap SYSTEM while powered off, and boot normally.
|
||||
Rollback means restoring the previous SYSTEM cartridge; it does not undo DATA
|
||||
file changes, application migrations, internal boot updates or EEPROM changes.
|
||||
|
||||
## Capture diagnostics and shut down
|
||||
|
||||
```sh
|
||||
bash /usr/share/fds/capture-hardware /tmp/recovery-capture
|
||||
cat /tmp/recovery-capture/status.tsv
|
||||
fds power status
|
||||
fds poweroff
|
||||
```
|
||||
|
||||
The collector records identities, USB topology, mounts, kernel messages, packages
|
||||
and boot events with checksums. Its output is temporary unless copied to a
|
||||
healthy DATA cartridge after explicitly activating that DATA with `fds data use N`.
|
||||
Do not activate the damaged cartridge merely to save a report. Eject writable
|
||||
DATA after copying, or let `fds poweroff` perform the normal verified shutdown.
|
||||
See [Shutdown](power.md) when shutdown reports a blocking DATA error.
|
||||
|
||||
Recovery is independent of SYSTEM, but it still depends on working internal
|
||||
boot storage, the kernel and firmware. An external rescue medium is required
|
||||
when that layer fails. EEPROM preparation and rollback are described separately
|
||||
in [EEPROM configuration](eeprom.md); no EEPROM is changed by these recovery tools.
|
||||
Reference in New Issue
Block a user