Files
fds-os/docs/developer/troubleshooting.md
T
2026-09-23 03:12:45 +08:00

247 lines
12 KiB
Markdown

# Troubleshooting the build
Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.
[Documentation index](README.md) · [First build](getting-started.md) · [Build reference](build-host.md)
Find the first failing command or `ERROR:` in the log, fix that problem, then
repeat the same step. Do not continue past a failed bootstrap and interpret later
missing files as independent problems. Run all commands below from the repository
root, as your regular user, unless explicitly shown otherwise.
## Find the right log
| Failing action | Where to look |
| --- | --- |
| `make bootstrap` | `out/logs/bootstrap.log` |
| Void build-container creation | `out/logs/xbps-bootstrap.log` |
| `make smoke-test` | `out/logs/smoke-test.log` |
| GNU hello build | `out/logs/xbps-hello.log` |
| `make check` | Terminal output; this target does not save a log automatically |
For example, after a failed smoke-test:
```sh
tail -n 80 out/logs/smoke-test.log
rg -n 'ERROR:|FAILED|error:|No space left|Permission denied' out/logs/
df -h .
```
A search with no matches exits 1. Read the surrounding log too: upstream tools
may report failures using other wording. Save logs before re-running if you need
the previous output; see [Development](development.md#preserve-logs-and-identify-inputs).
## Unsupported host, root execution, or missing commands
- `M0 requires an x86_64 Linux build host` or `supports Arch Linux`: use the
supported host described in [Your first build](getting-started.md). Changing
the check alone does not port the build system to another platform.
- `Run as a normal user, not root`: leave the root shell and run the build from
your normal account. Only host package installation uses sudo.
- `Missing host command`: install the prerequisites from the first-build guide,
then repeat bootstrap. `readelf` comes from binutils, `bwrap` from bubblewrap,
and `flock` from util-linux.
- `checkout path without whitespace`: move or obtain the checkout at a path
with no spaces, tabs, or other whitespace, then use that directory consistently.
If a previous root-run left files that your account cannot write, inspect their
ownership before correcting it. Do not respond by running all later builds as root.
## Bubblewrap or user namespace failure
Reproduce the same capability check bootstrap uses:
```sh
bwrap --ro-bind / / --unshare-user --uid 0 --gid 0 true
```
If it reports `Operation not permitted` or a namespace error, the host kernel,
container policy, or security configuration prevents the unprivileged container.
Use an Arch host that permits this operation, or have the host administrator
resolve the restriction. Repeated downloads, sudo builds, and a different Pi
will not resolve this preflight failure. Success is a zero exit status with no
output; run `make bootstrap` again once that works.
## Void submodule missing, wrong, or modified
Inspect before changing anything:
```sh
cat VOID_PACKAGES_COMMIT
git submodule status vendor/void-packages
git -C vendor/void-packages status --short
git -C vendor/void-packages diff HEAD --
```
If the submodule is missing, `make bootstrap` initializes it. The directory must
be part of a Git checkout; an unpacked source archive is insufficient. If a clone
was interrupted by a network error, restore connectivity and repeat bootstrap.
`Void checkout mismatch` means the checked-out commit differs from the explicit
pin. A `+` in `git submodule status` means it differs from the parent repository's
gitlink. These are two separate checks. Do not update the pin merely to silence
the error; first determine why the commits differ.
`Tracked Void files were changed` means upstream source has local edits. Preserve
and review the diff. Move intentional FDS changes into the overlay design or a
separate patch for review; restore upstream only after saving work you need.
Generated ignored files such as `etc/conf`, `hostdir`, and the masterdir are
normal. Never use a broad reset or clean command without checking what it removes.
## Configuration or overlay conflict
For `etc/conf differs from config/xbps-src.conf`, compare the two:
```sh
diff -u .host/void-packages/etc/conf config/xbps-src.conf
```
Put the desired settings in `config/xbps-src.conf`, then follow
[the explicit synchronization procedure](development.md#change-build-configuration-deliberately).
A `diff` exit status of 1 means differences were found, not that the comparison failed.
For `Stale overlay`, compare the named `.host/void-packages/srcpkgs/fds-*`
directory with its source under `packages/`. Preserve any unique edits in the
source first. Move that specific generated copy into a backup directory under
`out/`, outside `srcpkgs/`, then run `./tools/prepare-void` again. Do not move or
remove unrelated upstream packages. The Dasung package is now an active overlay. After changing it, reconcile that
generated copy by this same procedure before rebuilding.
`Overlay would replace an upstream package` is an intentional protection. Choose
an FDS-owned package name and design an overlay; do not disable the check or edit
the tracked upstream template in place.
## Download failure, checksum mismatch, or apparent stall
Bootstrap and package builds need network access. The first run can spend much
of its time downloading compiler packages. Read the latest log and check disk
space before assuming that a quiet download is a deadlock:
```sh
tail -n 40 out/logs/bootstrap.log
tail -n 40 out/logs/xbps-hello.log
du -sh out/downloads .host/void-packages/hostdir
df -h .
```
A log or directory may not exist if its step has not started. Bootstrap's static
XBPS download has retries and a timeout; an incomplete transfer uses a `.part`
filename. A retry re-downloads that partial file. Completed cached archives are
checksum-checked on every bootstrap.
For a checksum failure, retain the error and compare the named file against
`config/host-tools.conf`. If the cached completed archive is corrupt, move that
specific archive to a backup location and repeat bootstrap so it downloads again.
Do not change the expected hash to match an unexplained download. If a clean
re-download still disagrees, stop and investigate the source.
A Void mirror timeout is separate from the source pin: the templates are pinned,
but bootstrap and missing build dependencies still use rolling binary repositories.
An unavailable historical dependency may need a reviewed input/mirror solution;
blindly changing the source commit does not establish an equivalent build.
## XBPS command not found or package index seems empty
The XBPS tools are intentionally not installed in the host's `/usr/bin`.
`make bootstrap` creates `.host/xbps/usr/bin/`. FDS build helpers set PATH for you.
For manual queries, use the explicit paths in the [package guide](packages.md).
When querying or indexing ARM output, set `XBPS_ARCH=aarch64`. The host's static
musl executable can otherwise default to its own architecture and ignore the ARM
package. `make smoke-test` uses the correct override. `tools/build-package` exports
packages but does not update `out/packages/aarch64-repodata`; use the indexing
command in the package guide after a standalone package build.
## Rust target, linker, or architecture error
Check the selected toolchain and local configuration:
```sh
rustup show active-toolchain
rustup target list --installed
cat rust-toolchain.toml
cat .cargo/config.toml
printenv RUSTFLAGS CARGO_BUILD_TARGET CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER
```
`printenv` exits nonzero when a listed variable is unset; that is normal here.
Bootstrap installs the required target and rustfmt for the pinned toolchain.
Repeat it if those components are missing.
Build from the repository root with the exact Cargo command in
[Development](development.md#edit-and-rebuild-the-rust-program). Environment
variables, command-line `+toolchain` overrides, or outside Cargo configuration
can change the target, flags, or linker. Reconcile such overrides with the
checked-in `rust-lld` and static CRT configuration. Adding an arbitrary system
linker or glibc library is not a repair for a static-musl build.
## Exec format error, ldd output, and QEMU skips
`./out/fds-smoketest` normally gives `Exec format error` on x86_64: it is an ARM
program. Run it explicitly with optional `qemu-aarch64`, as described in
[the first-build guide](getting-started.md#7-optionally-execute-the-arm-rust-program).
Do not change the target to x86_64 merely to make direct execution work.
`ldd: not a dynamic executable` with exit 1 is expected for the static program.
Use `./tools/verify-elf out/fds-smoketest aarch64 static` for the actual check.
Host `ldd` cannot establish that a foreign ARM program will run.
`SKIP: ARM execution` means QEMU is absent. Compilation and ELF checks still run.
If QEMU is present but fails, the smoke-test must fail; inspect its diagnostic.
The dynamic GNU hello program also requires an ARM glibc loader and libraries,
so the simple static-program QEMU command is not a complete hello runtime setup.
## Missing artifacts or check fails before any build
`make check` expects the Cargo release artifact in `target/`. Run the Rust build
or `make smoke-test` first. A bare Cargo build updates `target/`, while
`make smoke-test` also refreshes `out/fds-smoketest` and the output manifests.
If checksums fail after editing or replacing outputs, inspect what changed and
rebuild through smoke-test; do not edit the manifest to conceal a mismatch.
For a Rust formatting failure, run `cargo fmt --all`, inspect the source diff,
then repeat `make check`.
## Busy /tmp message during Void bootstrap
The initial verified bootstrap emitted
`rm: cannot remove '//tmp': Device or resource busy` during upstream cleanup.
The bubblewrap bind mount explained that instance; upstream exited successfully,
and container GCC, the cross build, and the public M0 commands all passed.
Do not treat every later `/tmp` error as harmless. Confirm a zero command exit
status and the final bootstrap PASS, then run smoke-test. If the command exits
nonzero, report it as a failure with the surrounding log.
## Rootfs, images, and runtime commands
M1 implements `make rootfs PROFILE=cli` and `make rootfs-test`. If the target is
missing, check that you are in the current checkout. Read [M1 rootfs](rootfs.md)
for prerequisites, shell access, logs, and retained failed-build directories.
Python 3.14+ is required for XBPS archive inspection. Supported profiles are
`cli`, `development`, and `recovery`; unknown names fail explicitly. Package
configuration must pass before an archive is published.
`tools/in-rootfs` supports ARM child execution through a private user-namespace
binary-format handler. It requires Linux 6.7+ and namespaced `binfmt_misc` support.
The workstation's global registry is unchanged. For a single ELF executable,
`--direct` invokes QEMU without installing even a private child-execution handler.
For example: `./tools/in-rootfs out/rootfs-aarch64 --direct /usr/bin/xbps-pkgdb -a`.
The tar is configured userspace, not a flashable image. M2 adds `make init-test`
for a full ARM boot and `make vm` for a temporary development shell. Follow
[Native init and ARM VM](init.md) for its dependencies, expected output, and
failure logs. For the full kernel/stage0/SYSTEM path, use `make all`,
`make boot-test`, then `make console-vm`. Run `fds bays` and `fds poweroff`
inside the booted FDS console, not on the Arch host. An exported ARM binary will
not execute directly on x86_64. Physical Pi acceptance remains deferred.
## What to include in a bug report
Include the exact failing command, its exit status if available, the relevant log
excerpt, host distribution and architecture, current Void pin, and whether this
is the first build or a previously working checkout. Mention local configuration
changes and toolchain overrides. Preserve the initial error; a later missing-file
message often only describes its consequence.