181 lines
8.8 KiB
Markdown
181 lines
8.8 KiB
Markdown
# Performance targets and measurement
|
||
|
||
Development reference and historical context. For current operating instructions, use the [user manual](../README.md). Acceptance applies only to the source and artifacts identified in each record.
|
||
|
||
|
||
[Documentation index](README.md) · [Boot](boot.md) · [Services](services.md)
|
||
|
||
**Physical Pi performance remains unmeasured.** The target budgets below are
|
||
requirements. M5 now records actual software boot events and provides full-system
|
||
ARM VM measurements; these are kept separate from physical targets. The numerical
|
||
example in the master plan remains illustrative, not a benchmark.
|
||
|
||
## Target budgets
|
||
|
||
| Transition | Target | Current measurement |
|
||
| --- | --- | --- |
|
||
| Kernel entry → usable CLI | Under 1.5 s desired; under 2.0 s maximum target | Not measured |
|
||
| s6 running → usable CLI | Under 400 ms | Not measured |
|
||
| Cold power → usable CLI | Under 3 s stretch goal; under 5 s hard goal | Not measured |
|
||
| CLI → usable WindowMaker | Under 2 s | Not measured |
|
||
| Idle power-off | Under 1 s | Not measured |
|
||
| Normal power-off with writable DATA | Under 1.5 s | Not measured |
|
||
|
||
“Usable” means accepting input, not merely drawing a banner or starting a process.
|
||
A quick screen followed by a blocked console does not meet the console target.
|
||
Shutdown must finish required data operations; losing writes is not an optimization.
|
||
|
||
## Timestamp boundaries
|
||
|
||
| Marker | Event |
|
||
| --- | --- |
|
||
| T0 | Power applied |
|
||
| T1 | Kernel entered |
|
||
| T2 | `fds-stage0` entered |
|
||
| T3 | Root transition (`switch_root` or equivalent) |
|
||
| T4 | s6 running |
|
||
| T5 | CLI accepts input |
|
||
| T6 | WindowMaker accepts input |
|
||
|
||
Kernel-to-console is T5 − T1; s6-to-console is T5 − T4; cold boot is T5 − T0.
|
||
For desktop activation after the console is already usable, record a separate
|
||
activation-request timestamp and measure through T6. Do not include arbitrary
|
||
user delay at the prompt as desktop startup time.
|
||
|
||
The implemented `fds-boottrace` records stage0 entry, SYSTEM discovery, root mount,
|
||
root switch, s6 stage-2 entry and first-console readiness. Desktop readiness is
|
||
recorded by the M8 desktop session after its startup checks and remains absent
|
||
when no desktop has started. Events use Linux
|
||
`CLOCK_BOOTTIME`; its origin is the kernel boot clock, not external power-on.
|
||
The s6 marker is the start of stage 2, after native PID 1 exists. Console readiness
|
||
is recorded immediately before Bash presents its first prompt; the VM test then
|
||
sends real commands and verifies their results. These instrumentation boundaries
|
||
are explicit approximations to T1/T4/T5, not external electrical/display measurements.
|
||
|
||
## Collect and compare a report
|
||
|
||
On FDS, as the ordinary console user:
|
||
|
||
```sh
|
||
fds boot-profile
|
||
fds --json boot-profile > ~/boot-profile.json
|
||
```
|
||
|
||
The report lists measured intervals and missing events. A direct-root M2 VM has
|
||
no stage0 events; they remain absent rather than becoming zero-duration samples.
|
||
The default home is temporary until DATA integration, so export any report you
|
||
want to retain before powering off. Host tests save their reports under `out/`.
|
||
|
||
To compare two saved reports on FDS:
|
||
|
||
```sh
|
||
fds-boottrace compare baseline.json current.json
|
||
```
|
||
|
||
The command requires the same platform and clock and returns nonzero for a
|
||
kernel-to-console regression greater than 100 ms. A justified regression can be
|
||
recorded with `--explain "Reason for the measured regression"`; keep the explanation
|
||
and raw reports with the change. This does not make unlike scenarios comparable.
|
||
|
||
On the workstation, run the actual ARM tool through the project emulator:
|
||
|
||
```sh
|
||
./tools/in-void qemu-aarch64 "$PWD/out/fds-boottrace" compare \
|
||
"$PWD/baseline.json" "$PWD/current.json"
|
||
```
|
||
|
||
`make console-test` boots the ordinary-user console, verifies temporary home and
|
||
read-only SYSTEM, collects a trace, and tests the regression threshold. A second
|
||
boot holds eudev behind a FIFO gate: the test uses the prompt successfully before
|
||
releasing the gate. There is no sleep or artificial readiness delay in the target.
|
||
The gate exists only in a disposable test image.
|
||
|
||
## Physical measurement procedure
|
||
|
||
Each benchmark record should contain:
|
||
|
||
1. Hardware identity: Pi revision, firmware/EEPROM version, NVMe and SYSTEM media,
|
||
USB hubs/devices, power supply/battery conditions, and display setup.
|
||
2. Software identity: FDS revision, kernel configuration/version, image digest,
|
||
package/input records, active profile, and relevant boot configuration.
|
||
3. Scenario: cold power-on versus reboot, number and classes of inserted devices,
|
||
DATA presence, network state, and whether media or caches were already warm.
|
||
4. Instrumentation: where each marker was captured, timestamp resolution, clock
|
||
source, and how external power-on timing aligns with kernel timestamps.
|
||
5. Repeated raw samples, sample count, a typical value and observed range, with
|
||
failed or incomplete runs recorded rather than silently discarded.
|
||
|
||
Use a monotonic time source for intervals within the running OS. T0 occurs before
|
||
the kernel can timestamp anything, so cold-power timing needs external capture
|
||
or another explicitly justified measurement method. Do not subtract unrelated
|
||
clock readings and present the result as a precise interval.
|
||
|
||
Boot with one SYSTEM, with DATA, and with all 12 bays populated should be separate
|
||
scenarios. Missing SYSTEM followed by insertion is also a separate readiness test.
|
||
A single favorable run cannot establish behavior under the hardware stress cases.
|
||
|
||
## Regression and optimization rules
|
||
|
||
The master plan requires any boot regression greater than 100 ms to be explained
|
||
or fixed. Compare like-for-like hardware, images, scenarios, and measurement
|
||
boundaries; keep the raw measurements with the change report.
|
||
|
||
Optimization should remove work from the console's dependency path or make that
|
||
work faster. Prepare service databases and caches during image construction.
|
||
Start optional networking and desktop services when needed. Handle device readiness
|
||
through events and explicit dependencies. Do not add sleeps for a retro effect,
|
||
wait for all USB devices before the console, or hide unfinished work behind READY.
|
||
|
||
Rust's current `opt-level=z` is a starting configuration, not evidence that it is
|
||
the fastest choice for future control programs. Real programs and hardware must
|
||
guide later comparisons.
|
||
|
||
## Recorded VM comparison
|
||
|
||
The final M5 comparison is stored in `out/rust-profiles.x_wgx1kp/`, with nine
|
||
serial logs, nine raw reports, executable digests and `comparison.json`.
|
||
It uses the real Pi kernel and SYSTEM EROFS, QEMU 11.1.1 TCG, `virt`, `max` CPU,
|
||
1 GiB RAM and two virtual CPUs. Only stage0's Rust optimization level changes;
|
||
the kernel and SYSTEM image are held constant. Each level has three measured boots.
|
||
|
||
| Stage0 optimization | Static executable size | Median kernel-clock → console | Median s6 stage 2 → console |
|
||
| --- | ---: | ---: | ---: |
|
||
| `z` | 474,128 bytes | 1,788.575 ms | 714.436 ms |
|
||
| `2` | 517,688 bytes | 1,692.750 ms | 665.673 ms |
|
||
| `3` | 533,224 bytes | 1,714.568 ms | 702.154 ms |
|
||
|
||
These are **emulator measurements on a development workstation**, including its
|
||
scheduling noise. Three samples do not establish a hardware winner. Keep `z`
|
||
as the current default and repeat on the assembled Pi before selecting an
|
||
optimization level for physical boot speed. In particular, these numbers do not
|
||
establish either the physical 2-second console target or the 400 ms s6 target.
|
||
|
||
Reproduce the comparison after building the current SYSTEM and boot inputs:
|
||
|
||
```sh
|
||
make performance-test
|
||
```
|
||
|
||
The helper creates separate build directories and initramfs files for each level;
|
||
it does not overwrite the normal tool or initramfs outputs. Read
|
||
`out/rust-profiles-latest/comparison.json` for the measured artifact identities.
|
||
The uncompressed initramfs remains the default. All four formats boot in the M4
|
||
matrix, but compression speed must still be compared on physical NVMe/Pi hardware.
|
||
|
||
M10 records shutdown events and VM observations in [its validation report](m10-validation.md).
|
||
M11 records repeated empty/twelve-device VM comparisons, including the
|
||
reviewed greater-than-100 ms samples, in [its validation report](m11-validation.md).
|
||
No physical firmware, cold-power, desktop or shutdown timing is claimed here.
|
||
|
||
M12's [repeated console checkpoint](m12-validation.md#repeated-console-timing-checkpoint)
|
||
compares five alternating pairs with fixed kernel/initramfs inputs after the
|
||
startup clock floor and release-tool packaging changes. The median increased
|
||
29.435310 ms; the initial greater-than-100 ms single-sample difference was
|
||
investigated and its raw result retained.
|
||
|
||
The subsequent [Clap migration checkpoint](clap-validation.md#executable-size-and-startup-measurements)
|
||
passed five alternating before/after boot pairs with a −37.276280 ms median
|
||
change and repeated the three-sample `z`/`2`/`3` comparison on the updated
|
||
stage0. These are image-level VM observations; the default remains `z` and
|
||
physical Pi measurements are still deferred.
|