Files
Codex fc62400157
ci / prune-artifacts (pull_request) Skipped
ci / kernel-build (pull_request) Skipped
ci / patches-apply (pull_request) Successful in 37s
ci / prune-artifacts (push) Skipped
ci / kernel-build (push) Skipped
ci / test (pull_request) Successful in 26s
ci / test (push) Successful in 27s
ci / bench (pull_request) Successful in 29s
ci / bench (push) Successful in 22s
ci / quality (pull_request) Successful in 1m54s
ci / patches-apply (push) Successful in 40s
ci / mcdc (pull_request) Successful in 32s
ci / mcdc (push) Successful in 22s
ci / badges (pull_request) Skipped
ci / kernel-symbols (push) Successful in 1m58s
ci / qemu-tools (pull_request) Successful in 5m7s
ci / kernel-symbols (pull_request) Successful in 5m28s
ci / quality (push) Successful in 1m49s
ci / badges (push) Skipped
ci / qemu-tools (push) Successful in 5m15s
Use a pinned BusyBox source mirror
Try the Buildroot release mirror after a failed primary download, retaining
the existing SHA256, bounded attempts and verified-cache behavior. Keep the
new timeout, corruption and production-call controls in regular CI.

Fixes #41.
2026-11-03 14:35:42 -06:00
..
2026-11-03 14:35:42 -06:00
2026-11-03 14:35:42 -06:00

The Device Simulator

A QEMU virtual machine that boots the real forward-ported kernel and real userspace: the 86 Panel (init, daemons, networking, OTA, watchdog, display) developed and tested with no board attached. The third simulator in the stack (three-way split: root README), deliberately not named "sim": it runs the whole machine above the kernel entry point on real binaries: bring your own, or drop prebuilt payloads in payload/. Decision record: ADR-0006.

The Boundary

There is no RV1106 machine model in QEMU and everything below the kernel is closed rkbin blobs plus mask ROM, so the VM enters at -kernel zImage on -M virt,highmem=off (single Cortex-A7, 256M: the RV1106G3's shape).

Emulated / substituted Not emulated (stays bench / sim/ territory)
Kernel boot, init ordering, switch_root BootROM, idblock/DDR-init, SPL, U-Boot
A/B outcome (warden.slot= cmdline) Real BCB A/B selection, bootcount auto-revert
Storage: virtio-blk with the device's exact blkdevparts= layout + /dev/block/by-name/ contract eMMC controller itself
Network: virtio-net (slirp, hostfwd 22/80/28443) GMAC, AIC8800 wifi, usb0 gadget
Display: virtio-gpu 720x720 via fbdev emulation VOP/RGB666 pipeline, CH32V003 panel init, RGA blits
Touch: virtio-tablet (QMP input-send-event) GT911 on I2C3
Watchdog: i6300esb (PCI), -action watchdog=reset DW watchdog @0xff5a0000, HPMCU supervisor
RS485: pci-serial chardev bridged to sim/'s ModbusSlave Real UART4 timing/electrical behavior
RTC: PL031 (--rtc reproduces the no-RTC 2021-clock incident class) The unpopulated backup-cell reality

"Boots/works under emulation" is never evidence of "works on silicon." The VM narrows which claims need a panel; on-device claims still need on-device evidence. Conversely, the VM is the first environment that runs production binaries on a non-RV1106 memory map: it found flare-edge #106 (fatal SIGBUS in flared's HPMCU probe) and #107 (Y2038 time_t truncation) on its first two boots of real userspace.

Documented guest deviations from production, set by stage-2 init: WARDEN_FLARE_INSECURE=1 (the desk mock portal is plain HTTP) and WARDEN_HPMCU=0 (no mailbox SRAM on virt; flared >= flare-edge#106 fix required, or the daemon dies of SIGBUS).

The VM's kernel is ahead of the panels' in one way that shows in the UI. It is built from build/warden_defconfig plus configs/virt.fragment, so it has CONFIG_BRIDGE; the vendor 5.10 kernel most panels still run does not. A Client-mode access point is bridged to the wired uplink and needs that, so the Wi-Fi page offers it in the VM and says "Needs Router mode: this kernel cannot bridge" on a 5.10 panel. Both are correct for the kernel underneath. The difference disappears as panels move to the 6.18 build, but until then a screenshot from here is not evidence about a 5.10 panel's Wi-Fi page.

Quick Start

# 1. kernel: canonical build boots the VM as-is; the fragment variant adds
#    the scenario devices (PCI serial, watchdog, WireGuard, virtio-gpu/input)
WORK=$HOME/kbuild-out CROSS_COMPILE=arm-linux-gnueabihf- \
  WARDEN_KCONFIG_FRAGMENT=qemu/configs/virt.fragment bash build/build-kernel.sh

# 2. initramfs (sha256-pinned static busybox + qemu/rootfs/) and A/B disk;
#    RIG_BUSYBOX=panel builds the image's own BusyBox instead (qemu/busybox/)
bash qemu/mkinitramfs.sh
bash qemu/mkimage.sh                  # options: --portal-url --state K=V --fw-version

# 3. run (see run.sh header for all flags)
bash qemu/run.sh --kernel $HOME/kbuild-out/linux-6.18.46/arch/arm/boot/zImage --shell

Payload: drop static musl armv7 binaries into qemu/payload/ (see its README). Stage-2 init starts every panel daemon it finds, in the panel's own S-number order (warden-ai, warden-automation, warden-modbus, warden-mikrotik, warden-asic, warden-starlink, warden-stratum, warden-flared), then warden-ui (the LVGL fbdev+evdev build from flare-edge tools/build-ui-vm.sh). The quickest faithful payload is the daemons a bench panel already runs: flare-edge tools/rig-pull-daemons.sh --host <panel> copies them in. The RS485 UART is always present (a null chardev without --rs485), so warden-modbus polls a port with nothing on it, as on a panel with an empty header, and never the control channel.

A mosquitto in the payload starts the panel's device bus first (socket /run/warden/bus/bus.sock, one credential file per bus user under /run/warden/bus/cred/), as S10bus does before every bus client on a panel; see payload/README.md. The rig broker takes any user and loads no plugin. With a test-flavor warden-ui and wthctl (flare-edge tools/wthctl.c, the test hook client) in the payload as well, a flow's override step pins a state document inside the UI: qmp.py hands the request to the control bridge (@hookdoc pieces, then @hook override), which sends it to test/req/ui as bus user testhook and answers the hook's reply. The run's first hook request waits until the hook answers a ping (retried while the bridge reports no answer, up to HOOK_READY_S): right after boot the UI is still taking the broker's retained replay, and an override sent then was lost (flare-edge #2602).

The VT cursor is kept off (vt.global_cursor_default=0): fbcon shares the virtio-gpu framebuffer with warden-ui and its blinking cursor would otherwise show up in screendumps at random (issue #18).

The guest carries the panel's own /etc/passwd, /etc/shadow and /etc/group (root's md5-crypt of the documented default password), so a screen that verifies the root password against /etc/shadow behaves as it does on a panel instead of rejecting every attempt. qemu_stage_rootfs() (lib.sh) forces etc/shadow to mode 0600 on every stage: git tracks only the executable bit, so the checked-out source file's own mode depends on the checking-out umask and cannot be trusted to arrive non-world-readable.

--append warden.initchain boots the UI the way a panel does (flare-edge #2210). The caller lays the panel's own scripts over the rootfs with ROOTFS_OVERLAY=<dir> mkimage.sh (only etc/ and usr/ at its top, no symlinks). Stage-2 init then starts /etc/init.d/S00-S09 with start before the bus (S03warden-accounts creates the platform accounts, and the bus hands cred/ui to warden-ui), and S10-S99 after the payload daemons in place of its direct root start of warden-ui. S99lvgl then runs warden-ui-run, which drops the UI to its account and respawns it. Rig differences: the splash handover is marked done (no S09splash) and warden-ui-run passes -b fbdev, not the panel's -b drm. A chain boot with no script prints WARDEN-QEMU-INITCHAIN-EMPTY and starts no UI. tests/ui-drive.sh --init-chain <dir> does both steps, and the drive verb assert_ui_chain judges the result: one warden-ui and one warden-ui-run, the UI's four uids all warden-ui's, no safe-mode flag, and no exit or refusal in /tmp/warden-ui.log. tests/scripts/init-chain-ui.txt runs the check at about T+120 s.

A flow seed may contain qemu-flow-setup.sh. Stage-2 init runs it inside the disposable guest after mounting userdata and before starting daemons or the UI. A nonzero exit stops the boot. Use it only for test devices and commands that the virt machine does not provide.

Scenarios

All take the virt-fragment <zImage>; FLARE_EDGE=<checkout> where noted.

Scenario Needs Proves
stage-rootfs-perms.sh - offline, no VM: qemu_stage_rootfs() always lands etc/shadow at 0600, even staged from a source copy deliberately left 0644
seed-dir.sh - offline, no VM: mkimage.sh's SEED_DIR hook lands every seeded file under userdata/warden with its original mode (a 0600 secret included), a seeded file beats a same-named --state value, a SEED_DIR that isn't a directory fails closed, and so does an individual entry that is a symlink, a subdirectory, or named outside [A-Za-z0-9_.-]+ -- a hyphenated key such as gas-plant.devices still seeds cleanly
payload-dir.sh - offline, no VM: mkimage.sh's PAYLOAD_DIR hook puts the run's own payload set (symlinked entries included, as regular 0755 files; its README.md skipped) into /usr/bin of both slots and nothing from qemu/payload, the default still reads qemu/payload, a PAYLOAD_DIR that is not a directory fails closed, and ui-drive.sh looks for warden-ui in the same directory
boot-smoke.sh - sentinel-asserting boot; runs in CI inside kernel-build
portal-scenario.sh FLARE_EDGE real flared against the desk mock portal: authenticated check-in, desired-state pull, signed tier-1 .wfw download; verify/stage/APPLYING as a dry run (no WARDEN_FW_ALLOW_APPLY)
ota-apply.sh FLARE_EDGE the FULL apply: the .wfw's bootable rootfs payload is written to rootfs_b (run.sh --allow-apply gates it per boot), the AvbABData in misc flips, and slot _b boots the applied version
ui-shot.sh - display+touch, headless: QMP-screendumps the 720x720 UI, taps the Metrics tab via input-send-event, asserts the frame changed (qmp.py is the QMP client)
ui-drive.sh <script> - the same rig for a SEQUENCE: boots once, runs a qmp.py drive script of taps/swipes/screenshots in panel pixels, and FAILS if warden-ui died on the way (stage-2 init announces the exit on the console). tests/scripts/nav-stress.txt is the navigation regression: it reproduces the s_row_left overflow that segfaulted the UI on returning to Settings > Apps; tests/scripts/home-leaves-fullscreen.txt proves the home verb clears dashboard fullscreen, the state a hardware wake tap leaves behind (flare-edge #176); tests/scripts/fullscreen-toggle-tracks-real-state.txt proves a fullscreen toggle right after home (or a real tap) reads the dashboard's actual state rather than a belief home bypassed -- test_qmp_drive.py's FullscreenToggleTracksRealState pins the same contract offline
real-image-boot.sh matched rootfs.img + oem.img an ACTUAL flare-edge build (placed by mkimage.sh --rootfs-image/--oem-image) boots its own init chain to getty; binaries predating known fixes reproduce their bugs faithfully, a time machine for field issues
test-ui-drive-rs485.sh - offline, no VM: a fake run.sh and (for one case) a fake socat stand in so ui-drive.sh --rs485-devices's startup fails closed instead of printing a false "== rs485 simulator: ..." over a bus nothing is serving, for each of a qemu rs.sock that never appears, a socat that never links rs.pty, and an mbsim.py that dies before opening rs.ctl; also proves cleanup() escalates to SIGKILL for a sim process that ignores SIGTERM
rootfs-overlay.sh - offline, no VM: mkimage.sh's ROOTFS_OVERLAY hook lays a directory of panel files (flare-edge's S03warden-accounts, S99lvgl, warden-ui-run) over the rootfs of both slots with their modes kept, beside the skeleton's own files; without it there is no /etc/init.d; an overlay that is not a directory, names anything but etc/ and usr/ at its top, or holds a symlink fails closed and writes no image; ui-drive.sh --init-chain refuses a missing directory or one without etc/init.d/S99lvgl before boot
init-chain-test.sh - offline, no VM: warden_init_chain (rootfs/etc/warden-lib.sh) starts the executable S<NN>* scripts of a number range in name order, names a failing one and goes on, and counts them; warden_rig_bus_owners hands a bus credential to its account only when the rootfs has it; sbin/init runs S00-S09 before the bus and S10-S99 instead of the direct UI start on a warden.initchain boot (flare-edge #2210)
run-sh-args-test.sh - offline, no VM: a fake qemu-system-arm captures run.sh's own argv and pins the ctl/rs485 pci-serial ordering contract rootfs/sbin/init's ttyS0-vs-ttyS1 alias depends on -- the ctl device always enumerates before rs485, warden.ctl only lands on the cmdline when --ctl is given, and an rs485 pci-serial device (real or null-backed) is always present so the port count never shifts
busybox-panel-test.sh - offline, no VM: the panel BusyBox inputs (qemu/busybox/). panel.config carries every setting of the image's applet patch, rig.fragment changes only STATIC, USE_BB_PWD_GRP and USE_BB_SHADOW, a Kconfig rewrite of any setting and an applet the panel lacks fail the build, applets.txt and help.txt agree, and RIG_BUSYBOX refuses an unknown value, and accepts a prebuilt panel BUSYBOX only when its .inputs match this tree (flare-edge#2210)
busybox-panel-help.sh [BIN] an ARM runner (binfmt or QEMU_USER=qemu-arm) the built panel BusyBox against the bench capture: its applet list and every applet's --help equal the panel's own binary's. CI runs it in qemu-tools after build-busybox.sh
console-shell-test.sh - offline, no VM: warden_console_shell (warden.shell) runs setsid cttyhack sh where cttyhack exists, else setsid -c sh on the console tty read from sysfs (the panel BusyBox has no cttyhack), else a plain sh. CI runs it under bash's sh and busybox sh
ctl-bridge-test.sh - offline, no VM: runs the control bridge (warden_ctl_bridge in rootfs/etc/warden-lib.sh) against a fake warden-ui that answers some commands late. A reply slower than the old 5 s bound still arrives (the bound is now 30 s, WARDEN_CTL_WAIT_S), a timeout answers bridge: no reply from warden-ui ... (qmp.py raises it as a rig fault), and a timed-out command's late reply is drained instead of being read as the next command's answer (flare-edge #1448). A fake wthctl stands in for the hook client behind @hookdoc/@hook (flare-edge #2054): the pieces join into one request whose args are used once, only plain words reach the client, and a missing client or a request with no reply answers a bridge: line. CI runs it under bash's sh and busybox sh
init-rig-bus-test.sh - offline, no VM: warden_rig_bus in rootfs/etc/warden-lib.sh against stub brokers. No broker in the payload creates nothing; a broker that listens gets the panel's layout (the bus.sock link, a 0400 64-hex credential per bus user, a config without password file, ACL or plugin); a broker that exits or never listens is reported within the bound; init starts the bus before the payload daemons. WARDEN_TEST_MOSQUITTO_DIR adds a real mosquitto carrying one message between two users logged in by credential file (flare-edge #1754)
agent-test.sh - offline, no VM: the rig agent (warden_agent in rootfs/etc/warden-lib.sh, armed by run.sh --agent and warden.agent) answers @run ID SECS B64 with <<BEGIN ID>>, the base64 of the script's output and <<END ID RC>>; @chunk pieces join into one script used once; a script past its deadline is killed with its process group and answers 124; a malformed request answers one <<ERR ...>> line and runs nothing; init puts the agent's port after ctl and RS485. It is the guest shell of flare-edge tools/soak/rig-qemu.py (M9 soak-lite). CI runs it under bash's sh and busybox sh
watchdog (run.sh --watchdog) - arm /dev/watchdog, don't pet: the VM resets ~30 s later (verified)

Scenario fine print:

  • OTA: the BCB slot CHOICE and the physical reset stay emulated by the harness (ADR-0006 boundary); the VM exports WARDEN_HARD_RESET=0 so flared's post-apply reset surfaces as a reported error, not a /dev/mem fault.
  • Touch injection holds 200 ms: an instantaneous press+release lands inside one LVGL poll and never clicks.
  • Watchdog + a flared payload don't mix: flared pets only while the UI heartbeat is fresh.

Gotchas

  • AF_UNIX socket paths cap at ~108 chars; keep --rs485/--qmp paths short.
  • A serial port that is closed discards incoming bytes: hold ONE fd open across write and read when scripting the guest side of the RS485 bridge.
  • highmem=off and -global virtio-mmio.force-legacy=false are required (32-bit ECAM reach; virtio-1-only gpu/input); both live ONLY in run.sh, which every script (boot smoke included) delegates to.
  • Never pass earlyprintk: DEBUG_UART_PHYS is the RV1106's 0xff4c0000.

Requirements

qemu-system-arm (Debian 13 ships QEMU 10), curl, cpio, mkfs.ext4, debugfs (both ship in e2fsprogs; seed-dir.sh reads a built partition back with it), gcc-arm-linux-gnueabihf (kernel build), python3 (+cryptography for the portal scenario's .wfw signing). CI: the hosted qemu-tools job builds the tooling; the boot smoke runs inside the (also hosted, dispatch-only) kernel-build job, which apt-installs its own toolchain and qemu (ADR-0007).