review: iteration-1 fixes across CI, bridge, VM harness, and docs

CI/pipeline:
- KERNEL_TARBALL passed as a YAML env literal '~' was never tilde-expanded
  and would have failed every hosted kernel-build dispatch; the path is now
  exported from the shell. Verified reproducible before the fix.
- Every job gets timeout-minutes; boot smoke uses timeout -k so a wedged
  qemu is SIGKILLed instead of holding the job.
- Tarball fetch + fail-closed sha256 verification deduplicated into
  build/fetch-kernel-tarball.sh (with curl retries), used by build-kernel.sh
  and both CI jobs. busybox fetch gains retries too.
- ccache layer for kernel-build (cache keyed on defconfig+patches) recovers
  the incremental-compile speed the ephemeral-runner move cost.
- build-kernel.sh now asserts every fragment option survived olddefconfig —
  merge_config -m pastes text and Kconfig silently drops unmet symbols.

rs485-bridge:
- pending-buffer cap (2x max RTU ADU) instead of unbounded growth;
  explicit accept-loop error handling with backoff instead of .flatten();
  per-arm inline bounds instead of the string-keyed lookup whose default
  would have mis-bounded a future get-input; control-socket cleanup errors
  surfaced; flag-shaped values rejected in arg parsing; doc example uses a
  private mktemp dir. Test timing margins widened for contended runners
  (gap 25->120ms, 60x margin on the split-frame test).

VM harness:
- stage-1/stage-2 boot scripts share one validated slot parser and one
  by-name populator (qemu/rootfs/etc/warden-lib.sh) — the duplicated
  parser had already diverged on validation; userdata/oem mount failures
  now fail fast with a greppable sentinel; udhcpc fallback keys off the
  interface actually having an address; switch_root applet guarded.
- boot-smoke delegates the qemu invocation to run.sh (machine shape lives
  in ONE place); run.sh port 0 disables a hostfwd.
- mkimage: unknown partition names fail at build time; DISK_END is a max,
  not last-entry; --state keys validated as filenames.
- portal-scenario: mock readiness is asserted (no silent fall-through),
  hostfwd port collisions retried, mount-failure sentinel fails fast.
- ui-shot: fixed sleeps replaced with bounded screendump polling; the
  repaint assertion is real and documented as such. qmp.py loses its
  module-global and gains argv validation.

Docs/scrub: bench-host paths and the site AP name removed from six more
port docs and two evidence tables; path-bearing build artifacts (.elf,
.map) untracked (the 154-byte firmware .bin is path-free and stays);
ADR-0003 marked visibility-superseded by ADR-0007; stale section
cross-reference fixed; flare-edge noted as private for outside readers;
stale root-level review report removed per the new workspace rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018HUayid7W5w7jBdb9Rrj1K

qemu/README.md

@@ -89,7 +89,8 @@ stage-2 init when present.
 - A serial port that is closed discards incoming bytes: hold ONE fd open
   across write and read when scripting the guest side of the RS485 bridge.
 - `highmem=off` and `-global virtio-mmio.force-legacy=false` are load-bearing
-  (32-bit ECAM reach; virtio-1-only gpu/input) — both live in run.sh.
+  (32-bit ECAM reach; virtio-1-only gpu/input) — both live ONLY in run.sh,
+  which every script (boot smoke included) delegates to.
 - Never pass `earlyprintk`: DEBUG_UART_PHYS is the RV1106's 0xff4c0000.
 
 ## Host requirements

qemu/lib.sh

@@ -18,7 +18,7 @@ qemu_get_busybox() {
   BB="${BUSYBOX:-$out/busybox-armv7l}"
   if [ ! -f "$BB" ]; then
     qemu_log "downloading $BB_URL"
-    curl -fSL "$BB_URL" -o "$BB"
+    curl --retry 3 --retry-delay 5 -fSL "$BB_URL" -o "$BB"
   fi
   [ -f "$sha_file" ] || {
     echo "FATAL: no pinned sha256 for busybox (expected $sha_file) — refusing to build from an unverified binary" >&2

qemu/mkimage.sh

@@ -30,7 +30,17 @@ FW_VERSION="0.0.1"
 while [ $# -gt 0 ]; do
   case "$1" in
     --portal-url) PORTAL_URL="${2:?--portal-url needs a value}"; shift 2 ;;
-    --state)      STATE_KV+=("${2:?--state needs KEY=VALUE}"); shift 2 ;;
+    --state)
+      case "${2:?--state needs KEY=VALUE}" in
+        *=*) ;;
+        *) echo "FATAL: --state needs KEY=VALUE, got '$2'" >&2; exit 1 ;;
+      esac
+      case "${2%%=*}" in
+        *[!A-Za-z0-9_.]*|'')
+          echo "FATAL: --state key '${2%%=*}' must match [A-Za-z0-9_.]+ (it becomes a filename)" >&2
+          exit 1 ;;
+      esac
+      STATE_KV+=("$2"); shift 2 ;;
     --fw-version) FW_VERSION="${2:?--fw-version needs a value}"; shift 2 ;;
     *) echo "FATAL: unknown argument '$1' (usage: mkimage.sh [--portal-url URL] [--state KEY=VALUE]... [--fw-version V])" >&2; exit 1 ;;
   esac
@@ -83,7 +93,11 @@ place_partition() {
     rootfs_a|rootfs_b) stage="$ROOT" ;;
     userdata)          stage="$UDATA" ;;
     oem_a|oem_b)       stage="$SCRATCH/empty" ;;
-    *)                 stage="" ;;   # boot-chain partition: left zeroed
+    # Boot-chain partitions the VM never reads: present at the right offsets,
+    # left zeroed. Enumerated (not a wildcard) so a typo'd name in
+    # blkdevparts.conf fails HERE, not as a confusing mount error at boot.
+    env|idblock|uboot|misc|boot_a|boot_b|recovery) stage="" ;;
+    *) echo "FATAL: unknown partition name '$name' in blkdevparts.conf" >&2; exit 1 ;;
   esac
   # dd in 4K blocks — every offset in the canonical layout is 4K-aligned;
   # assert rather than assume, a misaligned write would corrupt a neighbor.
@@ -91,7 +105,8 @@ place_partition() {
     echo "FATAL: partition $name not 4K-aligned (off=$off size=$size)" >&2
     exit 1
   fi
-  DISK_END=$((off + size))
+  # Max, not last: blkdevparts grammar permits explicit @offsets out of order.
+  [ $((off + size)) -gt "$DISK_END" ] && DISK_END=$((off + size))
   [ -z "$stage" ] && return 0
   local img="$SCRATCH/$name.img"
   mkfs_part "$stage" "$size" "$img"

qemu/rootfs/etc/rc

@@ -1,37 +1,19 @@
 #!/bin/busybox sh
 # Stage-1 rc: sourced by /init (still PID 1, initramfs root) when a virtio
-# disk is present. Emulates U-Boot's slot choice — parse warden.slot= from the
-# cmdline, mount that rootfs, switch_root into it. This is an EMULATION of the
-# A/B selection outcome, not the BCB/bootcount mechanism itself.
+# disk is present. Emulates U-Boot's slot choice — mount the validated slot's
+# rootfs and switch_root into it. This is an EMULATION of the A/B selection
+# outcome, not the BCB/bootcount mechanism itself.
+#
+# Every guarded failure path `return`s to /init (valid in a sourced script;
+# /init then falls through to shell/poweroff). The final exec is the one
+# unguardable step: if switch_root itself fails to launch, the shell — PID 1 —
+# exits and the kernel panics; the applet-existence check below catches the
+# only preventable variant of that.
 
-# /dev/block/by-name/<PARTNAME> symlinks: the contract flare-edge slotctl.rs
-# relies on. blkdevparts= gives every vda partition a PARTNAME in sysfs.
-mkdir -p /dev/block/by-name
-for uev in /sys/class/block/vda*/uevent; do
-    [ -f "$uev" ] || continue
-    partname=""
-    devname=""
-    while IFS='=' read -r k v; do
-        case "$k" in
-            PARTNAME) partname="$v" ;;
-            DEVNAME)  devname="$v" ;;
-        esac
-    done < "$uev"
-    [ -n "$partname" ] && [ -n "$devname" ] \
-        && ln -sf "/dev/$devname" "/dev/block/by-name/$partname"
-done
+. /etc/warden-lib.sh
 
-# Slot select: whole-token parse of warden.slot= (never a substring match).
-slot="_a"
-for tok in $(cat /proc/cmdline); do
-    case "$tok" in
-        warden.slot=*) slot="${tok#warden.slot=}" ;;
-    esac
-done
-case "$slot" in
-    _a|_b) ;;
-    *) echo "rc: bad warden.slot='$slot', falling back to _a"; slot="_a" ;;
-esac
+warden_populate_by_name
+slot="$(warden_slot)"
 
 root="/dev/block/by-name/rootfs${slot}"
 if [ ! -e "$root" ]; then
@@ -49,6 +31,11 @@ if [ ! -x /mnt/sbin/init ]; then
     umount /mnt
     return 0
 fi
+if ! command -v switch_root >/dev/null; then
+    echo "rc: busybox lacks switch_root — staying in initramfs"
+    umount /mnt
+    return 0
+fi
 
 echo "rc: switching root to rootfs${slot} ($root)"
 exec switch_root /mnt /sbin/init

qemu/rootfs/etc/warden-lib.sh

@@ -0,0 +1,42 @@
+# shellcheck shell=sh
+# Shared helpers for the VM's stage-1 (/init + /etc/rc, initramfs) and stage-2
+# (/sbin/init, disk rootfs) boot scripts. Present in both filesystems because
+# both are staged from the same qemu/rootfs/ skeleton. ONE copy of each rule —
+# the slot-validation drift between two hand-copied parsers was a real
+# review finding.
+
+# Populate /dev/block/by-name/<PARTNAME> symlinks from sysfs uevents — the
+# contract flare-edge's slotctl.rs relies on. blkdevparts= gives every vda
+# partition a PARTNAME.
+warden_populate_by_name() {
+    mkdir -p /dev/block/by-name
+    for uev in /sys/class/block/vda*/uevent; do
+        [ -f "$uev" ] || continue
+        partname=""
+        devname=""
+        while IFS='=' read -r k v; do
+            case "$k" in
+                PARTNAME) partname="$v" ;;
+                DEVNAME)  devname="$v" ;;
+            esac
+        done < "$uev"
+        [ -n "$partname" ] && [ -n "$devname" ] \
+            && ln -sf "/dev/$devname" "/dev/block/by-name/$partname"
+    done
+}
+
+# Parse warden.slot= from the cmdline (whole-token, never substring) and
+# VALIDATE it — echoes "_a" or "_b", falling back to _a with a warning.
+warden_slot() {
+    slot="_a"
+    for tok in $(cat /proc/cmdline); do
+        case "$tok" in
+            warden.slot=*) slot="${tok#warden.slot=}" ;;
+        esac
+    done
+    case "$slot" in
+        _a|_b) ;;
+        *) echo "warden-lib: bad warden.slot='$slot', falling back to _a" >&2; slot="_a" ;;
+    esac
+    echo "$slot"
+}

qemu/rootfs/sbin/init

@@ -13,33 +13,29 @@ mount -t proc proc /proc
 mount -t sysfs sysfs /sys
 mount -t tmpfs tmpfs /tmp
 
-# Fresh devtmpfs — repopulate the by-name contract (slotctl.rs depends on it).
-mkdir -p /dev/block/by-name
-for uev in /sys/class/block/vda*/uevent; do
-    [ -f "$uev" ] || continue
-    partname=""
-    devname=""
-    while IFS='=' read -r k v; do
-        case "$k" in
-            PARTNAME) partname="$v" ;;
-            DEVNAME)  devname="$v" ;;
-        esac
-    done < "$uev"
-    [ -n "$partname" ] && [ -n "$devname" ] \
-        && ln -sf "/dev/$devname" "/dev/block/by-name/$partname"
-done
+. /etc/warden-lib.sh
 
-# Slot (whole-token parse, same rule as stage 1).
-slot="_a"
-for tok in $(cat /proc/cmdline); do
-    case "$tok" in
-        warden.slot=*) slot="${tok#warden.slot=}" ;;
-    esac
-done
+# Fresh devtmpfs — repopulate the by-name contract; same VALIDATED slot rule
+# as stage 1 (shared helper, so the two can never drift).
+warden_populate_by_name
+slot="$(warden_slot)"
 
 # The device's matched mounts: persistent state and the slot's oem partition.
-mount -t ext4 /dev/block/by-name/userdata /userdata || echo "init: userdata mount failed"
-mount -t ext4 "/dev/block/by-name/oem${slot}" /oem || echo "init: oem${slot} mount failed"
+# Fail-fast: a scenario against an image whose userdata cannot mount would
+# otherwise burn its whole deadline before failing generically. warden.shell
+# still gets a shell for post-mortem.
+mount_fatal() {
+    if ! mount -t ext4 "/dev/block/by-name/$1" "$2"; then
+        echo "WARDEN-QEMU-MOUNT-FAILED $1"
+        if grep -qw warden.shell /proc/cmdline; then
+            echo "warden.shell: post-mortem shell (exit powers off)"
+            setsid cttyhack sh
+        fi
+        poweroff -f
+    fi
+}
+mount_fatal userdata /userdata
+mount_fatal "oem${slot}" /oem
 mkdir -p /userdata/warden
 
 # RS485: warden-modbus hardcodes /dev/ttyS4 at compile time; alias it to the
@@ -47,10 +43,13 @@ mkdir -p /userdata/warden
 [ -c /dev/ttyS0 ] && ln -sf /dev/ttyS0 /dev/ttyS4
 
 # Network: slirp user-mode net on eth0 (DHCP, fallback to QEMU's static map).
+# The fallback keys off the interface actually having an address — udhcpc
+# exiting 0 only proves a lease, not that the hook script applied it.
 ip link set lo up
 if [ -e /sys/class/net/eth0 ]; then
     ip link set eth0 up
-    if ! udhcpc -i eth0 -n -q -t 5 -T 2 >/dev/null 2>&1; then
+    udhcpc -i eth0 -n -q -t 5 -T 2 >/dev/null 2>&1 || true
+    if ! ip -4 addr show dev eth0 | grep -q 'inet '; then
         ip addr add 10.0.2.15/24 dev eth0 2>/dev/null
         ip route replace default via 10.0.2.2 dev eth0
         echo "nameserver 10.0.2.3" > /etc/resolv.conf

qemu/rs485-bridge/src/lib.rs

@@ -24,6 +24,11 @@ use warden_sim::ModbusSlave;
 /// response timeout is orders of magnitude larger.
 pub const DEFAULT_GAP: Duration = Duration::from_millis(10);
 
+/// Accumulation cap: a Modbus RTU ADU is at most 256 bytes, so anything past
+/// 2x that without an inter-frame gap is a misbehaving master streaming
+/// continuously — drop the buffer instead of growing without bound.
+const MAX_PENDING: usize = 512;
+
 /// The shared bus: the slave plus its declared dimensions. The sim's register
 /// setters panic on out-of-range indices (deliberate test-harness semantics);
 /// the control channel must bounds-check first so a typo in a scenario script
@@ -64,7 +69,17 @@ pub fn pump_serial(
                 }
                 return Ok(());
             }
-            Ok(n) => buf.extend_from_slice(&chunk[..n]),
+            Ok(n) => {
+                buf.extend_from_slice(&chunk[..n]);
+                if buf.len() > MAX_PENDING {
+                    eprintln!(
+                        "rs485: {} bytes buffered with no inter-frame gap — discarding \
+                         (misbehaving master streaming continuously?)",
+                        buf.len()
+                    );
+                    buf.clear();
+                }
+            }
             Err(e)
                 if e.kind() == std::io::ErrorKind::WouldBlock
                     || e.kind() == std::io::ErrorKind::TimedOut =>
@@ -127,10 +142,10 @@ pub fn handle_control_line(line: &str, bus: &Bus) -> String {
     if words.next().is_some() {
         return format!("err trailing arguments after '{cmd}'");
     }
-    let bound = |cmd: &str| match cmd {
-        "holding" | "input" | "get-holding" => bus.regs,
-        _ => bus.bits,
-    };
+    // Each arm states its own bound (bus.regs for register space, bus.bits for
+    // bit space) INLINE — a previous string-keyed lookup defaulted silently to
+    // the bit bound, which would have handed a future `get-input` command the
+    // wrong range and reintroduced the out-of-range panic this check prevents.
     let mut s = bus.slave.lock().unwrap();
     match (cmd, arg) {
         ("ping", None) => "ok".into(),
@@ -152,39 +167,63 @@ pub fn handle_control_line(line: &str, bus: &Bus) -> String {
             }
             _ => format!("err bad exception code '{c}'"),
         },
-        ("holding" | "input" | "coil" | "discrete", Some(kv)) => {
-            let (addr, val) = match kv.split_once('=') {
-                Some((a, v)) => (parse_u16(a), parse_u16(v)),
-                None => (None, None),
-            };
-            match (addr, val) {
-                (Some(a), _) if (a as usize) >= bound(cmd) => {
-                    format!("err address {a} out of range (0..{})", bound(cmd))
+        ("holding" | "input", Some(kv)) => match parse_addr_val(kv, bus.regs) {
+            Ok((a, v)) => {
+                if cmd == "holding" {
+                    s.set_holding(a, v);
+                } else {
+                    s.set_input(a, v);
                 }
-                (Some(a), Some(v)) => {
-                    match cmd {
-                        "holding" => s.set_holding(a as usize, v),
-                        "input" => s.set_input(a as usize, v),
-                        "coil" => s.set_coil(a as usize, v != 0),
-                        _ => s.set_discrete(a as usize, v != 0),
-                    }
-                    "ok".into()
+                "ok".into()
+            }
+            Err(e) => e,
+        },
+        ("coil" | "discrete", Some(kv)) => match parse_addr_val(kv, bus.bits) {
+            Ok((a, v)) => {
+                if cmd == "coil" {
+                    s.set_coil(a, v != 0);
+                } else {
+                    s.set_discrete(a, v != 0);
                 }
-                _ => format!("err expected <addr>=<value>, got '{kv}'"),
+                "ok".into()
             }
-        }
-        ("get-holding" | "get-coil", Some(a)) => match parse_u16(a) {
-            Some(a) if (a as usize) >= bound(cmd) => {
-                format!("err address {a} out of range (0..{})", bound(cmd))
-            }
-            Some(a) if cmd == "get-holding" => format!("ok {}", s.holding(a as usize)),
-            Some(a) => format!("ok {}", u8::from(s.coil(a as usize))),
-            None => format!("err bad address '{a}'"),
+            Err(e) => e,
+        },
+        ("get-holding", Some(a)) => match parse_addr(a, bus.regs) {
+            Ok(a) => format!("ok {}", s.holding(a)),
+            Err(e) => e,
+        },
+        ("get-coil", Some(a)) => match parse_addr(a, bus.bits) {
+            Ok(a) => format!("ok {}", u8::from(s.coil(a))),
+            Err(e) => e,
         },
         _ => format!("err unknown or malformed command '{line}'"),
     }
 }
 
+/// Parse "<addr>=<value>" with the address bounds-checked against `bound`.
+fn parse_addr_val(kv: &str, bound: usize) -> Result<(usize, u16), String> {
+    let Some((a, v)) = kv.split_once('=') else {
+        return Err(format!("err expected <addr>=<value>, got '{kv}'"));
+    };
+    let (Some(a), Some(v)) = (parse_u16(a), parse_u16(v)) else {
+        return Err(format!("err expected <addr>=<value>, got '{kv}'"));
+    };
+    if (a as usize) >= bound {
+        return Err(format!("err address {a} out of range (0..{bound})"));
+    }
+    Ok((a as usize, v))
+}
+
+/// Parse a bare address, bounds-checked against `bound`.
+fn parse_addr(a: &str, bound: usize) -> Result<usize, String> {
+    match parse_u16(a) {
+        Some(v) if (v as usize) < bound => Ok(v as usize),
+        Some(v) => Err(format!("err address {v} out of range (0..{bound})")),
+        None => Err(format!("err bad address '{a}'")),
+    }
+}
+
 fn parse_u16(s: &str) -> Option<u16> {
     if let Some(h) = s.strip_prefix("0x").or_else(|| s.strip_prefix("0X")) {
         u16::from_str_radix(h, 16).ok()
@@ -200,10 +239,13 @@ mod tests {
     use std::time::Duration;
     use warden_sim::modbus::{crc_ok, read_holding};
 
-    // Test gap is larger than DEFAULT_GAP so a loaded CI runner cannot split
-    // a frame that the test wrote in two deliberate chunks.
-    const GAP: Duration = Duration::from_millis(25);
-    const SETTLE: Duration = Duration::from_millis(100);
+    // Test gap is much larger than DEFAULT_GAP so a loaded CI runner cannot
+    // split a frame the test wrote in two deliberate chunks: the 2ms
+    // inter-chunk pause has a 60x margin against the 120ms dispatch gap
+    // (25ms gave only 12.5x and was flagged as a flake risk on contended
+    // 2-vCPU hosted runners).
+    const GAP: Duration = Duration::from_millis(120);
+    const SETTLE: Duration = Duration::from_millis(400);
 
     fn bus() -> Bus {
         let b = Bus::new(1, 16, 16);
@@ -221,7 +263,9 @@ mod tests {
     }
 
     fn read_reply(master: &UnixStream) -> Vec<u8> {
-        master.set_read_timeout(Some(Duration::from_secs(2))).unwrap();
+        master
+            .set_read_timeout(Some(Duration::from_secs(2)))
+            .unwrap();
         let mut buf = [0u8; 256];
         let n = (&*master).read(&mut buf).expect("expected a reply frame");
         buf[..n].to_vec()
@@ -274,7 +318,10 @@ mod tests {
         with_pump(&s, |master| {
             (&*master).write_all(&read_holding(1, 2, 1)).unwrap();
             master
-                .set_read_timeout(Some(Duration::from_millis(200)))
+                // Well past GAP: the dropped frame must have been dispatched
+                // (and answered with silence) before the next request is
+                // written, or the two would merge in the pending buffer.
+                .set_read_timeout(Some(Duration::from_millis(500)))
                 .unwrap();
             let mut buf = [0u8; 16];
             assert!(

qemu/rs485-bridge/src/main.rs

@@ -2,10 +2,14 @@
 //! there); this file only parses arguments, connects sockets, and spawns the
 //! control listener.
 //!
-//! Typical use (matches qemu/run.sh --rs485):
+//! Typical use (matches qemu/run.sh --rs485). Put the sockets in a private
+//! per-run directory (mktemp -d) — short (AF_UNIX caps paths at ~108 chars)
+//! and not guessable/pre-creatable by other local users, unlike a fixed
+//! /tmp name:
 //!
-//!   qemu/run.sh --kernel ... --rs485 /tmp/warden-rs485.sock &
-//!   rs485-bridge --serial /tmp/warden-rs485.sock --control /tmp/warden-rs485-ctl.sock
+//!   d=$(mktemp -d /tmp/rs485.XXXXXX)
+//!   qemu/run.sh --kernel ... --rs485 "$d/serial.sock" &
+//!   rs485-bridge --serial "$d/serial.sock" --control "$d/ctl.sock"
 
 use std::io::{BufRead, BufReader, Write};
 use std::os::unix::net::{UnixListener, UnixStream};
@@ -31,10 +35,19 @@ fn main() {
 
     let mut args = std::env::args().skip(1);
     while let Some(a) = args.next() {
-        let mut val = |name: &str| args.next().unwrap_or_else(|| {
-            eprintln!("{name} needs a value");
-            usage()
-        });
+        let mut val = |name: &str| {
+            let v = args.next().unwrap_or_else(|| {
+                eprintln!("{name} needs a value");
+                usage()
+            });
+            // A following flag means the value was omitted — report the real
+            // problem instead of swallowing the flag as a bogus value.
+            if v.starts_with("--") {
+                eprintln!("{name} needs a value, got flag '{v}'");
+                usage()
+            }
+            v
+        };
         match a.as_str() {
             "--serial" => serial = Some(val("--serial")),
             "--control" => control = Some(val("--control")),
@@ -55,14 +68,32 @@ fn main() {
     let bus: &'static Bus = Box::leak(Box::new(Bus::new(address, regs, bits)));
 
     if let Some(path) = control {
-        let _ = std::fs::remove_file(&path); // stale socket from a previous run
+        // Clear a stale socket from a previous run. A failure here that is not
+        // "nothing to remove" (e.g. someone else's file behind /tmp's sticky
+        // bit) will make the bind below fail — surface both errors.
+        let removed = std::fs::remove_file(&path);
         let listener = UnixListener::bind(&path).unwrap_or_else(|e| {
             eprintln!("FATAL: cannot bind control socket {path}: {e}");
+            if let Err(re) = removed {
+                if re.kind() != std::io::ErrorKind::NotFound {
+                    eprintln!("       (removing the pre-existing file also failed: {re})");
+                }
+            }
             exit(1);
         });
         eprintln!("rs485: control socket at {path}");
         std::thread::spawn(move || {
-            for conn in listener.incoming().flatten() {
+            // Explicit error handling: `.flatten()` would turn a persistent
+            // accept() failure (fd exhaustion etc.) into a silent hot loop.
+            for conn in listener.incoming() {
+                let conn = match conn {
+                    Ok(c) => c,
+                    Err(e) => {
+                        eprintln!("rs485: control accept failed: {e} — backing off");
+                        std::thread::sleep(Duration::from_millis(200));
+                        continue;
+                    }
+                };
                 let reader = BufReader::new(conn.try_clone().expect("clone control conn"));
                 let mut writer = conn;
                 for line in reader.lines() {

qemu/run.sh

@@ -18,9 +18,9 @@
 #   --qmp SOCK         QMP unix socket (screendump, input-send-event, quit)
 #   --display MODE     off (default, -nographic) | on (gtk window) | headless
 #                      (virtio-gpu without a window; screendump via --qmp)
-#   --ssh-port N       hostfwd 127.0.0.1:N -> guest :22   (default 2222)
-#   --http-port N      hostfwd 127.0.0.1:N -> guest :80   (default 8080)
-#   --api-port N       hostfwd 127.0.0.1:N -> guest :28443 (default 28443)
+#   --ssh-port N       hostfwd 127.0.0.1:N -> guest :22   (default 2222; 0 disables)
+#   --http-port N      hostfwd 127.0.0.1:N -> guest :80   (default 8080; 0 disables)
+#   --api-port N       hostfwd 127.0.0.1:N -> guest :28443 (default 28443; 0 disables)
 #   --shell            interactive shell in the guest instead of daemon hold
 set -euo pipefail
 
@@ -76,13 +76,19 @@ fi
 # NOTE: never add `earlyprintk` — the config's DEBUG_UART_PHYS is the RV1106's
 # 0xff4c0000, which does not exist on -M virt.
 APPEND="console=ttyAMA0 rdinit=/init"
+# Port 0 disables a forward — a boot smoke needs no host ports and must not
+# fail on a busy default port.
+NETDEV="user,id=n0"
+[ "$SSH_PORT" != 0 ]  && NETDEV="$NETDEV,hostfwd=tcp:127.0.0.1:${SSH_PORT}-:22"
+[ "$HTTP_PORT" != 0 ] && NETDEV="$NETDEV,hostfwd=tcp:127.0.0.1:${HTTP_PORT}-:80"
+[ "$API_PORT" != 0 ]  && NETDEV="$NETDEV,hostfwd=tcp:127.0.0.1:${API_PORT}-:28443"
 ARGS=(
   -M "virt,highmem=off" -cpu cortex-a7 -smp 1 -m 256M
   # virtio-mmio defaults to the legacy (0.9) transport; virtio-gpu and
   # virtio-input are VERSION_1-only devices and never bind without this.
   -global "virtio-mmio.force-legacy=false"
   -kernel "$KERNEL" -initrd "$INITRD"
-  -netdev "user,id=n0,hostfwd=tcp:127.0.0.1:${SSH_PORT}-:22,hostfwd=tcp:127.0.0.1:${HTTP_PORT}-:80,hostfwd=tcp:127.0.0.1:${API_PORT}-:28443"
+  -netdev "$NETDEV"
   -device "virtio-net-device,netdev=n0"
   -no-reboot
 )

qemu/tests/boot-smoke.sh

@@ -27,13 +27,15 @@ command -v qemu-system-arm >/dev/null || {
 LOG="$(mktemp "${TMPDIR:-/tmp}/warden-qemu-smoke.XXXXXX")"
 trap 'rm -f "$LOG"' EXIT
 
-# NOTE: never pass `earlyprintk` — the config's DEBUG_UART_PHYS is the RV1106's
-# 0xff4c0000, which does not exist on -M virt.
-timeout 180 qemu-system-arm \
-  -M virt,highmem=off -cpu cortex-a7 -smp 1 -m 256M \
-  -kernel "$ZIMAGE" -initrd "$INITRD" \
-  -append "console=ttyAMA0 rdinit=/init" \
-  -nographic -no-reboot </dev/null | tee "$LOG" || {
+# Delegate the qemu invocation to run.sh (--no-disk) so the machine shape
+# (-M virt,highmem=off, cpu, memory, virtio topology) lives in exactly one
+# place — the two hand-copied invocations had already drifted once.
+# timeout -k: a wedged qemu that ignores SIGTERM gets SIGKILLed 10s later
+# instead of holding the job until the workflow-level timeout.
+timeout -k 10 180 bash "$QDIR/run.sh" \
+  --kernel "$ZIMAGE" --initrd "$INITRD" --no-disk \
+  --ssh-port 0 --http-port 0 --api-port 0 \
+  </dev/null | tee "$LOG" || {
     echo "FATAL: qemu exited non-zero (or hung until the 180s timeout)" >&2
     exit 1
   }

qemu/tests/portal-scenario.sh

@@ -62,11 +62,19 @@ python3 "$FLARE_EDGE/tools/mock-flare-portal.py" \
   --port "$PORT" --device "$DEVICE_ID:$API_KEY" \
   --wfw "$WORK/offer.wfw" > "$WORK/mock.log" 2>&1 &
 MOCK_PID=$!
+mock_ready=0
 for _ in $(seq 1 50); do
-  curl -so /dev/null "http://127.0.0.1:$PORT/" && break
+  curl -so /dev/null "http://127.0.0.1:$PORT/" && { mock_ready=1; break; }
   kill -0 "$MOCK_PID" 2>/dev/null || { echo "FATAL: mock portal died:" >&2; cat "$WORK/mock.log" >&2; exit 1; }
   sleep 0.2
 done
+# The loop must not fall through silently: an alive-but-unresponsive mock
+# would otherwise surface 420s later as an unrelated assertion timeout.
+[ "$mock_ready" = 1 ] || {
+  echo "FATAL: mock portal never answered on :$PORT within 10s" >&2
+  tail -10 "$WORK/mock.log" >&2
+  exit 1
+}
 echo "== mock portal on :$PORT, device $DEVICE_ID"
 
 # 2. image seeded with the portal URL + credentials.
@@ -79,11 +87,34 @@ bash "$QDIR/mkimage.sh" \
   --state "flare.api_key=$API_KEY" \
   --state "flare.site=qemu-devsim"
 
-# 3. boot the VM headless (daemons run; console log to file).
-bash "$QDIR/run.sh" --kernel "$ZIMAGE" \
-  --ssh-port $((PORT + 1)) --http-port $((PORT + 2)) --api-port $((PORT + 3)) \
-  > "$WORK/console.log" 2>&1 &
-QEMU_PID=$!
+# 3. boot the VM headless (daemons run; console log to file). Random hostfwd
+#    ports can collide with another process — detect the early qemu bind
+#    failure and retry with a fresh base rather than failing spuriously.
+QEMU_PID=""
+for _attempt in 1 2 3; do
+  VMBASE=$((20000 + RANDOM % 20000))
+  : > "$WORK/console.log"
+  bash "$QDIR/run.sh" --kernel "$ZIMAGE" \
+    --ssh-port "$VMBASE" --http-port $((VMBASE + 1)) --api-port $((VMBASE + 2)) \
+    > "$WORK/console.log" 2>&1 &
+  QEMU_PID=$!
+  sleep 3
+  if kill -0 "$QEMU_PID" 2>/dev/null; then
+    break
+  fi
+  if grep -aq 'Could not set up host forwarding' "$WORK/console.log"; then
+    echo "== hostfwd port collision on base $VMBASE — retrying"
+    QEMU_PID=""
+    continue
+  fi
+  echo "FATAL: VM died at launch:" >&2
+  tail -20 "$WORK/console.log" >&2
+  exit 1
+done
+if [ -z "$QEMU_PID" ] || ! kill -0 "$QEMU_PID" 2>/dev/null; then
+  echo "FATAL: could not launch the VM after 3 port attempts" >&2
+  exit 1
+fi
 
 # 4. assert: rootfs up, and the portal saw — from OUR device id — an
 #    authenticated check-in, the firmware desired-state pull, and the signed
@@ -99,6 +130,11 @@ while [ $SECONDS -lt $deadline ]; do
   grep -aq "GET /api/v1/devices/$DEVICE_ID/firmware -> 200" "$WORK/mock.log" && ok_fw=1
   grep -aq "GET /api/v1/devices/$DEVICE_ID/firmware/assets/.* -> 200" "$WORK/mock.log" && ok_asset=1
   [ $ok_report -eq 1 ] && [ $ok_fw -eq 1 ] && [ $ok_asset -eq 1 ] && break
+  grep -aq 'WARDEN-QEMU-MOUNT-FAILED' "$WORK/console.log" && {
+    echo "FATAL: guest partition mount failed (bad image?):" >&2
+    grep -a 'WARDEN-QEMU-MOUNT-FAILED' "$WORK/console.log" >&2
+    exit 1
+  }
   kill -0 "$QEMU_PID" 2>/dev/null || { echo "FATAL: VM exited early" >&2; tail -30 "$WORK/console.log" >&2; exit 1; }
   sleep 2
 done

qemu/tests/qmp.py

@@ -11,7 +11,7 @@ import sys
 import time
 
 
-def rpc(sock, obj):
+def rpc(sock, sock_file, obj):
     sock.sendall((json.dumps(obj) + "\n").encode())
     while True:
         line = sock_file.readline()
@@ -29,15 +29,20 @@ def main():
     if len(sys.argv) < 3:
         sys.exit(__doc__)
     path, cmd = sys.argv[1], sys.argv[2]
-    global sock_file
+    need = {"screendump": 4, "tap": 5, "quit": 3}
+    if cmd not in need:
+        sys.exit(f"unknown command {cmd}\n{__doc__}")
+    if len(sys.argv) < need[cmd]:
+        sys.exit(f"{cmd}: missing argument(s)\n{__doc__}")
+
     s = socket.socket(socket.AF_UNIX)
     s.connect(path)
-    sock_file = s.makefile("r")
-    sock_file.readline()  # greeting banner
-    rpc(s, {"execute": "qmp_capabilities"})
+    f = s.makefile("r")
+    f.readline()  # greeting banner
+    rpc(s, f, {"execute": "qmp_capabilities"})
 
     if cmd == "screendump":
-        rpc(s, {"execute": "screendump", "arguments": {"filename": sys.argv[3]}})
+        rpc(s, f, {"execute": "screendump", "arguments": {"filename": sys.argv[3]}})
     elif cmd == "tap":
         x, y = int(sys.argv[3]), int(sys.argv[4])
         press = [
@@ -46,16 +51,14 @@ def main():
             {"type": "btn", "data": {"down": True, "button": "left"}},
         ]
         release = [{"type": "btn", "data": {"down": False, "button": "left"}}]
-        rpc(s, {"execute": "input-send-event", "arguments": {"events": press}})
+        rpc(s, f, {"execute": "input-send-event", "arguments": {"events": press}})
         # Hold the press across several LVGL indev poll periods (33 ms each):
         # an instantaneous press+release lands inside one poll and no click
         # is ever registered.
         time.sleep(0.2)
-        rpc(s, {"execute": "input-send-event", "arguments": {"events": release}})
+        rpc(s, f, {"execute": "input-send-event", "arguments": {"events": release}})
     elif cmd == "quit":
         s.sendall(b'{"execute":"quit"}\n')
-    else:
-        sys.exit(f"unknown command {cmd}")
 
 
 if __name__ == "__main__":

qemu/tests/ui-shot.sh

@@ -1,9 +1,10 @@
 #!/usr/bin/env bash
 # Display + touch scenario: boot the VM headless with virtio-gpu, wait for the
-# LVGL UI (fbdev build) to start, screendump over QMP, inject an absolute
-# touch tap (virtio-tablet), screendump again. Asserts the first frame is
-# non-blank; reports (does not assert) whether the tap changed pixels — the
-# device-level analogue of flare-edge's Xvfb/xdotool sim-test.sh.
+# LVGL UI (fbdev build) to render a real frame, then inject an absolute touch
+# tap on the Metrics tab (virtio-tablet) and ASSERT the frame changed — the
+# device-level analogue of flare-edge's Xvfb/xdotool sim-test.sh. Readiness is
+# polled from screendumps on bounded deadlines, never guessed with fixed
+# sleeps: TCG renders CPU-bound and a loaded host can be arbitrarily slow.
 #
 # FAILS CLOSED on missing prerequisites.
 #
@@ -40,11 +41,30 @@ trap cleanup EXIT
 bash "$QDIR/mkinitramfs.sh"
 bash "$QDIR/mkimage.sh"
 
-PORT=$((21000 + RANDOM % 20000))
-bash "$QDIR/run.sh" --kernel "$ZIMAGE" --display headless --qmp "$WORK/qmp.sock" \
-  --ssh-port "$PORT" --http-port $((PORT + 1)) --api-port $((PORT + 2)) \
-  > "$WORK/console.log" 2>&1 &
-QEMU_PID=$!
+# Random hostfwd ports can collide — detect qemu's early bind failure and
+# retry with a fresh base rather than failing spuriously.
+for _attempt in 1 2 3; do
+  PORT=$((21000 + RANDOM % 20000))
+  : > "$WORK/console.log"
+  bash "$QDIR/run.sh" --kernel "$ZIMAGE" --display headless --qmp "$WORK/qmp.sock" \
+    --ssh-port "$PORT" --http-port $((PORT + 1)) --api-port $((PORT + 2)) \
+    > "$WORK/console.log" 2>&1 &
+  QEMU_PID=$!
+  sleep 3
+  kill -0 "$QEMU_PID" 2>/dev/null && break
+  if grep -aq 'Could not set up host forwarding' "$WORK/console.log"; then
+    echo "== hostfwd port collision on base $PORT — retrying"
+    QEMU_PID=""
+    continue
+  fi
+  echo "FATAL: VM died at launch:" >&2
+  tail -20 "$WORK/console.log" >&2
+  exit 1
+done
+if [ -z "$QEMU_PID" ] || ! kill -0 "$QEMU_PID" 2>/dev/null; then
+  echo "FATAL: could not launch the VM after 3 port attempts" >&2
+  exit 1
+fi
 
 deadline=$((SECONDS + 120))
 while [ $SECONDS -lt $deadline ]; do
@@ -57,36 +77,55 @@ grep -aq 'init: starting warden-ui' "$WORK/console.log" || {
   tail -25 "$WORK/console.log" >&2
   exit 1
 }
-sleep 8   # let LVGL render the first frames
 
 qmp() { python3 "$HERE/qmp.py" "$WORK/qmp.sock" "$@"; }
 
-qmp screendump "$WORK/shot1.ppm"
+# Frame is "real" once it has more than a handful of distinct colors (a blank
+# or console-only frame has very few).
+frame_rendered() { # $1 = ppm path
+  python3 - "$1" <<'EOF'
+import sys
+data = open(sys.argv[1], "rb").read()
+parts = data.split(b"\n", 3)               # P6 header: magic, dims, maxval, raw RGB
+pixels = parts[3] if len(parts) == 4 else b""
+distinct = len(set(pixels[i:i+3] for i in range(0, min(len(pixels), 3*720*720), 3)))
+print(f"{sys.argv[1]}: {len(pixels)} bytes, {distinct} distinct colors")
+sys.exit(0 if distinct > 16 else 1)
+EOF
+}
+
+# Poll for the first rendered frame (bounded, no guessed sleep).
+rendered=0
+deadline=$((SECONDS + 90))
+while [ $SECONDS -lt $deadline ]; do
+  qmp screendump "$WORK/shot1.ppm"
+  if frame_rendered "$WORK/shot1.ppm"; then rendered=1; break; fi
+  sleep 3
+done
+[ "$rendered" = 1 ] || {
+  echo "FATAL: UI never rendered a non-blank frame within 90s" >&2
+  exit 1
+}
+
 # Tap the "Metrics" tab: pixel (373,40) of 720x720 scaled to the QMP absolute
-# range 0..32767 — switching tabs must repaint the content area.
+# range 0..32767 — switching tabs must repaint the content area. Poll for the
+# repaint rather than guessing a delay.
 qmp tap 16975 1820
-sleep 3
-qmp screendump "$WORK/shot2.ppm"
+changed=0
+deadline=$((SECONDS + 30))
+while [ $SECONDS -lt $deadline ]; do
+  sleep 2
+  qmp screendump "$WORK/shot2.ppm"
+  if ! cmp -s "$WORK/shot1.ppm" "$WORK/shot2.ppm"; then changed=1; break; fi
+done
 
 mkdir -p "$OUTDIR"
 cp "$WORK/shot1.ppm" "$OUTDIR/ui-shot1.ppm"
-cp "$WORK/shot2.ppm" "$OUTDIR/ui-shot2.ppm"
+cp "$WORK/shot2.ppm" "$OUTDIR/ui-shot2.ppm" 2>/dev/null || true
 
-# Non-blank: more than one distinct pixel value in the raw PPM payload.
-python3 - "$WORK/shot1.ppm" <<'EOF'
-import sys
-data = open(sys.argv[1], "rb").read()
-# P6 header: magic, dims, maxval, then raw RGB
-parts = data.split(b"\n", 3)
-pixels = parts[3] if len(parts) == 4 else b""
-distinct = len(set(pixels[i:i+3] for i in range(0, min(len(pixels), 3*720*720), 3)))
-print(f"shot1: {len(pixels)} bytes of pixels, {distinct} distinct colors")
-sys.exit(0 if distinct > 1 else 1)
-EOF
-
-if cmp -s "$WORK/shot1.ppm" "$WORK/shot2.ppm"; then
-  echo "FATAL: tapping the Metrics tab did not change the frame — touch is not reaching the UI" >&2
+[ "$changed" = 1 ] || {
+  echo "FATAL: tapping the Metrics tab did not change the frame within 30s — touch is not reaching the UI" >&2
   exit 1
-fi
+}
 echo "tap on the Metrics tab repainted the frame (touch reached the UI)"
 echo "UI-SHOT-PASS (screenshots in $OUTDIR/ui-shot{1,2}.ppm)"