qemu: drive scripted UI scenarios

ui-shot.sh proves touch reaches the UI in one tap. Verifying a UI change needs
a SEQUENCE -- swipe through the app rows, open a submenu, tap a tab, bring up
the keyboard -- and booting per step costs about a minute under TCG, so:

- qmp.py gains a `drive` mode: one connection, one boot, a script of
  tap/swipe/fling/shot/sleep steps in PANEL PIXELS rather than the raw
  0..32767 tablet axis. Swipes interpolate their motion, because LVGL decides
  a gesture from the movement between indev polls and a press-then-release
  with nothing in between is a click, not a scroll.
- ui-drive.sh runs such a script against a booted VM and collects the
  screenshots.

It also FAILS on a UI that died mid-script. warden-ui crashing leaves its last
frame in the framebuffer, so screendumps keep returning a plausible picture of
a program that no longer exists; stage-2 init now announces the exit and its
status on the console, and ui-drive.sh greps for that after the run. This is
what caught the SIGSEGV behind flare-edge#125.

Stage-2 init also mounts devpts. The UI's Terminal page opens a PTY, so
without it that page could only ever report "no PTY available" -- it rendered,
which made a screenshot scenario look fine while the one thing the page does
was untestable.

tests/scripts/nav-stress.txt is the first committed drive script: the
navigation sequence that reproduces flare-edge#125.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn

qemu/README.md

@@ -68,6 +68,7 @@ All take the virt-fragment `<zImage>`; `FLARE_EDGE=<checkout>` where noted.
 | `portal-scenario.sh` | `FLARE_EDGE` | real flared against the desk mock portal: authenticated check-in, desired-state pull, signed tier-1 `.wfw` download; verify/stage/APPLYING as a dry run (no `WARDEN_FW_ALLOW_APPLY`) |
 | `ota-apply.sh` | `FLARE_EDGE` | the FULL apply: the `.wfw`'s bootable rootfs payload is written to rootfs_b (`run.sh --allow-apply` gates it per boot), the AvbABData in `misc` flips, and slot `_b` boots the applied version |
 | `ui-shot.sh` | - | display+touch, headless: QMP-screendumps the 720x720 UI, taps the Metrics tab via `input-send-event`, asserts the frame changed (`qmp.py` is the QMP client) |
+| `ui-drive.sh <script>` | - | the same rig for a SEQUENCE: boots once, runs a `qmp.py drive` script of taps/swipes/screenshots in panel pixels, and FAILS if warden-ui died on the way (stage-2 init announces the exit on the console). `tests/scripts/nav-stress.txt` is the navigation regression: it reproduces the s_row_left overflow that segfaulted the UI on returning to Settings > Apps |
 | `real-image-boot.sh` | matched `rootfs.img` + `oem.img` | an ACTUAL flare-edge build (placed by `mkimage.sh --rootfs-image/--oem-image`) boots its own init chain to getty; binaries predating known fixes reproduce their bugs faithfully, a time machine for field issues |
 | watchdog (`run.sh --watchdog`) | - | arm `/dev/watchdog`, don't pet: the VM resets ~30 s later (verified) |
 

qemu/rootfs/sbin/init

@@ -13,6 +13,12 @@ exec </dev/console >/dev/console 2>&1
 mount -t proc proc /proc
 mount -t sysfs sysfs /sys
 mount -t tmpfs tmpfs /tmp
+# The UI's Terminal page opens a PTY (posix_openpt), which needs devpts mounted
+# and /dev/ptmx pointing into it. Without this the page can only ever report
+# "no PTY available" -- it renders, so a screenshot scenario looks fine, while
+# the one thing the page does is untestable in the VM.
+mkdir -p /dev/pts
+mount -t devpts devpts /dev/pts 2>/dev/null
 
 # shellcheck source=qemu/rootfs/etc/warden-lib.sh disable=SC1091
 . /etc/warden-lib.sh
@@ -93,7 +99,21 @@ done
 # virt.fragment kernel.
 if [ -x /usr/bin/warden-ui ] && [ -c /dev/fb0 ]; then
     echo "init: starting warden-ui (fbdev)"
-    /usr/bin/warden-ui > /tmp/warden-ui.log 2>&1 &
+    # Announce the exit on the CONSOLE, not just in the log. A UI that dies
+    # mid-scenario otherwise looks exactly like a UI that stopped repainting:
+    # the framebuffer holds its last frame, screendumps keep working, and the
+    # scenario reports a stale picture as the current state. With this, a crash
+    # is one grep away for any scenario driving the VM from outside, and the
+    # signal or status that caused it is on the line.
+    (
+        /usr/bin/warden-ui > /tmp/warden-ui.log 2>&1
+        rc=$?
+        echo "init: warden-ui EXITED rc=$rc"
+        # 128+n is a signal death (139 = SIGSEGV); dump the tail so the
+        # scenario's console log carries the UI's own last words.
+        echo "init: warden-ui log tail:"
+        tail -n 20 /tmp/warden-ui.log 2>/dev/null
+    ) &
 fi
 
 if grep -qw warden.shell /proc/cmdline; then

qemu/tests/qmp.py

@@ -4,12 +4,32 @@
     qmp.py <socket> screendump <out.ppm>
     qmp.py <socket> tap <x> <y>          # absolute 0..32767 (virtio-tablet)
     qmp.py <socket> quit
+    qmp.py <socket> drive <script> <outdir> [--size N]
+
+`drive` runs a whole interaction on ONE connection, so the VM boots once and
+the scenario costs a couple of seconds per step instead of a full boot. Its
+script is one command per line, '#' comments and blank lines ignored, and
+coordinates are PANEL PIXELS (0..size-1, default 720) rather than the
+0..32767 absolute axis the raw `tap` takes -- the point of the mode is to write
+"tap the gear at 47,676" straight off a screenshot.
+
+    shot NAME              screendump to <outdir>/NAME.ppm
+    tap X Y                press, hold past several LVGL poll periods, release
+    swipe X1 Y1 X2 Y2 [MS] drag, interpolated so LVGL sees real motion
+    fling X1 Y1 X2 Y2      swipe fast enough to leave momentum behind
+    sleep SECONDS          let animations settle or timers fire
+    echo TEXT              progress marker in the driver's own output
+
+A step that names an unknown command is a FATAL error rather than a skip: a
+silently-ignored line in a scenario is a test that proves nothing.
 """
 import json
 import socket
 import sys
 import time
 
+AXIS_MAX = 32767
+
 
 def rpc(sock, sock_file, obj):
     sock.sendall((json.dumps(obj) + "\n").encode())
@@ -25,16 +45,112 @@ def rpc(sock, sock_file, obj):
         # asynchronous events are interleaved; skip them
 
 
+def send_events(s, f, events):
+    rpc(s, f, {"execute": "input-send-event", "arguments": {"events": events}})
+
+
+def abs_ev(axis, value):
+    return {"type": "abs", "data": {"axis": axis, "value": value}}
+
+
+def btn_ev(down):
+    return {"type": "btn", "data": {"down": down, "button": "left"}}
+
+
+def to_axis(px, size):
+    """Panel pixels -> the tablet's absolute axis, clamped to the panel."""
+    px = max(0, min(size - 1, int(px)))
+    return int(px * AXIS_MAX / (size - 1))
+
+
+def do_tap(s, f, ax, ay, hold=0.2):
+    send_events(s, f, [abs_ev("x", ax), abs_ev("y", ay), btn_ev(True)])
+    # Hold the press across several LVGL indev poll periods (33 ms each): an
+    # instantaneous press+release lands inside one poll and no click is ever
+    # registered.
+    time.sleep(hold)
+    send_events(s, f, [btn_ev(False)])
+
+
+def do_swipe(s, f, x1, y1, x2, y2, size, ms=400, steps=None):
+    """Drag with interpolated motion.
+
+    LVGL decides a gesture from the movement BETWEEN indev polls, so a press at
+    the start and a release at the end (with nothing in between) reads as a
+    click on whatever was under the finger, not a scroll. The interpolation
+    below is the whole reason a scroll can be tested at all; the step count is
+    derived from the duration so the poll period always sees a few pixels of
+    travel.
+    """
+    ax1, ay1 = to_axis(x1, size), to_axis(y1, size)
+    ax2, ay2 = to_axis(x2, size), to_axis(y2, size)
+    if steps is None:
+        steps = max(6, int(ms / 25))
+    send_events(s, f, [abs_ev("x", ax1), abs_ev("y", ay1), btn_ev(True)])
+    time.sleep(0.05)
+    for i in range(1, steps + 1):
+        t = i / steps
+        send_events(s, f, [
+            abs_ev("x", int(ax1 + (ax2 - ax1) * t)),
+            abs_ev("y", int(ay1 + (ay2 - ay1) * t)),
+        ])
+        time.sleep(ms / 1000.0 / steps)
+    send_events(s, f, [btn_ev(False)])
+
+
+def drive(s, f, script_path, outdir, size):
+    import os
+    os.makedirs(outdir, exist_ok=True)
+    with open(script_path) as fh:
+        lines = fh.readlines()
+
+    for lineno, raw in enumerate(lines, 1):
+        line = raw.split("#", 1)[0].strip()
+        if not line:
+            continue
+        parts = line.split()
+        cmd, args = parts[0], parts[1:]
+
+        if cmd == "shot":
+            name = args[0]
+            path = os.path.join(os.path.abspath(outdir), f"{name}.ppm")
+            rpc(s, f, {"execute": "screendump", "arguments": {"filename": path}})
+            print(f"  [{lineno}] shot {name}", flush=True)
+        elif cmd == "tap":
+            x, y = int(args[0]), int(args[1])
+            do_tap(s, f, to_axis(x, size), to_axis(y, size))
+            print(f"  [{lineno}] tap {x},{y}", flush=True)
+        elif cmd == "swipe":
+            ms = int(args[4]) if len(args) > 4 else 400
+            do_swipe(s, f, int(args[0]), int(args[1]), int(args[2]), int(args[3]), size, ms)
+            print(f"  [{lineno}] swipe {args[0]},{args[1]} -> {args[2]},{args[3]}", flush=True)
+        elif cmd == "fling":
+            do_swipe(s, f, int(args[0]), int(args[1]), int(args[2]), int(args[3]),
+                     size, ms=120)
+            print(f"  [{lineno}] fling {args[0]},{args[1]} -> {args[2]},{args[3]}", flush=True)
+        elif cmd == "sleep":
+            time.sleep(float(args[0]))
+            print(f"  [{lineno}] sleep {args[0]}", flush=True)
+        elif cmd == "echo":
+            print(f"  [{lineno}] {' '.join(args)}", flush=True)
+        else:
+            sys.exit(f"FATAL: {script_path}:{lineno}: unknown command '{cmd}'")
+
+
 def main():
     if len(sys.argv) < 3:
         sys.exit(__doc__)
     path, cmd = sys.argv[1], sys.argv[2]
-    need = {"screendump": 4, "tap": 5, "quit": 3}
+    need = {"screendump": 4, "tap": 5, "quit": 3, "drive": 5}
     if cmd not in need:
         sys.exit(f"unknown command {cmd}\n{__doc__}")
     if len(sys.argv) < need[cmd]:
         sys.exit(f"{cmd}: missing argument(s)\n{__doc__}")
 
+    size = 720
+    if "--size" in sys.argv:
+        size = int(sys.argv[sys.argv.index("--size") + 1])
+
     s = socket.socket(socket.AF_UNIX)
     s.connect(path)
     f = s.makefile("r")
@@ -44,19 +160,9 @@ def main():
     if cmd == "screendump":
         rpc(s, f, {"execute": "screendump", "arguments": {"filename": sys.argv[3]}})
     elif cmd == "tap":
-        x, y = int(sys.argv[3]), int(sys.argv[4])
+        do_tap(s, f, int(sys.argv[3]), int(sys.argv[4]))
-        press = [
+    elif cmd == "drive":
-            {"type": "abs", "data": {"axis": "x", "value": x}},
+        drive(s, f, sys.argv[3], sys.argv[4], size)
-            {"type": "abs", "data": {"axis": "y", "value": y}},
-            {"type": "btn", "data": {"down": True, "button": "left"}},
-        ]
-        release = [{"type": "btn", "data": {"down": False, "button": "left"}}]
-        rpc(s, f, {"execute": "input-send-event", "arguments": {"events": press}})
-        # Hold the press across several LVGL indev poll periods (33 ms each):
-        # an instantaneous press+release lands inside one poll and no click
-        # is ever registered.
-        time.sleep(0.2)
-        rpc(s, f, {"execute": "input-send-event", "arguments": {"events": release}})
     elif cmd == "quit":
         s.sendall(b'{"execute":"quit"}\n')
 

qemu/tests/scripts/nav-stress.txt

@@ -0,0 +1,55 @@
+# Navigation stress: the sequence that used to kill warden-ui.
+#
+# Settings > Apps reads the PROJECTED MENU TABLE (app.c's all_proj, populated
+# lazily on that page's first build). Walking the app rows writes one timestamp
+# per row into app.c's s_row_left. While that array was sized 8 and the shipped
+# registry projected 18 rows, leaving any row from index 8 up wrote past it --
+# straight over all_count and all_proj, which the linker places immediately
+# after. Coming back to Settings > Apps then walked a tick value as a `const
+# char *` and the process died with SIGSEGV (rc=139).
+#
+# So the order matters and is the whole test: populate the table, walk the late
+# rows, come BACK. A run that only walks rows never triggers it, because with
+# all_count still zero the page's lazy init quietly repairs the array.
+#
+# Run: ui-drive.sh <zImage> tests/scripts/nav-stress.txt <outdir>
+# PASS is ui-drive.sh finding no "warden-ui EXITED" on the console.
+
+echo == populate the projected menu table
+tap 47 676
+sleep 2
+tap 464 40
+sleep 4
+shot 01-apps-first
+
+echo == back to column 0, where the rail is on screen
+tap 174 40
+sleep 2
+
+echo == walk the app rows, the late ones included
+tap 47 512
+sleep 2
+tap 47 592
+sleep 2
+swipe 47 560 47 220 500
+sleep 1
+tap 47 420
+sleep 2
+tap 47 500
+sleep 2
+swipe 47 560 47 220 500
+sleep 1
+tap 47 380
+sleep 2
+tap 47 520
+sleep 2
+shot 02-late-rows
+
+echo == and back: this is the step that used to segfault
+tap 47 676
+sleep 2
+tap 464 40
+sleep 4
+shot 03-apps-again
+sleep 3
+shot 04-still-alive

qemu/tests/ui-drive.sh

@@ -0,0 +1,132 @@
+#!/usr/bin/env bash
+# Drive the LVGL UI through a scripted interaction and collect screenshots.
+#
+# ui-shot.sh proves touch reaches the UI in one tap; this is the same rig for
+# work that needs a SEQUENCE — swipe through the app rows, open a submenu, tap a
+# tab, bring up the keyboard — with a screendump wherever the script asks for
+# one. One boot serves the whole script, because booting per step (TCG, no KVM)
+# costs about a minute and a real interaction is thirty steps.
+#
+# It also FAILS on a UI that died mid-script. The framebuffer keeps its last
+# frame when warden-ui crashes, so screendumps carry on returning a plausible
+# picture of a program that no longer exists; stage-2 init announces the exit on
+# the console (see rootfs/sbin/init) and this greps for it after the run.
+#
+# FAILS CLOSED on missing prerequisites.
+#
+# Usage: ui-drive.sh <zImage-virt> <script> [out-dir]
+#   <script> is a qmp.py `drive` script: see its docstring for the commands.
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"     # qemu/tests/
+QDIR="$(cd "$HERE/.." && pwd)"                           # qemu/
+ZIMAGE="${1:-}"
+SCRIPT="${2:-}"
+OUTDIR="${3:-$QDIR/out/ui-drive}"
+
+if [ -z "$ZIMAGE" ] || [ ! -f "$ZIMAGE" ]; then
+  echo "FATAL: usage: $0 <zImage> <script> [out-dir]: the virt.fragment kernel variant" >&2
+  exit 1
+fi
+if [ -z "$SCRIPT" ] || [ ! -f "$SCRIPT" ]; then
+  echo "FATAL: no drive script at '$SCRIPT'" >&2
+  exit 1
+fi
+[ -x "$QDIR/payload/warden-ui" ] || {
+  echo "FATAL: no qemu/payload/warden-ui: build it with flare-edge tools/build-ui-vm.sh" >&2
+  exit 1
+}
+command -v qemu-system-arm >/dev/null || {
+  echo "FATAL: qemu-system-arm not on PATH: see qemu/README.md" >&2
+  exit 1
+}
+
+mkdir -p "$OUTDIR"
+# Short-named scratch: AF_UNIX socket paths are capped at ~108 chars.
+WORK="$(mktemp -d /tmp/wqd.XXXXXX)"
+QEMU_PID=""
+cleanup() {
+  if [ -n "$QEMU_PID" ]; then
+    python3 "$HERE/qmp.py" "$WORK/qmp.sock" quit 2>/dev/null || true
+    sleep 1
+    kill "$QEMU_PID" 2>/dev/null || true
+  fi
+  cp "$WORK/console.log" "$OUTDIR/console.log" 2>/dev/null || true
+  rm -rf "$WORK"
+}
+trap cleanup EXIT
+
+bash "$QDIR/mkinitramfs.sh"
+bash "$QDIR/mkimage.sh"
+
+# Random hostfwd ports can collide. Detect qemu's early bind failure and retry
+# with a fresh base rather than failing spuriously.
+for _attempt in 1 2 3; do
+  PORT=$((21000 + RANDOM % 20000))
+  : > "$WORK/console.log"
+  bash "$QDIR/run.sh" --kernel "$ZIMAGE" --display headless --qmp "$WORK/qmp.sock" \
+    --ssh-port "$PORT" --http-port $((PORT + 1)) --api-port $((PORT + 2)) \
+    > "$WORK/console.log" 2>&1 &
+  QEMU_PID=$!
+  sleep 3
+  kill -0 "$QEMU_PID" 2>/dev/null && break
+  if grep -aq 'Could not set up host forwarding' "$WORK/console.log"; then
+    echo "== hostfwd port collision on base $PORT, retrying"
+    QEMU_PID=""
+    continue
+  fi
+  echo "FATAL: VM died at launch:" >&2
+  tail -20 "$WORK/console.log" >&2
+  exit 1
+done
+if [ -z "$QEMU_PID" ] || ! kill -0 "$QEMU_PID" 2>/dev/null; then
+  echo "FATAL: could not launch the VM after 3 port attempts" >&2
+  exit 1
+fi
+
+echo "== waiting for warden-ui"
+deadline=$((SECONDS + 180))
+while [ $SECONDS -lt $deadline ]; do
+  grep -aq 'init: starting warden-ui' "$WORK/console.log" && break
+  kill -0 "$QEMU_PID" 2>/dev/null || { echo "FATAL: VM exited early" >&2; tail -25 "$WORK/console.log" >&2; exit 1; }
+  sleep 2
+done
+grep -aq 'init: starting warden-ui' "$WORK/console.log" || {
+  echo "FATAL: warden-ui never started (no fb0? wrong kernel?)" >&2
+  tail -25 "$WORK/console.log" >&2
+  exit 1
+}
+
+# A started process is not a rendered frame. Poll screendumps until the panel
+# stops being a single flat colour, on a bounded deadline: TCG renders CPU-bound
+# and a loaded host can be arbitrarily slow, so this is never a fixed sleep.
+echo "== waiting for the first real frame"
+deadline=$((SECONDS + 180))
+ready=0
+while [ $SECONDS -lt $deadline ]; do
+  python3 "$HERE/qmp.py" "$WORK/qmp.sock" screendump "$WORK/probe.ppm" 2>/dev/null || { sleep 2; continue; }
+  colors="$(python3 -c "
+import sys
+d=open('$WORK/probe.ppm','rb').read()
+print(len(set(d[i:i+3] for i in range(15, len(d), 3))))
+" 2>/dev/null || echo 0)"
+  [ "${colors:-0}" -gt 32 ] && { ready=1; break; }
+  sleep 3
+done
+[ "$ready" = 1 ] || {
+  echo "FATAL: the UI never rendered a real frame" >&2
+  tail -25 "$WORK/console.log" >&2
+  exit 1
+}
+
+echo "== driving $SCRIPT"
+python3 "$HERE/qmp.py" "$WORK/qmp.sock" drive "$SCRIPT" "$OUTDIR"
+
+# The UI must still be alive: see the header. Checked AFTER the script so a
+# crash caused by the interaction is caught, which is the usual case.
+if grep -aq 'warden-ui EXITED' "$WORK/console.log"; then
+  echo "FATAL: warden-ui DIED during the run:" >&2
+  grep -a -A22 'warden-ui EXITED' "$WORK/console.log" >&2
+  exit 1
+fi
+
+echo "UI-DRIVE-PASS (screenshots in $OUTDIR)"