qemu: start the device bus when the payload carries a broker
ci / test (push) Successful in 28s
ci / prune-artifacts (push) Skipped
ci / patches-apply (push) Successful in 1m2s
ci / mcdc (push) Successful in 29s
ci / bench (push) Successful in 12s
ci / mcdc (pull_request) Successful in 11s
ci / prune-artifacts (pull_request) Skipped
ci / kernel-build (pull_request) Skipped
ci / kernel-build (push) Skipped
ci / quality (push) Successful in 27s
ci / badges (push) Skipped
ci / qemu-tools (pull_request) Successful in 1m9s
ci / patches-apply (pull_request) Successful in 57s
ci / test (pull_request) Successful in 26s
ci / bench (pull_request) Successful in 16s
ci / quality (pull_request) Successful in 1m29s
ci / kernel-symbols (pull_request) Successful in 6m50s
ci / qemu-tools (push) Successful in 2m15s
ci / kernel-symbols (push) Successful in 4m42s
ci / badges (pull_request) Skipped

A panel runs one mosquitto on /run/warden/bus/bus.sock (flare-edge S10bus)
and every daemon and the UI talk over it; the rig had no broker, so the
Rust warden-modbus and the UI's bus client could not run there (flare-edge
#1754, the #1050 QEMU half). Stage-2 init now starts the broker before the
payload daemons when the payload carries a static mosquitto, with the
panel's paths: the socket under sock/ with the bus.sock link, and a fresh
0400 credential file per bus user under cred/. The rig broker takes any
user and loads no plugin (the rig is a test instrument; a static broker
loads none). A payload without a broker boots as before.

qemu/tests/init-rig-bus-test.sh covers no broker, a listening broker (layout,
modes, config), a broker that exits or never listens, bad user names and the
init order, under sh and busybox sh; with WARDEN_TEST_MOSQUITTO_DIR a real
mosquitto carries a message between two users logged in by credential file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KerYPVh72hhNgKi3wS5ijG
This commit is contained in:
BFE Engineering
2026-10-26 15:32:51 -06:00
co-authored by Claude Opus 5.5
parent 13cd0c63be
commit 5494b87dc3
6 changed files with 293 additions and 0 deletions
+8
View File
@@ -155,6 +155,14 @@ jobs:
run: bash qemu/tests/init-power-signals-test.sh
- name: rig init daemon order matches the panel (flare-edge#348)
run: bash qemu/tests/init-daemon-order-test.sh
- name: rig bus starts when the payload carries a broker (flare-edge#1754)
# A stub broker stands in for mosquitto (the runner has none), and
# once more under busybox sh, the rig's own shell.
run: |
bash qemu/tests/init-rig-bus-test.sh
if command -v busybox >/dev/null; then
WARDEN_TEST_SH="busybox sh" bash qemu/tests/init-rig-bus-test.sh
fi
- name: rig control bridge waits out a slow UI (flare-edge#1448)
run: bash qemu/tests/ctl-bridge-test.sh
- name: mkimage.sh SEED_DIR regression test
+6
View File
@@ -73,6 +73,11 @@ daemons a bench panel already runs: flare-edge `tools/rig-pull-daemons.sh
chardev without `--rs485`), so `warden-modbus` polls a port with nothing on
it, as on a panel with an empty header, and never the control channel.
A `mosquitto` in the payload starts the panel's device bus first (socket
`/run/warden/bus/bus.sock`, one credential file per bus user under
`/run/warden/bus/cred/`), as S10bus does before every bus client on a panel;
see `payload/README.md`. The rig broker takes any user and loads no plugin.
The VT cursor is kept off (`vt.global_cursor_default=0`): fbcon shares the
virtio-gpu framebuffer with warden-ui and its blinking cursor would otherwise
show up in screendumps at random (issue #18).
@@ -107,6 +112,7 @@ All take the virt-fragment `<zImage>`; `FLARE_EDGE=<checkout>` where noted.
| `test-ui-drive-rs485.sh` | - | offline, no VM: a fake `run.sh` and (for one case) a fake `socat` stand in so `ui-drive.sh --rs485-devices`'s startup fails closed instead of printing a false "== rs485 simulator: ..." over a bus nothing is serving, for each of a qemu rs.sock that never appears, a socat that never links rs.pty, and an mbsim.py that dies before opening rs.ctl; also proves `cleanup()` escalates to SIGKILL for a sim process that ignores SIGTERM |
| `run-sh-args-test.sh` | - | offline, no VM: a fake `qemu-system-arm` captures run.sh's own argv and pins the ctl/rs485 pci-serial ordering contract rootfs/sbin/init's ttyS0-vs-ttyS1 alias depends on -- the ctl device always enumerates before rs485, warden.ctl only lands on the cmdline when --ctl is given, and an rs485 pci-serial device (real or null-backed) is always present so the port count never shifts |
| `ctl-bridge-test.sh` | - | offline, no VM: runs the control bridge (`warden_ctl_bridge` in rootfs/etc/warden-lib.sh) against a fake warden-ui that answers some commands late. A reply slower than the old 5 s bound still arrives (the bound is now 30 s, `WARDEN_CTL_WAIT_S`), a timeout answers `bridge: no reply from warden-ui ...` (qmp.py raises it as a rig fault), and a timed-out command's late reply is drained instead of being read as the next command's answer (flare-edge #1448) |
| `init-rig-bus-test.sh` | - | offline, no VM: `warden_rig_bus` in rootfs/etc/warden-lib.sh against stub brokers. No broker in the payload creates nothing; a broker that listens gets the panel's layout (the `bus.sock` link, a 0400 64-hex credential per bus user, a config without password file, ACL or plugin); a broker that exits or never listens is reported within the bound; init starts the bus before the payload daemons. `WARDEN_TEST_MOSQUITTO_DIR` adds a real mosquitto carrying one message between two users logged in by credential file (flare-edge #1754) |
| watchdog (`run.sh --watchdog`) | - | arm `/dev/watchdog`, don't pet: the VM resets ~30 s later (verified) |
Scenario fine print:
+7
View File
@@ -26,6 +26,13 @@ musl binary, so flare-edge `tools/rig-pull-daemons.sh --host <panel>` copies
release and each md5 in `PULLED.txt`), minus `warden-ui`, which the rig
needs as the fbdev+evdev build.
A `mosquitto` here (the static armhf broker of flare-edge
`tools/mosquitto-build.sh armhf`) gives the guest the panel's device bus:
stage-2 init starts it on `/run/warden/bus/bus.sock` before any daemon and
writes a credential file per bus user to `/run/warden/bus/cred/<user>`
(`warden_rig_bus` in `qemu/rootfs/etc/warden-lib.sh`; the broker log is
`/tmp/mosquitto.log`). Without it the guest has no bus, as before.
Stage-2 init starts every `warden-*` daemon it finds here, in the panel's
own S-number order (ai, automation, modbus, mikrotik, asic, starlink,
stratum, flared), then `warden-ui` (the UI also needs `--display
+91
View File
@@ -148,3 +148,94 @@ _wcb_wait_reply() {
_wcb_n=$((_wcb_n + 1))
done
}
# The rig's device bus (flare-edge #1754, the #1050 QEMU half). A panel runs
# one mosquitto on /run/warden/bus/bus.sock (flare-edge S10bus) and every
# daemon, the UI included, talks over it. The rig starts the same broker
# when the payload carries one (/usr/bin/mosquitto, the static armhf build of
# flare-edge tools/mosquitto-build.sh); a payload without it boots exactly as
# before, with no bus.
#
# Same paths as the panel, so the production clients need no rig switch:
# the socket at <bus>/sock/bus.sock with the <bus>/bus.sock link clients
# dial, and one credential file per bus user at <bus>/cred/<user>, which a
# client reads at every CONNECT. What the rig leaves out, on purpose: the
# password file and ACL (the broker takes any user; the rig is a test
# instrument and every process in it runs as root anyway) and the retain gate
# plugin (a static broker loads no plugins). The credential files hold a
# fresh random secret all the same, so a client finds what it expects.
#
# WARDEN_RIG_BUS_USERS lists the bus users that get a credential file: the
# panel's /etc/warden/bus/users and bundled-users (flare-edge busgen output).
# WARDEN_RIG_BROKER, WARDEN_BUS_RUN, WARDEN_RIG_BUS_LOG and
# WARDEN_RIG_BUS_WAIT_S exist for the host test only.
warden_rig_bus() {
_wrb_broker="${WARDEN_RIG_BROKER:-/usr/bin/mosquitto}"
[ -x "$_wrb_broker" ] || return 0
_wrb_dir="${WARDEN_BUS_RUN:-/run/warden/bus}"
_wrb_log="${WARDEN_RIG_BUS_LOG:-/tmp/mosquitto.log}"
_wrb_users="${WARDEN_RIG_BUS_USERS:-flared ui io modbus appd telemetry automation.main}"
if ! mkdir -p "$_wrb_dir/cred" "$_wrb_dir/sock"; then
echo "init: rig bus FAILED: cannot create $_wrb_dir"
return 1
fi
chmod 0751 "$_wrb_dir"
chmod 0710 "$_wrb_dir/cred"
# The panel's broker settings (flare-edge
# overlay-flare-warden-ui/etc/warden/bus/mosquitto.conf) minus the
# password file, ACL and plugin lines (see above). `user root` keeps the
# broker from dropping to an account the rig rootfs does not have.
cat > "$_wrb_dir/mosquitto.conf" <<WRB_CONF
# QEMU rig broker, written by warden_rig_bus (qemu/rootfs/etc/warden-lib.sh).
user root
listener 0 $_wrb_dir/sock/bus.sock
allow_anonymous true
pid_file $_wrb_dir/sock/mosquitto.pid
use_username_as_clientid true
persistence false
message_size_limit 131072
max_packet_size 135168
max_connections 128
max_queued_messages 1000
max_queued_bytes 2097152
max_inflight_messages 20
max_qos 1
max_keepalive 60
sys_interval 0
connection_messages false
log_dest stderr
log_type error
log_type warning
WRB_CONF
for _wrb_u in $_wrb_users; do
case "$_wrb_u" in
''|*/*|.*) echo "init: rig bus: bad user name '$_wrb_u' skipped"; continue ;;
esac
rm -f "$_wrb_dir/cred/$_wrb_u"
(umask 077; od -An -tx1 -N32 /dev/urandom | tr -d ' \n' > "$_wrb_dir/cred/$_wrb_u")
chmod 0400 "$_wrb_dir/cred/$_wrb_u"
done
ln -sfn sock/bus.sock "$_wrb_dir/bus.sock"
ln -sfn sock/mosquitto.pid "$_wrb_dir/mosquitto.pid"
"$_wrb_broker" -c "$_wrb_dir/mosquitto.conf" > "$_wrb_log" 2>&1 &
_wrb_pid=$!
# Clients retry a missing socket, but a daemon started before the broker
# listens logs a refused first dial: wait for the socket (0.1 s polls,
# bounded) so the payload daemons find it on their first try.
_wrb_n=0
_wrb_limit=$(( ${WARDEN_RIG_BUS_WAIT_S:-10} * 10 ))
while [ ! -S "$_wrb_dir/sock/bus.sock" ]; do
if ! kill -0 "$_wrb_pid" 2>/dev/null; then
echo "init: rig bus FAILED: the broker exited; log tail:"
tail -n 5 "$_wrb_log" 2>/dev/null
return 1
fi
if [ "$_wrb_n" -ge "$_wrb_limit" ]; then
echo "init: rig bus FAILED: no socket after $((_wrb_limit / 10)) s"
return 1
fi
sleep 0.1
_wrb_n=$((_wrb_n + 1))
done
echo "init: rig bus up on $_wrb_dir/bus.sock (broker pid $_wrb_pid)"
}
+5
View File
@@ -108,6 +108,11 @@ export WARDEN_HARD_RESET=0
# Production init scripts create this volatile socket directory before their
# daemons start. The compact QEMU rootfs must provide the same contract.
mkdir -p /run/warden
# The device bus, when the payload carries a broker (warden_rig_bus): up
# before the daemons, as S10bus is before every bus client on the panel. A
# failed start is logged and the boot goes on: the clients keep dialing, and
# a flow that needs the bus fails on its own checks.
warden_rig_bus || true
# OTA apply is opt-in per boot (run.sh --allow-apply): writing rootfs_b is
# safe inside disk.img but must never be the default posture.
if grep -qw warden.fwapply /proc/cmdline; then
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env bash
# shellcheck disable=SC2317 # the checks and the cleanup run through check() and trap
# Host test for the rig's device bus, warden_rig_bus in
# qemu/rootfs/etc/warden-lib.sh (flare-edge #1754, #1050). Checks:
# - a payload without a broker boots as before: nothing created, status 0
# - with a broker: the panel's layout (sock/bus.sock and the bus.sock link,
# one 0400 credential file of 64 hex digits per bus user, each its own),
# a config with the listener and no password file, ACL or plugin
# - a broker that exits, or never listens, is reported and returns 1
# within the wait bound
# - a user name that is not a plain file name gets no file
# - /sbin/init starts the bus before the payload daemons
# A stub broker (python3, binds the configured listener) stands in for
# mosquitto. With WARDEN_TEST_MOSQUITTO_DIR set to a directory holding
# mosquitto, mosquitto_pub and mosquitto_sub (flare-edge
# tools/build-host-mosquitto.sh), the real broker also carries one message
# between two users that log in with their credential files.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
LIB="$HERE/../rootfs/etc/warden-lib.sh"
INIT="$HERE/../rootfs/sbin/init"
SH="${WARDEN_TEST_SH:-sh}"
WORK="$(mktemp -d)" || exit 1
cleanup() {
# The brokers outlive the shell that started them, as on the rig; each
# has this test's folder in its argv (the stub that never listens writes
# its pid instead).
pkill -f -- "$WORK/" 2>/dev/null
[ -f "$STUB/mute.pid" ] && kill "$(cat "$STUB/mute.pid")" 2>/dev/null
rm -rf "$WORK"
}
trap cleanup EXIT
STUB="$WORK/stubs"
mkdir -p "$STUB"
FAIL=0
check() { # check <label> <command...>
if "${@:2}"; then
echo "[PASS] $1"
else
echo "[FAIL] $1"
FAIL=1
fi
}
# Stub brokers: argv is "-c <conf>", as the lib calls mosquitto.
# good: binds the listener path from the config and stays.
# exits: dies at once. mute: stays and never listens.
cat > "$STUB/good" <<'PY'
#!/usr/bin/env python3
import socket, sys, time
conf = sys.argv[sys.argv.index("-c") + 1]
lines = open(conf).read().splitlines()
path = [l.split()[2] for l in lines if l.startswith("listener ")][0]
time.sleep(0.3)
s = socket.socket(socket.AF_UNIX)
s.bind(path)
s.listen(1)
print("stub broker listening", flush=True)
time.sleep(600)
PY
printf '#!/bin/sh\necho "Error: stub broker refused" >&2\nexit 3\n' > "$STUB/exits"
cat > "$STUB/mute" <<'EOF'
#!/bin/sh
echo $$ > "${0%/*}/mute.pid"
exec sleep 600
EOF
chmod +x "$STUB/good" "$STUB/exits" "$STUB/mute"
run_bus() { # run_bus <case> <broker> [VAR=value...]: output in <case>/out, status in <case>/rc
local name="$1" broker="$2"
shift 2
mkdir -p "$WORK/$name"
# $SH unquoted: WARDEN_TEST_SH may be "busybox sh". The lib is sourced
# inside, so the quoted script is meant literally.
# shellcheck disable=SC2086,SC2016
env WARDEN_RIG_BROKER="$broker" WARDEN_BUS_RUN="$WORK/$name/bus" \
WARDEN_RIG_BUS_LOG="$WORK/$name/broker.log" "$@" \
$SH -c '. "$1"; warden_rig_bus' sh "$LIB" > "$WORK/$name/out" 2>&1
echo $? > "$WORK/$name/rc"
}
rc_is() { [ "$(cat "$WORK/$1/rc")" = "$2" ]; }
out_has() { grep -qF -- "$2" "$WORK/$1/out"; }
# 1. No broker in the payload.
run_bus none "$WORK/no-such-broker"
check "no broker: status 0" rc_is none 0
check "no broker: nothing created" test ! -e "$WORK/none/bus"
check "no broker: silent" test ! -s "$WORK/none/out"
# 2. A broker that listens.
run_bus good "$STUB/good" WARDEN_RIG_BUS_USERS="flared ui modbus ../evil .hidden automation.main"
B="$WORK/good/bus"
check "broker up: status 0" rc_is good 0
check "broker up: reported" out_has good "init: rig bus up on $B/bus.sock"
check "broker up: socket listens" test -S "$B/sock/bus.sock"
check "broker up: bus.sock link resolves to the socket" test "$(readlink "$B/bus.sock")" = sock/bus.sock -a -S "$B/bus.sock"
creds_ok() {
local u f seen=""
for u in flared ui modbus automation.main; do
f="$B/cred/$u"
[ -f "$f" ] || { echo "missing $f"; return 1; }
[ "$(stat -c %a "$f")" = 400 ] || { echo "$f mode $(stat -c %a "$f")"; return 1; }
grep -qxE '[0-9a-f]{64}' "$f" || { echo "$f is not 64 hex digits"; return 1; }
case " $seen " in *" $(cat "$f") "*) echo "$f repeats a secret"; return 1 ;; esac
seen="$seen $(cat "$f")"
done
}
check "broker up: one 0400 64-hex credential per user, all different" creds_ok
check "broker up: a name with a slash gets no file" test ! -e "$B/evil"
check "broker up: a dot name gets no file" test ! -e "$B/cred/.hidden"
check "broker up: bad names reported" out_has good "bad user name '../evil' skipped"
check "broker up: cred/ is 0710" test "$(stat -c %a "$B/cred")" = 710
conf_ok() {
local c="$B/mosquitto.conf"
grep -qx "listener 0 $B/sock/bus.sock" "$c" &&
grep -qx 'allow_anonymous true' "$c" &&
grep -qx 'use_username_as_clientid true' "$c" &&
! grep -qE '^(password_file|acl_file|plugin)' "$c"
}
check "broker up: config has the listener and no password file, ACL or plugin" conf_ok
# 3. A broker that exits at once.
run_bus exits "$STUB/exits" WARDEN_RIG_BUS_WAIT_S=5
check "broker exits: status 1" rc_is exits 1
check "broker exits: reported with its log" out_has exits "init: rig bus FAILED: the broker exited"
check "broker exits: log tail shown" out_has exits "stub broker refused"
# 4. A broker that never listens.
start=$(date +%s)
run_bus mute "$STUB/mute" WARDEN_RIG_BUS_WAIT_S=1
took=$(( $(date +%s) - start ))
[ -f "$STUB/mute.pid" ] && kill "$(cat "$STUB/mute.pid")" 2>/dev/null
check "silent broker: status 1" rc_is mute 1
check "silent broker: reported" out_has mute "init: rig bus FAILED: no socket after 1 s"
check "silent broker: gave up within the bound (${took} s)" test "$took" -le 4
# 5. /sbin/init: the bus before the payload daemon loop.
order_ok() {
local bus loop
bus=$(grep -n '^warden_rig_bus' "$INIT" | head -1 | cut -d: -f1)
loop=$(grep -n '^for d in /usr/bin/warden-' "$INIT" | head -1 | cut -d: -f1)
[ -n "$bus" ] && [ -n "$loop" ] && [ "$bus" -lt "$loop" ]
}
check "init: the rig bus starts before the payload daemons" order_ok
# 6. The real broker, when one is given.
MQ="${WARDEN_TEST_MOSQUITTO_DIR:-}"
if [ -n "$MQ" ] && [ -x "$MQ/mosquitto" ] && [ -x "$MQ/mosquitto_sub" ] && [ -x "$MQ/mosquitto_pub" ]; then
run_bus real "$MQ/mosquitto" WARDEN_RIG_BUS_USERS="modbus ui"
R="$WORK/real/bus"
check "real broker: up" rc_is real 0
real_message() {
local got
"$MQ/mosquitto_sub" --unix "$R/bus.sock" -u ui -P "$(cat "$R/cred/ui")" \
-t 'modbus/state/test' -C 1 -W 10 > "$WORK/real/sub.out" 2>&1 &
local sub=$!
sleep 0.5
"$MQ/mosquitto_pub" --unix "$R/bus.sock" -u modbus -P "$(cat "$R/cred/modbus")" \
-t 'modbus/state/test' -m rig-bus-ok || return 1
wait "$sub"
got=$(cat "$WORK/real/sub.out")
[ "$got" = rig-bus-ok ] || { echo "subscriber got: $got"; return 1; }
}
check "real broker: ui receives what modbus publishes, both logged in by credential file" real_message
else
echo "[SKIP] real broker (set WARDEN_TEST_MOSQUITTO_DIR)"
fi
if [ "$FAIL" -eq 0 ]; then
echo "PASS: rig bus"
else
echo "FAIL: rig bus"
fi
exit "$FAIL"