Author SHA1 Message Date
NoahandClaude Opus 5 8066ef7d79 Stop the OTG state machines running blind in either role
Widens the #160 fix, which was scoped too narrowly to host role, and fixes
flare-edge#168.

Device role was deliberately left alone on the reasoning that the state
machine is only meaningless when the port is a host. That was wrong. On a
board with no extcon phandle the machine has no external role signal in
EITHER direction, and with bvalid pinned high by the VBUS bypass its charger
detection cannot be meaningful at all. In device role its verdict suspends
the phy underneath a bound gadget, so the USB recovery link dies.

Reproduced deterministically on warden-c8a3 with a host attached:

  device role, host attached   ccf=1  phy_sus=0x0    udc=configured  works
  -> host role                 ccf=1  phy_sus=0x0    udc=(none)
  -> back to gadget            ccf=0  phy_sus=0x1d1  udc=configured  SUSPENDED
  -> +15s / +30s               ccf=0  phy_sus=0x1d1  udc=configured  never recovers

and the 2207:0019 device disappears from the host for good.

Nothing notices, because every obvious signal lies: udc state reads
"configured", usb0 carrier reads 1, and DCTL RUN_STOP reads 1, so the gadget
really is asserting its pullup. It asserts into a phy whose analog front end
and 480M clock are off, so the SoC sees SE0 and the host sees no device at
all, not even an enumeration error.

It cannot self-heal either. bvalid is pinned high, so a host attaching later
produces no VBUS edge for the machine to trigger on: it ran once, powered the
phy off, and nothing re-evaluates. That is why a reboot appears to fix it and
why this read as a cable fault for weeks. It is not, and the cables were fine.

Gated on the extcon being self-allocated rather than on the board compatible,
so it describes the actual precondition: no external role signal exists.

Renamed from 29-usb2phy-no-otg-sm-in-host.patch, whose name now misstated the
scope.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVGTC78dgCGfRANNKjoPea
2026-09-23 10:41:29 -06:00
NoahandClaude Opus 5 f6add60a9c Keep the usb2 phy OTG state machines out of host role
Issue #160: the USB ethernet link on the OTG port stops passing traffic and
the host controller is later declared dead ("xHCI host not responding to stop
endpoint command", "HC died").

Root cause. The phy's OTG state machine and charger detection only make sense
while the port is a peripheral. This board has no ID source, so in host role
their inputs are meaningless: rockchip_usb2phy_otg_sm_work() reaches
OTG_STATE_B_IDLE with vbus asserted, schedules rockchip_chg_detect_work(), and
a DCP verdict calls rockchip_usb2phy_power_off(). That suspends the phy and
drops the 480M clkout the controller's core runs on, while a device is
enumerated and working.

The controller then stops clocking with its register block still readable, so
it reports itself healthy: USBCMD keeps RS set, USBSTS reads 0, PORTSC still
shows CCS and PED. Only MFINDEX gives it away by no longer advancing. Nothing
completes from that point, and the first transmit afterwards times out. The
recovery path cannot help either, because xhci_halt() needs the core clock to
update USBSTS.HCH, so it times out and xhci_reset() returns early without ever
writing USBCMD.HCRST.

dwc3 already calls phy_set_mode() on every role change, but this driver
implemented no .set_mode, so the phy never learned the role. Implement it,
track the role per port, and return early from both work functions in host
role. The check is inside the functions because both have direct callers
besides the scheduled work.

This keeps dr_mode "otg" and runtime role switching. Setting dr_mode to "host"
also avoids the fault, by skipping the same state machine, but compiles out
the gadget needed for flashing, recovery and pairing.

Measured on warden-c8a3, per arm, transmit packets and xhci interrupts
delivered after the stated idle:

  idle 0 s    +9, +10, +9 packets   +170, +176, +188 interrupts
  idle 120 s  +0, +0, +0 packets    +0, +0, +0 interrupts

Also drops 28-dwc3-otg-no-live-reinit.patch. It guarded dwc3_otg_host_init()
against re-entry on a live controller, but that path never executes here:
GSTS reads OTG IP clear and instrumentation logged no entries on either the
hardware irq or the guard site.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVGTC78dgCGfRANNKjoPea
2026-09-23 00:25:15 -06:00
NoahandClaude Opus 5 4a5481b6a5 Instrument the dwc3 OTG path to test the guard
The guard alone did not fix flare-edge#160: with dr_mode="otg" and the live
re-init skipped, the controller still died 14s after carrier-on under load
(carrier 54.1s, first Tx timeout 63.4s, HC died 68.4s).

That means either the OTG event path is not the trigger, or the harm comes from
somewhere else in it. dr_mode="host" removes far more than this one call: it
also skips dwc3_otgregs_init() and hands port-power control to xHCI instead of
the OTG block.

IRQ 47 is shared between dwc3-otg and xhci-hcd, so /proc/interrupts cannot say
whether OTG events fire at all. CONFIG_DYNAMIC_DEBUG is off, so dev_dbg is
compiled out and invisible.

So this build logs, ratelimited and always compiled:
  - every OTG hardware event with its OEVT value, role and restart flag
  - each time the guard skips a re-init on a live host

Run under load, that answers whether the OTG path is even active during the
failure window, instead of guessing a third patch.

Also worth measuring: load accelerates this dramatically -- 452-587s idle
versus 9-14s under line-rate traffic -- so idle soaks are not comparable to
load tests and earlier comparisons need re-reading with that in mind.

Refs flare-edge#160

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVGTC78dgCGfRANNKjoPea
2026-09-22 17:34:51 -06:00
NoahandClaude Opus 5 9ff60d60db Do not re-init OTG host registers on a live controller
dwc3_otg_host_init() carries the comment "should be called before Host
controller driver is started". It rewrites OCTL/OCFG, touches GUSB2PHYCFG and
re-asserts port power (OCTL.PrtPwrCtl).

dwc3_otg_irq() sets otg_restart_host ONLY when current_otg_role is already
DWC3_OTG_ROLE_HOST. The threaded handler then calls dwc3_otg_host_init()
unconditionally, so that path always ran against a live, running host -- the
exact condition the comment forbids. Re-asserting port power under a running
xHCI corrupts an in-flight command.

Observed on the RV1106 86-panel as the controller failing to answer a Stop
Endpoint command, after which xhci-hcd declares it dead and tears the bus down.
An r8152 USB NIC stalls a transfer and the controller died 8 times out of 8
between 452s and 587s. Compiling the OTG path out entirely (dr_mode=host)
survived 1810s with zero deaths, which localised it here.

On this board the event is always spurious: the dwc3 node deliberately carries
no extcon, because wiring it pins the role to peripheral on D1-modded panels,
and CONIDSTS reads 0 -- so the role cannot legitimately change while host is
current. Skip the re-init in that case and log it at debug level.

This does not remove the restart capability. dwc3_set_mode() still runs
immediately below, and if the role has genuinely changed dwc3_otg_update()
performs a proper dwc3_host_exit() before re-initialising. The only behaviour
removed is poking a running controller, which was never legitimate.

Refs flare-edge#160

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVGTC78dgCGfRANNKjoPea
2026-09-22 16:54:54 -06:00
BFE EngineeringandClaude Opus 5 e12bb2c554 Implement RV1106 USB2 PHY signal tuning
Cherry-picked onto the pinned SDK so the ONLY delta versus the pin is this
patch -- the xHCI controller-death investigation needs a controlled build.

The forward-port omitted the vendor driver's rv1106_usb2phy_tuning with the
note "the phy is functional without it". It is functional, but not reliable:
with these registers at reset defaults the HS link is electrically marginal
under sustained traffic, a bulk transfer eventually stalls, and the r8152
Tx-timeout recovery issues an xHCI Stop Endpoint command the controller never
answers. Measured carrier-on to death over four runs: 14s, 519s, 521s, 584s.

Refs flare-edge#160

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVGTC78dgCGfRANNKjoPea
2026-09-22 12:11:14 -06:00
NoahandClaude Sonnet 5 ac8658d1d5 qemu: add wait_stat verb for polling stat checks
assert_stat samples a stat field once. warden-ui's fps counter is a
rolling one-second window, so a single sample right after a page opens
can read 0 even though the UI is live (flare-edge #44). Add wait_stat
FIELD OP VALUE TIMEOUT, the same polling pattern as wait_hit/wait_json,
built on a shared eval_stat() that assert_stat now uses too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SMRwnkPp1upR6QFWCZouE7
2026-09-14 15:13:44 -06:00
Noah cdf491caa9 Retry pinned BusyBox fetches (#554) 2026-09-11 11:39:07 -06:00
Noah 2efe36f582 Wait for QEMU input releases (#546) 2026-09-11 11:15:02 -06:00
Noah 55660b7c01 Observe QEMU swipe presses (#546) 2026-09-11 10:31:09 -06:00
Noah 7e0089826b Harden QEMU input and credentials (#516 #519) 2026-09-11 09:49:09 -06:00
Noah bf39a74337 Create QEMU daemon runtime directory 2026-09-11 02:35:21 -06:00
NoahandClaude Fable 5.1 2b6e8a2098 qemu and build: review fixes across the rig driver, boot script, and fetch helpers
Bounded waits and validated arguments in run.sh and ui-drive.sh, a seeded
settings directory and root-only staged rootfs permissions with their own
tests, qmp.py and imgtools.py hardening, the fetch scripts checking what they
download, and ASCII typography throughout. Each fix carries its test under
qemu/tests or tests/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-09 19:17:54 -06:00
NoahandClaude Fable 5.1 bda6c6c633 qemu: two tests the review pass left on disk
tests/scripts/fullscreen-toggle-tracks-real-state.txt pins the debug
channel's `fullscreen toggle` deciding from the dashboard's real state
rather than a private flag that `home` and a real tap left stale (rig:
14 ok). tests/test-ui-drive-rs485.sh runs offline with a fake run.sh and
socat and proves ui-drive.sh --rs485-devices fails closed when the
socket, pty or control socket never appears, dumps mbsim.log on failure,
and kills a simulator that ignores SIGTERM. README lists both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-09 13:54:35 -06:00
NoahandClaude Fable 5.1 8805b6106c qemu: review pass over the rig driver and boot script
Four review passes with fixes between them (flare-edge's flow-framework
review, 2026-09-09). qmp.py: drive() split out of a 330-line dispatcher,
every verb guarded so a raising verb records a fatal row instead of ending
the run, the shot path sanitised, the rs485 and wait verbs judged through
shared helpers; imgtools.py: a bench subcommand for phash/structural
timings and a colour probe that samples instead of scanning the frame;
ui-drive.sh: the boot poll no longer walks every pixel per tick and the
simulator's control socket path is passed as one word. Offline tests: 8.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-09 13:52:47 -06:00
NoahandClaude Fable 5.1 48be35dc7b qemu: rs485 silence|restore takes a simulated device off the bus mid-run
ui-drive.sh --rs485-devices fixed the roster for the whole boot, so no
script could show the guest noticing a device go quiet. mbsim.py now
serves a control socket (flare-edge --control); ui-drive.sh opens it next
to the pty and hands its path to qmp.py drive (--rs485-control), whose new
verb `rs485 silence|restore ADDR` sends one command and judges the reply.
A run without a simulated bus records the step as fatal rather than a
silent pass. Rig: unit 5 silenced reads online=false after 65 s in the
status json, restored reads online=true after 64 s (flare-edge #184).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-09 06:55:07 -06:00
NoahandClaude Fable 5.1 cab6c25c7e qemu: wait_hit and wait_region poll a settling page
A hit or region check fired once, so a check one second after `nav`
raced a page still laying itself out (flare-edge #175). wait_hit X Y CLASS
TIMEOUT_S [box=...] [TEXT] and wait_region NAME TOLERANCE TIMEOUT_S run
the same judgement every 0.5 s until it holds or the deadline passes, and
report how long they waited. The assert_hit judgement moves into
judge_hit so both verbs mean the same thing by a match; a missing
reference or foreign tolerance stays fatal on the first answer, since
waiting cannot fix either. Offline tests cover both verbs and the timeout.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 17:33:19 -06:00
NoahandClaude Fable 5.1 40de48092f qemu: regression script for home leaving fullscreen
tests/scripts/home-leaves-fullscreen.txt taps the dashboard into
fullscreen, proves the rail gear's pixel now resolves to the 720x720 canvas,
sends `home` and proves the gear's own box is back; then the same from
another page. The `home` verb used to reach Dashboard/Dashboard without
leaving fullscreen, which is the state a hardware wake tap leaves behind
(flare-edge #176). Runs green on the rig: 20 ok, 0 failed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 17:18:43 -06:00
NoahandClaude Fable 5.1 e60d47846b qemu: simulated Modbus devices on the rig's RS485 bus
ui-drive.sh --rs485-devices ADDR:SLUG[,...] attaches the VM's RS485 UART
to a socket, turns it into a pty with socat and serves the named corpus
profiles on it with flare-edge tools/modbus-sim/mbsim.py, so warden-modbus
in the guest discovers and identifies devices as it would real controllers.
Verified: unit 5 as smartgen-hgm6100n4g appears in the status json as
SmartGen HGM6110N, online, 8 ms, and RS485/Devices lists it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 17:03:13 -06:00
NoahandClaude Fable 5.1 a512d56650 qemu: scroll and home verbs in the driver
scroll X Y DY and home are judged by the channel's own ack, like nav and
wake: the deterministic stand-in for a swipe that only brings a control
into view (flare-edge #174), and the fresh-boot state a live panel needs
before each flow (flare-edge #173).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 16:48:09 -06:00
NoahandClaude Fable 5.1 bd5c5af9db qemu: structural hash relative to the background, taps that land on the pixel
imgtools.structural() marked a cell occupied when its grey exceeded an
absolute 10/255, and the WardenOS page background is grey 16: every cell
of every region read occupied and no structural check could ever fail
(#19). Occupancy is now grey deviating from the crop's own median by more
than DEVIATION_THRESHOLD, or edge energy above EDGE_THRESHOLD. Measured on
real captures: a switch knob left/right differs in 240 of 256 cells (was
0), a dark card reads its icon and text and nothing else. Every committed
reference is recaptured with flare-edge tools/flow-run-all.sh --capture.

qmp.py to_axis() truncated the pixel-to-axis conversion and LVGL's
evdev calibration truncates on the way back, so many pixels landed one
short (130 -> 5924 -> 129) and a tap could miss the control hit had just
confirmed at that pixel (#21). It now rounds up to the smallest axis value
that truncates to the requested pixel; test_qmp_drive.py asserts the round
trip for every pixel at three panel sizes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 13:53:59 -06:00
NoahandClaude Fable 5.1 f809c6e5a3 qemu: stage the panel's root credential files
The busybox skeleton had no /etc/shadow, so Settings/Security's root
password check (grep root: /etc/shadow, mkpasswd -m md5 with the same
salt) rejected the documented default password every time; the flow that
covers it could only record the rejection. The guest now carries
/etc/passwd, /etc/shadow (the overlay's root line, md5-crypt of the
default password with salt wardenrs, verified equal to openssl passwd -1)
and /etc/group.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 11:54:49 -06:00
NoahandClaude Fable 5.1 e6f936ee8d qemu: one image and initramfs per ui-drive run
mkinitramfs.sh, mkimage.sh and run.sh all read $OUT, and ui-drive.sh left
it at the shared default, so two concurrent runs built and booted the same
qemu/out/disk.img and wrote their userdata into it: a VM seeded with
role=1 read back client because the unseeded run next to it had rebuilt
the image (#20). Each run now builds and boots its own image under
$TMPDIR (real disk; /tmp is a tmpfs on the dev box and the run refuses to
start with under 1 GB free there), keeps the pinned busybox shared and
read-only through BUSYBOX, and removes the image on exit.

Verified: a seeded router-mode flow, the unseeded client-mode flow and a
third flow booted at once, 25/25, 25/25 and 10/10, each reading its own
role.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3G6m9Aw5RyVY4ZowtKzEj
2026-09-08 07:01:20 -06:00
NoahandClaude Fable 5.1 7727fbc888 qemu: the bridge waits for a fresh reply instead of sleeping
The control bridge sent a command, slept 0.3 s and read /tmp/warden-ui.dbg,
so a page that took longer to build handed the host the previous command's
reply as if it were this one (flare-edge #152). It now removes the old
reply before sending and waits, up to 5 s, for warden-ui to rename the new
one into place, the same recipe tools/warden-ctl and flow-run-hw.sh use.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-08 01:17:41 -06:00
NoahandClaude Fable 5.1 9b6428db47 qemu: start every panel daemon, and keep Modbus off the control channel
Stage-2 init started only warden-flared and warden-modbus from the
payload, so every daemon-backed screen (Automation, MikroTik, Mining,
RS485, Starlink, Services) sat in its "not running" state on the rig
while the bench panel showed the daemons up with nothing attached. It now
starts every warden-* daemon it finds in the panel's own S-number order
(ai, automation, modbus, mikrotik, asic, starlink, stratum, flared), each
with no arguments as start-stop-daemon runs them on the panel. flare-edge
tools/rig-pull-daemons.sh copies a panel's static binaries in.

Doing that exposed a latent alias: init pointed warden-modbus's /dev/ttyS4
at /dev/ttyS0 whenever ttyS0 existed, and with --ctl that port IS the
control channel, so Modbus polls arrived inside the wake reply and the
bridge hung. run.sh now always attaches an RS485 pci-serial (a null
chardev without --rs485), and init aliases ttyS4 to ttyS1 when warden.ctl
is on the cmdline, ttyS0 otherwise.

Verified on a boot with all eight daemons: status json reports
automation.engine, mikrotik.backend, mining.available and rs485.available
true with the Modbus scan sweeping the empty UART; six daemon-backed pages
matched screenshots of the bench panel structurally (0/256 cells differ).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-08 00:47:08 -06:00
NoahandClaude Fable 5.1 eb16a301c0 qemu: wake verb, idle stat, and no fbcon cursor in the framebuffer
- tests/qmp.py: `wake` is a judged verb (the reply must be the channel's
  own ack) and `assert_stat` knows `idle` (0 awake, 1 dimmed, 2 asleep).
  warden-ui's sleep.c swallows the touch that wakes a dimmed or blanked
  panel, so every compiled flow script now opens with wake (flare-edge
  #148); tests cover both.
- run.sh: vt.global_cursor_default=0. fbcon shares the virtio-gpu
  framebuffer with warden-ui and its cursor blinked an 8x2 block onto the
  top-left corner in some boots and not others, which made a structural
  pixel reference fail by one cell (#18). The panel has no fbcon on its
  display, so this only makes the rig match it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-07 23:39:07 -06:00
NoahandClaude Fable 5.1 86a9544dcc qemu: json/stat/region channels, seeding, driver tests
- rootfs/sbin/init: the control bridge answers `@cat PATH` locally so the
  driver can read /tmp/warden-web-status.json out of the guest. That file
  has no trailing newline; the bridge adds one so the sentinel stays on
  its own line and the line-based reader never blocks.
- tests/qmp.py: wait_json/assert_json (dotted paths, eq/ne/contains/
  len_eq/len_ge/gt/lt), assert_stat off the FIFO's stats reply,
  capture_region, assert_region NAME [TOLERANCE] and assert_ocr. A
  tolerance other than the captured one, a reference box that does not
  fit the screendump, a missing reference or a missing tesseract is FATAL
  for that step and the run continues (flare-edge #147).
- tests/imgtools.py: P6 reader, crop, perceptual and structural hashes,
  compare, with a self-test.
- tests/test_qmp_drive.py: drive() with QMP and the control channel
  faked, pinning the per-step ok/fail/fatal contract.
- mkimage.sh SEED_DIR and ui-drive.sh --seed/--refs: settings fixtures
  staged into userdata before warden-ui starts, and a reference store
  handed to the driver.
- ci: the driver tests and the imgtools self-test run in qemu-tools.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-07 23:26:01 -06:00
NoahandClaude Fable 5.1 8a57057053 Bridge warden-ui's debug channel out of the qemu VM
The rig could drive the UI and detect one outcome: the process died. Nothing
could ask the UI what page it was on or what a tap would land on, because
that channel is a FIFO inside the guest and the initramfs is busybox-only
with no sshd. Scenarios therefore asserted nothing and screenshots went
unread.

run.sh --ctl exposes a second pci-serial port as a unix socket, the same
device the RS485 bridge already rides, listed first so it is always ttyS0.
It also puts warden.ctl on the kernel command line, and init bridges only
when that marker is present: a VM launched with --rs485 alone has a ttyS0
too, and that one is the Modbus wire. The bridge relays one command line in
and the FIFO's reply out, then a sentinel so the reader needs no timeout.

qmp.py gains the channel verbs (nav, page, stats, hit, assert_page,
assert_hit), records every step to results.jsonl as ok/fail/fatal, continues
past an assertion mismatch so one run reports every broken expectation, and
checks the console after EVERY step for the stage-2 init's EXITED line so a
crash is pinned to the step that caused it. assert_hit matches the widget's
bounding box: an icon has no usable caption and two list rows share a class,
but the geometry the UI itself resolved is exact.

The vocabulary is what tools/warden-ctl already speaks over SSH to a real
panel, so a script that runs here runs there. Verified end to end on the rig
(11/11 verbs round-tripped) and against the bench panel, where the same
commands returned byte-identical results.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-07 21:54:52 -06:00
NoahandClaude Opus 5 3ea6c837e3 Pack the VERBOSE splash variant into resource.img
U-Boot draws the boot splash from resource.img by name, and now draws a second
image by name when the splash is tapped: the same logo with its bottom-right
mark reading VERBOSE instead of GRAPHIC. Without --logo-verbose that image is
absent from the FIT and a tap changes nothing on screen, even though the boot
itself would still go verbose.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013aHKWzT5EF86RFKRMtAv9n
2026-09-07 14:48:43 -06:00
NoahandClaude Opus 5 f0204f2b53 Fix mkimage -B probe defeated by pipefail
The probe piped the candidate binary into grep to look for -B support.
mkimage with no arguments prints usage and exits non-zero, and this script
runs under `set -euo pipefail`, so the pipeline reported failure even when
grep matched. Every candidate was rejected and the script failed closed with
"no mkimage on PATH supports -B" while a capable mkimage 2025.01 sat first
on PATH, making it impossible to build a boot.img at all.

Neutralise the probed command's exit status before the pipe. Sentinel
comments now bracket the probe so the new regression test extracts and
exercises the shipping code rather than a copy of it.

Closes #17

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-04 17:06:54 -06:00
NoahandClaude Opus 5 caa12d17ff Correct the FIT alignment to 512, and stop gating on 2048
054790d claimed the vendor aligns FIT sub-images to 0x800 and made that a build
gate. Both were wrong, and the reasoning was thin: two payloads from one build,
generalised into an invariant.

What U-Boot actually requires. IMAGE_ALIGN_SIZE is 512 (include/image.h:955-958)
and the read is blk_off = (FIT_ALIGN(fdt_totalsize) + offset) / blksz
(arch/arm/mach-rockchip/fit.c:331), a truncating divide by the 512-byte eMMC
block. So the metadata size must be a multiple of 512 -- otherwise FIT_ALIGN
rounds it up and EVERY payload is read late, including ones whose own position
is perfectly aligned -- and each data-position must be a multiple of 512.
Nothing in the FIT or RESC path references 2048.

Why the 2048 gate was actively harmful: of the eight boot.img files on this
machine, four are 512-aligned but sit at data-position % 2048 = 1536, including
vendor RELEASE_TEST builds that boot. The gate would have rejected images the
vendor shipped. 0x800 is the vendor's -p value -- the file position of the FIRST
payload -- not an alignment; 054790d moved it into the -B slot.

And it broke the build inside the SDK. mk-bootimg.sh resolves mkimage with a
bare `command -v`, project/build.sh:64 prepends the SDK tool directory to PATH,
and the SDK vendors mkimage 2017.09, which has no -B and exits 255 with
"invalid option -- 'B'" under set -euo pipefail. The 6.18 image built only
because it was made from a normal shell that found host mkimage 2025.01. The
flag is now feature-detected; the vendor packer 512-aligns natively, so omitting
it there is correct rather than a fallback.

9387cff's -B 0x200 was right, and better derived than 054790d credited: the
proven image's resource sits at 8748544, a multiple of 512 but not of 1024 or
2048, which pins the alignment at exactly 512 rather than bounding it.

The gate now asserts what U-Boot enforces -- metadata % 512 and every
data-position % 512 -- and the embedded-data guard drops from 65536 to >= 4096
to match FIT_FDT_MAX_SIZE (SZ_4K, fit.c:24). At 65536 a 5000-byte header passed
the build and returned "No fit blob" on a panel.

The already-flashed 6.18 boot.img is unaffected: 2048 is a multiple of 512, so
it satisfies the real requirement, which is why it booted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-04 16:39:44 -06:00
NoahandClaude Opus 5 054790dde6 Align every FIT sub-image to 0x800, and check it
mkimage -p only places the FIRST payload. The rest are packed contiguously, and
U-Boot reads each by dividing its data-position by the 512-byte block size with a
truncating divide -- so a payload that is not block-aligned is read from the
wrong offset. Measured: without -B this script emitted the kernel at 0x9A94,
data-position % 512 = 148, which is exactly the failing shape.

That failure is on record. flare-edge 9387cff (2026-09-01) hit it on a boot_b
FIT: "unaligned sub-images fail the RESC loader's sha256 and the logo silently
falls back to 'No resource file'". It landed on -B 0x200, inferred from noticing
the proven image was "512-aligned" without checking for a larger factor.

The proven images are aligned to 0x800, not merely to 512. Measured on the pair
that boots this board today: boot.img fdt/kernel/resource at 0x800 / 0x12800 /
0x3A7800, recovery.img at 0x800 / 0x12800 / 0x3F8800 -- every one a multiple of
2048. So 0x800 satisfies everything 0x200 does, since every 0x800 multiple is a
0x200 multiple, and additionally reproduces the vendor layout exactly. The cost
is at most 2 KiB of padding. Where a remembered rule and a booting image
disagree, match the image.

The alignment is now also asserted at build time rather than trusted. It runs on
every build because the failure it catches is silent -- a wrong offset does not
fail the build, it fails on a panel, and only sometimes visibly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-04 16:16:33 -06:00
NoahandClaude Opus 5 3d8a683d68 Pin where buildroot comes from
The firmware builds against sysdrv/source/buildroot/buildroot-2025.02.8 in the
vendor SDK. That tree is not in the vendor checkout -- the SDK ships 2023.02.6 --
it was not in this manifest, and nothing anywhere recorded its origin. A clean
rebuild on another machine silently fell back to the vendor's older buildroot
and produced a different userspace, which is flare-edge#135.

fetch-buildroot-tarball.sh follows fetch-kernel-tarball.sh exactly: pinned URL,
pinned sha256, fails closed on a missing pin. Buildroot signs releases with GPG
rather than publishing a .sha256, so the pin was computed from the tarball and
is what the script verifies against.

The manifest now says out loud that two of the inputs are tarballs rather than
git trees, so "which buildroot" has an answer in the same place as "which LVGL".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 21:24:10 -06:00
NoahandClaude Opus 5 b6aee456e8 Note where the VM's kernel is ahead of the panels
The VM builds warden_defconfig plus virt.fragment, so it has CONFIG_BRIDGE; the
vendor 5.10 kernel most panels still run does not. A Client-mode access point is
bridged to the wired uplink and needs that, so the Wi-Fi page offers it here and
declines on a 5.10 panel.

Both answers are correct for the kernel underneath, which is exactly why it is
worth writing down: a screenshot from the VM is not evidence about that page on
a 5.10 panel until the fleet has moved to the 6.18 build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 19:18:08 -06:00
NoahandClaude Opus 5 120016f54a Enable bridging in the kernel config
The vendor 5.10 kernel is built without CONFIG_BRIDGE, and that absence has
shaped the firmware: the panel's access point has to be its own subnet with its
own DHCP pool instead of joining the LAN, and warden_router_set_zone() treats a
second LAN port as impossible. Both carry a code comment saying "this kernel has
no bridge".

Turning it on is what lets a Client-mode panel run an access point the way an
ordinary ceiling AP does -- wlan0 and the wired uplink in one broadcast domain,
addresses handed out by whatever router is already upstream.

Verified on the bench panel: before, `ip link add type bridge` returned
"Operation not supported"; on this kernel the bridge is created and eth0
enslaves into it (/sys/class/net/br0/brif/eth0). WireGuard, iptables and VLAN
still work.

Also fixes a warden-modbus path in architecture.md left stale by the apps/ move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 18:16:37 -06:00
NoahandClaude Opus 5 ba2bba082b Record the 6.18 hardware verification
The series was built, packaged with mk-bootimg.sh, and booted on the bench
panel: display, backlight, Goodix touch, RGA, eth0 and the usb0 gadget all
work, with no kernel faults. M4/M5/M6 in this file are stale.

The same test is what proved the three missing subsystems were real rather than
a reading of the config: wireguard, iptables and VLAN each failed on the panel
before the defconfig fix and each works after it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 17:38:12 -06:00
NoahandClaude Opus 5 a367991998 Own the vendored trees by pinning them
LVGL and the vendor RV1106 SDK are what this platform is built against, and
nothing in this repo said WHICH ones. The two live outside every worktree as
shared checkouts, so "the version we build against" was whatever happened to be
on the machine.

They are pinned here now, by exact commit, with one script that obtains and
verifies them. Not copied: between them they are ~21 GB, the vendor SDK bakes
absolute paths into its host tools so it has to sit at one stable path anyway,
and a 21 GB git repo would be unusable while still not making anything
reproducible. What makes a build reproducible is knowing exactly which tree was
used, which is a commit id -- the same reasoning build/fetch-kernel-tarball.sh
already applies to the kernel, where a sha256 pin stands in for vendoring the
tarball.

Two behaviours worth stating, because both were wrong in the first draft:

- A checkout is detected by `.git` EXISTING, not by it being a directory. LVGL
  is checked out as a worktree here, where `.git` is a file.
- Local modifications are reported and are NOT a failure. The vendor SDK is
  supposed to carry them -- tools/build-firmware.sh applies our sdk-patches
  series into that tree on every build -- so a pristine checkout is the
  unusual state. Only a MISSING or DRIFTED tree fails.

Nothing is ever reset automatically: a tree off its pin is reported, because a
local change to a vendor tree is usually someone mid-debug, not something to
throw away on their behalf.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 16:07:05 -06:00
NoahandClaude Opus 5 13280d07cf Build a bootable image, and boot with a network
Two gaps between "this SDK builds a kernel" and "this SDK can replace the
vendor SDK for the device".

BOOT IMAGE. build-kernel.sh emitted a zImage and a dtb and stopped, so
producing something the board's U-Boot would actually boot still meant going
through the vendor tree. mk-bootimg.sh packages the pair the way M2 bring-up
established (kernel/docs/m2-boot-on-c8a3.md): an EXTERNAL-DATA FIT
(mkimage -E -p 0x800), a mandatory `resource` multi sub-image carrying
rk-kernel.dtb plus any logos, and the sysmem sentinel load addresses. Each of
those was learned from a specific failure -- an embedded-data FIT is "No fit
blob", a missing resource image is "Failed to load DTB, ret=-19", real load
addresses collide -- so the script also ASSERTS the metadata stayed small,
because an embedded-data FIT looks perfectly fine until a panel will not come
back.

NETWORKING. Diffing this defconfig's expansion against the kernel actually
shipping on the panel found three whole subsystems missing, none of which fail
at build time and none of which are visible until the unit is in the field:

  - WIREGUARD + NET_UDP_TUNNEL: flared's mesh to FLARE. Without it wg0 never
    comes up.
  - VLAN_8021Q: the MikroTik app configures tagged ports the panel terminates.
  - NETFILTER and legacy iptables: every rule in S35iptables, and NAT for
    router mode.

The netfilter half carried a trap worth naming. 6.18 split the legacy tables
out behind NETFILTER_XTABLES_LEGACY and IP_NF_IPTABLES_LEGACY, symbols that do
not exist in 5.10 -- so copying the vendor kernel's symbol list verbatim gives
a kernel where IP_NF_FILTER and IP_NF_NAT silently stay off and `iptables` has
no filter or nat table at all. Our userspace drives legacy iptables, not nft.

The result now shows ZERO regressions against the shipping 5.10 kernel across
mesh, gadget, firewall, VLAN, storage, net core, display/input, RGA/NPU,
wifi/BT and watchdog.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 15:52:42 -06:00
NoahandClaude Opus 5 7b395da7de qemu: tee the UI log to the console
A scenario driving the VM from outside can only see the console, and the UI's
own log was going to a file inside the guest -- reachable only after the
process exited. That made a whole class of question unanswerable while the UI
was still running: what the UI thought it was doing at the moment a screenshot
looked wrong.

It paid for itself immediately: a dashboard fullscreen that kept reverting was
a full UI rebuild firing 553 ms after the tap, which the console showed
outright (flare-edge: the applier's first-tick rebuild).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 14:25:24 -06:00
NoahandClaude Opus 5 f245917570 qemu: drive scripted UI scenarios
ui-shot.sh proves touch reaches the UI in one tap. Verifying a UI change needs
a SEQUENCE -- swipe through the app rows, open a submenu, tap a tab, bring up
the keyboard -- and booting per step costs about a minute under TCG, so:

- qmp.py gains a `drive` mode: one connection, one boot, a script of
  tap/swipe/fling/shot/sleep steps in PANEL PIXELS rather than the raw
  0..32767 tablet axis. Swipes interpolate their motion, because LVGL decides
  a gesture from the movement between indev polls and a press-then-release
  with nothing in between is a click, not a scroll.
- ui-drive.sh runs such a script against a booted VM and collects the
  screenshots.

It also FAILS on a UI that died mid-script. warden-ui crashing leaves its last
frame in the framebuffer, so screendumps keep returning a plausible picture of
a program that no longer exists; stage-2 init now announces the exit and its
status on the console, and ui-drive.sh greps for that after the run. This is
what caught the SIGSEGV behind flare-edge#125.

Stage-2 init also mounts devpts. The UI's Terminal page opens a PTY, so
without it that page could only ever report "no PTY available" -- it rendered,
which made a screenshot scenario look fine while the one thing the page does
was untestable.

tests/scripts/nav-stress.txt is the first committed drive script: the
navigation sequence that reproduces flare-edge#125.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T2D2KtdgwbhbF6Mo64eUrn
2026-09-03 11:50:20 -06:00
42 changed files with 5964 additions and 49 deletions
+71 -1
View File
@@ -110,7 +110,77 @@ jobs:
sudo apt-get update -qq && sudo apt-get install -y -qq shellcheck
shellcheck -x qemu/*.sh qemu/tests/*.sh build/*.sh \
qemu/rootfs/etc/warden-lib.sh qemu/rootfs/etc/rc \
qemu/rootfs/sbin/init qemu/rootfs/init
qemu/rootfs/sbin/init qemu/rootfs/init tests/mk-bootimg/*.sh \
tests/fetch-vendor/*.sh tests/fetch-buildroot-tarball/*.sh
- name: cache apt archives (python3-pil)
# Same cost class as the busybox binary cached below: a system package
# plus its libjpeg/libpng transitive deps, downloaded fresh on every
# push otherwise.
uses: actions/cache@v4
with:
path: /var/cache/apt/archives
key: apt-archives-python3-pil-${{ runner.os }}
- name: ui-drive driver and image tools (offline)
# qmp.py's drive() with QMP and the control channel faked, plus
# imgtools' self-test: the per-step ok/fail/fatal contract and the
# region reference math, no VM needed.
run: |
sudo apt-get install -y -qq python3-pil
python3 qemu/tests/imgtools.py selftest
python3 qemu/tests/test_qmp_drive.py
- name: imgtools bench (smoke, printed for trend-watching)
# Same pattern as the sim/rs485-bridge bench job above: no stored
# baseline yet, just a number in the log so a phash/structural
# regression (DCT size, downscale filter, occupancy thresholds)
# is visible instead of only showing up as an unexplained slower
# flow run later.
run: python3 qemu/tests/imgtools.py bench
- name: mk-bootimg probe regression tests
# Guards issue #17 (mkimage's non-zero exit sinking the probe's grep
# pipeline under set -o pipefail) on every push/PR, not only on the
# next workflow_dispatch that happens to exercise mk-bootimg.sh for
# real via kernel-build.
run: bash tests/mk-bootimg/run-probe-tests.sh
- name: run.sh argv ordering regression test
# Pins the CTL-before-RS485 pci-serial argv order that
# rootfs/sbin/init's ttyS0-vs-ttyS1 alias depends on: a swap here
# reproduces run.sh:131-133's own incident, Modbus frames landing
# on the debug channel. All offline (a stub qemu-system-arm on
# PATH), so it runs on every push/PR, not only a real boot.
run: bash qemu/tests/run-sh-args-test.sh
- name: mkimage.sh SEED_DIR regression test
# Only ui-drive.sh --seed (a real VM boot) exercises this hook
# otherwise; this builds the same unprivileged mkfs.ext4 image and
# reads it back with debugfs, no VM needed.
run: bash qemu/tests/seed-dir.sh
- name: qemu_stage_rootfs permission regression test
# Git tracks only the executable bit, so a fresh checkout can land
# the source etc/shadow world-readable under a permissive umask;
# this pins the staged copy at 0600 regardless of the source mode.
run: bash qemu/tests/stage-rootfs-perms.sh
- name: fetch-vendor regression tests
# --check state machine (MISSING/OK/DRIFTED), --help, and the
# clone stall guard, against local throwaway repos: no network.
run: bash tests/fetch-vendor/run-fetch-vendor-tests.sh
- name: fetch-buildroot-tarball regression tests
# Retry-on-mismatch, cleanup, and the already-verified
# short-circuit, against a fake curl on PATH: no network.
run: bash tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh
- name: mk-bootimg boot.img validation regression tests
# Guards issue #22 (a missing/erroring fdtget silently skipping the
# data-position check) plus the FIT metadata and per-image
# data-position %512 checks and the embedded-data-FIT check.
run: bash tests/mk-bootimg/run-boot-img-validate-tests.sh
- name: mk-bootimg --help regression test
# Pins --help against its own header comment so a hardcoded line
# range can't silently start printing code again the next time the
# header grows or shrinks (the bug fetch-vendor.sh's --help had).
run: bash tests/mk-bootimg/run-help-tests.sh
- name: qemu-tools CI wiring regression test
# Catches a regression test shipping in this job without this job
# ever calling it -- the exact gap run-probe-tests.sh sat in before
# the step above wired it in.
run: bash tests/mk-bootimg/run-ci-wiring-tests.sh
- name: cache pinned busybox
uses: actions/cache@v4
with:
+1 -1
View File
@@ -52,7 +52,7 @@ scenario tests (portal, OTA apply, display + touch, watchdog).
| Directory | Contents |
|---|---|
| `patches/` | the RV1106 forward-port onto pristine linux-6.18.46, subsystem-split |
| `build/` | hermetic kernel build: pinned fetch -> apply patches -> `zImage` + dtb |
| `build/` | hermetic kernel build: pinned fetch -> apply patches -> `zImage` + dtb; `vendor.manifest` pins the third-party trees this platform builds against (LVGL, the vendor RV1106 SDK) to exact commits, and `fetch-vendor.sh` obtains and verifies them |
| `qemu/` | device simulator: QEMU `-M virt` boots the real kernel and real userspace |
| `sim/` | register-level hardware models (Rust): membus, HPMCU, CRU, Modbus, RGA, NPU |
| `drivers/` | hardened hardware-facing drivers: HAL seams, test harnesses |
+1
View File
@@ -0,0 +1 @@
695360d10d038122f11c76c35f3bd10561d949af2bc90468659f4ec5ee6687cc buildroot-2025.02.8.tar.xz
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Fetch (with retries) and sha256-verify the pristine Buildroot tarball into $1.
#
# WHY THIS EXISTS. The firmware builds against
# sysdrv/source/buildroot/buildroot-2025.02.8 inside the vendor SDK. That tree
# is NOT in the vendor checkout -- the SDK ships 2023.02.6 -- and nothing
# recorded where it came from, so the build was reproducible only on the one
# machine that happened to have the directory (flare-edge#135). Buildroot signs
# its releases with GPG rather than publishing a .sha256, so the pin here was
# computed from the downloaded tarball and is what this script verifies against.
#
# Same shape as fetch-kernel-tarball.sh, deliberately: a version bump edits this
# file and the pin beside it, nothing else. FAILS CLOSED on a missing pin.
#
# Not wired into build-firmware.sh or CI yet -- tools/build-firmware.sh stages
# sdk-patches/buildroot/ onto whatever buildroot tree the vendor SDK already
# has, it does not yet extract this pinned tarball over it. Exercised today by
# tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh and by
# running the script by hand; wiring it into the real build path is a separate
# change (it has to reconcile the pristine tree with the existing SDK buildroot
# checkout first).
#
# Usage: fetch-buildroot-tarball.sh <destination-path>
set -euo pipefail
BRVER=2025.02.8
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # build/
SHA_FILE="$HERE/buildroot-$BRVER.tar.xz.sha256"
URL="https://buildroot.org/downloads/buildroot-$BRVER.tar.xz"
TB="${1:?usage: fetch-buildroot-tarball.sh <destination-path>}"
[ -f "$SHA_FILE" ] || {
echo "FATAL: no pinned sha256 for buildroot-$BRVER (expected $SHA_FILE):" >&2
echo " refusing an unverified tarball" >&2
exit 1
}
WANT="$(awk '{print $1; exit}' "$SHA_FILE")"
verify() {
[ -f "$TB" ] || return 1
local got
got="$(sha256sum "$TB" | awk '{print $1}')"
[ "$got" = "$WANT" ]
}
if verify; then
echo "buildroot-$BRVER: already present and verified"
exit 0
fi
for attempt in 1 2 3; do
echo "== fetching buildroot-$BRVER (attempt $attempt)"
# --retry only re-fires once curl decides a transfer has failed; a connection
# that opens and then stalls (blackholed route, hung proxy) never reaches
# that decision and would otherwise block forever. --connect-timeout bounds
# the handshake, --max-time bounds the whole request.
if curl -fsSL --retry 2 --connect-timeout 20 --max-time 120 -o "$TB" "$URL" && verify; then
echo "buildroot-$BRVER: sha256 verified"
exit 0
fi
rm -f "$TB"
done
echo "FATAL: could not fetch a buildroot-$BRVER tarball matching $WANT" >&2
exit 1
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# Obtain (or verify) the third-party trees pinned in build/vendor.manifest.
#
# These trees are shared between worktrees and are far too large to live in git
# (the vendor SDK alone is 21 GB, and its host tools bake in absolute paths, so
# it must sit at ONE stable path anyway). What belongs in git is the answer to
# "which commit" -- that is the manifest, and this is the only thing that reads
# it.
#
# fetch-vendor.sh --check [DIR] report each tree's state; non-zero if any
# is missing or off its pin (default: check)
# fetch-vendor.sh --fetch [DIR] clone what is missing, then check
#
# DIR defaults to $WARDEN_VENDOR_DIR, else the conventional shared location
# alongside a flare-edge checkout. Nothing is ever moved or reset: a tree that
# has drifted is REPORTED, because a local change to a vendor tree is usually
# someone's debugging session, not something to silently throw away.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MANIFEST="$HERE/vendor.manifest"
MODE="check"
DIR=""
while [ $# -gt 0 ]; do
case "$1" in
--check) MODE="check"; shift ;;
--fetch) MODE="fetch"; shift ;;
# Print the header comment (line 1 is the shebang, so start at 2) and
# stop at the first line of code rather than a hardcoded line count --
# a fixed range silently starts printing code again the next time the
# header comment grows or shrinks.
-h|--help) awk '/^set /{exit} NR>1{print}' "$0"; exit 0 ;;
*) DIR="$1"; shift ;;
esac
done
[ -r "$MANIFEST" ] || { echo "FATAL: no manifest at $MANIFEST" >&2; exit 1; }
command -v git >/dev/null || { echo "FATAL: git not on PATH" >&2; exit 1; }
# A stalled clone (dead peer, wedged proxy) must not hang this script forever
# with no way for a caller to tell "still working" from "wedged" -- the
# luckfox-pico tree alone is ~21 GB, so a plain wall-clock timeout would also
# abort a clone that is merely slow. Abort only on a stall instead: git's http
# transport already aborts a transfer whose average speed drops below
# LOW_SPEED_LIMIT bytes/sec for LOW_SPEED_TIME seconds straight, so a slow but
# progressing clone is never penalized. Both are overridable for a link that
# is legitimately slow.
: "${WARDEN_VENDOR_LOW_SPEED_LIMIT:=1000}"
: "${WARDEN_VENDOR_LOW_SPEED_TIME:=60}"
export GIT_HTTP_LOW_SPEED_LIMIT="$WARDEN_VENDOR_LOW_SPEED_LIMIT"
export GIT_HTTP_LOW_SPEED_TIME="$WARDEN_VENDOR_LOW_SPEED_TIME"
if [ -z "$DIR" ]; then
DIR="${WARDEN_VENDOR_DIR:-$HOME/projects/scada/flare-edge}"
fi
# Where each tree conventionally lives. lvgl sits under ui/ because that is the
# shared tree flare-edge worktrees symlink into (ui-src/lvgl).
tree_path() {
case "$1" in
lvgl) echo "$DIR/ui/lvgl" ;;
luckfox-pico) echo "$DIR/sdk" ;;
*) echo "$DIR/$1" ;;
esac
}
rc=0
while IFS=$'\t' read -r name url commit desc; do
case "$name" in ''|\#*) continue ;; esac
path="$(tree_path "$name")"
# -e, not -d: a git worktree or submodule has a .git FILE, and lvgl is
# checked out as one here.
if [ ! -e "$path/.git" ]; then
if [ "$MODE" = "fetch" ]; then
echo "== cloning $name -> $path"
mkdir -p "$(dirname "$path")"
git clone --quiet "$url" "$path" || { echo " FAILED to clone $name" >&2; rc=1; continue; }
git -C "$path" checkout --quiet "$commit" || { echo " FAILED to check out $commit" >&2; rc=1; continue; }
else
echo "MISSING $name ($path)"
rc=1
continue
fi
fi
have="$(git -C "$path" rev-parse HEAD 2>/dev/null)"
if [ "$have" = "$commit" ]; then
# Local changes are reported but are NOT a failure. The vendor SDK is
# expected to carry them: tools/build-firmware.sh applies our sdk-patches
# series into that tree as part of every build, so a clean luckfox-pico
# checkout is the unusual state, not the healthy one.
dirty=""
[ -n "$(git -C "$path" status --porcelain 2>/dev/null | head -1)" ] && dirty=" (locally modified)"
echo "OK $name ${commit:0:12}$dirty"
else
echo "DRIFTED $name want ${commit:0:12} have ${have:0:12} ($path)"
echo " not reset automatically: check whether that is deliberate, then"
echo " either 'git -C $path checkout $commit' or update the manifest."
rc=1
fi
done < "$MANIFEST"
exit "$rc"
+242
View File
@@ -0,0 +1,242 @@
#!/usr/bin/env bash
# Package a bootable boot.img from a kernel this SDK built.
#
# build-kernel.sh produces a zImage and a dtb; this turns them into the image
# the board's U-Boot will actually boot, which was the last thing keeping the
# device build tied to the vendor SDK. The format is not negotiable and was
# established the hard way during M2 bring-up (kernel/docs/m2-boot-on-c8a3.md):
#
# - EXTERNAL-DATA FIT (`mkimage -E -p 0x800`). The FDT metadata stays tiny and
# the kernel/fdt/resource payloads are appended after it. An ordinary
# embedded-data FIT is rejected outright: "FIT: No fit blob".
# - A `resource` MULTI SUB-IMAGE IS MANDATORY. Rockchip's U-Boot reads the DTB
# over its RESC: path from a resource image holding `rk-kernel.dtb` (plus
# any boot logos). Without one: "Failed to load DTB, ret=-19".
# - SYSMEM SENTINEL LOAD ADDRESSES: fdt load 0xffffff00, kernel load/entry
# 0xffffff01. These tell U-Boot's sysmem to place the images itself. Real
# low addresses collide and come back as "No fit blob".
#
# The board's U-Boot has CONFIG_FIT_SIGNATURE off, so the image is not signed.
#
# Usage:
# mk-bootimg.sh --kernel zImage --dtb rv1106-warden.dtb --out boot.img
# [--logo FILE] [--logo-kernel FILE] [--logo-recovery FILE]
# [--logo-verbose FILE]
# [--resource-tool PATH]
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
KERNEL=""; DTB=""; OUT=""; RTOOL=""
LOGO=""; LOGO_KERNEL=""; LOGO_RECOVERY=""; LOGO_VERBOSE=""
while [ $# -gt 0 ]; do
case "$1" in
--kernel) KERNEL="${2:?}"; shift 2 ;;
--dtb) DTB="${2:?}"; shift 2 ;;
--out) OUT="${2:?}"; shift 2 ;;
--logo) LOGO="${2:?}"; shift 2 ;;
--logo-kernel) LOGO_KERNEL="${2:?}"; shift 2 ;;
--logo-recovery) LOGO_RECOVERY="${2:?}"; shift 2 ;;
# The same image as --logo but with a VERBOSE mark. U-Boot draws it by
# name when the splash is tapped, so it must be packed even though nothing
# displays it on an ordinary boot.
--logo-verbose) LOGO_VERBOSE="${2:?}"; shift 2 ;;
--resource-tool) RTOOL="${2:?}"; shift 2 ;;
# Print the header comment (line 1 is the shebang, so start at 2) and
# stop at the first line of code rather than a hardcoded line count --
# a fixed range silently starts printing code again the next time the
# header comment grows or shrinks (see build/fetch-vendor.sh's own
# --help, which had this exact bug).
-h|--help) awk '/^set /{exit} NR>1{print}' "$0"; exit 0 ;;
*) echo "FATAL: unknown argument '$1'" >&2; exit 1 ;;
esac
done
[ -n "$KERNEL" ] && [ -f "$KERNEL" ] || { echo "FATAL: --kernel: no such file" >&2; exit 1; }
[ -n "$DTB" ] && [ -f "$DTB" ] || { echo "FATAL: --dtb: no such file" >&2; exit 1; }
[ -n "$OUT" ] || { echo "FATAL: --out is required" >&2; exit 1; }
command -v mkimage >/dev/null || {
echo "FATAL: mkimage not on PATH (Debian/Ubuntu: u-boot-tools)" >&2; exit 1; }
# fdtget backs the post-build alignment assertions below (the whole point of
# which is that a misaligned image boots fine in CI and fails on a panel), so
# its absence must fail the build rather than silently skip those checks.
command -v fdtget >/dev/null || {
echo "FATAL: fdtget not on PATH (Debian/Ubuntu: device-tree-compiler)" >&2; exit 1; }
# resource_tool is a Rockchip host tool. It has no free-standing source here, so
# it is taken from the vendor SDK when one is present rather than vendored as a
# binary; --resource-tool overrides. FAILS CLOSED, because a boot.img built
# without a resource image looks fine and does not boot.
if [ -z "$RTOOL" ]; then
for c in \
"$HERE/../tools/resource_tool" \
"${WARDEN_VENDOR_SDK:-$HERE/../../../sdk}/sysdrv/tools/pc/toolkits/resource_tool" \
"$HOME/projects/scada/flare-edge/sdk/sysdrv/tools/pc/toolkits/resource_tool"
do
[ -x "$c" ] && { RTOOL="$c"; break; }
done
fi
[ -n "$RTOOL" ] && [ -x "$RTOOL" ] || {
echo "FATAL: resource_tool not found; pass --resource-tool PATH" >&2; exit 1; }
WORKDIR="$(mktemp -d)"
trap 'rm -rf "$WORKDIR"' EXIT
# resource_tool packs files under a root dir by their names, and U-Boot looks
# for the DTB under the fixed name rk-kernel.dtb.
cp "$DTB" "$WORKDIR/rk-kernel.dtb"
RES_FILES=(rk-kernel.dtb)
for pair in "$LOGO:logo.bmp" "$LOGO_KERNEL:logo_kernel.bmp" \
"$LOGO_RECOVERY:logo_recovery.bmp" "$LOGO_VERBOSE:logo_verbose.bmp"; do
src="${pair%%:*}"; dst="${pair##*:}"
[ -n "$src" ] || continue
[ -f "$src" ] || { echo "FATAL: logo '$src' not found" >&2; exit 1; }
cp "$src" "$WORKDIR/$dst"
RES_FILES+=("$dst")
done
echo "== resource.img (${RES_FILES[*]})"
( cd "$WORKDIR" && "$RTOOL" --pack --root=. --image=resource.img "${RES_FILES[@]}" >/dev/null )
[ -s "$WORKDIR/resource.img" ] || { echo "FATAL: resource_tool produced nothing" >&2; exit 1; }
cp "$KERNEL" "$WORKDIR/kernel"
cp "$WORKDIR/resource.img" "$WORKDIR/resource"
cp "$DTB" "$WORKDIR/fdt"
cat > "$WORKDIR/boot.its" <<'ITS'
/dts-v1/;
/ {
description = "FIT image with Linux kernel, FDT blob and resource";
images {
fdt {
data = /incbin/("fdt");
type = "flat_dt";
arch = "arm";
compression = "none";
load = <0xffffff00>;
hash { algo = "sha256"; };
};
kernel {
data = /incbin/("kernel");
type = "kernel";
arch = "arm";
os = "linux";
compression = "none";
entry = <0xffffff01>;
load = <0xffffff01>;
hash { algo = "sha256"; };
};
resource {
data = /incbin/("resource");
type = "multi";
arch = "arm";
compression = "none";
hash { algo = "sha256"; };
};
};
configurations {
default = "conf";
conf {
rollback-index = <0x0>;
kernel = "kernel";
fdt = "fdt";
multi = "resource";
};
};
};
ITS
# - SUB-IMAGE ALIGNMENT (`-B 0x200`). `-p` places only the FIRST payload; the
# rest are packed contiguously. U-Boot reads each with
# `blk_off = (FIT_ALIGN(fdt_totalsize) + offset) / blksz` (fit.c:331), a
# truncating divide by the 512-byte eMMC block, where FIT_ALIGN rounds to
# IMAGE_ALIGN_SIZE = 512 (include/image.h:955-958). So TWO things must hold:
# the metadata size must be a multiple of 512, or FIT_ALIGN(T) != T and
# EVERY payload is read late by the difference; and each data-position must
# be a multiple of 512, or the divide drops the remainder.
#
# Measured: without -B this script emitted the kernel at data-position
# 0x9A94 (% 512 = 148) with metadata 1064 (% 512 = 40) -- both faults at
# once. flare-edge 9387cff hit exactly this on hardware: "resource: sha256
# Bad hash" then "No resource file: logo.bmp", fixed by -B 0x200 and
# confirmed by a clean boot.
#
# 512, NOT 2048. An earlier revision of this comment claimed the vendor
# aligns to 0x800 and gated on it. That was drawn from two payloads in one
# build. Across the vendor's own release images on this machine, every
# data-position is a multiple of 512 and only some are multiples of 2048
# (four of eight RELEASE_TEST boot.img files sit at % 2048 = 1536), so a
# 2048 gate rejects images the vendor shipped and U-Boot boots. 0x800 is the
# vendor's -p value -- the absolute position of the first payload -- not an
# alignment.
#
# A -B-capable mkimage is REQUIRED, not preferred. The SDK vendors 2017.09,
# which has no -B, and project/build.sh:64 puts it first on PATH -- so the
# wrong one is what a build inside the SDK environment picks up. Measured:
# that binary produces metadata of 1064 bytes here, not a multiple of 512,
# so FIT_ALIGN rounds it up and every payload is read late. It does NOT
# 512-align this .its natively. Rather than silently emit an image that
# fails on a panel, pick the first mkimage on PATH that supports -B and say
# plainly when there is none.
# >>> mkimage-probe (extracted verbatim by tests/mk-bootimg/run-probe-tests.sh)
MKIMAGE=""
_IFS_SAVE="$IFS"; IFS=:
for _d in $PATH; do
_c="$_d/mkimage"
[ -x "$_c" ] || continue
# mkimage with no args prints its usage and exits non-zero; under the
# pipefail set at the top of this file that failure would sink the whole
# pipeline, so the probe would reject every mkimage including good ones.
if { "$_c" 2>&1 || true; } | grep -q -- '-B '; then MKIMAGE="$_c"; break; fi
done
IFS="$_IFS_SAVE"
# <<< mkimage-probe
if [ -z "$MKIMAGE" ]; then
echo "FATAL: no mkimage on PATH supports -B (sub-image alignment)." >&2
echo " The SDK vendors mkimage 2017.09, which does not, and its output" >&2
echo " is not 512-aligned here. Install u-boot-tools (2025.01 works)" >&2
echo " and make sure it precedes the SDK tools on PATH." >&2
exit 1
fi
echo "== FIT (external data, -E -p 0x800 -B 0x200) using $MKIMAGE"
( cd "$WORKDIR" && "$MKIMAGE" -f boot.its -E -p 0x800 -B 0x200 boot.img >/dev/null )
# Assert what U-Boot actually requires, on every build: the failure is silent --
# a misread offset does not fail the build, it fails on a panel, and sometimes
# only as a missing logo. Computed once and reused below (the embedded-data-FIT
# check further down needs the same value) so a future fix to how this is read
# cannot land in one check and not the other.
meta="$(od -An -tu4 -j4 -N4 --endian=big "$WORKDIR/boot.img" | tr -d ' ')"
if [ $(( meta % 512 )) -ne 0 ]; then
echo "FATAL: FIT metadata is $meta bytes, not a multiple of 512;" >&2
echo " FIT_ALIGN would round it up and every payload reads late" >&2
exit 1
fi
for _n in fdt kernel resource; do
# fdtget's presence is checked up front; a failure here means the FIT this
# script just built is malformed, not that the field is legitimately
# absent (mkimage -E gives every one of these images a data-position). Fail
# loud rather than treat an empty read as nothing to check.
if ! _pos="$(fdtget -t u "$WORKDIR/boot.img" "/images/$_n" data-position 2>&1)"; then
echo "FATAL: fdtget could not read /images/$_n data-position: $_pos" >&2
exit 1
fi
if [ $(( _pos % 512 )) -ne 0 ]; then
echo "FATAL: /images/$_n data-position $_pos is not 512-aligned;" >&2
echo " U-Boot's truncating block divide would read the wrong offset" >&2
exit 1
fi
done
# A FIT whose metadata swelled to the size of the whole image is an
# embedded-data build, which this U-Boot rejects. Catch it here rather than on
# a panel that will not come back.
total="$(stat -c %s "$WORKDIR/boot.img")"
if [ "${meta:-0}" -ge 4096 ] || [ "${meta:-0}" -ge "$total" ]; then
echo "FATAL: FIT metadata is ${meta} bytes of a ${total}-byte image: that is an" >&2
echo " embedded-data FIT and U-Boot will report 'No fit blob'." >&2
exit 1
fi
install -m 0644 "$WORKDIR/boot.img" "$OUT"
echo "boot.img: $OUT ($(stat -c %s "$OUT") bytes, FIT metadata ${meta} bytes)"
+34
View File
@@ -0,0 +1,34 @@
# Third-party trees this platform is built against, pinned to an exact commit.
#
# WHY A PIN AND NOT A COPY. Between them these are ~21 GB; the vendor SDK alone
# is 21 GB of build tree with absolute paths baked into its host tools. Copying
# that into git would make every clone of this repo unusable and would still not
# make the result reproducible. What makes a build reproducible is knowing
# EXACTLY which tree was used, and that is a commit id -- the same reasoning
# build/fetch-kernel-tarball.sh already applies to the kernel, where a sha256
# pin stands in for vendoring 150 MB of tarball.
#
# So this file is the single place that answers "which LVGL, which vendor SDK",
# and fetch-vendor.sh is the only thing that acts on it. A checkout that has
# drifted off its pin is reported, never silently used.
#
# NOT EVERYTHING HERE IS A GIT TREE. Two build inputs are pinned tarballs
# instead, each with its own fetch-and-verify script beside this file, because a
# release tarball has no commit to name:
#
# linux-6.18.46 build/fetch-kernel-tarball.sh
# buildroot-2025.02.8 build/fetch-buildroot-tarball.sh
#
# The buildroot one matters more than it looks. The firmware builds against
# sysdrv/source/buildroot/buildroot-2025.02.8 inside the vendor SDK, and the
# vendor SDK ships 2023.02.6 -- so that tree is not in the vendor checkout, was
# not in this manifest, and had no recorded origin at all. A clean rebuild
# silently used the vendor's older buildroot and produced a different userspace
# (flare-edge#135). Our delta on top of it is captured in the flare-edge repo at
# sdk-patches/buildroot/.
#
# Format: name<TAB>url<TAB>commit<TAB>description
# Blank lines and lines starting with '#' are ignored.
lvgl https://github.com/lvgl/lvgl.git 066d8db0b54819223357731f68961a90b3d785b4 LVGL v9.5.0-383-g066d8db0b: the UI toolkit warden-ui links against
luckfox-pico https://github.com/LuckfoxTECH/luckfox-pico.git 824b817f889c2cbff1d48fcdb18ab494a68f69d1 Vendor RV1106 SDK: buildroot userspace, U-Boot and the host packaging tools. Being replaced by this repo; still the source of the rootfs and the FIT/resource host tools.
+62
View File
@@ -1308,3 +1308,65 @@ CONFIG_DEBUG_UART_PHYS=0xff4c0000
CONFIG_DEBUG_UART_VIRT=0xff4c0000
CONFIG_DEBUG_UART_8250_WORD=y
CONFIG_EARLY_PRINTK=y
#
# WardenOS platform networking.
#
# These were absent, and their absence is not visible until the panel is in
# the field: the kernel boots, the UI runs, and then the mesh never comes up,
# every iptables rule fails to load, and a tagged port carries nothing. The
# shipping 5.10 kernel has all three, so a 6.18 image without them is a
# REGRESSION against what is on the panel today -- caught by diffing the two
# device configs, not by anything that fails at build time.
#
# WireGuard: flared's mesh to FLARE (wg0). Needs the UDP tunnel helper.
CONFIG_WIREGUARD=y
CONFIG_NET_UDP_TUNNEL=y
# VLAN: the MikroTik app configures tagged ports and the panel terminates them.
CONFIG_VLAN_8021Q=y
# Bridging. Not in the vendor kernel, and its absence is why the panel's access
# point has to be its own subnet with its own DHCP pool rather than joining the
# LAN's broadcast domain (see the comment in ui-src/src/warden/router.c). With a
# bridge the panel can run an AP the way a UniFi AP does: wlan0 and the LAN port
# in one broadcast domain, addresses handed out by whatever router is already
# upstream. That is a capability this kernel adds, not one it restores.
CONFIG_BRIDGE=y
CONFIG_BRIDGE_IGMP_SNOOPING=y
CONFIG_LLC=y
CONFIG_STP=y
# Netfilter: S35iptables and router mode. Scoped to what the scripts actually
# use -- filter + nat tables, MASQUERADE, and the state/limit matches -- rather
# than the vendor kernel's full 124-symbol suite.
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_NF_CONNTRACK=y
CONFIG_NF_NAT=y
CONFIG_NF_NAT_MASQUERADE=y
CONFIG_NETFILTER_XTABLES=y
# ...which in turn gates on the legacy xtables core, another 6.18 split.
CONFIG_NETFILTER_XTABLES_LEGACY=y
CONFIG_NETFILTER_XT_MATCH_STATE=y
CONFIG_NETFILTER_XT_MATCH_LIMIT=y
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_NAT=y
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=y
CONFIG_NF_DEFRAG_IPV4=y
# No NF_CONNTRACK_IPV4 symbol here -- IPv4 conntrack has been unconditional in
# NF_CONNTRACK's core since well before 6.18 (net/ipv4/netfilter/Kconfig has no
# such config). Only stale arch defconfigs (e.g. keystone_defconfig) still set
# it; Kconfig drops an unknown symbol with no warning, so it looked live but
# did nothing. Left out on purpose so this file does not claim a gate that
# does not exist. If a future kernel bump reintroduces a real symbol by this
# name, catch it by diffing the expanded .config, not by functional test alone
# -- see PORT-STATUS.md's own history of a silent-drop hiding a real gap.
CONFIG_IP_NF_IPTABLES=y
# 6.18 SPLIT THE LEGACY TABLES OUT. IP_NF_FILTER and IP_NF_NAT depend on
# IP_NF_IPTABLES_LEGACY, which did not exist in 5.10 -- so copying the vendor
# kernel's symbol list verbatim yields a kernel where `iptables` has no filter
# or nat table and every rule in S35iptables fails. Our userspace drives legacy
# iptables, not nft, so this is the table set to enable.
CONFIG_IP_NF_IPTABLES_LEGACY=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_MANGLE=y
+1 -1
View File
@@ -17,7 +17,7 @@ tested (or not) differently. Today:
| HPMCU / RISC-V coproc | `flared/src/hpmcu.rs` | via devmem + firmware blob load | `WARDEN_HPMCU_FW` redirects the blob path only | env gate disables it |
| NPU load | `ui-src/.../sysmon.c` | read `/proc/rknpu/load` | **none**, literal path | file absent -> "NPU absent" |
| RGA (2D blit) | `ui-src/.../warden_rga.c` | `librga improcess` + dma-heap ioctl | compile-time `#if WARDEN_USE_RGA` | `#if` off -> LVGL software path |
| RS485 daemon | `warden-modbus/modbus_engine.c` | `open("/dev/ttyS4")` | recompile `-DRS485_PORT=<pty>` | (recompiled for a pty) |
| RS485 daemon | `apps/warden-modbus/modbus_engine.c` | `open("/dev/ttyS4")` | recompile `-DRS485_PORT=<pty>` | (recompiled for a pty) |
| RS485 panel client | `ui-src/.../modbus.c` | `AF_UNIX` socket | `WARDEN_MODBUS_SOCK` env override | socket absent -> "unavailable" |
| Relays / GPIO | `ui-src/.../relays.c` | `/sys/class/gpio` sysfs | **none**, literal paths | path absent -> "unavailable" |
| Slot metadata | `flared/src/slotctl.rs` | `misc` partition + `/proc/cmdline` | `WARDEN_MISC_DEV`, `WARDEN_CMDLINE_FILE` env overrides | (redirected to scratch files) |
+1 -1
View File
@@ -13,7 +13,7 @@ execute code on private infrastructure (ADR-0007).
| `mcdc` | ubuntu-latest | 100% MC/DC enforced on every `drivers/*/test` (gcc-14 `-fcondition-coverage`). |
| `bench` | ubuntu-latest | Smoke-runs the sim + rs485-bridge micro-benchmarks; emits ns/op trend JSON. |
| `patches-apply` | ubuntu-latest | Fetches pristine linux-6.18.46 (cached, sha256-verified) and applies `patches/*` in order. |
| `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`; builds the initramfs (pinned busybox) and the A/B disk image. |
| `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`, `tests/mk-bootimg/*.sh`, `tests/fetch-vendor/*.sh`, `tests/fetch-buildroot-tarball/*.sh`; runs the offline regression tests for the mk-bootimg probe, boot.img validation, run.sh argv ordering, mkimage's SEED_DIR hook, staged rootfs permissions, fetch-vendor, fetch-buildroot-tarball, and its own CI wiring; builds the initramfs (pinned busybox) and the A/B disk image. |
| `quality` | ubuntu-latest | Codacy-style grade computed in-pipeline: clippy, cppcheck, shellcheck, ruff, lizard, jscpd, cargo-audit feed `tools/quality/score.py` (SQALE debt ratio + a separate worst-of security axis; SonarQube's published thresholds). Uploads `quality.json`; fails if the security grade is worse than C. |
| `kernel-build` | ubuntu-latest, **dispatch-only** | apt-installs the cross toolchain + qemu, `build/build-kernel.sh` -> `zImage` + `rv1106-warden.dtb`, QEMU `-M virt` boot smoke (fail-closed), artifact upload (best-effort). Trigger: `gh workflow run ci.yml`. |
| `prune-artifacts` | ubuntu-latest, dispatch-only | Deletes `kernel-rv1106` artifacts beyond the newest 3. |
+29
View File
@@ -159,3 +159,32 @@ build-m2.sh reproducible M2 build (zImage + dtb)
```
The full ported tree lives in `flare-edge/research/linux-6.18.46/` (scratch); this dir is
the durable, reviewable capture, to become a proper patch series as milestones land.
## Hardware verification (2026-09-03)
The current series in `../../patches/` was built with `build/build-kernel.sh` +
`build/warden_defconfig` (rockchip gcc 8.3), packaged with the new
`build/mk-bootimg.sh`, flashed to the bench panel's inactive slot armed for a
single try, and booted. Working on real silicon: display (`/dev/fb0`,
`/dev/dri/card0`), backlight, Goodix touch, RGA (`/dev/rga`), eth0, the usb0
gadget, all userspace daemons, and zero kernel faults in dmesg.
M4 (display) and part of M6 (RGA) above are STALE: both are in the series and
verified here. M5 (wifi) and the rest of M6 (watchdog, HPMCU, full USB-OTG
dual role) are still open -- this run gives no evidence for them.
**What the same test found missing, and why it matters.** Diffing this
defconfig's expansion against the kernel actually shipping on a panel showed
three subsystems absent, none of which fail at build time:
ip link add type wireguard -> Operation not supported
iptables -L -> FAIL (no filter or nat table)
/proc/net/vlan -> absent
That is the FLARE mesh, every rule in S35iptables plus router-mode NAT, and the
MikroTik tagged-port work. The netfilter half hid behind a 6.18 rename: the
legacy tables moved under NETFILTER_XTABLES_LEGACY / IP_NF_IPTABLES_LEGACY, so
copying the vendor kernel's symbol list verbatim leaves IP_NF_FILTER and
IP_NF_NAT silently off. Fixed in warden_defconfig and re-verified on the same
panel: wireguard device creates, filter and nat tables load, MASQUERADE and the
state match apply, and a VLAN interface creates.
@@ -1,12 +1,14 @@
* RV1106 USB2 PHY: single OTG port at 0xff3e0000 (ported from the vendor driver;
* fields map 1:1 to 6.18 except utmi_iddig -> utmi_id, and the 5.10-only
* iddig_output/iddig_en/bvalid_grf_sel are dropped). Signal-quality phy_tuning is
* left off for now (guarded, so NULL is safe): the phy is functional without it.
* iddig_output/iddig_en/bvalid_grf_sel are dropped). Signal-quality phy_tuning
* was originally left off here; it is now implemented (patches/27-usb-phy-tuning.patch)
* because "functional without it" turned out not to mean reliable -- see #160.
*/
static const struct rockchip_usb2phy_cfg rv1106_phy_cfgs[] = {
{
.reg = 0xff3e0000,
.num_ports = 1,
.phy_tuning = rv1106_usb2phy_tuning,
.clkout_ctl = { 0x0058, 4, 4, 1, 0 },
.port_cfgs = {
[USB2PHY_PORT_OTG] = {
+128
View File
@@ -0,0 +1,128 @@
diff --git a/drivers/phy/rockchip/phy-rockchip-inno-usb2.c b/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
--- a/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
+++ b/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
@@ -247,6 +247,7 @@
struct device *dev;
struct regmap *grf;
struct regmap *usbgrf;
+ void __iomem *phy_base;
struct clk_bulk_data *clks;
struct clk *clk480m;
struct clk_hw clk480m_hw;
@@ -1417,6 +1418,19 @@
return -EINVAL;
}
+ /*
+ * RV1106 tunes signal quality through the PHY's own register block, which
+ * is this node's "reg" resource and is distinct from the GRF regmap above.
+ * Map it only where a tuning callback needs it; every other SoC here tunes
+ * through the GRF and must keep working with phy_base left NULL.
+ */
+ if (of_device_is_compatible(np, "rockchip,rv1106-usb2phy")) {
+ rphy->phy_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(rphy->phy_base))
+ return dev_err_probe(dev, PTR_ERR(rphy->phy_base),
+ "failed to map phy registers\n");
+ }
+
rphy->phy_reset = devm_reset_control_get_optional(dev, "phy");
if (IS_ERR(rphy->phy_reset))
return PTR_ERR(rphy->phy_reset);
@@ -1506,6 +1520,30 @@
return ret;
}
+/*
+ * Direct PHY-register accessors. Most SoCs here tune through the GRF regmap,
+ * but the RV1106's signal-quality registers live in the PHY's own block, which
+ * is mapped separately into rphy->phy_base.
+ */
+static inline void phy_update_bits(void __iomem *reg, u32 mask, u32 val)
+{
+ u32 tmp = readl(reg);
+
+ tmp &= ~mask;
+ tmp |= val & mask;
+ writel(tmp, reg);
+}
+
+static inline void phy_set_bits(void __iomem *reg, u32 bits)
+{
+ writel(readl(reg) | bits, reg);
+}
+
+static inline void phy_clear_bits(void __iomem *reg, u32 bits)
+{
+ writel(readl(reg) & ~bits, reg);
+}
+
static int rk3128_usb2phy_tuning(struct rockchip_usb2phy *rphy)
{
/* Turn off differential receiver in suspend mode */
@@ -2291,10 +2329,65 @@
* iddig_output/iddig_en/bvalid_grf_sel are dropped). Signal-quality phy_tuning is
* left off for now (guarded, so NULL is safe) — the phy is functional without it.
*/
+/*
+ * RV1106 USB2 PHY signal-quality tuning, ported from the vendor 5.10 driver
+ * (drivers/phy/rockchip/phy-rockchip-inno-usb2.c, rv1106_usb2phy_tuning).
+ *
+ * The 6.18 forward-port originally omitted this ("the phy is functional without
+ * it"). It is functional, but not reliable: with these registers left at reset
+ * defaults the HS link is electrically marginal under sustained traffic. On a
+ * bench panel a USB-Ethernet dongle would stall a bulk transfer after a few
+ * minutes; the r8152 Tx-timeout recovery then issues an xHCI Stop Endpoint
+ * command which the controller never answers, and xhci-hcd declares the
+ * controller dead and tears the bus down (flare-edge issue #160). Measured
+ * carrier-on to death across four runs: 14s, 519s, 521s, 584s -- a stochastic
+ * spread, not a timer, which is what marginal signalling looks like.
+ *
+ * Pre-emphasis strength is silicon-revision dependent. The vendor driver picks
+ * it from rockchip_get_cpu_version(), which does not exist in mainline; it
+ * derives from RV1106_OS_REG1 (0xff020204) & 0x7. Our panels read 1, i.e. the
+ * non-zero branch, so 3'b001 is pinned here. If a board ever reports version 0
+ * it wants 3'b011 instead -- see the vendor driver.
+ */
+static int rv1106_usb2phy_tuning(struct rockchip_usb2phy *rphy)
+{
+ /* Mapped only for this compatible; refuse to poke a NULL base. */
+ if (!rphy->phy_base)
+ return -ENODEV;
+
+ /* Always enable pre-emphasis in SOF & EOP & chirp & non-chirp state */
+ phy_update_bits(rphy->phy_base + 0x30, GENMASK(2, 0), 0x07);
+
+ /* Set Tx HS pre_emphasize strength to 3'b001 (cpu_version != 0) */
+ phy_update_bits(rphy->phy_base + 0x40, GENMASK(5, 3), (0x01 << 3));
+
+ /* Set RX Squelch trigger point configure to 4'b0000(112.5 mV) */
+ phy_update_bits(rphy->phy_base + 0x64, GENMASK(6, 3), (0x00 << 3));
+
+ /* Turn off differential receiver by default to save power */
+ phy_clear_bits(rphy->phy_base + 0x100, BIT(6));
+
+ /* Set 45ohm HS ODT value to 5'b10111 to increase driver strength */
+ phy_update_bits(rphy->phy_base + 0x11c, GENMASK(4, 0), 0x17);
+
+ /* Set Tx HS eye height tuning to 3'b011(462 mV) */
+ phy_update_bits(rphy->phy_base + 0x124, GENMASK(4, 2), (0x03 << 2));
+
+ /* Bypass Squelch detector calibration */
+ phy_update_bits(rphy->phy_base + 0x1a4, GENMASK(7, 4), (0x01 << 4));
+ phy_update_bits(rphy->phy_base + 0x1b4, GENMASK(7, 4), (0x01 << 4));
+
+ /* Set HS disconnect detect mode to single ended detect mode */
+ phy_set_bits(rphy->phy_base + 0x70, BIT(2));
+
+ return 0;
+}
+
static const struct rockchip_usb2phy_cfg rv1106_phy_cfgs[] = {
{
.reg = 0xff3e0000,
.num_ports = 1,
+ .phy_tuning = rv1106_usb2phy_tuning,
.clkout_ctl = { 0x0058, 4, 4, 1, 0 },
.port_cfgs = {
[USB2PHY_PORT_OTG] = {
+136
View File
@@ -0,0 +1,136 @@
diff --git a/drivers/phy/rockchip/phy-rockchip-inno-usb2.c b/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
--- a/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
+++ b/drivers/phy/rockchip/phy-rockchip-inno-usb2.c
@@ -203,6 +203,7 @@
* @event_nb: hold event notification callback.
* @state: define OTG enumeration states before device reset.
* @mode: the dr_mode of the controller.
+ * @host_mode: the port is currently acting as a USB host.
*/
struct rockchip_usb2phy_port {
struct phy *phy;
@@ -222,6 +223,7 @@
struct notifier_block event_nb;
enum usb_otg_state state;
enum usb_dr_mode mode;
+ bool host_mode;
};
/**
@@ -239,6 +241,8 @@
* @dcd_retries: The retry count used to track Data contact
* detection process.
* @edev: extcon device for notification registration
+ * @edev_self_allocated: no extcon phandle in DT, so the OTG state machine
+ * has no external role signal at all
* @irq: muxed interrupt for single irq configuration
* @phy_cfg: phy register configuration, assigned by driver data.
* @ports: phy port instance.
@@ -257,6 +261,7 @@
enum power_supply_type chg_type;
u8 dcd_retries;
struct extcon_dev *edev;
+ bool edev_self_allocated;
int irq;
const struct rockchip_usb2phy_cfg *phy_cfg;
struct rockchip_usb2phy_port ports[USB2PHY_NUM_PORTS];
@@ -459,6 +464,14 @@
if (ret)
return dev_err_probe(rphy->dev, ret,
"failed to register extcon device\n");
+
+ /*
+ * Nothing external drives the role on this board. The extcon
+ * below is ours, and no code ever sets a cable state on it, so
+ * the OTG state machine's view of the world is permanently
+ * blank rather than merely stale.
+ */
+ rphy->edev_self_allocated = true;
}
rphy->edev = edev;
@@ -649,11 +662,52 @@
return 0;
}
+/*
+ * The OTG port's state machines only make sense while the port is a
+ * peripheral. In host role their inputs are meaningless on a board with no
+ * ID source: charger detection drives the analog front end of a port that is
+ * already enumerating a device, and its DCP verdict ends in
+ * rockchip_usb2phy_power_off(), which suspends the phy and drops the 480M
+ * clkout that the host controller's core runs on. The controller then stops
+ * clocking with its registers still readable, so it reports itself healthy
+ * while no transfer ever completes.
+ *
+ * dwc3 already calls phy_set_mode() on every role change, but this driver
+ * implemented no .set_mode, so the phy never learned the role. Track it and
+ * keep both state machines out of host role.
+ */
+static int rockchip_usb2phy_set_mode(struct phy *phy, enum phy_mode mode,
+ int submode)
+{
+ struct rockchip_usb2phy_port *rport = phy_get_drvdata(phy);
+
+ if (rport->port_id != USB2PHY_PORT_OTG)
+ return 0;
+
+ switch (mode) {
+ case PHY_MODE_USB_HOST:
+ if (!rport->host_mode) {
+ rport->host_mode = true;
+ cancel_delayed_work_sync(&rport->otg_sm_work);
+ cancel_delayed_work_sync(&rport->chg_work);
+ }
+ break;
+ case PHY_MODE_USB_DEVICE:
+ rport->host_mode = false;
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
static const struct phy_ops rockchip_usb2phy_ops = {
.init = rockchip_usb2phy_init,
.exit = rockchip_usb2phy_exit,
.power_on = rockchip_usb2phy_power_on,
.power_off = rockchip_usb2phy_power_off,
+ .set_mode = rockchip_usb2phy_set_mode,
.owner = THIS_MODULE,
};
@@ -667,6 +721,15 @@
unsigned long delay;
bool vbus_attach, sch_work, notify_charger;
+ /*
+ * This machine can only get the answer wrong here. In host role its
+ * verdict suspends a phy that is enumerating a device (#160); with a
+ * self-allocated extcon it has no role signal in EITHER direction, and
+ * in device role it strands the gadget the same way (#168).
+ */
+ if (rport->host_mode || rphy->edev_self_allocated)
+ return;
+
vbus_attach = property_enabled(rphy->grf,
&rport->port_cfg->utmi_bvalid);
@@ -825,6 +888,15 @@
bool is_dcd, tmout, vout, vbus_attach;
unsigned long delay;
+ /*
+ * Never probe the front end of a port that is hosting a device, and
+ * never at all on a board whose bvalid is pinned high by the VBUS
+ * bypass: charger detection cannot be meaningful there, and its DCP
+ * verdict powers the phy off underneath a working link (#160, #168).
+ */
+ if (rport->host_mode || rphy->edev_self_allocated)
+ return;
+
vbus_attach = property_enabled(rphy->grf, &rport->port_cfg->utmi_bvalid);
dev_dbg(&rport->phy->dev, "chg detection work state = %d\n",
+2
View File
@@ -18,6 +18,8 @@ with `../build/warden_defconfig` -> `zImage` + `rv1106-warden.dtb`).
| `15-pinctrl.patch` | `pinctrl-rockchip` RV1106 iomux |
| `20-arch-dts-bindings.patch` | `arch/arm` DTs (`rv1106.dtsi`, `rv1106-warden.dts`, pinctrl), mach, configs, `include/dt-bindings` |
| `25-usb-phy.patch` | inno-usb2 phy |
| `27-usb-phy-tuning.patch` | inno-usb2 phy: RV1106 signal-quality register tuning |
| `29-usb2phy-no-blind-otg-sm.patch` | inno-usb2 phy: stop the OTG/charger state machines running blind |
| `30-drm-vop-panel-rgb.patch` | VOP (`rgb_dclk_pol`=0 + WIN1 scanout), panel-simple bus-format, rockchip_rgb |
| `35-mailbox-hpmcu.patch` | rockchip-mailbox (rv1106 num_chans=1) |
| `40-pvtm-soc.patch` | PVTM (core+pmu) + `include/soc/rockchip/pvtm.h` |
+41 -7
View File
@@ -37,6 +37,15 @@ Documented guest deviations from production, set by stage-2 init:
`WARDEN_HPMCU=0` (no mailbox SRAM on virt; flared >= flare-edge#106 fix
required, or the daemon dies of SIGBUS).
**The VM's kernel is ahead of the panels' in one way that shows in the UI.**
It is built from `build/warden_defconfig` plus `configs/virt.fragment`, so it
has `CONFIG_BRIDGE`; the vendor 5.10 kernel most panels still run does not.
A Client-mode access point is bridged to the wired uplink and needs that, so
the Wi-Fi page offers it in the VM and says "Needs Router mode: this kernel
cannot bridge" on a 5.10 panel. Both are correct for the kernel underneath.
The difference disappears as panels move to the 6.18 build, but until then a
screenshot from here is not evidence about a 5.10 panel's Wi-Fi page.
## Quick Start
```sh
@@ -54,9 +63,27 @@ bash qemu/run.sh --kernel $HOME/kbuild-out/linux-6.18.46/arch/arm/boot/zImage --
```
Payload: drop static musl armv7 binaries into `qemu/payload/` (see its
README); `warden-flared`, `warden-modbus`, and `warden-ui` (the LVGL
fbdev+evdev build from flare-edge `tools/build-ui-vm.sh`) are started by
stage-2 init when present.
README). Stage-2 init starts every panel daemon it finds, in the panel's own
S-number order (`warden-ai`, `warden-automation`, `warden-modbus`,
`warden-mikrotik`, `warden-asic`, `warden-starlink`, `warden-stratum`,
`warden-flared`), then `warden-ui` (the LVGL fbdev+evdev build from
flare-edge `tools/build-ui-vm.sh`). The quickest faithful payload is the
daemons a bench panel already runs: flare-edge `tools/rig-pull-daemons.sh
--host <panel>` copies them in. The RS485 UART is always present (a null
chardev without `--rs485`), so `warden-modbus` polls a port with nothing on
it, as on a panel with an empty header, and never the control channel.
The VT cursor is kept off (`vt.global_cursor_default=0`): fbcon shares the
virtio-gpu framebuffer with warden-ui and its blinking cursor would otherwise
show up in screendumps at random (issue #18).
The guest carries the panel's own `/etc/passwd`, `/etc/shadow` and
`/etc/group` (root's md5-crypt of the documented default password), so a
screen that verifies the root password against `/etc/shadow` behaves as it
does on a panel instead of rejecting every attempt. `qemu_stage_rootfs()`
(`lib.sh`) forces `etc/shadow` to mode 0600 on every stage: git tracks only
the executable bit, so the checked-out source file's own mode depends on the
checking-out umask and cannot be trusted to arrive non-world-readable.
## Scenarios
@@ -64,11 +91,16 @@ All take the virt-fragment `<zImage>`; `FLARE_EDGE=<checkout>` where noted.
| Scenario | Needs | Proves |
|---|---|---|
| `stage-rootfs-perms.sh` | - | offline, no VM: `qemu_stage_rootfs()` always lands `etc/shadow` at 0600, even staged from a source copy deliberately left 0644 |
| `seed-dir.sh` | - | offline, no VM: `mkimage.sh`'s `SEED_DIR` hook lands every seeded file under userdata/warden with its original mode (a 0600 secret included), a seeded file beats a same-named `--state` value, a `SEED_DIR` that isn't a directory fails closed, and so does an individual entry that is a symlink, a subdirectory, or named outside `[A-Za-z0-9_.-]+` -- a hyphenated key such as `gas-plant.devices` still seeds cleanly |
| `boot-smoke.sh` | - | sentinel-asserting boot; runs in CI inside kernel-build |
| `portal-scenario.sh` | `FLARE_EDGE` | real flared against the desk mock portal: authenticated check-in, desired-state pull, signed tier-1 `.wfw` download; verify/stage/APPLYING as a dry run (no `WARDEN_FW_ALLOW_APPLY`) |
| `ota-apply.sh` | `FLARE_EDGE` | the FULL apply: the `.wfw`'s bootable rootfs payload is written to rootfs_b (`run.sh --allow-apply` gates it per boot), the AvbABData in `misc` flips, and slot `_b` boots the applied version |
| `ui-shot.sh` | - | display+touch, headless: QMP-screendumps the 720x720 UI, taps the Metrics tab via `input-send-event`, asserts the frame changed (`qmp.py` is the QMP client) |
| `ui-drive.sh <script>` | - | the same rig for a SEQUENCE: boots once, runs a `qmp.py drive` script of taps/swipes/screenshots in panel pixels, and FAILS if warden-ui died on the way (stage-2 init announces the exit on the console). `tests/scripts/nav-stress.txt` is the navigation regression: it reproduces the s_row_left overflow that segfaulted the UI on returning to Settings > Apps; `tests/scripts/home-leaves-fullscreen.txt` proves the `home` verb clears dashboard fullscreen, the state a hardware wake tap leaves behind (flare-edge #176); `tests/scripts/fullscreen-toggle-tracks-real-state.txt` proves a `fullscreen toggle` right after `home` (or a real tap) reads the dashboard's actual state rather than a belief `home` bypassed -- `test_qmp_drive.py`'s `FullscreenToggleTracksRealState` pins the same contract offline |
| `real-image-boot.sh` | matched `rootfs.img` + `oem.img` | an ACTUAL flare-edge build (placed by `mkimage.sh --rootfs-image/--oem-image`) boots its own init chain to getty; binaries predating known fixes reproduce their bugs faithfully, a time machine for field issues |
| `test-ui-drive-rs485.sh` | - | offline, no VM: a fake `run.sh` and (for one case) a fake `socat` stand in so `ui-drive.sh --rs485-devices`'s startup fails closed instead of printing a false "== rs485 simulator: ..." over a bus nothing is serving, for each of a qemu rs.sock that never appears, a socat that never links rs.pty, and an mbsim.py that dies before opening rs.ctl; also proves `cleanup()` escalates to SIGKILL for a sim process that ignores SIGTERM |
| `run-sh-args-test.sh` | - | offline, no VM: a fake `qemu-system-arm` captures run.sh's own argv and pins the ctl/rs485 pci-serial ordering contract rootfs/sbin/init's ttyS0-vs-ttyS1 alias depends on -- the ctl device always enumerates before rs485, warden.ctl only lands on the cmdline when --ctl is given, and an rs485 pci-serial device (real or null-backed) is always present so the port count never shifts |
| watchdog (`run.sh --watchdog`) | - | arm `/dev/watchdog`, don't pet: the VM resets ~30 s later (verified) |
Scenario fine print:
@@ -95,7 +127,9 @@ Scenario fine print:
## Requirements
`qemu-system-arm` (Debian 13 ships QEMU 10), `curl`, `cpio`, `mkfs.ext4`,
`gcc-arm-linux-gnueabihf` (kernel build), `python3` (+`cryptography` for the
portal scenario's `.wfw` signing). CI: the hosted `qemu-tools` job builds the
tooling; the boot smoke runs inside the (also hosted, dispatch-only)
`kernel-build` job, which apt-installs its own toolchain and qemu (ADR-0007).
`debugfs` (both ship in `e2fsprogs`; `seed-dir.sh` reads a built partition
back with it), `gcc-arm-linux-gnueabihf` (kernel build), `python3`
(+`cryptography` for the portal scenario's `.wfw` signing). CI: the hosted
`qemu-tools` job builds the tooling; the boot smoke runs inside the (also
hosted, dispatch-only) `kernel-build` job, which apt-installs its own
toolchain and qemu (ADR-0007).
+32 -7
View File
@@ -22,15 +22,34 @@ qemu_get_busybox() {
echo "FATAL: no pinned sha256 for busybox (expected $sha_file): refusing to build from an unverified binary" >&2
exit 1
}
if [ ! -f "$BB" ]; then
qemu_log "downloading $BB_URL"
curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$BB_URL" -o "$BB"
fi
local want got
want="$(cat "$sha_file")"
got="$(sha256sum "$BB" | awk '{print $1}')"
[ "$want" = "$got" ] || { echo "busybox sha256 mismatch: want $want got $got" >&2; exit 1; }
qemu_log "busybox sha256 verified"
if [ -f "$BB" ]; then
got="$(sha256sum "$BB" | awk '{print $1}')"
if [ "$want" = "$got" ]; then
qemu_log "busybox sha256 verified"
return
fi
qemu_log "cached busybox failed verification; fetching a clean copy"
fi
local attempt tmp="${BB}.download.$$"
rm -f "$tmp"
for attempt in 1 2 3; do
qemu_log "downloading $BB_URL (attempt $attempt/3)"
if curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$BB_URL" -o "$tmp"; then
got="$(sha256sum "$tmp" | awk '{print $1}')"
if [ "$want" = "$got" ]; then
mv -f "$tmp" "$BB"
qemu_log "busybox sha256 verified"
return
fi
echo "busybox sha256 mismatch on attempt $attempt: want $want got $got" >&2
fi
rm -f "$tmp"
done
echo "FATAL: could not fetch pinned busybox after 3 verified attempts" >&2
exit 1
}
# Stage the shared rootfs skeleton (qemu/rootfs/ + busybox) into $1.
@@ -44,6 +63,12 @@ qemu_stage_rootfs() {
cp -a "$QEMU_DIR/rootfs/." "$root/"
chmod 0755 "$root/init" "$root/sbin/init" "$root/etc/rc" \
"$root/usr/share/udhcpc/default.script"
# git only tracks the executable bit, so a checkout lands etc/shadow at
# whatever the umask gives a non-executable file (644 under the common
# 022) -- root's crypt hash world-readable. Force the normal shadow mode
# here, once, for every caller (mkimage.sh and mkinitramfs.sh both stage
# through this function).
chmod 0600 "$root/etc/shadow"
}
# Parse a "SIZE[@OFFSET](NAME)" blkdevparts entry list (without the "vda:"
+51 -2
View File
@@ -15,8 +15,14 @@
# flare-edge build's rootfs.img/oem.img matched pair) into slot A instead of
# the busybox skeleton; slot B keeps the skeleton as a known-good fallback.
# Env:
# BUSYBOX path to a local busybox binary (skips the download; still verified)
# OUT output dir (default: qemu/out); image at $OUT/disk.img
# BUSYBOX path to a local busybox binary (skips the download; still verified)
# OUT output dir (default: qemu/out); image at $OUT/disk.img
# SEED_DIR a directory of pre-built userdata/warden files (one file per
# settings key, e.g. flare-edge tools/seed-fixtures.py's output)
# copied in VERBATIM, after --portal-url/--state. Lets a caller
# seed an arbitrary key set -- a flow spec's setup.seed can name
# anything settings.c reads -- without growing --state into a
# multi-value flag; last one written wins, same as --state.
set -euo pipefail
QEMU_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -85,6 +91,49 @@ for kv in ${STATE_KV[@]+"${STATE_KV[@]}"}; do
printf '%s\n' "${kv#*=}" > "$UDATA/warden/${kv%%=*}"
done
# SEED_DIR: a pre-built set of settings files (see the Env note above), copied
# in whole rather than re-parsed here -- seed-fixtures.py already wrote them
# in the exact format settings.c reads (filename = key, mode 0600 for a
# secret), so re-deriving that here would be a second place to keep in sync
# with settings.c. Applied after --state so a seeded file can override a
# same-named --state value; `cp -a` preserves the 0600 on a secret entry.
if [ -n "${SEED_DIR:-}" ]; then
[ -d "$SEED_DIR" ] || { echo "FATAL: SEED_DIR '$SEED_DIR' is not a directory" >&2; exit 1; }
# Validate every entry before cp -a touches any of them: `cp -a` preserves
# a symlink rather than following it, so one placed in SEED_DIR would land
# as a live symlink under /userdata/warden that a later read (flared's
# settings loader, running inside the guest) resolves through. Fail closed
# on the first bad entry the same way --state fails closed above, rather
# than copy it into the image and let it surface as a confusing read
# later. Only a top-level plain file matches "one file per settings key"
# (the Env note above) -- seed-fixtures.py never nests a subdirectory, and
# neither should anything else pointed at this hook.
#
# Charset is seed-fixtures.py's KEY_RE, not --state's stricter
# [A-Za-z0-9_.]+ above: seed-fixtures.py deliberately also allows '-' for
# keys like "gas-plant.devices" (flare-edge#151), and committed flow
# specs (gas_compression, gas_plant, liquid_pumping, power_generation)
# already seed hyphenated keys through this exact path -- --state's
# charset would reject every one of them.
while IFS= read -r -d '' entry; do
base="$(basename "$entry")"
case "$base" in
*[!A-Za-z0-9_.-]*)
echo "FATAL: SEED_DIR entry '$base' must match [A-Za-z0-9_.-]+ (it becomes a filename)" >&2
exit 1 ;;
esac
if [ -L "$entry" ]; then
echo "FATAL: SEED_DIR entry '$base' is a symlink (refusing to copy it into userdata verbatim)" >&2
exit 1
fi
[ -f "$entry" ] || {
echo "FATAL: SEED_DIR entry '$base' is not a plain file (refusing to copy it into userdata verbatim)" >&2
exit 1
}
done < <(find "$SEED_DIR" -mindepth 1 -maxdepth 1 -print0)
cp -a "$SEED_DIR"/. "$UDATA/warden/"
fi
mkdir -p "$SCRATCH/empty"
# mkfs an ext4 partition image of exactly $2 bytes from staged dir $1.
+13 -4
View File
@@ -20,7 +20,16 @@ Then:
cp <flare-edge>/target/armv7-unknown-linux-musleabihf/release/warden-flared qemu/payload/
```
Stage-2 init starts `warden-flared`, `warden-modbus`, and `warden-ui` (the
UI also needs `--display on|headless` + the virt.fragment kernel for
/dev/fb0) automatically when present (logs land in `/tmp/<name>.log` inside
the guest). An empty payload is valid: the image boots busybox-only.
Or take them from a panel that already runs them: every daemon is a static
musl binary, so flare-edge `tools/rig-pull-daemons.sh --host <panel>` copies
`/usr/bin/warden-*` straight into this directory (recording the panel's
release and each md5 in `PULLED.txt`), minus `warden-ui`, which the rig
needs as the fbdev+evdev build.
Stage-2 init starts every `warden-*` daemon it finds here, in the panel's
own S-number order (ai, automation, modbus, mikrotik, asic, starlink,
stratum, flared), then `warden-ui` (the UI also needs `--display
on|headless` + the virt.fragment kernel for /dev/fb0). Logs land in
`/tmp/<name>.log` inside the guest, readable over the control bridge with
`ctl @cat /tmp/<name>.log`. An empty payload is valid: the image boots
busybox-only.
+8
View File
@@ -0,0 +1,8 @@
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
mail:x:8:
www-data:x:33:
operator:x:37:
nogroup:x:65534:
+9
View File
@@ -0,0 +1,9 @@
root:x:0:0:root:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/false
bin:x:2:2:bin:/bin:/bin/false
sys:x:3:3:sys:/dev:/bin/false
sync:x:4:100:sync:/bin:/bin/sync
mail:x:8:8:mail:/var/spool/mail:/bin/false
www-data:x:33:33:www-data:/var/www:/bin/false
operator:x:37:37:Operator:/var:/bin/false
nobody:x:65534:65534:nobody:/home:/bin/false
+9
View File
@@ -0,0 +1,9 @@
root:$1$wardenrs$F.aYwbBz69WjSvwa3RNRF1:19664::::::
daemon:*:::::::
bin:*:::::::
sys:*:::::::
sync:*:::::::
mail:*:::::::
www-data:*:::::::
operator:*:::::::
nobody:*:::::::
+141 -4
View File
@@ -10,9 +10,26 @@
exec </dev/console >/dev/console 2>&1
/bin/busybox --install -s /bin
# mkfs.ext4 -d preserves the checkout owner's uid. Production owns shadow as
# root, and the UI intentionally rejects any other owner before verifying it.
chown 0:0 /etc/shadow
chmod 0600 /etc/shadow
root_hash="$(awk -F: '$1 == "root" { print $2 }' /etc/shadow)"
test_hash="$(printf '%s' root | /usr/bin/mkpasswd -m md5 -S wardenrs 2>/dev/null)"
if [ -z "$root_hash" ] || [ "$test_hash" != "$root_hash" ]; then
echo "FATAL: QEMU root credential verifier is unavailable"
poweroff -f
fi
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t tmpfs tmpfs /tmp
# The UI's Terminal page opens a PTY (posix_openpt), which needs devpts mounted
# and /dev/ptmx pointing into it. Without this the page can only ever report
# "no PTY available" -- it renders, so a screenshot scenario looks fine, while
# the one thing the page does is untestable in the VM.
mkdir -p /dev/pts
mount -t devpts devpts /dev/pts 2>/dev/null
# shellcheck source=qemu/rootfs/etc/warden-lib.sh disable=SC1091
. /etc/warden-lib.sh
@@ -41,8 +58,12 @@ mount_fatal "oem${slot}" /oem
mkdir -p /userdata/warden
# RS485: warden-modbus hardcodes /dev/ttyS4 at compile time; alias it to the
# VM's pci-serial UART when one is present (needs the virt.fragment kernel).
[ -c /dev/ttyS0 ] && ln -sf /dev/ttyS0 /dev/ttyS4
# VM's RS485 pci-serial UART (needs the virt.fragment kernel). run.sh lists
# the control channel's port first when there is one (warden.ctl on the
# cmdline), so the RS485 UART is ttyS1 then and ttyS0 otherwise. Aliasing
# ttyS0 blindly put Modbus polls on the control channel.
if grep -qw warden.ctl /proc/cmdline; then rs485="/dev/ttyS1"; else rs485="/dev/ttyS0"; fi
[ -c "$rs485" ] && ln -sf "$rs485" /dev/ttyS4
# Network: slirp user-mode net on eth0 (DHCP, fallback to QEMU's static map).
# The fallback keys off the interface actually having an address: udhcpc
@@ -74,13 +95,26 @@ export WARDEN_HPMCU=0
# post-apply "reboot" surfaces as a clean flared error; the scenario harness
# performs the actual reboot into the applied slot.
export WARDEN_HARD_RESET=0
# Production init scripts create this volatile socket directory before their
# daemons start. The compact QEMU rootfs must provide the same contract.
mkdir -p /run/warden
# OTA apply is opt-in per boot (run.sh --allow-apply): writing rootfs_b is
# safe inside disk.img but must never be the default posture.
if grep -qw warden.fwapply /proc/cmdline; then
export WARDEN_FW_ALLOW_APPLY=1
echo "init: OTA APPLY ENABLED (warden.fwapply)"
fi
for d in /usr/bin/warden-flared /usr/bin/warden-modbus; do
# The same daemons the panel's SysV scripts start, in their S-number order
# (S92 ai, S93 automation, S94 modbus, S95 mikrotik, S96 asic/starlink/
# stratum, S97 flared), each with no arguments and no environment, exactly
# as start-stop-daemon runs them there. A daemon that cannot live on -M virt
# (warden-ai wants the NPU) exits into its log and the UI shows the same
# "not running" it would show on a panel whose daemon died; that is the
# panel's behaviour, not a rig substitute for it. warden-watchdog stays out:
# it and a payload flared do not mix (README).
for d in /usr/bin/warden-ai /usr/bin/warden-automation /usr/bin/warden-modbus \
/usr/bin/warden-mikrotik /usr/bin/warden-asic /usr/bin/warden-starlink \
/usr/bin/warden-stratum /usr/bin/warden-flared; do
if [ -x "$d" ]; then
name="$(basename "$d")"
echo "init: starting $name"
@@ -93,7 +127,110 @@ done
# virt.fragment kernel.
if [ -x /usr/bin/warden-ui ] && [ -c /dev/fb0 ]; then
echo "init: starting warden-ui (fbdev)"
/usr/bin/warden-ui > /tmp/warden-ui.log 2>&1 &
# Announce the exit on the CONSOLE, not just in the log. A UI that dies
# mid-scenario otherwise looks exactly like a UI that stopped repainting:
# the framebuffer holds its last frame, screendumps keep working, and the
# scenario reports a stale picture as the current state. With this, a crash
# is one grep away for any scenario driving the VM from outside, and the
# signal or status that caused it is on the line.
(
# tee, not a plain redirect: the UI's own log (LV_LOG_USER and friends)
# is the most useful thing there is when a scenario does not do what it
# should, and a scenario driving the VM from outside can only see the
# CONSOLE. The file is kept as well so the exit dump below still works.
/usr/bin/warden-ui 2>&1 | tee /tmp/warden-ui.log
rc=$?
echo "init: warden-ui EXITED rc=$rc"
# 128+n is a signal death (139 = SIGSEGV); dump the tail so the
# scenario's console log carries the UI's own last words.
echo "init: warden-ui log tail:"
tail -n 20 /tmp/warden-ui.log 2>/dev/null
) &
fi
# Control bridge: the UI's debug FIFO, reachable from OUTSIDE the VM.
#
# warden-ui answers nav/page/stats/hit on /tmp/warden-ui.ctl (warden_debug.c),
# but that FIFO lives in here and a scenario drives the VM from the host. On
# real hardware the same channel is reached over SSH (flare-edge
# tools/warden-ctl); this initramfs is busybox-only and has no sshd, so the
# equivalent seam is a second 16550 that run.sh --ctl exposes as a unix socket
# (pci-serial, the same device the RS485 bridge already rides). One command
# per line in, the FIFO's reply out, and a sentinel line so the reader knows
# the reply is complete without a timeout. The FIFO vocabulary itself is
# identical on both sides of that seam, which is what lets one flow script
# run against the sim and against a panel.
#
# One exception, answered by the bridge itself and never forwarded to the
# FIFO: `@cat PATH` replies with PATH's contents (or one "bridge: no such
# file: PATH" line if it is missing), then the same sentinel. This is how the
# json flow channel reads webstatus.c's /tmp/warden-web-status.json snapshot
# from OUTSIDE the VM -- on a panel that file is just as reachable over the
# SSH session tools/warden-ctl already has, so hardware needs no equivalent.
#
# run.sh lists the ctl port before any other pci-serial, so it is always the
# first 8250, and it says so with warden.ctl on the command line. The marker,
# not the mere presence of a ttyS0, is what arms the bridge: a VM launched
# with --rs485 alone also has a ttyS0, and that one is the Modbus wire.
if grep -qw warden.ctl /proc/cmdline && [ -c /dev/ttyS0 ]; then
ctl=/dev/ttyS0
echo "init: control bridge on $ctl"
(
# Opened ONCE, read-write, on fd 3. Reopening a serial port per line
# can block on carrier detect; one open at bridge start either works
# or fails visibly on the console. The tty stays in its default cooked
# mode: the host discards echoed input, and a whole line arrives per
# read.
exec 3<> "$ctl"
while IFS= read -r cmd <&3; do
[ -n "$cmd" ] || continue
case "$cmd" in
"@cat "*)
# A bridge-local command, never forwarded to warden-ui's
# FIFO: `@cat PATH` reads PATH directly off the GUEST's
# own filesystem and answers with it, which is how the
# json flow channel gets webstatus.c's snapshot out to
# the host driving the VM from outside. `-f` so a
# directory or device node reports as missing rather than
# cat hanging or erroring oddly.
path="${cmd#@cat }"
if [ -f "$path" ]; then
cat "$path" >&3
# Force a newline after the file's own bytes: the
# status json (webstatus.c) is written with NO
# trailing newline, and without this the sentinel
# below would land on the SAME line as the content
# and the reader (qmp.py Ctl.send, line-based) would
# block forever waiting for a line that never comes.
echo >&3
else
echo "bridge: no such file: $path" >&3
fi
;;
*)
if [ -p /tmp/warden-ui.ctl ]; then
# Remove the previous reply BEFORE sending, then wait
# for the new one to appear (the UI renames it into
# place whole). A fixed settle used to hand back the
# previous command's reply whenever a page took longer
# than 0.3 s to build (flare-edge #152); this is the
# same recipe tools/warden-ctl uses over SSH.
rm -f /tmp/warden-ui.dbg
printf '%s\n' "$cmd" > /tmp/warden-ui.ctl
n=0
while [ ! -s /tmp/warden-ui.dbg ] && [ "$n" -lt 100 ]; do
sleep 0.05
n=$((n + 1))
done
cat /tmp/warden-ui.dbg 2>/dev/null >&3
else
echo "bridge: warden-ui control FIFO not present" >&3
fi
;;
esac
echo "<<END>>" >&3
done
) &
fi
if grep -qw warden.shell /proc/cmdline; then
+34 -4
View File
@@ -14,6 +14,9 @@
# "device boots believing 2021" incident class
# --rs485 SOCK unix socket chardev for the RS485/Modbus bridge
# (pci-serial: needs the virt.fragment kernel)
# --ctl SOCK unix socket to warden-ui's debug channel (nav/page/hit),
# bridged by init from a pci-serial port; the rig-side
# twin of tools/warden-ctl over SSH on a real panel
# --watchdog add i6300esb watchdog, reset on expiry (fragment kernel)
# --qmp SOCK QMP unix socket (screendump, input-send-event, quit)
# --display MODE off (default, -nographic) | on (gtk window) | headless
@@ -32,7 +35,7 @@ QEMU_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
OUT="${OUT:-$QEMU_DIR/out}"
KERNEL="" INITRD="$OUT/initramfs.cpio.gz" DISK="" NO_DISK=0 SLOT="_a"
RTC="" RS485="" WATCHDOG=0 QMP="" DISPLAY_MODE="off" SHELL_FLAG=0 ALLOW_APPLY=0
RTC="" RS485="" CTL="" WATCHDOG=0 QMP="" DISPLAY_MODE="off" SHELL_FLAG=0 ALLOW_APPLY=0
SSH_PORT=2222 HTTP_PORT=8080 API_PORT=28443
EXTRA=()
@@ -45,6 +48,7 @@ while [ $# -gt 0 ]; do
--slot) SLOT="${2:?}"; shift 2 ;;
--rtc) RTC="${2:?}"; shift 2 ;;
--rs485) RS485="${2:?}"; shift 2 ;;
--ctl) CTL="${2:?}"; shift 2 ;;
--watchdog) WATCHDOG=1; shift ;;
--qmp) QMP="${2:?}"; shift 2 ;;
--display) DISPLAY_MODE="${2:?}"; shift 2 ;;
@@ -78,7 +82,12 @@ fi
# NOTE: never add `earlyprintk`: the config's DEBUG_UART_PHYS is the RV1106's
# 0xff4c0000, which does not exist on -M virt.
APPEND="console=ttyAMA0 rdinit=/init"
# vt.global_cursor_default=0: the virt kernel binds fbcon to the virtio-gpu
# framebuffer, and its blinking cursor lands on /dev/fb0 under warden-ui (an
# 8x2 block at the top-left, present in one boot and absent in the next). The
# panel has no fbcon on its display, and a pixel reference captured with the
# cursor in frame fails without it (issue #18).
APPEND="console=ttyAMA0 rdinit=/init vt.global_cursor_default=0"
# Port 0 disables a forward. A boot smoke needs no host ports and must not
# fail on a busy default port.
NETDEV="user,id=n0"
@@ -102,11 +111,32 @@ if [ -n "$DISK" ] && [ "$NO_DISK" -eq 0 ]; then
-device "virtio-blk-device,drive=vd0" )
fi
[ "$SHELL_FLAG" -eq 1 ] && APPEND="$APPEND warden.shell"
# Tells init the FIRST 8250 is the control bridge. Without this marker init
# bridges nothing, so a VM launched with --rs485 alone never has its Modbus
# wire mistaken for the debug channel.
[ -n "$CTL" ] && APPEND="$APPEND warden.ctl"
[ "$ALLOW_APPLY" -eq 1 ] && APPEND="$APPEND warden.fwapply"
[ -n "$RTC" ] && ARGS+=( -rtc "base=$RTC" )
[ "$WATCHDOG" -eq 1 ] && ARGS+=( -device i6300esb -action watchdog=reset )
[ -n "$RS485" ] && ARGS+=( -chardev "socket,id=rs485,path=$RS485,server=on,wait=off"
-device "pci-serial,chardev=rs485" )
# The UI's debug channel, bridged out of the VM: rootfs/sbin/init relays lines
# between this port and warden-ui's control FIFO (see qmp.py --ctl). Same
# pci-serial the RS485 bridge rides. Listed BEFORE rs485 so that, whichever
# combination is requested, this port enumerates as ttyS0 -- the first port
# init's bridge probes. Put rs485 first and the bridge would be talking to the
# Modbus wire.
[ -n "$CTL" ] && ARGS+=( -chardev "socket,id=ctl,path=$CTL,server=on,wait=off"
-device "pci-serial,chardev=ctl" )
# Without --rs485 the UART is still there, wired to nothing: warden-modbus
# then polls a port that never answers, which is what a panel with an empty
# RS485 header looks like, rather than failing to open a port that does not
# exist (a different UI state) or, worse, being aliased onto the control
# channel by init (which happened: Modbus frames in the wake reply).
if [ -n "$RS485" ]; then
ARGS+=( -chardev "socket,id=rs485,path=$RS485,server=on,wait=off"
-device "pci-serial,chardev=rs485" )
else
ARGS+=( -chardev "null,id=rs485" -device "pci-serial,chardev=rs485" )
fi
[ -n "$QMP" ] && ARGS+=( -qmp "unix:$QMP,server=on,wait=off" )
case "$DISPLAY_MODE" in
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env bash
set -euo pipefail
TEST_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
QEMU_DIR="$(cd "$TEST_DIR/.." && pwd)"
SCRATCH="$(mktemp -d "${TMPDIR:-/tmp}/busybox-fetch.XXXXXX")"
trap 'rm -rf "$SCRATCH"' EXIT
FAKE_BIN="$SCRATCH/bin"
FAKE_CURL_COUNT="$SCRATCH/curl-count"
mkdir -p "$FAKE_BIN" "$SCRATCH/qemu"
printf 'good payload' > "$SCRATCH/good"
sha256sum "$SCRATCH/good" | awk '{print $1}' > "$SCRATCH/qemu/busybox.sha256"
FAKE_CURL="$FAKE_BIN/curl"
apply_fake_curl() {
chmod 0755 "$FAKE_CURL"
export PATH="$FAKE_BIN:$PATH"
export FAKE_CURL_COUNT FAKE_CURL_MODE
}
# The path is resolved at runtime from this file's location.
# shellcheck disable=SC1091
source "$QEMU_DIR/lib.sh"
QEMU_DIR="$SCRATCH/qemu"
# Read by qemu_get_busybox from the sourced helper.
# shellcheck disable=SC2034
BB_URL="https://invalid.example/busybox"
# These single-quoted lines are the literal source of the fake curl program.
# shellcheck disable=SC2016
printf '%s\n' '#!/bin/sh' \
'count=0' \
'[ ! -f "$FAKE_CURL_COUNT" ] || count=$(cat "$FAKE_CURL_COUNT")' \
'count=$((count + 1))' \
'printf "%s\n" "$count" > "$FAKE_CURL_COUNT"' \
'out=' \
'while [ "$#" -gt 0 ]; do' \
' if [ "$1" = "-o" ]; then shift; out="$1"; fi' \
' shift' \
'done' \
'[ -n "$out" ] || exit 2' \
'if [ "$FAKE_CURL_MODE" = "flaky" ] && [ "$count" -gt 1 ]; then' \
' printf "good payload" > "$out"' \
'else' \
' printf "bad payload" > "$out"' \
'fi' > "$FAKE_CURL"
FAKE_CURL_MODE=flaky
apply_fake_curl
OUT="$SCRATCH/flaky" qemu_get_busybox
test "$(cat "$FAKE_CURL_COUNT")" = 2
test "$(sha256sum "$BB" | awk '{print $1}')" = "$(cat "$QEMU_DIR/busybox.sha256")"
printf '0\n' > "$FAKE_CURL_COUNT"
FAKE_CURL_MODE=bad
apply_fake_curl
if (OUT="$SCRATCH/always-bad" qemu_get_busybox) >"$SCRATCH/failure.log" 2>&1; then
echo "FAIL: an unverified download was accepted" >&2
exit 1
fi
test "$(cat "$FAKE_CURL_COUNT")" = 3
test ! -e "$SCRATCH/always-bad/busybox-armv7l"
grep -q 'after 3 verified attempts' "$SCRATCH/failure.log"
printf '0\n' > "$FAKE_CURL_COUNT"
mkdir -p "$SCRATCH/cached"
cp "$SCRATCH/good" "$SCRATCH/cached/busybox-armv7l"
OUT="$SCRATCH/cached" qemu_get_busybox
test "$(cat "$FAKE_CURL_COUNT")" = 0
echo "busybox fetch tests passed"
+563
View File
@@ -0,0 +1,563 @@
#!/usr/bin/env python3
"""PPM loading, cropping and image comparison for UI flow region asserts.
Standalone (no VM, no other module in this repo needed): a screendump is
QEMU's own P6 PPM file, and everything below works straight off that file
plus a JSON reference. Pure Python plus Pillow only.
imgtools.py compare REFS NAME SHOT.ppm
imgtools.py capture REFS NAME SHOT.ppm X Y W H TOL # TOL: exact|loose|structural
imgtools.py hash SHOT.ppm
imgtools.py bench [SHOT.ppm] [N] # per-call phash/structural timing, N iterations
imgtools.py selftest
`compare` and `capture` read/write a reference file: JSON mapping a region
name to {"x","y","w","h","tolerance","phash","structural"}. `capture` always
fills in both hashes so an existing entry's tolerance class can be changed
later without re-shooting the region.
Public API (imported directly by the driver, not just via the CLI above):
load_ppm(path) -> (w, h, rgb_bytes)
crop(img, x, y, w, h) -> img
phash(img) -> int 8x8 DCT of a 32x32 grey downscale, 64-bit
structural(img) -> bytes 16x16 edge/nonblack occupancy mask, 32 bytes
compare(ref_entry, cur_img, tolerance) -> (ok, detail)
`img` is always the (w, h, rgb_bytes) triple load_ppm/crop return -- there is
no separate image type. rgb_bytes is tightly packed row-major RGB, 3 bytes
per pixel, no padding, matching both PPM P6 and PIL's "RGB" raw layout.
"""
import math
import os
import sys
import json
from PIL import Image, ImageFilter
# occupancy thresholds for structural(): a cell is occupied when its grey is
# this far from the crop's median (its background), or its FIND_EDGES energy
# exceeds the edge threshold. Both are 0..255 greyscale/edge-magnitude
# averages over the cell.
# Validated on real captures: a switch knob left/right differs in 240/256
# cells, a dark card's icon and text stand out from its (7,13,29) ground.
DEVIATION_THRESHOLD = 28
EDGE_THRESHOLD = 24
# ---------------------------------------------------------------------------
# PPM
# ---------------------------------------------------------------------------
def load_ppm(path):
"""Read a binary PPM (P6), return (w, h, rgb_bytes).
Delegates to Pillow's own P6 decoder (Image.open, format auto-detected
from the magic) instead of re-parsing the PPM grammar by hand: skipping
whitespace runs and '#' comments, and normalising any maxval to 8-bit,
are Pillow's problem here, not ours. convert("RGB") guarantees the
tightly-packed 3-bytes-per-pixel layout every caller in this file
assumes regardless of the source channel depth.
"""
with Image.open(path) as im:
im = im.convert("RGB")
return im.width, im.height, im.tobytes()
def crop(img, x, y, w, h):
"""Crop (W,H,rgb) to the x,y,w,h box, return a new (w,h,rgb) image.
The bounds check stays explicit and fails loudly (see main()'s own
comment on that) rather than delegating to PIL's crop, which silently
zero-pads a box that runs outside the source image instead of raising --
exactly the kind of stale/mistyped region box this check exists to catch.
"""
width, height, data = img
if x < 0 or y < 0 or w <= 0 or h <= 0 or x + w > width or y + h > height:
raise ValueError(f"crop box {x},{y},{w}x{h} outside image {width}x{height}")
cropped = _to_pil(img).crop((x, y, x + w, y + h))
return w, h, cropped.tobytes()
def _to_pil(img):
w, h, data = img
return Image.frombytes("RGB", (w, h), data)
# ---------------------------------------------------------------------------
# perceptual hash (phash): 8x8 DCT of a 32x32 grey downscale
# ---------------------------------------------------------------------------
def _dct_basis(n):
"""n x n DCT-II basis matrix. Unnormalised -- fine since phash only
thresholds coefficients against each other, never compares magnitudes
across images."""
return [[math.cos(math.pi / n * (col + 0.5) * row) for col in range(n)] for row in range(n)]
_DCT32 = _dct_basis(32)
# phash() only ever reads the low-frequency _DCT_KEEP x _DCT_KEEP corner of
# the 32x32 DCT (see its docstring), so _dct2d_32 stops both passes at this
# many frequencies instead of computing all 1024 coefficients -- same 64
# coefficients out, roughly 6x less pure-Python multiply-add work per call.
_DCT_KEEP = 8
def _dct2d_32(rows):
"""2D DCT-II of a 32x32 list-of-lists via two separable 1D passes,
truncated to the low-frequency _DCT_KEEP x _DCT_KEEP corner -- the only
coefficients any caller reads."""
n = 32
# columns first: tmp[u][x] = DCT of column x at frequency u. Frequencies
# u >= _DCT_KEEP never feed a returned coefficient, so skip them.
tmp = [[0.0] * n for _ in range(_DCT_KEEP)]
for u in range(_DCT_KEEP):
b = _DCT32[u]
for x in range(n):
s = 0.0
for y in range(n):
s += rows[y][x] * b[y]
tmp[u][x] = s
# then rows: out[u][v] = DCT of tmp's row u at frequency v. Same cutoff
# on v; the y/x summations stay full since each kept coefficient still
# needs the whole 32-wide signal.
out = [[0.0] * _DCT_KEEP for _ in range(_DCT_KEEP)]
for u in range(_DCT_KEEP):
row = tmp[u]
for v in range(_DCT_KEEP):
b = _DCT32[v]
s = 0.0
for x in range(n):
s += row[x] * b[x]
out[u][v] = s
return out
def phash(img):
"""8x8-DCT perceptual hash of a 32x32 greyscale downscale, as a 64-bit
int. Classic pHash recipe: downscale, DCT, keep the low-frequency 8x8
corner, threshold each coefficient against the block's mean (DC term
excluded from the mean -- it is just overall brightness and would bias
every bit the same way)."""
small = _to_pil(img).convert("L").resize((32, 32), Image.LANCZOS)
px = small.load()
rows = [[px[x, y] for x in range(32)] for y in range(32)]
coeffs = _dct2d_32(rows)
block = [coeffs[u][v] for u in range(_DCT_KEEP) for v in range(_DCT_KEEP)]
mean = sum(block[1:]) / (len(block) - 1)
bits = 0
for c in block:
bits = (bits << 1) | (1 if c > mean else 0)
return bits
def hamming(a, b):
"""Bit-differences between two phash ints."""
return (a ^ b).bit_count()
# ---------------------------------------------------------------------------
# structural occupancy hash: 16x16 edge/nonblack mask
# ---------------------------------------------------------------------------
def structural(img):
"""16x16 binary occupancy mask as 32 bytes (256 bits, MSB first,
row-major). A cell is "occupied" if it carries ink relative to the crop's
OWN background -- its grey deviates from the crop's median by more than
DEVIATION_THRESHOLD -- or sits on an edge, so the mask is robust to a
recolour (still occupied) but sensitive to a shape disappearing (goes
from occupied to empty).
Relative to the median, not to black: the WardenOS page background is
(7,13,29), grey 16, and an absolute non-black test read every cell of
every region as occupied, so no structural check could ever fail
(SDK #19, flare-edge #156). Measured on real captures with this rule: a
Bluetooth switch off/on differs in 240 of 256 cells, a dark card reads
its icon and text and nothing else."""
grey = _to_pil(img).convert("L")
edges = grey.filter(ImageFilter.FIND_EDGES)
grey_small = grey.resize((16, 16), Image.BOX)
edge_small = edges.resize((16, 16), Image.BOX)
median = _median_grey(grey)
gpx, epx = grey_small.load(), edge_small.load()
bits = bytearray(32)
idx = 0
for y in range(16):
for x in range(16):
occupied = abs(gpx[x, y] - median) > DEVIATION_THRESHOLD or epx[x, y] > EDGE_THRESHOLD
if occupied:
bits[idx // 8] |= 1 << (7 - (idx % 8))
idx += 1
return bytes(bits)
def _median_grey(grey):
"""The crop's dominant luminance: the background of a card, the fill of a
switch, whatever most of the pixels are."""
hist = grey.histogram()
total = sum(hist)
acc = 0
for value, count in enumerate(hist):
acc += count
if acc * 2 >= total:
return value
return 0
def _structural_diff(a, b):
"""Count of differing bits between two 32-byte occupancy masks."""
return (int.from_bytes(a, "big") ^ int.from_bytes(b, "big")).bit_count()
# ---------------------------------------------------------------------------
# compare
# ---------------------------------------------------------------------------
def compare(ref_entry, cur_img, tolerance):
"""Compare cur_img (already cropped to the reference's box) against
ref_entry under the given tolerance class. Returns (ok, detail); detail
explains the verdict either way, for results.jsonl.
exact: phash hamming distance == 0
loose: phash hamming distance <= 6
structural: occupancy mask bytes equal (no distance, no tolerance)
"""
if tolerance in ("exact", "loose"):
if "phash" not in ref_entry:
return False, "reference missing 'phash' field"
try:
ref_hash = int(ref_entry["phash"], 16)
except (TypeError, ValueError):
return False, f"reference 'phash' is not valid hex: {ref_entry.get('phash')!r}"
cur_hash = phash(cur_img)
dist = hamming(ref_hash, cur_hash)
limit = 0 if tolerance == "exact" else 6
ok = dist <= limit
detail = f"phash hamming={dist} limit={limit} ref={ref_hash:016x} cur={cur_hash:016x}"
return ok, detail
if tolerance == "structural":
if "structural" not in ref_entry:
return False, "reference missing 'structural' field"
try:
ref_bits = bytes.fromhex(ref_entry["structural"])
except (TypeError, ValueError):
return False, f"reference 'structural' is not valid hex: {ref_entry.get('structural')!r}"
cur_bits = structural(cur_img)
ok = ref_bits == cur_bits
if ok:
detail = "structural mask matches (0/256 cells differ)"
else:
diff = _structural_diff(ref_bits, cur_bits)
detail = f"structural mask differs in {diff}/256 cells"
return ok, detail
return False, f"unknown tolerance class: {tolerance!r}"
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def _load_refs(path):
if not os.path.exists(path):
return {}
with open(path) as f:
return json.load(f)
def _save_refs(path, refs):
with open(path, "w") as f:
json.dump(refs, f, indent=2, sort_keys=True)
f.write("\n")
def cmd_compare(argv):
if len(argv) != 3:
print("usage: imgtools.py compare REFS NAME SHOT.ppm", file=sys.stderr)
return 2
refs_path, name, shot_path = argv
refs = _load_refs(refs_path)
if name not in refs:
print(f"FAIL {name}: no such reference (run capture first)")
return 1
entry = refs[name]
for key in ("x", "y", "w", "h", "tolerance"):
if key not in entry:
print(f"FAIL {name}: reference entry missing '{key}'")
return 1
w, h, data = load_ppm(shot_path)
cropped = crop((w, h, data), entry["x"], entry["y"], entry["w"], entry["h"])
ok, detail = compare(entry, cropped, entry["tolerance"])
print(f"{'PASS' if ok else 'FAIL'} {name}: {detail}")
return 0 if ok else 1
def cmd_capture(argv):
if len(argv) != 8:
print("usage: imgtools.py capture REFS NAME SHOT.ppm X Y W H TOL", file=sys.stderr)
return 2
refs_path, name, shot_path, x, y, w, h, tol = argv
if tol not in ("exact", "loose", "structural"):
print(f"usage: TOL must be exact|loose|structural, got {tol!r}", file=sys.stderr)
return 2
x, y, w, h = int(x), int(y), int(w), int(h)
iw, ih, data = load_ppm(shot_path)
cropped = crop((iw, ih, data), x, y, w, h)
entry = {
"x": x, "y": y, "w": w, "h": h,
"tolerance": tol,
"phash": f"{phash(cropped):016x}",
"structural": structural(cropped).hex(),
}
refs = _load_refs(refs_path)
refs[name] = entry
_save_refs(refs_path, refs)
print(f"captured {name}: {entry}")
return 0
def cmd_hash(argv):
if len(argv) != 1:
print("usage: imgtools.py hash SHOT.ppm", file=sys.stderr)
return 2
w, h, data = load_ppm(argv[0])
img = (w, h, data)
print(f"{w}x{h} phash={phash(img):016x} structural={structural(img).hex()}")
return 0
def cmd_bench(argv):
"""Time phash()/structural() -- the per-step cost of every
assert_region/wait_region/capture_region in a flow run (wait_region polls
at 0.5s intervals, calling back into these on every poll), so a change to
the DCT size, the downscale filter, or the occupancy thresholds shows up
as a number here instead of only as a slower flow run nobody investigates.
usage: imgtools.py bench [SHOT.ppm] [N]
SHOT.ppm: a real screendump to benchmark against a representative crop
of; default is a synthetic 720x720 image (the virt.fragment screendump
size) so bench never depends on a committed fixture or a live rig.
N: iterations per function, default 200.
"""
import timeit
if len(argv) > 2:
print("usage: imgtools.py bench [SHOT.ppm] [N]", file=sys.stderr)
return 2
shot_path = next((a for a in argv if not a.isdigit()), None)
n = int(next((a for a in argv if a.isdigit()), "200"))
if shot_path:
w, h, data = load_ppm(shot_path)
else:
w, h = 720, 720
data = _make_test_rgb(w, h, (220, 20, 20))
img = (w, h, data)
# a representative crop, not the whole screen: every real region assert
# crops first, and structural()'s edge filter cost scales with crop
# size even though phash's fixed 32x32 downscale mostly doesn't.
cw, ch = min(200, w), min(200, h)
region = crop(img, 0, 0, cw, ch)
def percentiles(samples):
s = sorted(samples)
p50 = s[min(len(s) - 1, int(len(s) * 0.50))]
p95 = s[min(len(s) - 1, int(len(s) * 0.95))]
return s[0], p50, p95, s[-1]
print(f"bench: {cw}x{ch} region, n={n} iterations")
for label, fn in (("phash", lambda: phash(region)),
("structural", lambda: structural(region))):
for _ in range(5):
fn() # warm up: first call can carry one-off costs (e.g. module-
# level caches settling) that don't belong in the steady-state
# population below
samples = timeit.repeat(fn, repeat=n, number=1)
lo, p50, p95, hi = percentiles(samples)
print(f" {label:<10} min={lo * 1000:7.3f}ms p50={p50 * 1000:7.3f}ms "
f"p95={p95 * 1000:7.3f}ms max={hi * 1000:7.3f}ms")
return 0
# ---------------------------------------------------------------------------
# selftest
# ---------------------------------------------------------------------------
def _make_test_rgb(w, h, colour, shift=(0, 0)):
"""Black canvas with a coloured square near one corner: the self-test's
synthetic images, and bench's default when it isn't given a real
SHOT.ppm. colour=None means no square at all (removed shape)."""
from PIL import ImageDraw
img = Image.new("RGB", (w, h), (0, 0, 0))
if colour is not None:
draw = ImageDraw.Draw(img)
ox, oy = shift
x0, y0 = 8 + ox, 8 + oy
draw.rectangle([x0, y0, x0 + 20, y0 + 20], fill=colour)
return img.tobytes()
def _write_ppm(path, w, h, rgb_bytes):
with open(path, "wb") as f:
f.write(f"P6\n{w} {h}\n255\n".encode("ascii"))
f.write(rgb_bytes)
def _dct2d_32_brute(rows):
"""Unoptimised reference DCT: the full 32x32 transform with no early
cutoff. selftest()-only, so _dct2d_32's _DCT_KEEP truncation has
something independent to be checked against -- a future edit that moves
the cutoff on the wrong loop would otherwise change captured phash bits
with nothing catching it."""
n = 32
tmp = [[0.0] * n for _ in range(n)]
for u in range(n):
b = _DCT32[u]
for x in range(n):
s = 0.0
for y in range(n):
s += rows[y][x] * b[y]
tmp[u][x] = s
out = [[0.0] * n for _ in range(n)]
for u in range(n):
row = tmp[u]
for v in range(n):
b = _DCT32[v]
s = 0.0
for x in range(n):
s += row[x] * b[x]
out[u][v] = s
return out
def selftest():
import tempfile
w, h = 128, 128
failures = []
def check(label, cond):
print(f"[{'ok' if cond else 'FAIL'}] {label}")
if not cond:
failures.append(label)
with tempfile.TemporaryDirectory() as tmp:
refs_path = os.path.join(tmp, "refs.json")
base_path = os.path.join(tmp, "base.ppm")
_write_ppm(base_path, w, h, _make_test_rgb(w, h, (220, 20, 20)))
# capture the same corner box under all three tolerance classes
for tol in ("exact", "loose", "structural"):
rc = main(["capture", refs_path, f"corner-{tol}", base_path, "4", "4", "32", "32", tol])
check(f"capture corner-{tol} succeeds", rc == 0)
# load_ppm/crop sanity: the drawn pixel lands where expected
bw, bh, bdata = load_ppm(base_path)
check("load_ppm reads back the written size", (bw, bh) == (w, h))
cropped = crop((bw, bh, bdata), 4, 4, 32, 32)
px_off = ((8 - 4) * 32 + (8 - 4)) * 3
check("crop keeps the drawn pixel in place",
cropped[2][px_off:px_off + 3] == bytes((220, 20, 20)))
# a header QEMU never writes but the P6 grammar allows: a '#'
# comment line and a non-255 maxval. Pillow's decoder (load_ppm no
# longer hand-parses the header) must still read it correctly.
odd_path = os.path.join(tmp, "odd-header.ppm")
with open(odd_path, "wb") as f:
f.write(b"P6\n# generated for a selftest, not by QEMU\n2 2\n100\n")
f.write(bytes((100, 0, 0, 0, 100, 0, 0, 0, 100, 100, 100, 100)))
ow, oh, odata = load_ppm(odd_path)
check("load_ppm reads width/height past a comment line", (ow, oh) == (2, 2))
check("load_ppm scales a non-255 maxval channel to 8-bit",
odata[0:3] == bytes((255, 0, 0)))
# crop box outside the image must still fail loudly, not silently
# zero-pad the way PIL's own Image.crop does
try:
crop((bw, bh, bdata), bw - 4, bh - 4, 32, 32)
check("crop rejects a box past the image edge", False)
except ValueError:
check("crop rejects a box past the image edge", True)
# _dct2d_32's _DCT_KEEP truncation must land on the exact same
# coefficients an untruncated 32x32 DCT would produce
small = _to_pil((bw, bh, bdata)).convert("L").resize((32, 32), Image.LANCZOS)
px = small.load()
dct_rows = [[px[x, y] for x in range(32)] for y in range(32)]
fast, brute = _dct2d_32(dct_rows), _dct2d_32_brute(dct_rows)
check("_dct2d_32's truncated corner matches the untruncated DCT",
all(fast[u][v] == brute[u][v]
for u in range(_DCT_KEEP) for v in range(_DCT_KEEP)))
# identical shot: exact must pass
same_path = os.path.join(tmp, "same.ppm")
_write_ppm(same_path, w, h, _make_test_rgb(w, h, (220, 20, 20)))
rc = main(["compare", refs_path, "corner-exact", same_path])
check("exact passes for an identical shot", rc == 0)
# small shift: exact must fail, loose must still pass
shift_path = os.path.join(tmp, "shift.ppm")
_write_ppm(shift_path, w, h, _make_test_rgb(w, h, (220, 20, 20), shift=(1, 1)))
rc = main(["compare", refs_path, "corner-exact", shift_path])
check("exact fails for a slight shift", rc == 1)
rc = main(["compare", refs_path, "corner-loose", shift_path])
check("loose passes for a slight shift", rc == 0)
# colour change, same footprint: structural must still pass
colour_path = os.path.join(tmp, "colour.ppm")
_write_ppm(colour_path, w, h, _make_test_rgb(w, h, (20, 20, 220)))
rc = main(["compare", refs_path, "corner-structural", colour_path])
check("structural passes for a colour change", rc == 0)
# shape removed entirely: structural must fail
removed_path = os.path.join(tmp, "removed.ppm")
_write_ppm(removed_path, w, h, _make_test_rgb(w, h, None))
rc = main(["compare", refs_path, "corner-structural", removed_path])
check("structural fails when the shape is removed", rc == 1)
# bad-path coverage
rc = main(["compare", refs_path, "no-such-name", base_path])
check("compare on an unknown reference name fails loudly", rc == 1)
rc = main(["hash", base_path])
check("hash on a real ppm succeeds", rc == 0)
rc = main(["bench", base_path, "3"])
check("bench runs to completion", rc == 0)
if failures:
print(f"\n{len(failures)} check(s) failed:")
for label in failures:
print(f" - {label}")
return 1
print("\nall checks passed")
return 0
# ---------------------------------------------------------------------------
def main(argv):
if not argv:
print(__doc__, file=sys.stderr)
return 2
cmd, rest = argv[0], argv[1:]
if cmd == "selftest":
return selftest()
handlers = {"compare": cmd_compare, "capture": cmd_capture, "hash": cmd_hash, "bench": cmd_bench}
if cmd not in handlers:
print(f"unknown command: {cmd!r}", file=sys.stderr)
return 2
# A missing file, unreadable ref JSON, or malformed PPM/crop box is a
# prerequisite failure -- report it plainly and fail loudly (exit 1),
# never let it fall through as a silent pass or a raw traceback.
try:
return handlers[cmd](rest)
except (OSError, ValueError, json.JSONDecodeError) as exc:
print(f"FATAL: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+1169 -15
View File
File diff suppressed because it is too large Load Diff
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# Offline regression test for run.sh's own argv construction: no real QEMU,
# no kernel image. A fake `qemu-system-arm` placed first on PATH dumps the
# argv it was handed (one token per line) and exits, so this pins the exact
# contract run.sh:121-138 and rootfs/sbin/init:54,161 share without either
# side moving: init decides ttyS0 vs ttyS1 for the Modbus alias purely by
# grepping warden.ctl off /proc/cmdline, so run.sh has to keep two promises
# every single invocation -- the ctl pci-serial device, when present, comes
# BEFORE the rs485 one in argv (virt's PCI bus enumerates in that order),
# and an rs485 pci-serial device (real or null-backed) is always there so
# the port count init relies on never shifts.
#
# What is worth pinning: nothing else exercises this. test-ui-drive-rs485.sh
# stubs run.sh out entirely (a fake VM), and the only real boot in CI
# (boot-smoke.sh) passes neither --ctl nor --rs485, so a swapped
# `[ -n "$CTL" ]`/`[ -n "$RS485" ]` block, or a dropped null-chardev
# fallback, would reach a panel as Modbus polls landing on the debug channel
# (run.sh:131-133's own incident) before anything here caught it.
#
# bash run-sh-args-test.sh
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RUN_SH="$HERE/../run.sh"
SCRATCH="$(mktemp -d /tmp/runshargs.XXXXXX)"
trap 'rm -rf "$SCRATCH"' EXIT
FAIL=0
pass() { printf '[PASS] %s\n' "$1"; }
fail() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
KERNEL="$SCRATCH/fake-zImage"
INITRD="$SCRATCH/fake-initramfs.cpio.gz"
: > "$KERNEL"
: > "$INITRD"
BIN="$SCRATCH/bin"
mkdir -p "$BIN"
cat > "$BIN/qemu-system-arm" <<'STUB'
#!/usr/bin/env bash
# Stand-in for the real binary: record argv, one token per line, and exit
# straight away. $ARGV_CAPTURE names where -- run.sh always `exec`s this as
# its very last step, so nothing downstream of it ever runs.
printf '%s\n' "$@" > "$ARGV_CAPTURE"
STUB
chmod +x "$BIN/qemu-system-arm"
# run_case ARGV_FILE EXTRA_ARGS...: invoke the real run.sh --no-disk (so
# nothing under the real qemu/out/ is ever touched) with the fake binary
# first on PATH, capturing its argv into ARGV_FILE. Fails the case loudly if
# run.sh itself exits nonzero -- a silent empty capture would otherwise look
# just like "the assertions below simply found nothing".
run_case() {
local argv_file="$1"; shift
local out rc
out="$(cd "$SCRATCH" && PATH="$BIN:$PATH" ARGV_CAPTURE="$argv_file" \
bash "$RUN_SH" --kernel "$KERNEL" --initrd "$INITRD" --no-disk "$@" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || { fail "run.sh exited $rc for: $* -- output: $out"; return 1; }
[ -s "$argv_file" ] || { fail "run.sh produced no captured argv for: $*"; return 1; }
return 0
}
# chardev_line ARGV_FILE PREFIX: 1-indexed line number of the first argv
# token starting with PREFIX (the socket/null chardev spec, which always
# immediately follows the "-chardev" token it belongs to), or empty.
chardev_line() { grep -n -m1 "^$2" "$1" | cut -d: -f1; }
# --- case: neither --ctl nor --rs485 -> null-backed rs485, no ctl device ---
argv="$SCRATCH/argv-neither.txt"
if run_case "$argv"; then
if ! grep -qF 'id=ctl' "$argv"; then
pass "neither flag: no ctl chardev/device at all"
else
fail "neither flag: a ctl chardev/device appeared unrequested"
fi
if grep -qF -- '-append' "$argv" && ! grep -qw 'warden.ctl' "$argv"; then
pass "neither flag: -append omits warden.ctl"
else
fail "neither flag: -append should omit warden.ctl"
fi
if [ -n "$(chardev_line "$argv" 'null,id=rs485')" ] \
&& grep -qF 'pci-serial,chardev=rs485' "$argv"; then
pass "neither flag: null-backed rs485 pci-serial device is still present"
else
fail "neither flag: expected a null-backed rs485 device (port count must not shift)"
fi
fi
# --- case: --rs485 alone -> real rs485 device, still no ctl device ---------
argv="$SCRATCH/argv-rs485-only.txt"
if run_case "$argv" --rs485 "$SCRATCH/rs.sock"; then
if ! grep -qF 'id=ctl' "$argv"; then
pass "rs485 only: no ctl chardev/device"
else
fail "rs485 only: a ctl chardev/device appeared unrequested"
fi
if ! grep -qw 'warden.ctl' "$argv"; then
pass "rs485 only: -append omits warden.ctl"
else
fail "rs485 only: -append should omit warden.ctl"
fi
if [ -n "$(chardev_line "$argv" "socket,id=rs485,path=$SCRATCH/rs.sock,")" ]; then
pass "rs485 only: rs485 chardev carries the requested socket path"
else
fail "rs485 only: rs485 chardev did not carry the requested socket path"
fi
fi
# --- case: --ctl alone -> ctl device first, null-backed rs485 still present,
# and warden.ctl on the cmdline ------------------------------------------
argv="$SCRATCH/argv-ctl-only.txt"
if run_case "$argv" --ctl "$SCRATCH/ctl.sock"; then
ctl_ln="$(chardev_line "$argv" "socket,id=ctl,path=$SCRATCH/ctl.sock,")"
rs_ln="$(chardev_line "$argv" 'null,id=rs485')"
if [ -n "$ctl_ln" ] && [ -n "$rs_ln" ] && [ "$ctl_ln" -lt "$rs_ln" ]; then
pass "ctl only: ctl chardev (line $ctl_ln) precedes the null rs485 chardev (line $rs_ln)"
else
fail "ctl only: expected ctl chardev before a null-backed rs485 chardev, got ctl=$ctl_ln rs485=$rs_ln"
fi
if grep -qw 'warden.ctl' "$argv"; then
pass "ctl only: -append carries warden.ctl"
else
fail "ctl only: -append should carry warden.ctl"
fi
fi
# --- case: --ctl and --rs485 together -> ctl device still enumerates first -
argv="$SCRATCH/argv-both.txt"
if run_case "$argv" --ctl "$SCRATCH/ctl.sock" --rs485 "$SCRATCH/rs.sock"; then
ctl_ln="$(chardev_line "$argv" "socket,id=ctl,path=$SCRATCH/ctl.sock,")"
rs_ln="$(chardev_line "$argv" "socket,id=rs485,path=$SCRATCH/rs.sock,")"
if [ -n "$ctl_ln" ] && [ -n "$rs_ln" ] && [ "$ctl_ln" -lt "$rs_ln" ]; then
pass "both flags: ctl chardev (line $ctl_ln) precedes the rs485 chardev (line $rs_ln)"
else
fail "both flags: expected ctl chardev before rs485 chardev, got ctl=$ctl_ln rs485=$rs_ln"
fi
if grep -qw 'warden.ctl' "$argv"; then
pass "both flags: -append carries warden.ctl"
else
fail "both flags: -append should carry warden.ctl"
fi
fi
if [ "$FAIL" -eq 0 ]; then
echo "ALL RUN.SH ARGV TESTS PASSED"
exit 0
else
echo "RUN.SH ARGV TESTS FAILED"
exit 1
fi
@@ -0,0 +1,39 @@
# `fullscreen toggle` must read the dashboard's REAL state, not a belief the
# FIFO's own `fullscreen` verb tracked on the side.
#
# warden_debug.c's `fullscreen` command used to keep a private `static bool
# on` as its only record of state, and nothing kept it in step with the real
# dashboard: `home` clears fullscreen by calling
# warden_screen_overview_set_fullscreen(false) directly (every hardware run's
# preamble, since a live panel is wherever the last flow or an operator left
# it), and a real tap on the dashboard toggles it through the UI's own
# tap-catcher. Either way the FIFO's belief was left stale. A `fullscreen
# toggle` right after either one then decided its next state from that stale
# belief: if `on` was still true from an earlier `fullscreen on`, toggling
# set it to false -- which coincidentally MATCHED the already-off real state,
# so the dashboard never actually turned on, yet the reply still said
# "fullscreen off: ok". See test_qmp_drive.py's FullscreenToggleTracksRealState
# for the same scenario driven offline against a faked control channel (both
# the pre-fix and fixed shapes).
#
# `wake` first, as every compiled flow does: by the time the control bridge
# is up the rig may already sit dimmed, and sleep.c swallows the touch that
# wakes a dimmed panel (flare-edge #148).
wake
sleep 1
assert_page Dashboard/Dashboard
assert_hit 47 676 obj box=12,640,72x72
ctl fullscreen on
sleep 1
assert_hit 47 676 widget box=0,0,720x720
home
sleep 1
assert_page Dashboard/Dashboard
# home must have cleared the REAL state -- this is home-leaves-fullscreen.txt's
# own check, repeated here so a failure below is unambiguously the toggle's
# fault and not a home regression.
assert_hit 47 676 obj box=12,640,72x72
# The dashboard is really off now. A correct toggle from here can only go on.
ctl fullscreen toggle
sleep 1
assert_hit 47 676 widget box=0,0,720x720
@@ -0,0 +1,35 @@
# `home` must leave dashboard fullscreen, not only reach Dashboard/Dashboard.
#
# On the dashboard a tap anywhere toggles fullscreen (screen_overview.c's
# tap-catcher). On a panel a flow's `wake` is a real finger on the glass, so
# a flow that woke the panel on the dashboard left the next flow a 720x720
# canvas over the chrome: the rail gear at (47,676) resolved to
# `widget box=0,0,720x720` instead of `obj box=12,640,72x72`, and every
# hit against chrome failed from there (bench sample 2026-09-08, flare-edge
# #176). The runner's `home` preamble now clears fullscreen last, after the
# dashboard is the page again, so this is the regression guard for it.
#
# `wake` first, as every compiled flow does: by the time the control bridge is
# up the rig may already sit dimmed, and sleep.c swallows the touch that wakes
# a dimmed panel (flare-edge #148) -- the tap below must reach the tap-catcher.
wake
sleep 1
assert_page Dashboard/Dashboard
assert_hit 47 676 obj box=12,640,72x72
tap 360 360
sleep 1
assert_hit 47 676 widget box=0,0,720x720
home
sleep 1
assert_page Dashboard/Dashboard
assert_hit 47 676 obj box=12,640,72x72
# The same from another page: fullscreen on, leave, come home.
tap 360 360
sleep 1
assert_hit 47 676 widget box=0,0,720x720
nav Mining Devices/ASICs
sleep 1
home
sleep 1
assert_page Dashboard/Dashboard
assert_hit 47 676 obj box=12,640,72x72
+55
View File
@@ -0,0 +1,55 @@
# Navigation stress: the sequence that used to kill warden-ui.
#
# Settings > Apps reads the PROJECTED MENU TABLE (app.c's all_proj, populated
# lazily on that page's first build). Walking the app rows writes one timestamp
# per row into app.c's s_row_left. While that array was sized 8 and the shipped
# registry projected 18 rows, leaving any row from index 8 up wrote past it --
# straight over all_count and all_proj, which the linker places immediately
# after. Coming back to Settings > Apps then walked a tick value as a `const
# char *` and the process died with SIGSEGV (rc=139).
#
# So the order matters and is the whole test: populate the table, walk the late
# rows, come BACK. A run that only walks rows never triggers it, because with
# all_count still zero the page's lazy init quietly repairs the array.
#
# Run: ui-drive.sh <zImage> tests/scripts/nav-stress.txt <outdir>
# PASS is ui-drive.sh finding no "warden-ui EXITED" on the console.
echo == populate the projected menu table
tap 47 676
sleep 2
tap 464 40
sleep 4
shot 01-apps-first
echo == back to column 0, where the rail is on screen
tap 174 40
sleep 2
echo == walk the app rows, the late ones included
tap 47 512
sleep 2
tap 47 592
sleep 2
swipe 47 560 47 220 500
sleep 1
tap 47 420
sleep 2
tap 47 500
sleep 2
swipe 47 560 47 220 500
sleep 1
tap 47 380
sleep 2
tap 47 520
sleep 2
shot 02-late-rows
echo == and back: this is the step that used to segfault
tap 47 676
sleep 2
tap 464 40
sleep 4
shot 03-apps-again
sleep 3
shot 04-still-alive
+190
View File
@@ -0,0 +1,190 @@
#!/usr/bin/env bash
# Regression test for mkimage.sh's SEED_DIR hook (see its Env note and the
# block right after --state is applied): a caller-supplied directory of
# pre-built userdata/warden files, copied in whole and applied AFTER --state
# so a seeded file can override a same-named --state value, after each entry
# is validated the same way --state's own KEY=VALUE is validated -- a
# symlink, a non-plain-file entry (a subdirectory included), or a name
# outside [A-Za-z0-9_.-]+ fails closed before cp -a runs. Nothing else in
# the qemu test suite ever sets SEED_DIR -- the CI qemu-tools job runs
# mkimage.sh unseeded, and only ui-drive.sh --seed exercises this path, and
# only when booting a real VM with a flare-edge checkout on hand -- so this
# is the only offline coverage of it.
#
# Builds a real disk image the same way mkimage.sh always does (unprivileged
# mkfs.ext4 -d), then reads the userdata partition back with debugfs -R
# (read-only, no mount or loop device needed) to check what actually landed
# on disk rather than trusting the script's own log output.
#
# bash seed-dir.sh
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # qemu/tests/
QEMU_DIR="$(cd "$HERE/.." && pwd)" # qemu/
FAIL=0
pass() { printf '[PASS] %s\n' "$1"; }
fail() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
# mkfs.ext4 and debugfs both live in sbin, which user shells on Debian don't
# put on PATH -- same fix mkimage.sh itself applies.
PATH="$PATH:/usr/sbin:/sbin"
command -v debugfs >/dev/null || {
echo "FATAL: debugfs (e2fsprogs) not found: needed to read the userdata partition back" >&2
exit 1
}
# shellcheck source=../lib.sh disable=SC1091
. "$QEMU_DIR/lib.sh"
# shellcheck source=../blkdevparts.conf disable=SC1091
. "$QEMU_DIR/blkdevparts.conf"
SCRATCH="$(mktemp -d "${TMPDIR:-/tmp}/warden-qemu-seed-dir.XXXXXX")"
trap 'rm -rf "$SCRATCH"' EXIT
# userdata's byte offset/size come from the same blkdevparts string
# mkimage.sh itself parses, not a hardcoded number: a future layout change
# doesn't strand this test.
USERDATA_OFF=""
USERDATA_SIZE=""
capture_userdata() { [ "$1" = userdata ] && { USERDATA_OFF="$2"; USERDATA_SIZE="$3"; }; return 0; }
qemu_each_partition capture_userdata
[ -n "$USERDATA_OFF" ] || { echo "FATAL: no 'userdata' entry in blkdevparts.conf" >&2; exit 1; }
# extract_userdata DISK OUTFILE: pull the userdata partition window out of a
# built disk image. Sparse output so an otherwise near-empty 1G partition
# costs kilobytes of scratch space, not a real gigabyte, per scenario.
extract_userdata() {
dd if="$1" of="$2" bs=4096 skip=$((USERDATA_OFF / 4096)) \
count=$((USERDATA_SIZE / 4096)) conv=sparse status=none
}
# Reuse an already-verified busybox (read-only) so this test stays offline
# wherever a prior build has already produced one; only a checkout that has
# never run mkimage.sh falls back to the same fetch+verify mkimage.sh always
# does, once, shared by every scenario below.
BUSYBOX_BIN="$QEMU_DIR/out/busybox-armv7l"
if [ ! -f "$BUSYBOX_BIN" ]; then
OUT="$SCRATCH" qemu_get_busybox
BUSYBOX_BIN="$BB"
fi
# --- scenario 1: seed applied verbatim, secret mode preserved, seed beats a same-named --state ---
SEED="$SCRATCH/seed"
mkdir -p "$SEED"
printf 'plain-value\n' > "$SEED/plain.key"
printf 'secret-value\n' > "$SEED/secret.key"
chmod 0600 "$SEED/secret.key"
printf 'seeded-value\n' > "$SEED/override.key"
OUT1="$SCRATCH/out1"
if OUT="$OUT1" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$SEED" \
bash "$QEMU_DIR/mkimage.sh" --state "override.key=state-value" \
> "$SCRATCH/mkimage1.log" 2>&1; then
UIMG="$SCRATCH/userdata1.img"
extract_userdata "$OUT1/disk.img" "$UIMG"
plain_stat="$(debugfs -R "stat /warden/plain.key" "$UIMG" 2>/dev/null)"
if [ -n "$plain_stat" ]; then
pass "SEED_DIR: plain.key landed under userdata/warden"
else
fail "SEED_DIR: plain.key missing from userdata/warden"
fi
secret_stat="$(debugfs -R "stat /warden/secret.key" "$UIMG" 2>/dev/null)"
secret_mode="$(printf '%s' "$secret_stat" | grep -oE 'Mode: *[0-7]+' | grep -oE '[0-7]+$')"
if [ "$secret_mode" = "0600" ]; then
pass "SEED_DIR: secret.key kept mode 0600 through cp -a"
else
fail "SEED_DIR: secret.key mode '$secret_mode', want 0600"
fi
override_content="$(debugfs -R "cat /warden/override.key" "$UIMG" 2>/dev/null)"
if [ "$override_content" = "seeded-value" ]; then
pass "SEED_DIR: seeded override.key beats the same-named --state value"
else
fail "SEED_DIR: override.key = '$override_content', want 'seeded-value' (seed must apply after --state)"
fi
else
fail "SEED_DIR: mkimage.sh exited nonzero with a valid seed dir (see $SCRATCH/mkimage1.log)"
fi
# --- scenario 2: SEED_DIR that is not a directory fails closed ---
NOTADIR="$SCRATCH/notadir"
: > "$NOTADIR"
OUT2="$SCRATCH/out2"
err2="$(OUT="$OUT2" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$NOTADIR" \
bash "$QEMU_DIR/mkimage.sh" 2>&1 1>/dev/null)"
rc2=$?
if [ "$rc2" -ne 0 ] && printf '%s' "$err2" | grep -qF "FATAL: SEED_DIR '$NOTADIR' is not a directory"; then
pass "SEED_DIR: a non-directory path fails closed with the FATAL message"
else
fail "SEED_DIR: non-directory path gave rc=$rc2, stderr='$err2' (want nonzero + the FATAL message)"
fi
# --- scenario 3: a symlink entry fails closed instead of being copied verbatim ---
SEED3="$SCRATCH/seed3"
mkdir -p "$SEED3"
printf 'plain-value\n' > "$SEED3/plain.key"
ln -s /etc/passwd "$SEED3/evil.key"
OUT3="$SCRATCH/out3"
err3="$(OUT="$OUT3" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$SEED3" \
bash "$QEMU_DIR/mkimage.sh" 2>&1 1>/dev/null)"
rc3=$?
if [ "$rc3" -ne 0 ] && printf '%s' "$err3" | grep -qF "FATAL: SEED_DIR entry 'evil.key' is a symlink"; then
pass "SEED_DIR: a symlink entry fails closed instead of being copied verbatim"
else
fail "SEED_DIR: symlink entry gave rc=$rc3, stderr='$err3' (want nonzero + the symlink FATAL message)"
fi
[ -e "$OUT3/disk.img" ] && fail "SEED_DIR: a disk image was written despite the symlink entry"
# --- scenario 4: an entry with a character outside [A-Za-z0-9_.-]+ fails closed ---
SEED4="$SCRATCH/seed4"
mkdir -p "$SEED4"
printf 'x\n' > "$SEED4/bad key"
OUT4="$SCRATCH/out4"
err4="$(OUT="$OUT4" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$SEED4" \
bash "$QEMU_DIR/mkimage.sh" 2>&1 1>/dev/null)"
rc4=$?
if [ "$rc4" -ne 0 ] && printf '%s' "$err4" | grep -qF "FATAL: SEED_DIR entry 'bad key' must match [A-Za-z0-9_.-]+"; then
pass "SEED_DIR: an entry name outside [A-Za-z0-9_.-]+ fails closed"
else
fail "SEED_DIR: bad-name entry gave rc=$rc4, stderr='$err4' (want nonzero + the charset FATAL message)"
fi
# --- scenario 5: a subdirectory entry fails closed (not a plain file) ---
SEED5="$SCRATCH/seed5"
mkdir -p "$SEED5/subdir"
printf 'x\n' > "$SEED5/subdir/leaf.key"
OUT5="$SCRATCH/out5"
err5="$(OUT="$OUT5" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$SEED5" \
bash "$QEMU_DIR/mkimage.sh" 2>&1 1>/dev/null)"
rc5=$?
if [ "$rc5" -ne 0 ] && printf '%s' "$err5" | grep -qF "FATAL: SEED_DIR entry 'subdir' is not a plain file"; then
pass "SEED_DIR: a subdirectory entry fails closed instead of being recursed into"
else
fail "SEED_DIR: subdirectory entry gave rc=$rc5, stderr='$err5' (want nonzero + the plain-file FATAL message)"
fi
# --- scenario 6: a hyphenated key (seed-fixtures.py's KEY_RE, e.g.
# "gas-plant.devices") still seeds cleanly -- guards against tightening the
# charset to --state's stricter [A-Za-z0-9_.]+ by mistake, which would
# reject keys committed flow specs already seed through this path ---
SEED6="$SCRATCH/seed6"
mkdir -p "$SEED6"
printf 'r5\n' > "$SEED6/gas-plant.devices"
OUT6="$SCRATCH/out6"
if OUT="$OUT6" BUSYBOX="$BUSYBOX_BIN" SEED_DIR="$SEED6" \
bash "$QEMU_DIR/mkimage.sh" > "$SCRATCH/mkimage6.log" 2>&1; then
UIMG6="$SCRATCH/userdata6.img"
extract_userdata "$OUT6/disk.img" "$UIMG6"
hyphen_content="$(debugfs -R "cat /warden/gas-plant.devices" "$UIMG6" 2>/dev/null)"
if [ "$hyphen_content" = "r5" ]; then
pass "SEED_DIR: a hyphenated key (gas-plant.devices) still seeds cleanly"
else
fail "SEED_DIR: gas-plant.devices = '$hyphen_content', want 'r5'"
fi
else
fail "SEED_DIR: mkimage.sh rejected a valid hyphenated key (see $SCRATCH/mkimage6.log)"
fi
[ "$FAIL" -eq 0 ] && echo "ALL SEED_DIR TESTS PASSED" || echo "SEED_DIR TESTS FAILED"
[ "$FAIL" -eq 0 ]
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env bash
# Offline regression test for qemu_stage_rootfs() (qemu/lib.sh): the staged
# etc/shadow must come out mode 0600 regardless of the mode the SOURCE
# qemu/rootfs/etc/shadow happens to carry in the working tree. Git tracks
# only the executable bit, so a fresh checkout can land that source file at
# anything a non-executable blob gets under the checking-out user's umask
# (644 under the common 022) -- world readable, exposing root's crypt hash
# to any unprivileged process in the guest. The test stages from an isolated
# copy of qemu/rootfs with etc/shadow deliberately set to 0644 first, so it
# still catches the regression even when the real working tree's copy
# already happens to be 0600 locally (that local mode is never what ships;
# only what git tracks does). No real busybox or QEMU needed: a stub binary
# is enough to exercise the staging function itself.
#
# bash stage-rootfs-perms.sh
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REAL_QEMU_DIR="$(cd "$HERE/.." && pwd)"
SCRATCH="$(mktemp -d /tmp/wqperm.XXXXXX)"
trap 'rm -rf "$SCRATCH"' EXIT
FAIL=0
pass() { printf '[PASS] %s\n' "$1"; }
fail() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
# shellcheck source=../lib.sh disable=SC1091
. "$REAL_QEMU_DIR/lib.sh"
# Isolated QEMU_DIR: a copy of the real rootfs skeleton, source etc/shadow
# forced to 0644 to simulate the permissive-umask checkout this test must
# catch regardless of what the working tree's own copy happens to be.
QEMU_DIR="$SCRATCH/qemu"
mkdir -p "$QEMU_DIR"
cp -a "$REAL_QEMU_DIR/rootfs" "$QEMU_DIR/rootfs"
chmod 0644 "$QEMU_DIR/rootfs/etc/shadow"
# Stand in for a verified busybox download: qemu_stage_rootfs only installs
# it, never reads its content.
BB="$SCRATCH/fake-busybox"
printf '#!/bin/sh\nexit 0\n' > "$BB"
chmod 0755 "$BB"
ROOT="$SCRATCH/root"
qemu_stage_rootfs "$ROOT"
shadow_mode="$(stat -c '%a' "$ROOT/etc/shadow")"
if [ "$shadow_mode" = "600" ]; then
pass "etc/shadow staged at 0600"
else
fail "etc/shadow staged at $shadow_mode, want 600"
fi
# Control: passwd/group carry no secrets and stay world-readable, same as
# every other Linux system -- confirms the fix targets shadow specifically
# rather than locking the whole /etc tree down.
passwd_mode="$(stat -c '%a' "$ROOT/etc/passwd")"
if [ "$passwd_mode" = "644" ] || [ "$passwd_mode" = "664" ]; then
pass "etc/passwd untouched by the shadow chmod (mode $passwd_mode)"
else
fail "etc/passwd unexpectedly mode $passwd_mode"
fi
[ "$FAIL" -eq 0 ] && echo "ALL STAGE-ROOTFS PERM TESTS PASSED" || echo "STAGE-ROOTFS PERM TESTS FAILED"
[ "$FAIL" -eq 0 ]
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env bash
# Offline regression test for ui-drive.sh's rs485 simulator startup and its
# process cleanup: no real QEMU or kernel image. A fake qemu/run.sh stands in
# for the VM (it optionally binds the rs.sock qemu's --rs485 chardev would
# create) and, for one case, a fake socat stands in ahead of it on PATH. This
# is the same "copy the real script beside stand-in siblings" trick
# tests/flows/run-flow-run-all-tests.sh uses for flow-run-rig.sh, one layer
# down at ui-drive.sh's own dependencies.
#
# What is worth pinning:
# - the rs.sock/rs.pty/rs.ctl waits fail closed instead of printing
# "== rs485 simulator: ..." over a bus nothing is actually serving --
# before this fix, a socat or mbsim.py that never came up looked, from
# ui-drive.sh's own output, exactly like a live one, and every later
# rs485 assertion failed for a reason buried in a log nobody was pointed
# at.
# - cleanup() escalates to SIGKILL for a sim process that ignores SIGTERM,
# instead of leaving it to outlive the script (an accumulation of
# orphaned simulator processes on a loaded CI runner).
#
# bash test-ui-drive-rs485.sh
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SCRATCH="$(mktemp -d /tmp/uidrs485.XXXXXX)"
trap 'rm -rf "$SCRATCH"' EXIT
FAIL=0
pass() { printf '[PASS] %s\n' "$1"; }
fail() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
ZIMAGE="$SCRATCH/fake-zImage"
SCRIPT="$SCRATCH/fake.txt"
: > "$ZIMAGE"
printf 'wake\n' > "$SCRIPT"
# build_rig WORKDIR: lay out WORKDIR/qemu with a copy of the real (fixed)
# ui-drive.sh and qmp.py next to stand-in mkinitramfs.sh/mkimage.sh/run.sh/
# payload, so ui-drive.sh's own HERE/QDIR resolve inside WORKDIR and nothing
# under the real qemu/out/ is ever touched.
build_rig() {
local work="$1"
mkdir -p "$work/qemu/tests" "$work/qemu/payload"
cp "$HERE/ui-drive.sh" "$work/qemu/tests/ui-drive.sh"
cp "$HERE/qmp.py" "$work/qemu/tests/qmp.py"
chmod +x "$work/qemu/tests/ui-drive.sh"
printf '#!/usr/bin/env bash\nexit 0\n' > "$work/qemu/mkinitramfs.sh"
printf '#!/usr/bin/env bash\nexit 0\n' > "$work/qemu/mkimage.sh"
cat > "$work/qemu/run.sh" <<'EOS'
#!/usr/bin/env bash
# Stand-in for qemu/run.sh: no real VM, just enough to drive ui-drive.sh's
# rs485 startup logic offline. TEST_MAKE_RS_SOCK=1 binds a listening AF_UNIX
# socket at --rs485's path (as qemu's server=on,wait=off chardev would);
# unset/0 leaves it absent, simulating qemu never wiring up the bus. Stays
# up 8s -- past ui-drive.sh's 3s post-launch liveness check and the rs485
# waits' poll budgets -- then exits, so a run that gets past the rs485 block
# still fails fast (VM exited early) instead of idling out the 180s deadline.
rs485_sock=""
while [ $# -gt 0 ]; do
case "$1" in
--rs485) rs485_sock="$2"; shift 2 ;;
*) shift ;;
esac
done
exec python3 -c '
import os, socket, sys, time
sock_path = sys.argv[1] if len(sys.argv) > 1 else ""
if sock_path and os.environ.get("TEST_MAKE_RS_SOCK") == "1":
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.bind(sock_path)
s.listen(1)
time.sleep(8)
' "$rs485_sock"
EOS
chmod +x "$work/qemu/mkinitramfs.sh" "$work/qemu/mkimage.sh" "$work/qemu/run.sh"
: > "$work/qemu/payload/warden-ui"
chmod +x "$work/qemu/payload/warden-ui"
}
# run_rig WORKDIR: invoke the copied ui-drive.sh with a bounded outer timeout
# (the test's own safety net, matching run-flow-run-all-tests.sh) and stash
# combined output + exit code in the globals below.
RIG_OUT=""
RIG_RC=0
run_rig() {
local work="$1"
RIG_OUT="$(cd "$work" && timeout -k 5 40 bash "$work/qemu/tests/ui-drive.sh" \
--rs485-devices "5:bare" "$ZIMAGE" "$SCRIPT" "$work/out" 2>&1)"
RIG_RC=$?
}
# --- case A: qemu's rs.sock never appears -----------------------------------
work="$SCRATCH/a"; build_rig "$work"
FLARE_EDGE="$SCRATCH/flare-edge-a"
mkdir -p "$FLARE_EDGE/tools/modbus-sim"
printf '#!/usr/bin/env python3\nimport sys; sys.exit(1)\n' > "$FLARE_EDGE/tools/modbus-sim/mbsim.py"
# A VAR=val prefix on a function call exports VAR into that one call only
# (and any children it spawns) and restores whatever VAR held before once the
# call returns -- the same per-case scoping a wrapping subshell gave us, but
# without a subshell: run_rig's RIG_OUT/RIG_RC writes land here directly, so
# nothing needs to be re-serialized as text and re-parsed back out below.
FLARE_EDGE="$FLARE_EDGE" TEST_MAKE_RS_SOCK='' run_rig "$work"
[ "$RIG_RC" -ne 0 ] \
&& pass "case A: never-appeared rs.sock fails the run (rc=$RIG_RC)" \
|| fail "case A: never-appeared rs.sock fails the run: rc=$RIG_RC, out: $RIG_OUT"
printf '%s\n' "$RIG_OUT" | grep -qF 'FATAL: rs485 bus socket never appeared' \
&& pass "case A: FATAL names the missing rs.sock" \
|| fail "case A: FATAL names the missing rs.sock: $RIG_OUT"
printf '%s\n' "$RIG_OUT" | grep -qF '== rs485 simulator:' \
&& fail "case A: printed the rs485-live success line over a bus that was never up: $RIG_OUT" \
|| pass "case A: never claims the bus is live"
# --- case B: rs.sock is up but socat never creates rs.pty -------------------
work="$SCRATCH/b"; build_rig "$work"
FLARE_EDGE="$SCRATCH/flare-edge-b"
mkdir -p "$FLARE_EDGE/tools/modbus-sim"
printf '#!/usr/bin/env python3\nimport sys; sys.exit(1)\n' > "$FLARE_EDGE/tools/modbus-sim/mbsim.py"
FAKEBIN="$SCRATCH/fakebin-b"; mkdir -p "$FAKEBIN"
printf '#!/usr/bin/env bash\necho "FAKE SOCAT: simulated failure" >&2\nexit 1\n' > "$FAKEBIN/socat"
chmod +x "$FAKEBIN/socat"
FLARE_EDGE="$FLARE_EDGE" TEST_MAKE_RS_SOCK=1 PATH="$FAKEBIN:$PATH" run_rig "$work"
[ "$RIG_RC" -ne 0 ] \
&& pass "case B: socat never linking rs.pty fails the run (rc=$RIG_RC)" \
|| fail "case B: socat never linking rs.pty fails the run: rc=$RIG_RC, out: $RIG_OUT"
printf '%s\n' "$RIG_OUT" | grep -qF 'FATAL: rs485 socat never created rs.pty' \
&& pass "case B: FATAL names the missing rs.pty" \
|| fail "case B: FATAL names the missing rs.pty: $RIG_OUT"
# --- case C: rs.sock+rs.pty are up but mbsim.py crashes before rs.ctl -------
work="$SCRATCH/c"; build_rig "$work"
FLARE_EDGE="$SCRATCH/flare-edge-c"
mkdir -p "$FLARE_EDGE/tools/modbus-sim"
cat > "$FLARE_EDGE/tools/modbus-sim/mbsim.py" <<'EOS'
#!/usr/bin/env python3
import sys
sys.stderr.write("FAKE MBSIM: simulated crash before opening the control socket\n")
sys.exit(1)
EOS
FLARE_EDGE="$FLARE_EDGE" TEST_MAKE_RS_SOCK=1 TEST_MBSIM_PID_FILE='' run_rig "$work"
[ "$RIG_RC" -ne 0 ] \
&& pass "case C: mbsim.py crashing before rs.ctl fails the run (rc=$RIG_RC)" \
|| fail "case C: mbsim.py crashing before rs.ctl fails the run: rc=$RIG_RC, out: $RIG_OUT"
printf '%s\n' "$RIG_OUT" | grep -qF 'FATAL: rs485 simulator (mbsim.py) never came up' \
&& pass "case C: FATAL names the dead simulator" \
|| fail "case C: FATAL names the dead simulator: $RIG_OUT"
printf '%s\n' "$RIG_OUT" | grep -qF 'FAKE MBSIM: simulated crash' \
&& pass "case C: mbsim.log is dumped on failure" \
|| fail "case C: mbsim.log is dumped on failure: $RIG_OUT"
# --- case D: a sim process that ignores SIGTERM is still reaped ------------
work="$SCRATCH/d"; build_rig "$work"
FLARE_EDGE="$SCRATCH/flare-edge-d"
mkdir -p "$FLARE_EDGE/tools/modbus-sim"
cat > "$FLARE_EDGE/tools/modbus-sim/mbsim.py" <<'EOS'
#!/usr/bin/env python3
# Comes up cleanly (binds --control so ui-drive.sh's rs.ctl wait passes),
# then ignores SIGTERM -- the process cleanup() must still SIGKILL.
import argparse, os, signal, socket, sys, time
ap = argparse.ArgumentParser()
ap.add_argument('--port')
ap.add_argument('--control')
ap.add_argument('--device', action='append', default=[])
args = ap.parse_args()
signal.signal(signal.SIGTERM, signal.SIG_IGN)
if args.control:
try:
os.unlink(args.control)
except FileNotFoundError:
pass
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.bind(args.control)
s.listen(1)
pidfile = os.environ.get('TEST_MBSIM_PID_FILE')
if pidfile:
with open(pidfile, 'w') as f:
f.write(str(os.getpid()))
while True:
time.sleep(1)
EOS
PIDFILE="$SCRATCH/mbsim-d.pid"
FLARE_EDGE="$FLARE_EDGE" TEST_MAKE_RS_SOCK=1 TEST_MBSIM_PID_FILE="$PIDFILE" run_rig "$work"
if [ -s "$PIDFILE" ]; then
mbsim_pid="$(cat "$PIDFILE")"
if ! kill -0 "$mbsim_pid" 2>/dev/null; then
pass "case D: a SIGTERM-ignoring sim process is dead once ui-drive.sh returns (SIGKILL fallback)"
else
fail "case D: pid $mbsim_pid is still alive after ui-drive.sh returned -- SIGKILL fallback did not fire"
kill -KILL "$mbsim_pid" 2>/dev/null || true # don't leak it out of the test either
fi
else
fail "case D: the fake mbsim.py never wrote its pid file, out: $RIG_OUT"
fi
[ "$FAIL" -eq 0 ] && echo "ALL UI-DRIVE RS485 TESTS PASSED" || echo "UI-DRIVE RS485 TESTS FAILED"
[ "$FAIL" -eq 0 ]
+1147
View File
File diff suppressed because it is too large Load Diff
+334
View File
@@ -0,0 +1,334 @@
#!/usr/bin/env bash
# Drive the LVGL UI through a scripted interaction and collect screenshots.
#
# ui-shot.sh proves touch reaches the UI in one tap; this is the same rig for
# work that needs a SEQUENCE (swipe through the app rows, open a submenu, tap a
# tab, bring up the keyboard) with a screendump wherever the script asks for
# one. One boot serves the whole script, because booting per step (TCG, no KVM)
# costs about a minute and a real interaction is thirty steps.
#
# It also FAILS on a UI that died mid-script. The framebuffer keeps its last
# frame when warden-ui crashes, so screendumps carry on returning a plausible
# picture of a program that no longer exists; stage-2 init announces the exit on
# the console (see rootfs/sbin/init) and this greps for it after the run.
#
# FAILS CLOSED on missing prerequisites.
#
# Usage: ui-drive.sh [--seed FILE] [--refs FILE] [--rs485-devices LIST] <zImage-virt> <script> [out-dir]
# <script> is a qmp.py `drive` script: see its docstring for the commands.
# --seed FILE a seed manifest (flat YAML key: value, see flare-edge
# tools/seed-fixtures.py) run through that tool and staged
# into userdata/warden BEFORE boot, via mkimage.sh's SEED_DIR
# hook -- so a screen's first read at startup already sees
# it, not a value written after the race is already lost.
# Needs FLARE_EDGE=<checkout> to find the tool (same
# convention as ota-apply.sh / portal-scenario.sh).
# --refs FILE passed straight through to qmp.py drive's --refs (the
# region/ocr reference JSON); see qmp.py's own docstring.
# --rs485-devices ADDR:SLUG[,ADDR:SLUG...]
# put simulated Modbus devices on the VM's RS485 bus: the
# UART is attached to a unix socket (run.sh --rs485), socat
# turns it into a pty, and flare-edge tools/modbus-sim/mbsim.py
# serves the named corpus profiles on it, so warden-modbus in
# the guest discovers and identifies devices the way it does
# on a panel with real controllers on the header. Needs
# FLARE_EDGE and socat.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # qemu/tests/
QDIR="$(cd "$HERE/.." && pwd)" # qemu/
SEED_FILE=""
REFS_FILE=""
RS485_DEVICES=""
ARGS=()
while [ $# -gt 0 ]; do
case "$1" in
--seed) SEED_FILE="${2:?--seed needs a manifest path}"; shift 2 ;;
--refs) REFS_FILE="${2:?--refs needs a path}"; shift 2 ;;
--rs485-devices) RS485_DEVICES="${2:?--rs485-devices needs ADDR:SLUG[,...]}"; shift 2 ;;
--) shift; ARGS+=("$@"); break ;;
-*) echo "FATAL: unknown option '$1' (usage: $0 [--seed FILE] [--refs FILE] [--rs485-devices LIST] <zImage> <script> [out-dir])" >&2; exit 1 ;;
*) ARGS+=("$1"); shift ;;
esac
done
ZIMAGE="${ARGS[0]:-}"
SCRIPT="${ARGS[1]:-}"
OUTDIR="${ARGS[2]:-$QDIR/out/ui-drive}"
if [ -z "$ZIMAGE" ] || [ ! -f "$ZIMAGE" ]; then
echo "FATAL: usage: $0 [--seed FILE] [--refs FILE] [--rs485-devices LIST] <zImage> <script> [out-dir]: the virt.fragment kernel variant" >&2
exit 1
fi
if [ -z "$SCRIPT" ] || [ ! -f "$SCRIPT" ]; then
echo "FATAL: no drive script at '$SCRIPT'" >&2
exit 1
fi
if [ -n "$SEED_FILE" ] && [ ! -f "$SEED_FILE" ]; then
echo "FATAL: no seed manifest at '$SEED_FILE'" >&2
exit 1
fi
if [ -n "$SEED_FILE" ] && { [ -z "${FLARE_EDGE:-}" ] || [ ! -f "$FLARE_EDGE/tools/seed-fixtures.py" ]; }; then
echo "FATAL: --seed needs FLARE_EDGE to point at a flare-edge checkout (seed-fixtures.py not found under '${FLARE_EDGE:-}')" >&2
exit 1
fi
[ -x "$QDIR/payload/warden-ui" ] || {
echo "FATAL: no qemu/payload/warden-ui: build it with flare-edge tools/build-ui-vm.sh" >&2
exit 1
}
command -v qemu-system-arm >/dev/null || {
echo "FATAL: qemu-system-arm not on PATH: see qemu/README.md" >&2
exit 1
}
mkdir -p "$OUTDIR"
# Short-named scratch: AF_UNIX socket paths are capped at ~108 chars.
WORK="$(mktemp -d /tmp/wqd.XXXXXX)"
# Every run gets its own image and initramfs: mkinitramfs.sh, mkimage.sh and
# run.sh all read $OUT, and with the shared default (qemu/out) two concurrent
# runs built and booted the SAME disk.img, so a seeded VM could read back
# another run's state (SDK #20). The image is sparse but is written by the
# guest, and /tmp is a tmpfs on the dev box, so this lives under $TMPDIR on
# real disk; the pinned busybox stays shared and read-only through BUSYBOX.
avail_kb="$(df --output=avail -k "${TMPDIR:-/tmp}" | tail -1 | tr -d ' ')"
if [ "${avail_kb:-0}" -lt 1048576 ]; then
echo "FATAL: ${TMPDIR:-/tmp} has under 1 GB free; point TMPDIR at real disk (see flows README)" >&2
exit 1
fi
mkdir -p "$QDIR/out"
RUN_OUT="$(mktemp -d "${TMPDIR:-/tmp}/wqd-out.XXXXXX")"
export OUT="$RUN_OUT"
export BUSYBOX="${BUSYBOX:-$QDIR/out/busybox-armv7l}"
QEMU_PID=""
SIM_PIDS=""
# SIGTERM first (lets qemu/socat/mbsim.py close their sockets and pty
# cleanly), then SIGKILL if that didn't take: a process wedged on a full pty
# buffer or an uninterruptible I/O wait under host load must not outlive this
# script, or a long batch (flow-run-all.sh --jobs N) slowly accumulates
# orphaned simulator/VM processes competing with the runs still in flight.
reap() {
local p="$1"
kill "$p" 2>/dev/null || return 0
for _i in $(seq 1 20); do
kill -0 "$p" 2>/dev/null || return 0
sleep 0.1
done
kill -KILL "$p" 2>/dev/null || true
}
# Poll for PATH to appear, checking every 0.1s for up to 5 seconds -- the
# rs.sock/rs.pty/rs.ctl handshake budget below, now set in one place instead
# of three copies that could drift out of step with each other. When PID is
# given, also stop the moment PID has died: a process that's already gone
# will never create the path, so there is no reason to spend the rest of the
# budget waiting on it. The exit status carries no verdict -- each call site
# still makes its own existence (and, for rs.ctl, liveness) check right after
# this returns, exactly as it did before the loop was pulled out.
wait_for_path() {
local path="$1" pid="${2:-}" _i
for _i in $(seq 1 50); do
[ -e "$path" ] && return 0
if [ -n "$pid" ]; then
kill -0 "$pid" 2>/dev/null || return 0
fi
sleep 0.1
done
return 0
}
cleanup() {
for p in $SIM_PIDS; do reap "$p"; done
if [ -n "$QEMU_PID" ]; then
# `timeout` here is a second, independent bound on top of qmp.py's own
# QMP_TIMEOUT_S socket timeout: whichever one it is that stalls, this
# call must not itself keep cleanup() from reaching reap "$QEMU_PID"
# below -- the one thing meant to guarantee a wedged qemu-system-arm
# cannot outlive this script.
timeout -k 5 25 python3 "$HERE/qmp.py" "$WORK/qmp.sock" quit 2>/dev/null || true
sleep 1
reap "$QEMU_PID"
fi
cp "$WORK/console.log" "$OUTDIR/console.log" 2>/dev/null || true
cp "$WORK/mbsim.log" "$OUTDIR/mbsim.log" 2>/dev/null || true
rm -rf "$WORK" "$RUN_OUT"
}
trap cleanup EXIT
bash "$QDIR/mkinitramfs.sh"
if [ -n "$SEED_FILE" ]; then
# Stage the settings files BEFORE mkimage.sh builds the disk: userdata is
# baked into the image up front (see qemu/README.md's boundary table --
# there is no mount-after-boot step this rig could inject files through),
# so anything a screen reads at startup has to be in place before -kernel
# even runs, not written into a running VM.
python3 "$FLARE_EDGE/tools/seed-fixtures.py" "$SEED_FILE" "$WORK/seed"
SEED_DIR="$WORK/seed" bash "$QDIR/mkimage.sh"
else
bash "$QDIR/mkimage.sh"
fi
# Random hostfwd ports can collide. Detect qemu's early bind failure and retry
# with a fresh base rather than failing spuriously.
for _attempt in 1 2 3; do
PORT=$((21000 + RANDOM % 20000))
: > "$WORK/console.log"
bash "$QDIR/run.sh" --kernel "$ZIMAGE" --display headless --qmp "$WORK/qmp.sock" \
--ctl "$WORK/ctl.sock" ${RS485_DEVICES:+--rs485 "$WORK/rs.sock"} \
--ssh-port "$PORT" --http-port $((PORT + 1)) --api-port $((PORT + 2)) \
> "$WORK/console.log" 2>&1 &
QEMU_PID=$!
sleep 3
kill -0 "$QEMU_PID" 2>/dev/null && break
if grep -aq 'Could not set up host forwarding' "$WORK/console.log"; then
echo "== hostfwd port collision on base $PORT, retrying"
QEMU_PID=""
continue
fi
echo "FATAL: VM died at launch:" >&2
tail -20 "$WORK/console.log" >&2
exit 1
done
if [ -z "$QEMU_PID" ] || ! kill -0 "$QEMU_PID" 2>/dev/null; then
echo "FATAL: could not launch the VM after 3 port attempts" >&2
exit 1
fi
if [ -n "$RS485_DEVICES" ]; then
# qemu created rs.sock at launch (server=on,wait=off). socat gives the
# simulator the serial device it expects; mbsim then answers the guest's
# Modbus polls with the corpus profiles' own register maps.
command -v socat >/dev/null || { echo "FATAL: --rs485-devices needs socat" >&2; exit 1; }
[ -n "${FLARE_EDGE:-}" ] && [ -f "$FLARE_EDGE/tools/modbus-sim/mbsim.py" ] || {
echo "FATAL: --rs485-devices needs FLARE_EDGE to point at a flare-edge checkout (mbsim.py)" >&2; exit 1; }
wait_for_path "$WORK/rs.sock"
[ -S "$WORK/rs.sock" ] || {
echo "FATAL: rs485 bus socket never appeared at $WORK/rs.sock (qemu's --rs485 chardev never came up)" >&2
tail -25 "$WORK/console.log" >&2
exit 1
}
socat "UNIX-CONNECT:$WORK/rs.sock" "PTY,link=$WORK/rs.pty,raw,echo=0" > "$WORK/socat.log" 2>&1 &
socat_pid=$!
SIM_PIDS="$SIM_PIDS $socat_pid"
# Break out the moment socat dies rather than always spending the full
# poll budget: a socat that never links the pty is usually already gone
# (bad UNIX-CONNECT target, no pty node available), and kill -0 catches
# that in one tick instead of five seconds.
wait_for_path "$WORK/rs.pty" "$socat_pid"
[ -e "$WORK/rs.pty" ] || {
echo "FATAL: rs485 socat never created rs.pty (see $WORK/socat.log)" >&2
cat "$WORK/socat.log" >&2
exit 1
}
dev_args=()
IFS=',' read -r -a _devs <<< "$RS485_DEVICES"
for d in "${_devs[@]}"; do dev_args+=(--device "$d"); done
# --control is the runtime lever: qmp.py's `rs485 silence|restore ADDR`
# verb talks to it, so a flow can take a device off the bus mid-run.
python3 "$FLARE_EDGE/tools/modbus-sim/mbsim.py" --port "$WORK/rs.pty" --control "$WORK/rs.ctl" "${dev_args[@]}" > "$WORK/mbsim.log" 2>&1 &
mbsim_pid=$!
SIM_PIDS="$SIM_PIDS $mbsim_pid"
wait_for_path "$WORK/rs.ctl" "$mbsim_pid"
{ [ -S "$WORK/rs.ctl" ] && kill -0 "$mbsim_pid" 2>/dev/null; } || {
echo "FATAL: rs485 simulator (mbsim.py) never came up (see $WORK/mbsim.log)" >&2
cat "$WORK/mbsim.log" >&2
exit 1
}
echo "== rs485 simulator: $RS485_DEVICES on $WORK/rs.pty"
fi
echo "== waiting for warden-ui"
deadline=$((SECONDS + 180))
while [ $SECONDS -lt $deadline ]; do
grep -aq 'init: starting warden-ui' "$WORK/console.log" && break
kill -0 "$QEMU_PID" 2>/dev/null || { echo "FATAL: VM exited early" >&2; tail -25 "$WORK/console.log" >&2; exit 1; }
sleep 2
done
grep -aq 'init: starting warden-ui' "$WORK/console.log" || {
echo "FATAL: warden-ui never started (no fb0? wrong kernel?)" >&2
tail -25 "$WORK/console.log" >&2
exit 1
}
# A started process is not a rendered frame. Poll screendumps until the panel
# stops being a single flat colour, on a bounded deadline: TCG renders CPU-bound
# and a loaded host can be arbitrarily slow, so this is never a fixed sleep.
#
# The count only has to clear a fixed threshold (32), not be exact, so this
# samples every 32nd pixel instead of all ~518K (720x720) of them, and bails
# the moment the threshold clears instead of always finishing the scan. A
# still-flat splash reads identically under any stride -- every sampled pixel
# is the same colour either way, so sampling can never manufacture a false
# "ready" -- and a rendered frame has enough variety that a sparse sample
# clears 32 distinct colours just as reliably. This loop runs on the same
# host CPU the comment above names as the bottleneck, on every poll tick of
# every flow's boot, so the per-tick cost matters; a plain byte loop here also
# avoids adding a Pillow dependency to a boot check that today has none (see
# qmp.py's own imgtools/Pillow deferral -- a plain nav/tap flow must keep
# booting on a host without Pillow installed).
echo "== waiting for the first real frame"
deadline=$((SECONDS + 180))
ready=0
while [ $SECONDS -lt $deadline ]; do
python3 "$HERE/qmp.py" "$WORK/qmp.sock" screendump "$WORK/probe.ppm" 2>/dev/null || { sleep 2; continue; }
colors="$(python3 -c "
d = open('$WORK/probe.ppm', 'rb').read()
colors = set()
for i in range(15, len(d), 32 * 3):
colors.add(d[i:i+3])
if len(colors) > 32:
break
print(len(colors))
" 2>/dev/null || echo 0)"
[ "${colors:-0}" -gt 32 ] && { ready=1; break; }
sleep 3
done
[ "$ready" = 1 ] || {
echo "FATAL: the UI never rendered a real frame" >&2
tail -25 "$WORK/console.log" >&2
exit 1
}
# The control bridge (init -> warden-ui's debug FIFO, see run.sh --ctl) comes
# up with the UI; a script's first `page`/`hit` must not race it. Bounded, and
# fail-closed: a scenario that asserts on UI state needs the channel, and a
# silently absent one would turn every assertion into an infrastructure error
# dressed as a test result.
echo "== waiting for the control bridge"
deadline=$((SECONDS + 60))
until grep -aq 'init: control bridge on' "$WORK/console.log"; do
[ $SECONDS -lt $deadline ] || {
echo "FATAL: the control bridge never announced itself (run.sh --ctl / init marker)" >&2
tail -25 "$WORK/console.log" >&2
exit 1
}
sleep 1
done
echo "== driving $SCRIPT"
# Every step lands in results.jsonl (ok / fail / fatal); an assertion mismatch
# is a `fail` and the run continues, so one run reports every broken
# expectation. The driver's exit status is the verdict; capture it rather than
# let `set -e` skip the backstop and the summary below.
drive_rc=0
DRIVE_ARGS=(--ctl "$WORK/ctl.sock" --console "$WORK/console.log")
# Only passed when given: qmp.py's own default (<outdir>/refs.json, see its
# docstring) is right for the common case of one refs file living next to a
# flow's other fixtures, and forcing a path here would just duplicate that
# default in two places.
[ -n "$REFS_FILE" ] && DRIVE_ARGS+=(--refs "$REFS_FILE")
python3 "$HERE/qmp.py" "$WORK/qmp.sock" drive "$SCRIPT" "$OUTDIR" \
${RS485_DEVICES:+--rs485-control "$WORK/rs.ctl"} \
"${DRIVE_ARGS[@]}" || drive_rc=$?
# The UI must still be alive: see the header. The driver checks this after
# every step and pins a crash to the step that caused it; this is the
# backstop for a death after the last step, or a driver that itself fell over.
if grep -aq 'warden-ui EXITED' "$WORK/console.log"; then
echo "FATAL: warden-ui DIED during the run:" >&2
grep -a -A22 'warden-ui EXITED' "$WORK/console.log" >&2
exit 1
fi
if [ "$drive_rc" -ne 0 ]; then
echo "UI-DRIVE-FAIL: see $OUTDIR/results.jsonl" >&2
grep -E '"status": "(fail|fatal)"' "$OUTDIR/results.jsonl" >&2 || true
exit 1
fi
echo "UI-DRIVE-PASS (results in $OUTDIR/results.jsonl, screenshots in $OUTDIR)"
@@ -0,0 +1,135 @@
#!/usr/bin/env bash
# Regression tests for build/fetch-buildroot-tarball.sh's retry/verify path.
#
# The script had never been exercised by anything (nothing calls it yet --
# see its own header) or by a test, so its retry-on-mismatch, cleanup, and
# already-verified short-circuit had never actually run. It also fetched with
# no --connect-timeout/--max-time, so a connection that opens and then stalls
# (a blackholed route, a hung proxy) would block forever instead of retrying;
# issue tracked separately.
#
# Runs the real script (copied into a fixture dir so its own HERE-relative pin
# lookup finds a pin we control) with a fake curl first on PATH, so no network
# is used and the outcome of each attempt is exact.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/fetch-buildroot-tarball.sh"
[ -f "$SCRIPT" ] || { echo "FAIL: script not found at $SCRIPT"; exit 1; }
FAIL=0
ok() { printf '[PASS] %s\n' "$1"; }
bad() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
WANT_SHA="$(printf 'good-bytes' | sha256sum | awk '{print $1}')"
# A private copy of the real script plus a pin file we control, in its own
# directory: the script resolves its pin relative to itself, so this is the
# shipping file under test, not a reimplementation of it.
setup() { # setup <case-name> -> prints the fixture dir
local dir="$TMP/$1"
mkdir -p "$dir/bin"
cp "$SCRIPT" "$dir/fetch-buildroot-tarball.sh"
printf '%s buildroot-2025.02.8.tar.xz\n' "$WANT_SHA" \
> "$dir/buildroot-2025.02.8.tar.xz.sha256"
printf '%s' "$dir"
}
run() { # run <dir> <dest> -> stdout+stderr on stdout, $? in $RC
local dir="$1" dest="$2"
OUT="$(PATH="$dir/bin:$PATH" bash "$dir/fetch-buildroot-tarball.sh" "$dest" 2>&1)"
RC=$?
}
# --- case A: destination already matches the pin -> curl never runs -------
dir="$(setup case-a)"
printf 'good-bytes' > "$dir/dest.tar.xz"
cat > "$dir/bin/curl" <<'FAKE'
#!/bin/sh
echo "curl should not have run" >&2
exit 1
FAKE
chmod +x "$dir/bin/curl"
run "$dir" "$dir/dest.tar.xz"
if [ "$RC" -eq 0 ] && printf '%s' "$OUT" | grep -q 'already present and verified'; then
ok "already-verified destination: curl never invoked"
else
bad "already-verified destination: curl never invoked (rc=$RC, out=$OUT)"
fi
# --- case B: fails once, succeeds on retry -> verified, no extra attempts -
dir="$(setup case-b)"
cat > "$dir/bin/curl" <<FAKE
#!/bin/sh
cnt_file="$dir/curl-calls"
n=0
[ -f "\$cnt_file" ] && n=\$(cat "\$cnt_file")
n=\$((n + 1))
echo "\$n" > "\$cnt_file"
out="" prev=""
for a in "\$@"; do
[ "\$prev" = "-o" ] && out="\$a"
prev="\$a"
done
if [ "\$n" -lt 2 ]; then printf 'bad-bytes' > "\$out"
else printf 'good-bytes' > "\$out"
fi
FAKE
chmod +x "$dir/bin/curl"
run "$dir" "$dir/dest.tar.xz"
calls="$(cat "$dir/curl-calls" 2>/dev/null || echo 0)"
if [ "$RC" -eq 0 ] && [ "$calls" -eq 2 ] && printf '%s' "$OUT" | grep -q 'sha256 verified'; then
ok "mismatch then match: verified on attempt 2, stops retrying"
else
bad "mismatch then match: verified on attempt 2, stops retrying (rc=$RC calls=$calls out=$OUT)"
fi
# --- case C: every attempt mismatches -> fails closed, no partial file left
dir="$(setup case-c)"
cat > "$dir/bin/curl" <<'FAKE'
#!/bin/sh
out="" prev=""
for a in "$@"; do
[ "$prev" = "-o" ] && out="$a"
prev="$a"
done
printf 'always-bad' > "$out"
FAKE
chmod +x "$dir/bin/curl"
run "$dir" "$dir/dest.tar.xz"
if [ "$RC" -ne 0 ] && printf '%s' "$OUT" | grep -q 'FATAL: could not fetch'; then
ok "persistent mismatch: exits nonzero with FATAL"
else
bad "persistent mismatch: exits nonzero with FATAL (rc=$RC, out=$OUT)"
fi
if [ ! -e "$dir/dest.tar.xz" ]; then
ok "persistent mismatch: no partial/corrupt tarball left at the destination"
else
bad "persistent mismatch: no partial/corrupt tarball left at the destination"
fi
attempts="$(printf '%s' "$OUT" | grep -c '== fetching')"
if [ "$attempts" -eq 3 ]; then
ok "persistent mismatch: exactly 3 attempts"
else
bad "persistent mismatch: exactly 3 attempts (got $attempts)"
fi
# --- guard: the fetch carries a connect and overall timeout ---------------
# Without these, --retry never fires (it only re-attempts a transfer curl has
# already decided failed) and a connection that opens then stalls blocks
# forever -- exactly the routing failure this workspace sees from some hosts.
if grep -q -- '--connect-timeout' "$SCRIPT" && grep -q -- '--max-time' "$SCRIPT"; then
ok "fetch carries --connect-timeout and --max-time"
else
bad "fetch carries --connect-timeout and --max-time"
fi
if [ "$FAIL" -eq 0 ]; then
echo "ALL FETCH-BUILDROOT-TARBALL TESTS PASSED"
else
echo "FETCH-BUILDROOT-TARBALL TESTS FAILED"
fi
exit "$FAIL"
+304
View File
@@ -0,0 +1,304 @@
#!/usr/bin/env bash
# Regression and coverage tests for build/fetch-vendor.sh.
#
# The script had no test of any kind before this: not the --check/--fetch
# state machine (MISSING/OK/DRIFTED, the lvgl/luckfox-pico path mapping, the
# "locally modified" annotation), not --help, not the clone stall guard. Two
# concrete regressions motivate the first two cases:
#
# - --help sliced its own source with a hardcoded line range that stopped
# one line too late, so it printed "set -uo pipefail" -- the first line
# of code -- as the last line of help text.
# - git clone ran with no bound on a stalled transfer: a dead peer or a
# wedged proxy mid-clone (the luckfox-pico tree alone is ~21 GB) hung the
# script forever with no way for a caller to tell "still working" from
# "wedged". The fix sets GIT_HTTP_LOW_SPEED_LIMIT/TIME so a stalled
# transfer aborts while a merely slow one is left alone.
#
# --help runs the shipping script directly. The --fetch/--check cases run a
# copy of it in a private fixture dir (its manifest lookup is relative to
# itself, so a copy is how its own HERE-relative resolution can be pointed at
# a manifest we control) against local, throwaway origin repos -- no network,
# and the real git binary does the work throughout (a thin logging wrapper
# only intercepts "clone" to record the env it saw, then execs straight
# through), so these exercise the shipping script's actual git calls.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/fetch-vendor.sh"
[ -f "$SCRIPT" ] || { echo "FAIL: script not found at $SCRIPT"; exit 1; }
FAIL=0
ok() { printf '[PASS] %s\n' "$1"; }
bad() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
GITC() { git -c user.email=t@t.invalid -c user.name=t "$@"; }
# mkrepo1 <dir> -> one commit on main, prints its hash
mkrepo1() {
local dir="$1"
git init -q -b main "$dir"
printf 'v1\n' > "$dir/file.txt"
git -C "$dir" add file.txt
GITC -C "$dir" commit -q -m v1
git -C "$dir" rev-parse HEAD
}
# mkrepo2 <dir> -> two commits on main, prints "C1 C2" (C1 older)
mkrepo2() {
local dir="$1" c1 c2
git init -q -b main "$dir"
printf 'v1\n' > "$dir/file.txt"
git -C "$dir" add file.txt
GITC -C "$dir" commit -q -m v1
c1="$(git -C "$dir" rev-parse HEAD)"
printf 'v2\n' > "$dir/file.txt"
git -C "$dir" add file.txt
GITC -C "$dir" commit -q -m v2
c2="$(git -C "$dir" rev-parse HEAD)"
printf '%s %s\n' "$c1" "$c2"
}
# ---------------------------------------------------------------------------
# Regression: --help must not spill into the script's own code.
# ---------------------------------------------------------------------------
HELP_OUT="$(bash "$SCRIPT" --help)"
if printf '%s\n' "$HELP_OUT" | grep -q 'set -uo pipefail'; then
bad "--help does not print the script's own code"
else
ok "--help does not print the script's own code"
fi
LAST_LINE="$(printf '%s\n' "$HELP_OUT" | tail -1)"
if [ "$LAST_LINE" = "# someone's debugging session, not something to silently throw away." ]; then
ok "--help ends on the last comment line, not past it"
else
bad "--help ends on the last comment line, not past it (got: $LAST_LINE)"
fi
# ---------------------------------------------------------------------------
# Regression: the clone stall guard reaches git clone's environment.
# ---------------------------------------------------------------------------
REALGIT="$(command -v git)"
FAKEBIN="$WORK/fakebin"
mkdir -p "$FAKEBIN"
CAPTURE="$WORK/clone-env.txt"
cat > "$FAKEBIN/git" <<FAKEGIT
#!/bin/sh
if [ "\$1" = "clone" ]; then
printf '%s %s\n' "\$GIT_HTTP_LOW_SPEED_LIMIT" "\$GIT_HTTP_LOW_SPEED_TIME" >> "$CAPTURE"
fi
exec "$REALGIT" "\$@"
FAKEGIT
chmod +x "$FAKEBIN/git"
ORIGIN_GUARD="$WORK/origin-guard"
GUARD_C="$(mkrepo1 "$ORIGIN_GUARD")"
MANIFEST_GUARD="$WORK/manifest-guard"
printf 'widget\t%s\t%s\tguard test tree\n' "$ORIGIN_GUARD" "$GUARD_C" > "$MANIFEST_GUARD"
# fetch-vendor.sh finds its manifest next to itself, so give it a private
# fixture dir carrying a copy of the real script alongside our manifest.
GUARD_FIXTURE="$WORK/guard-fixture"
mkdir -p "$GUARD_FIXTURE"
cp "$SCRIPT" "$GUARD_FIXTURE/fetch-vendor.sh"
cp "$MANIFEST_GUARD" "$GUARD_FIXTURE/vendor.manifest"
: > "$CAPTURE"
PATH="$FAKEBIN:$PATH" bash "$GUARD_FIXTURE/fetch-vendor.sh" --fetch "$WORK/vendor-default" \
> "$WORK/guard-default.log" 2>&1
if [ "$(cat "$CAPTURE")" = "1000 60" ]; then
ok "clone runs with the default low-speed guard (1000 bytes/sec, 60s)"
else
bad "clone runs with the default low-speed guard (got: $(cat "$CAPTURE" 2>/dev/null))"
fi
: > "$CAPTURE"
PATH="$FAKEBIN:$PATH" WARDEN_VENDOR_LOW_SPEED_LIMIT=5 WARDEN_VENDOR_LOW_SPEED_TIME=9 \
bash "$GUARD_FIXTURE/fetch-vendor.sh" --fetch "$WORK/vendor-override" \
> "$WORK/guard-override.log" 2>&1
if [ "$(cat "$CAPTURE")" = "5 9" ]; then
ok "the low-speed guard is overridable"
else
bad "the low-speed guard is overridable (got: $(cat "$CAPTURE" 2>/dev/null))"
fi
# ---------------------------------------------------------------------------
# Coverage: the --check/--fetch state machine and the name-to-path mapping.
# ---------------------------------------------------------------------------
ORIGIN_WIDGET="$WORK/origin-widget"
read -r WIDGET_C1 WIDGET_C2 <<< "$(mkrepo2 "$ORIGIN_WIDGET")"
ORIGIN_LVGL="$WORK/origin-lvgl"
LVGL_C="$(mkrepo1 "$ORIGIN_LVGL")"
ORIGIN_SDK="$WORK/origin-sdk"
SDK_C="$(mkrepo1 "$ORIGIN_SDK")"
MANIFEST="$WORK/vendor.manifest"
write_manifest() { # write_manifest <widget-commit>
{
printf 'widget\t%s\t%s\tgeneric tree, default path mapping\n' "$ORIGIN_WIDGET" "$1"
printf 'lvgl\t%s\t%s\tlvgl name maps under ui/lvgl\n' "$ORIGIN_LVGL" "$LVGL_C"
printf 'luckfox-pico\t%s\t%s\tluckfox-pico name maps under sdk\n' "$ORIGIN_SDK" "$SDK_C"
} > "$MANIFEST"
}
STATE_FIXTURE="$WORK/state-fixture"
mkdir -p "$STATE_FIXTURE"
cp "$SCRIPT" "$STATE_FIXTURE/fetch-vendor.sh"
VENDOR_DIR="$WORK/vendor"
write_manifest "$WIDGET_C1"
cp "$MANIFEST" "$STATE_FIXTURE/vendor.manifest"
# Case 1: nothing cloned yet -> --check reports MISSING for all three, rc=1.
OUT="$(bash "$STATE_FIXTURE/fetch-vendor.sh" --check "$VENDOR_DIR")"; RC=$?
if [ "$RC" -ne 0 ] \
&& printf '%s\n' "$OUT" | grep -q '^MISSING widget' \
&& printf '%s\n' "$OUT" | grep -q '^MISSING lvgl' \
&& printf '%s\n' "$OUT" | grep -q '^MISSING luckfox-pico'; then
ok "--check reports MISSING and rc=1 when nothing is cloned"
else
bad "--check reports MISSING and rc=1 when nothing is cloned (rc=$RC)"
fi
# Case 2: --fetch clones each tree under its mapped path and checks out the pin.
OUT="$(bash "$STATE_FIXTURE/fetch-vendor.sh" --fetch "$VENDOR_DIR")"; RC=$?
if [ "$RC" -eq 0 ] \
&& [ -e "$VENDOR_DIR/widget/.git" ] \
&& [ -e "$VENDOR_DIR/ui/lvgl/.git" ] \
&& [ -e "$VENDOR_DIR/sdk/.git" ]; then
ok "--fetch clones lvgl under ui/lvgl and luckfox-pico under sdk"
else
bad "--fetch clones lvgl under ui/lvgl and luckfox-pico under sdk (rc=$RC)"
fi
if [ "$(git -C "$VENDOR_DIR/widget" rev-parse HEAD 2>/dev/null)" = "$WIDGET_C1" ]; then
ok "--fetch checks out the manifest-pinned commit"
else
bad "--fetch checks out the manifest-pinned commit"
fi
# Case 3: a checkout sitting at its pin --check's clean, rc=0.
OUT="$(bash "$STATE_FIXTURE/fetch-vendor.sh" --check "$VENDOR_DIR")"; RC=$?
if [ "$RC" -eq 0 ] && printf '%s\n' "$OUT" | grep -q "^OK widget ${WIDGET_C1:0:12}$"; then
ok "--check reports OK with no suffix for a clean checkout at the pin"
else
bad "--check reports OK with no suffix for a clean checkout at the pin"
fi
# Case 4: manifest moves to a commit the checkout is not on -> DRIFTED,
# rc=1, and the checkout itself is left untouched (never reset).
write_manifest "$WIDGET_C2"
cp "$MANIFEST" "$STATE_FIXTURE/vendor.manifest"
OUT="$(bash "$STATE_FIXTURE/fetch-vendor.sh" --check "$VENDOR_DIR")"; RC=$?
if [ "$RC" -ne 0 ] \
&& printf '%s\n' "$OUT" | grep -q "^DRIFTED widget want ${WIDGET_C2:0:12} have ${WIDGET_C1:0:12}"; then
ok "--check reports DRIFTED when HEAD does not match the pin"
else
bad "--check reports DRIFTED when HEAD does not match the pin"
fi
if [ "$(git -C "$VENDOR_DIR/widget" rev-parse HEAD 2>/dev/null)" = "$WIDGET_C1" ]; then
ok "a drifted checkout is reported, never reset"
else
bad "a drifted checkout is reported, never reset"
fi
# Case 5: back at the pin but with an uncommitted local change -> OK, but
# annotated, and still rc=0 (a dirty vendor tree is expected, not a failure).
write_manifest "$WIDGET_C1"
cp "$MANIFEST" "$STATE_FIXTURE/vendor.manifest"
echo "local debugging change" >> "$VENDOR_DIR/widget/file.txt"
OUT="$(bash "$STATE_FIXTURE/fetch-vendor.sh" --check "$VENDOR_DIR")"; RC=$?
if [ "$RC" -eq 0 ] && printf '%s\n' "$OUT" | grep -q "^OK widget ${WIDGET_C1:0:12} (locally modified)$"; then
ok "--check reports OK (locally modified) for a dirty checkout at the pin, rc=0"
else
bad "--check reports OK (locally modified) for a dirty checkout at the pin, rc=0"
fi
# ---------------------------------------------------------------------------
# Case 6: git clone fails (bad origin) -> reported, rc=1, no directory left
# behind for that tree, and -- the actual regression this guards -- the loop
# still reaches the remaining manifest entries and reports exactly one
# failure line for widget, not a second "FAILED to check out" once the
# clone's own continue has fired.
# ---------------------------------------------------------------------------
ORIGIN_BAD="$WORK/no-such-origin"
CLONEFAIL_FIXTURE="$WORK/clonefail-fixture"
mkdir -p "$CLONEFAIL_FIXTURE"
cp "$SCRIPT" "$CLONEFAIL_FIXTURE/fetch-vendor.sh"
{
printf 'widget\t%s\t%s\tbad origin, clone must fail\n' "$ORIGIN_BAD" "$WIDGET_C1"
printf 'lvgl\t%s\t%s\tlvgl name maps under ui/lvgl\n' "$ORIGIN_LVGL" "$LVGL_C"
printf 'luckfox-pico\t%s\t%s\tluckfox-pico name maps under sdk\n' "$ORIGIN_SDK" "$SDK_C"
} > "$CLONEFAIL_FIXTURE/vendor.manifest"
CLONEFAIL_DIR="$WORK/vendor-clonefail"
OUT="$(bash "$CLONEFAIL_FIXTURE/fetch-vendor.sh" --fetch "$CLONEFAIL_DIR" 2>&1)"; RC=$?
FAILED_COUNT="$(printf '%s\n' "$OUT" | grep -c 'FAILED to clone widget')"
if [ "$RC" -ne 0 ] && [ "$FAILED_COUNT" -eq 1 ] \
&& ! printf '%s\n' "$OUT" | grep -q 'FAILED to check out'; then
ok "--fetch reports FAILED to clone once and rc=1 for a bad origin"
else
bad "--fetch reports FAILED to clone once and rc=1 for a bad origin (rc=$RC, count=$FAILED_COUNT)"
fi
if [ ! -e "$CLONEFAIL_DIR/widget/.git" ]; then
ok "a failed clone leaves no checkout behind for that tree"
else
bad "a failed clone leaves no checkout behind for that tree"
fi
if [ -e "$CLONEFAIL_DIR/ui/lvgl/.git" ] && [ -e "$CLONEFAIL_DIR/sdk/.git" ]; then
ok "a clone failure on one tree does not stop the remaining trees from being fetched"
else
bad "a clone failure on one tree does not stop the remaining trees from being fetched"
fi
# ---------------------------------------------------------------------------
# Case 7: git checkout fails (pinned commit missing from the origin) ->
# reported, rc=1, the remaining trees still get fetched, and the checkout is
# left wherever the failed checkout left it -- a later --check must report
# that as DRIFTED, never mistake it for success.
# ---------------------------------------------------------------------------
BOGUS_COMMIT="deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"
CHECKOUTFAIL_FIXTURE="$WORK/checkoutfail-fixture"
mkdir -p "$CHECKOUTFAIL_FIXTURE"
cp "$SCRIPT" "$CHECKOUTFAIL_FIXTURE/fetch-vendor.sh"
{
printf 'widget\t%s\t%s\tcommit missing from the origin, checkout must fail\n' "$ORIGIN_WIDGET" "$BOGUS_COMMIT"
printf 'lvgl\t%s\t%s\tlvgl name maps under ui/lvgl\n' "$ORIGIN_LVGL" "$LVGL_C"
printf 'luckfox-pico\t%s\t%s\tluckfox-pico name maps under sdk\n' "$ORIGIN_SDK" "$SDK_C"
} > "$CHECKOUTFAIL_FIXTURE/vendor.manifest"
CHECKOUTFAIL_DIR="$WORK/vendor-checkoutfail"
OUT="$(bash "$CHECKOUTFAIL_FIXTURE/fetch-vendor.sh" --fetch "$CHECKOUTFAIL_DIR" 2>&1)"; RC=$?
if [ "$RC" -ne 0 ] && printf '%s\n' "$OUT" | grep -q "FAILED to check out $BOGUS_COMMIT"; then
ok "--fetch reports FAILED to check out for a commit missing from the origin"
else
bad "--fetch reports FAILED to check out for a commit missing from the origin (rc=$RC)"
fi
# The regression this guards: if the checkout failure's own "continue" were
# ever dropped, the same loop iteration falls through into the have-vs-pin
# comparison below and prints a bogus DRIFTED/OK line for widget in this same
# --fetch run, on top of the FAILED line above.
if ! printf '%s\n' "$OUT" | grep -qE '^(DRIFTED {2}|OK {7})widget'; then
ok "a checkout failure does not fall through to a DRIFTED/OK line in the same run"
else
bad "a checkout failure does not fall through to a DRIFTED/OK line in the same run"
fi
if [ -e "$CHECKOUTFAIL_DIR/widget/.git" ] \
&& [ -e "$CHECKOUTFAIL_DIR/ui/lvgl/.git" ] && [ -e "$CHECKOUTFAIL_DIR/sdk/.git" ]; then
ok "a checkout failure on one tree does not stop the remaining trees from being fetched"
else
bad "a checkout failure on one tree does not stop the remaining trees from being fetched"
fi
OUT="$(bash "$CHECKOUTFAIL_FIXTURE/fetch-vendor.sh" --check "$CHECKOUTFAIL_DIR" 2>&1)"; RC=$?
if [ "$RC" -ne 0 ] && printf '%s\n' "$OUT" | grep -q "^DRIFTED widget want ${BOGUS_COMMIT:0:12}"; then
ok "a later --check reports the failed checkout as DRIFTED, never as success"
else
bad "a later --check reports the failed checkout as DRIFTED, never as success"
fi
[ "$FAIL" -eq 0 ] && echo "All fetch-vendor tests passed." || echo "Some fetch-vendor tests failed."
exit "$FAIL"
+124
View File
@@ -0,0 +1,124 @@
#!/bin/bash
# Regression tests for the post-build boot.img validation in
# build/mk-bootimg.sh: the FIT metadata %512 check, the per-image
# data-position %512 check (and fdtget's own failure path, issue #22 --
# a missing or erroring fdtget used to be swallowed and treated as nothing
# to check), and the >=4096 embedded-data-FIT check.
#
# Runs the real script end to end with stub mkimage/resource_tool/fdtget so
# the arithmetic is exercised as it actually ships, not copied out and
# re-tested in isolation. The stub mkimage answers the -B capability probe
# and, for the real build invocation, writes a synthetic boot.img whose
# 4-byte big-endian metadata-size word and total length are test-controlled --
# the same field mk-bootimg.sh reads with `od -An -tu4 -j4 -N4 --endian=big`.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/mk-bootimg.sh"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
STUBS="$TMP/stubs"
mkdir -p "$STUBS"
cat > "$STUBS/mkimage" <<'EOF'
#!/bin/bash
if [ "$#" -eq 0 ]; then
echo "Usage: mkimage [-T type] -l image" >&2
echo " -B => align size in hex for FIT structure and header" >&2
exit 1
fi
out=""
for a in "$@"; do out="$a"; done
python3 -c '
import sys
out, meta, total = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
total = max(total, 8)
data = bytearray(total)
data[4:8] = (meta & 0xffffffff).to_bytes(4, "big")
open(out, "wb").write(bytes(data))
' "$out" "${MKBI_META:-512}" "${MKBI_TOTAL:-8192}"
EOF
chmod +x "$STUBS/mkimage"
cat > "$STUBS/resource_tool" <<'EOF'
#!/bin/bash
img=""
for a in "$@"; do
case "$a" in --image=*) img="${a#--image=}" ;; esac
done
echo stub > "$img"
EOF
chmod +x "$STUBS/resource_tool"
cat > "$STUBS/fdtget" <<'EOF'
#!/bin/bash
# Invoked as: fdtget -t u FILE /images/NAME data-position
node="$4"
name="${node#/images/}"
var="FDTGET_$(printf '%s' "$name" | tr '[:lower:]' '[:upper:]')_POS"
val="${!var-}"
if [ -z "$val" ] || [ "$val" = "FAIL" ]; then
echo "fdtget: FDT_ERR_NOTFOUND, $node data-position" >&2
exit 1
fi
echo "$val"
EOF
chmod +x "$STUBS/fdtget"
KERNEL="$TMP/zImage"; DTB="$TMP/rv1106-warden.dtb"
head -c 4096 /dev/urandom > "$KERNEL"
head -c 512 /dev/urandom > "$DTB"
fails=0
run_case() { # run_case <name> <want_rc:ok|fail> <want_grep>
local name="$1" want_rc="$2" want_grep="$3" out rc
out="$(PATH="$STUBS:$PATH" "$SCRIPT" --kernel "$KERNEL" --dtb "$DTB" \
--resource-tool "$STUBS/resource_tool" --out "$TMP/boot.img" 2>&1)"
rc=$?
if [ "$want_rc" = ok ] && [ "$rc" -ne 0 ]; then
echo "FAIL: $name (expected success, got rc=$rc: $out)"; fails=$((fails + 1)); return
fi
if [ "$want_rc" = fail ] && [ "$rc" -eq 0 ]; then
echo "FAIL: $name (expected failure, script exited 0: $out)"; fails=$((fails + 1)); return
fi
if ! printf '%s' "$out" | grep -qF -- "$want_grep"; then
echo "FAIL: $name (output missing '$want_grep'): $out"; fails=$((fails + 1)); return
fi
echo "PASS: $name"
}
# A well-formed image: metadata 512-aligned and small, every data-position
# 512-aligned. The baseline every failure case below is a single change from.
MKBI_META=512 MKBI_TOTAL=8192 \
FDTGET_FDT_POS=512 FDTGET_KERNEL_POS=1024 FDTGET_RESOURCE_POS=1536 \
run_case "well-formed image accepted" ok "FIT metadata 512 bytes"
# Metadata size itself not a multiple of 512 (the exact value measured from
# the SDK's vendored mkimage 2017.09, see the comment above the probe).
MKBI_META=1064 MKBI_TOTAL=8192 \
FDTGET_FDT_POS=512 FDTGET_KERNEL_POS=1024 FDTGET_RESOURCE_POS=1536 \
run_case "unaligned metadata size rejected" fail "not a multiple of 512"
# One sub-image's data-position not a multiple of 512; the FATAL must name it.
MKBI_META=512 MKBI_TOTAL=8192 \
FDTGET_FDT_POS=512 FDTGET_KERNEL_POS=148 FDTGET_RESOURCE_POS=1536 \
run_case "unaligned data-position rejected" fail "/images/kernel data-position 148 is not 512-aligned"
# fdtget itself fails (not on PATH, or the FIT it just built is malformed).
# Regression for issue #22: this used to be swallowed by `|| true` and
# treated as "nothing to check" instead of a build failure.
MKBI_META=512 MKBI_TOTAL=8192 \
FDTGET_FDT_POS=512 FDTGET_KERNEL_POS=FAIL FDTGET_RESOURCE_POS=1536 \
run_case "fdtget failure fails the build, not skips the check" fail \
"fdtget could not read /images/kernel data-position"
# Metadata swelled to >=4096 bytes: an embedded-data FIT, which this U-Boot
# rejects outright. 4096 is itself a multiple of 512 so this must be caught
# by the second check, not mistaken for the first.
MKBI_META=4096 MKBI_TOTAL=8192 \
FDTGET_FDT_POS=512 FDTGET_KERNEL_POS=1024 FDTGET_RESOURCE_POS=1536 \
run_case "embedded-data FIT rejected" fail "embedded-data FIT"
[ "$fails" -eq 0 ] && echo "All mk-bootimg boot.img validation tests passed." || echo "$fails test(s) failed."
exit "$fails"
+56
View File
@@ -0,0 +1,56 @@
#!/bin/bash
# Regression test: every offline regression-test script this job ships must
# actually be invoked by the qemu-tools CI job, not just committed.
#
# run-probe-tests.sh (guards issue #17) sat in the tree unwired into
# .github/workflows/ci.yml: it passed by hand but ran nowhere in CI, so a
# regression in the mkimage probe would only have surfaced on the next
# workflow_dispatch kernel build, not on every push/PR. The same gap later
# reopened for six more scripts written the same way, so this now checks
# every one of them (including itself) instead of only the first: it
# isolates the qemu-tools job and checks each script's basename appears in
# its steps.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
CI_YML="$HERE/../../.github/workflows/ci.yml"
[ -f "$CI_YML" ] || { echo "FAIL: no workflow file at $CI_YML"; exit 1; }
# Isolate the qemu-tools job: from its own header line up to (but not
# including) the next job at the same two-space indent.
job="$(awk '
/^ qemu-tools:/ { inside = 1; print; next }
inside && /^ [A-Za-z0-9_-]+:/ { exit }
inside { print }
' "$CI_YML")"
[ -n "$job" ] || { echo "FAIL: qemu-tools job not found in $CI_YML"; exit 1; }
# Every offline regression-test script the qemu-tools job owns. Add new
# scripts here when they are written, not only when someone remembers to
# wire them in -- that is the failure this test exists to catch.
scripts=(
"tests/mk-bootimg/run-probe-tests.sh"
"qemu/tests/run-sh-args-test.sh"
"qemu/tests/seed-dir.sh"
"qemu/tests/stage-rootfs-perms.sh"
"tests/fetch-vendor/run-fetch-vendor-tests.sh"
"tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh"
"tests/mk-bootimg/run-boot-img-validate-tests.sh"
"tests/mk-bootimg/run-help-tests.sh"
"tests/mk-bootimg/run-ci-wiring-tests.sh"
)
fail=0
for s in "${scripts[@]}"; do
base="$(basename "$s")"
if echo "$job" | grep -q -- "$base"; then
echo "PASS: qemu-tools job invokes $s"
else
echo "FAIL: qemu-tools job never runs $s"
fail=1
fi
done
exit "$fail"
+45
View File
@@ -0,0 +1,45 @@
#!/bin/bash
# Regression test: --help must not spill into build/mk-bootimg.sh's own code.
#
# build/fetch-vendor.sh's --help sliced its own source with a hardcoded line
# range that stopped one line too late, printing "set -uo pipefail" -- the
# first line of code -- as the last line of help text. mk-bootimg.sh's
# --help used the same hardcoded-range shape (currently pointed at the right
# span), so the same slip was one header-comment edit away here too. It now
# uses the same self-terminating awk pattern fetch-vendor.sh was fixed to
# use, so this pins both that it stays in sync as the header grows or
# shrinks and that it matches the header verbatim today.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/mk-bootimg.sh"
[ -f "$SCRIPT" ] || { echo "FAIL: script not found at $SCRIPT"; exit 1; }
fail=0
HELP_OUT="$(bash "$SCRIPT" --help)"
if printf '%s\n' "$HELP_OUT" | grep -q 'set -euo pipefail'; then
echo "FAIL: --help prints the script's own code"
fail=1
else
echo "PASS: --help does not print the script's own code"
fi
LAST_LINE="$(printf '%s\n' "$HELP_OUT" | tail -1)"
if [ "$LAST_LINE" = "# [--resource-tool PATH]" ]; then
echo "PASS: --help ends on the last header comment line, not past it"
else
echo "FAIL: --help ends on the last header comment line, not past it (got: $LAST_LINE)"
fail=1
fi
FIRST_LINE="$(printf '%s\n' "$HELP_OUT" | head -1)"
if [ "$FIRST_LINE" = "# Package a bootable boot.img from a kernel this SDK built." ]; then
echo "PASS: --help starts after the shebang, not on it"
else
echo "FAIL: --help starts after the shebang, not on it (got: $FIRST_LINE)"
fail=1
fi
exit "$fail"
+70
View File
@@ -0,0 +1,70 @@
#!/bin/bash
# Regression tests for the mkimage capability probe in build/mk-bootimg.sh.
#
# Guards issue #17: the probe ran the candidate binary and piped it into grep.
# mkimage with no arguments prints usage and exits non-zero, and mk-bootimg.sh
# runs under `set -euo pipefail`, so the pipeline reported failure even when
# grep matched. Every mkimage was rejected and the script failed closed with
# "no mkimage on PATH supports -B" while a capable mkimage sat first on PATH.
#
# The probe is extracted verbatim from mk-bootimg.sh between its sentinel
# comments, so these tests exercise the shipping code rather than a copy.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/mk-bootimg.sh"
PROBE="$(mktemp)"; TMP="$(mktemp -d)"
trap 'rm -rf "$PROBE" "$TMP"' EXIT
sed -n '/^# >>> mkimage-probe/,/^# <<< mkimage-probe/p' "$SCRIPT" > "$PROBE"
grep -q 'MKIMAGE=' "$PROBE" || { echo "FAIL: probe not extracted (sentinels missing)"; exit 1; }
# The probe needs grep on PATH, but the real /usr/bin must stay off it or the
# host's own mkimage would answer these tests. Give it a directory holding
# nothing but grep.
mkdir -p "$TMP/util"
ln -s "$(command -v grep)" "$TMP/util/grep"
# A faithful stand-in for real mkimage: usage on stderr, non-zero exit.
mk_fake() { # mk_fake <dir> <supports-B:0|1>
mkdir -p "$1"
{ echo '#!/bin/sh'
echo 'echo "Usage: mkimage [-T type] -l image" >&2'
[ "$2" = 1 ] && echo 'echo " -B => align size in hex for FIT structure and header" >&2'
echo 'exit 1'
} > "$1/mkimage"
chmod +x "$1/mkimage"
}
fails=0
check() { # check <name> <expected-basename-or-empty> <dir...>
local name="$1" want="$2"; shift 2
local path=""; for d in "$@"; do path="$path${path:+:}$d"; done
path="$path${path:+:}$TMP/util"
local got
# /bin/bash by absolute path: a PATH holding only the fixture dirs cannot
# locate the interpreter itself.
got="$(TEST_PATH="$path" PROBE="$PROBE" /bin/bash -c \
'set -euo pipefail; PATH="$TEST_PATH"; . "$PROBE"; echo "${MKIMAGE:-}"' 2>/dev/null)"
got="${got##*/}"
if [ "$got" = "$want" ]; then
echo "PASS: $name"
else
echo "FAIL: $name (wanted '${want:-<none>}', got '${got:-<none>}')"
fails=$((fails + 1))
fi
}
mk_fake "$TMP/good" 1
mk_fake "$TMP/old" 0
# The regression itself: a capable mkimage that exits non-zero must be accepted.
check "capable mkimage accepted despite non-zero exit" mkimage "$TMP/good"
# The guarantee the FATAL message depends on: no false positives.
check "mkimage without -B rejected" "" "$TMP/old"
# PATH order decides, and an incapable earlier entry must not stop the scan.
check "scan continues past an incapable mkimage" mkimage "$TMP/old" "$TMP/good"
check "no mkimage anywhere selects nothing" "" "$TMP/none"
[ "$fails" -eq 0 ] && echo "All mk-bootimg probe tests passed." || echo "$fails test(s) failed."
exit "$fails"