diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d19c783..e775d50 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -110,7 +110,16 @@ jobs: sudo apt-get update -qq && sudo apt-get install -y -qq shellcheck shellcheck -x qemu/*.sh qemu/tests/*.sh build/*.sh \ qemu/rootfs/etc/warden-lib.sh qemu/rootfs/etc/rc \ - qemu/rootfs/sbin/init qemu/rootfs/init + qemu/rootfs/sbin/init qemu/rootfs/init tests/mk-bootimg/*.sh \ + tests/fetch-vendor/*.sh tests/fetch-buildroot-tarball/*.sh + - name: cache apt archives (python3-pil) + # Same cost class as the busybox binary cached below: a system package + # plus its libjpeg/libpng transitive deps, downloaded fresh on every + # push otherwise. + uses: actions/cache@v4 + with: + path: /var/cache/apt/archives + key: apt-archives-python3-pil-${{ runner.os }} - name: ui-drive driver and image tools (offline) # qmp.py's drive() with QMP and the control channel faked, plus # imgtools' self-test: the per-step ok/fail/fatal contract and the @@ -119,6 +128,59 @@ jobs: sudo apt-get install -y -qq python3-pil python3 qemu/tests/imgtools.py selftest python3 qemu/tests/test_qmp_drive.py + - name: imgtools bench (smoke, printed for trend-watching) + # Same pattern as the sim/rs485-bridge bench job above: no stored + # baseline yet, just a number in the log so a phash/structural + # regression (DCT size, downscale filter, occupancy thresholds) + # is visible instead of only showing up as an unexplained slower + # flow run later. + run: python3 qemu/tests/imgtools.py bench + - name: mk-bootimg probe regression tests + # Guards issue #17 (mkimage's non-zero exit sinking the probe's grep + # pipeline under set -o pipefail) on every push/PR, not only on the + # next workflow_dispatch that happens to exercise mk-bootimg.sh for + # real via kernel-build. + run: bash tests/mk-bootimg/run-probe-tests.sh + - name: run.sh argv ordering regression test + # Pins the CTL-before-RS485 pci-serial argv order that + # rootfs/sbin/init's ttyS0-vs-ttyS1 alias depends on: a swap here + # reproduces run.sh:131-133's own incident, Modbus frames landing + # on the debug channel. All offline (a stub qemu-system-arm on + # PATH), so it runs on every push/PR, not only a real boot. + run: bash qemu/tests/run-sh-args-test.sh + - name: mkimage.sh SEED_DIR regression test + # Only ui-drive.sh --seed (a real VM boot) exercises this hook + # otherwise; this builds the same unprivileged mkfs.ext4 image and + # reads it back with debugfs, no VM needed. + run: bash qemu/tests/seed-dir.sh + - name: qemu_stage_rootfs permission regression test + # Git tracks only the executable bit, so a fresh checkout can land + # the source etc/shadow world-readable under a permissive umask; + # this pins the staged copy at 0600 regardless of the source mode. + run: bash qemu/tests/stage-rootfs-perms.sh + - name: fetch-vendor regression tests + # --check state machine (MISSING/OK/DRIFTED), --help, and the + # clone stall guard, against local throwaway repos: no network. + run: bash tests/fetch-vendor/run-fetch-vendor-tests.sh + - name: fetch-buildroot-tarball regression tests + # Retry-on-mismatch, cleanup, and the already-verified + # short-circuit, against a fake curl on PATH: no network. + run: bash tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh + - name: mk-bootimg boot.img validation regression tests + # Guards issue #22 (a missing/erroring fdtget silently skipping the + # data-position check) plus the FIT metadata and per-image + # data-position %512 checks and the embedded-data-FIT check. + run: bash tests/mk-bootimg/run-boot-img-validate-tests.sh + - name: mk-bootimg --help regression test + # Pins --help against its own header comment so a hardcoded line + # range can't silently start printing code again the next time the + # header grows or shrinks (the bug fetch-vendor.sh's --help had). + run: bash tests/mk-bootimg/run-help-tests.sh + - name: qemu-tools CI wiring regression test + # Catches a regression test shipping in this job without this job + # ever calling it -- the exact gap run-probe-tests.sh sat in before + # the step above wired it in. + run: bash tests/mk-bootimg/run-ci-wiring-tests.sh - name: cache pinned busybox uses: actions/cache@v4 with: diff --git a/README.md b/README.md index 8ff9927..7063d4a 100644 --- a/README.md +++ b/README.md @@ -52,13 +52,12 @@ scenario tests (portal, OTA apply, display + touch, watchdog). | Directory | Contents | |---|---| | `patches/` | the RV1106 forward-port onto pristine linux-6.18.46, subsystem-split | -| `build/` | hermetic kernel build: pinned fetch -> apply patches -> `zImage` + dtb | +| `build/` | hermetic kernel build: pinned fetch -> apply patches -> `zImage` + dtb; `vendor.manifest` pins the third-party trees this platform builds against (LVGL, the vendor RV1106 SDK) to exact commits, and `fetch-vendor.sh` obtains and verifies them | | `qemu/` | device simulator: QEMU `-M virt` boots the real kernel and real userspace | | `sim/` | register-level hardware models (Rust): membus, HPMCU, CRU, Modbus, RGA, NPU | | `drivers/` | hardened hardware-facing drivers: HAL seams, test harnesses | | `kernel/` | forward-port provenance and bring-up records (`patches/` is canonical) | | `tools/` | `config-lint` (static memory-map gates) and dev tooling | -| `build/vendor.manifest` | the third-party trees this platform builds against (LVGL, the vendor RV1106 SDK), pinned to exact commits; `build/fetch-vendor.sh` obtains and verifies them | | `docs/` | architecture, ADRs (`decisions/`), CI/CD | ## Architecture diff --git a/build/fetch-buildroot-tarball.sh b/build/fetch-buildroot-tarball.sh index 84a0b31..21e37da 100755 --- a/build/fetch-buildroot-tarball.sh +++ b/build/fetch-buildroot-tarball.sh @@ -12,6 +12,14 @@ # Same shape as fetch-kernel-tarball.sh, deliberately: a version bump edits this # file and the pin beside it, nothing else. FAILS CLOSED on a missing pin. # +# Not wired into build-firmware.sh or CI yet -- tools/build-firmware.sh stages +# sdk-patches/buildroot/ onto whatever buildroot tree the vendor SDK already +# has, it does not yet extract this pinned tarball over it. Exercised today by +# tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh and by +# running the script by hand; wiring it into the real build path is a separate +# change (it has to reconcile the pristine tree with the existing SDK buildroot +# checkout first). +# # Usage: fetch-buildroot-tarball.sh set -euo pipefail @@ -43,7 +51,11 @@ fi for attempt in 1 2 3; do echo "== fetching buildroot-$BRVER (attempt $attempt)" - if curl -fsSL --retry 2 -o "$TB" "$URL" && verify; then + # --retry only re-fires once curl decides a transfer has failed; a connection + # that opens and then stalls (blackholed route, hung proxy) never reaches + # that decision and would otherwise block forever. --connect-timeout bounds + # the handshake, --max-time bounds the whole request. + if curl -fsSL --retry 2 --connect-timeout 20 --max-time 120 -o "$TB" "$URL" && verify; then echo "buildroot-$BRVER: sha256 verified" exit 0 fi diff --git a/build/fetch-vendor.sh b/build/fetch-vendor.sh index 8392075..8eca88b 100755 --- a/build/fetch-vendor.sh +++ b/build/fetch-vendor.sh @@ -26,7 +26,11 @@ while [ $# -gt 0 ]; do case "$1" in --check) MODE="check"; shift ;; --fetch) MODE="fetch"; shift ;; - -h|--help) sed -n '2,18p' "$0"; exit 0 ;; + # Print the header comment (line 1 is the shebang, so start at 2) and + # stop at the first line of code rather than a hardcoded line count -- + # a fixed range silently starts printing code again the next time the + # header comment grows or shrinks. + -h|--help) awk '/^set /{exit} NR>1{print}' "$0"; exit 0 ;; *) DIR="$1"; shift ;; esac done @@ -34,6 +38,19 @@ done [ -r "$MANIFEST" ] || { echo "FATAL: no manifest at $MANIFEST" >&2; exit 1; } command -v git >/dev/null || { echo "FATAL: git not on PATH" >&2; exit 1; } +# A stalled clone (dead peer, wedged proxy) must not hang this script forever +# with no way for a caller to tell "still working" from "wedged" -- the +# luckfox-pico tree alone is ~21 GB, so a plain wall-clock timeout would also +# abort a clone that is merely slow. Abort only on a stall instead: git's http +# transport already aborts a transfer whose average speed drops below +# LOW_SPEED_LIMIT bytes/sec for LOW_SPEED_TIME seconds straight, so a slow but +# progressing clone is never penalized. Both are overridable for a link that +# is legitimately slow. +: "${WARDEN_VENDOR_LOW_SPEED_LIMIT:=1000}" +: "${WARDEN_VENDOR_LOW_SPEED_TIME:=60}" +export GIT_HTTP_LOW_SPEED_LIMIT="$WARDEN_VENDOR_LOW_SPEED_LIMIT" +export GIT_HTTP_LOW_SPEED_TIME="$WARDEN_VENDOR_LOW_SPEED_TIME" + if [ -z "$DIR" ]; then DIR="${WARDEN_VENDOR_DIR:-$HOME/projects/scada/flare-edge}" fi diff --git a/build/mk-bootimg.sh b/build/mk-bootimg.sh index 36ef6d7..e7c08f2 100755 --- a/build/mk-bootimg.sh +++ b/build/mk-bootimg.sh @@ -42,7 +42,12 @@ while [ $# -gt 0 ]; do # displays it on an ordinary boot. --logo-verbose) LOGO_VERBOSE="${2:?}"; shift 2 ;; --resource-tool) RTOOL="${2:?}"; shift 2 ;; - -h|--help) sed -n '2,25p' "$0"; exit 0 ;; + # Print the header comment (line 1 is the shebang, so start at 2) and + # stop at the first line of code rather than a hardcoded line count -- + # a fixed range silently starts printing code again the next time the + # header comment grows or shrinks (see build/fetch-vendor.sh's own + # --help, which had this exact bug). + -h|--help) awk '/^set /{exit} NR>1{print}' "$0"; exit 0 ;; *) echo "FATAL: unknown argument '$1'" >&2; exit 1 ;; esac done @@ -52,6 +57,11 @@ done [ -n "$OUT" ] || { echo "FATAL: --out is required" >&2; exit 1; } command -v mkimage >/dev/null || { echo "FATAL: mkimage not on PATH (Debian/Ubuntu: u-boot-tools)" >&2; exit 1; } +# fdtget backs the post-build alignment assertions below (the whole point of +# which is that a misaligned image boots fine in CI and fails on a panel), so +# its absence must fail the build rather than silently skip those checks. +command -v fdtget >/dev/null || { + echo "FATAL: fdtget not on PATH (Debian/Ubuntu: device-tree-compiler)" >&2; exit 1; } # resource_tool is a Rockchip host tool. It has no free-standing source here, so # it is taken from the vendor SDK when one is present rather than vendored as a @@ -160,11 +170,6 @@ ITS # vendor's -p value -- the absolute position of the first payload -- not an # alignment. # -# The flag is feature-detected because the SDK vendors mkimage 2017.09, -# which has no -B at all and dies with "invalid option -- 'B'". Its packer -# already 512-aligns, so omitting the flag there is correct rather than a -# fallback. project/build.sh prepends the SDK tool dir to PATH, so that -# binary IS what a build inside the SDK environment resolves. # A -B-capable mkimage is REQUIRED, not preferred. The SDK vendors 2017.09, # which has no -B, and project/build.sh:64 puts it first on PATH -- so the # wrong one is what a build inside the SDK environment picks up. Measured: @@ -198,16 +203,24 @@ echo "== FIT (external data, -E -p 0x800 -B 0x200) using $MKIMAGE" # Assert what U-Boot actually requires, on every build: the failure is silent -- # a misread offset does not fail the build, it fails on a panel, and sometimes -# only as a missing logo. -_meta="$(od -An -tu4 -j4 -N4 --endian=big "$WORKDIR/boot.img" | tr -d ' ')" -if [ $(( _meta % 512 )) -ne 0 ]; then - echo "FATAL: FIT metadata is $_meta bytes, not a multiple of 512;" >&2 +# only as a missing logo. Computed once and reused below (the embedded-data-FIT +# check further down needs the same value) so a future fix to how this is read +# cannot land in one check and not the other. +meta="$(od -An -tu4 -j4 -N4 --endian=big "$WORKDIR/boot.img" | tr -d ' ')" +if [ $(( meta % 512 )) -ne 0 ]; then + echo "FATAL: FIT metadata is $meta bytes, not a multiple of 512;" >&2 echo " FIT_ALIGN would round it up and every payload reads late" >&2 exit 1 fi for _n in fdt kernel resource; do - _pos="$(fdtget -t u "$WORKDIR/boot.img" "/images/$_n" data-position 2>/dev/null || true)" - [ -n "$_pos" ] || continue + # fdtget's presence is checked up front; a failure here means the FIT this + # script just built is malformed, not that the field is legitimately + # absent (mkimage -E gives every one of these images a data-position). Fail + # loud rather than treat an empty read as nothing to check. + if ! _pos="$(fdtget -t u "$WORKDIR/boot.img" "/images/$_n" data-position 2>&1)"; then + echo "FATAL: fdtget could not read /images/$_n data-position: $_pos" >&2 + exit 1 + fi if [ $(( _pos % 512 )) -ne 0 ]; then echo "FATAL: /images/$_n data-position $_pos is not 512-aligned;" >&2 echo " U-Boot's truncating block divide would read the wrong offset" >&2 @@ -218,7 +231,6 @@ done # A FIT whose metadata swelled to the size of the whole image is an # embedded-data build, which this U-Boot rejects. Catch it here rather than on # a panel that will not come back. -meta="$(od -An -tu4 -j4 -N4 --endian=big "$WORKDIR/boot.img" | tr -d ' ')" total="$(stat -c %s "$WORKDIR/boot.img")" if [ "${meta:-0}" -ge 4096 ] || [ "${meta:-0}" -ge "$total" ]; then echo "FATAL: FIT metadata is ${meta} bytes of a ${total}-byte image: that is an" >&2 diff --git a/build/warden_defconfig b/build/warden_defconfig index 1ca9e8c..88384b8 100644 --- a/build/warden_defconfig +++ b/build/warden_defconfig @@ -1351,7 +1351,14 @@ CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y CONFIG_NETFILTER_XT_NAT=y CONFIG_NETFILTER_XT_TARGET_MASQUERADE=y CONFIG_NF_DEFRAG_IPV4=y -CONFIG_NF_CONNTRACK_IPV4=y +# No NF_CONNTRACK_IPV4 symbol here -- IPv4 conntrack has been unconditional in +# NF_CONNTRACK's core since well before 6.18 (net/ipv4/netfilter/Kconfig has no +# such config). Only stale arch defconfigs (e.g. keystone_defconfig) still set +# it; Kconfig drops an unknown symbol with no warning, so it looked live but +# did nothing. Left out on purpose so this file does not claim a gate that +# does not exist. If a future kernel bump reintroduces a real symbol by this +# name, catch it by diffing the expanded .config, not by functional test alone +# -- see PORT-STATUS.md's own history of a silent-drop hiding a real gap. CONFIG_IP_NF_IPTABLES=y # 6.18 SPLIT THE LEGACY TABLES OUT. IP_NF_FILTER and IP_NF_NAT depend on # IP_NF_IPTABLES_LEGACY, which did not exist in 5.10 -- so copying the vendor diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 4208c91..7eb5b6f 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -13,7 +13,7 @@ execute code on private infrastructure (ADR-0007). | `mcdc` | ubuntu-latest | 100% MC/DC enforced on every `drivers/*/test` (gcc-14 `-fcondition-coverage`). | | `bench` | ubuntu-latest | Smoke-runs the sim + rs485-bridge micro-benchmarks; emits ns/op trend JSON. | | `patches-apply` | ubuntu-latest | Fetches pristine linux-6.18.46 (cached, sha256-verified) and applies `patches/*` in order. | -| `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`; builds the initramfs (pinned busybox) and the A/B disk image. | +| `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`, `tests/mk-bootimg/*.sh`, `tests/fetch-vendor/*.sh`, `tests/fetch-buildroot-tarball/*.sh`; runs the offline regression tests for the mk-bootimg probe, boot.img validation, run.sh argv ordering, mkimage's SEED_DIR hook, staged rootfs permissions, fetch-vendor, fetch-buildroot-tarball, and its own CI wiring; builds the initramfs (pinned busybox) and the A/B disk image. | | `quality` | ubuntu-latest | Codacy-style grade computed in-pipeline: clippy, cppcheck, shellcheck, ruff, lizard, jscpd, cargo-audit feed `tools/quality/score.py` (SQALE debt ratio + a separate worst-of security axis; SonarQube's published thresholds). Uploads `quality.json`; fails if the security grade is worse than C. | | `kernel-build` | ubuntu-latest, **dispatch-only** | apt-installs the cross toolchain + qemu, `build/build-kernel.sh` -> `zImage` + `rv1106-warden.dtb`, QEMU `-M virt` boot smoke (fail-closed), artifact upload (best-effort). Trigger: `gh workflow run ci.yml`. | | `prune-artifacts` | ubuntu-latest, dispatch-only | Deletes `kernel-rv1106` artifacts beyond the newest 3. | diff --git a/kernel/rv1106-enablement/PORT-STATUS.md b/kernel/rv1106-enablement/PORT-STATUS.md index b827c0b..550da2e 100644 --- a/kernel/rv1106-enablement/PORT-STATUS.md +++ b/kernel/rv1106-enablement/PORT-STATUS.md @@ -160,16 +160,18 @@ build-m2.sh reproducible M2 build (zImage + dtb) The full ported tree lives in `flare-edge/research/linux-6.18.46/` (scratch); this dir is the durable, reviewable capture, to become a proper patch series as milestones land. -## Hardware verification of the CURRENT series (2026-09-03, bench panel) +## Hardware verification (2026-09-03) -The series in `../../patches/` was built with `build/build-kernel.sh` + +The current series in `../../patches/` was built with `build/build-kernel.sh` + `build/warden_defconfig` (rockchip gcc 8.3), packaged with the new `build/mk-bootimg.sh`, flashed to the bench panel's inactive slot armed for a single try, and booted. Working on real silicon: display (`/dev/fb0`, `/dev/dri/card0`), backlight, Goodix touch, RGA (`/dev/rga`), eth0, the usb0 gadget, all userspace daemons, and zero kernel faults in dmesg. -M4/M5/M6 above are STALE: display and RGA are in the series and verified here. +M4 (display) and part of M6 (RGA) above are STALE: both are in the series and +verified here. M5 (wifi) and the rest of M6 (watchdog, HPMCU, full USB-OTG +dual role) are still open -- this run gives no evidence for them. **What the same test found missing, and why it matters.** Diffing this defconfig's expansion against the kernel actually shipping on a panel showed diff --git a/qemu/README.md b/qemu/README.md index 9b37250..15e0c07 100644 --- a/qemu/README.md +++ b/qemu/README.md @@ -80,7 +80,10 @@ show up in screendumps at random (issue #18). The guest carries the panel's own `/etc/passwd`, `/etc/shadow` and `/etc/group` (root's md5-crypt of the documented default password), so a screen that verifies the root password against `/etc/shadow` behaves as it -does on a panel instead of rejecting every attempt. +does on a panel instead of rejecting every attempt. `qemu_stage_rootfs()` +(`lib.sh`) forces `etc/shadow` to mode 0600 on every stage: git tracks only +the executable bit, so the checked-out source file's own mode depends on the +checking-out umask and cannot be trusted to arrive non-world-readable. ## Scenarios @@ -88,6 +91,8 @@ All take the virt-fragment ``; `FLARE_EDGE=` where noted. | Scenario | Needs | Proves | |---|---|---| +| `stage-rootfs-perms.sh` | - | offline, no VM: `qemu_stage_rootfs()` always lands `etc/shadow` at 0600, even staged from a source copy deliberately left 0644 | +| `seed-dir.sh` | - | offline, no VM: `mkimage.sh`'s `SEED_DIR` hook lands every seeded file under userdata/warden with its original mode (a 0600 secret included), a seeded file beats a same-named `--state` value, a `SEED_DIR` that isn't a directory fails closed, and so does an individual entry that is a symlink, a subdirectory, or named outside `[A-Za-z0-9_.-]+` -- a hyphenated key such as `gas-plant.devices` still seeds cleanly | | `boot-smoke.sh` | - | sentinel-asserting boot; runs in CI inside kernel-build | | `portal-scenario.sh` | `FLARE_EDGE` | real flared against the desk mock portal: authenticated check-in, desired-state pull, signed tier-1 `.wfw` download; verify/stage/APPLYING as a dry run (no `WARDEN_FW_ALLOW_APPLY`) | | `ota-apply.sh` | `FLARE_EDGE` | the FULL apply: the `.wfw`'s bootable rootfs payload is written to rootfs_b (`run.sh --allow-apply` gates it per boot), the AvbABData in `misc` flips, and slot `_b` boots the applied version | @@ -95,6 +100,7 @@ All take the virt-fragment ``; `FLARE_EDGE=` where noted. | `ui-drive.sh