qemu, build: bound the busybox and kernel fetches
ci / kernel-symbols (push) Successful in 7m23s
ci / bench (pull_request) Successful in 31s
ci / mcdc (pull_request) Successful in 38s
ci / patches-apply (pull_request) Successful in 59s
ci / kernel-symbols (pull_request) Successful in 5m20s
ci / bench (push) Successful in 34s
ci / patches-apply (push) Successful in 1m6s
ci / qemu-tools (push) Successful in 2m37s
ci / prune-artifacts (push) Skipped
ci / kernel-build (push) Skipped
ci / prune-artifacts (pull_request) Skipped
ci / kernel-build (pull_request) Skipped
ci / qemu-tools (pull_request) Successful in 2m46s
ci / test (push) Successful in 40s
ci / mcdc (push) Successful in 33s
ci / test (pull_request) Successful in 43s
ci / quality (push) Successful in 1m56s
ci / badges (push) Skipped
ci / quality (pull_request) Successful in 1m27s
ci / badges (pull_request) Skipped

qemu_get_busybox and fetch-kernel-tarball.sh ran curl with no connect
timeout, stall limit or total cap. A server that answered and then went
silent held the flare-edge QEMU UI gate build job until its 30 minute
timeout, with no retry (flare-edge #1588, run 7391 job 63953).

- qemu/lib.sh: each attempt gives up below 1 KiB/s for 20 s, or after
  60 s in all; curl retries once, the 3-attempt loop stays. Limits are
  overridable through BB_FETCH_* for the tests.
- build/fetch-kernel-tarball.sh: 20 s connect timeout, abort below
  10 KiB/s for 30 s, 600 s cap. The download lands in a temp file and is
  renamed on success, so a failed fetch no longer leaves a partial
  tarball that the next run takes as present and rejects on its sha256.
- Tests: busybox-fetch.sh serves a stalling local HTTP endpoint to the
  real curl (hung for the full 60 s guard before, fails in about 9 s
  after); new tests/fetch-kernel-tarball suite (bounded argv, no partial
  file, refetch after a failure, reuse of a matching tarball; 6 of 9
  checks red before). Both wired into CI and shellcheck.

Refs blueflare/flare-edge#1588

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KerYPVh72hhNgKi3wS5ijG
This commit is contained in:
BFE Engineering
2026-10-23 00:23:47 -06:00
co-authored by Claude Opus 5.5
parent 7d18896a0d
commit 44b96074ed
5 changed files with 199 additions and 3 deletions
+9 -1
View File
@@ -112,7 +112,7 @@ jobs:
qemu/rootfs/etc/warden-lib.sh qemu/rootfs/etc/rc \
qemu/rootfs/sbin/init qemu/rootfs/init tests/mk-bootimg/*.sh \
tests/fetch-vendor/*.sh tests/fetch-buildroot-tarball/*.sh \
tests/unpack-buildroot/*.sh
tests/fetch-kernel-tarball/*.sh tests/unpack-buildroot/*.sh
- name: qemu init shellcheck regression test
run: bash tests/qemu-init/run-init-shellcheck-tests.sh
- name: cache apt archives (python3-pil)
@@ -175,6 +175,14 @@ jobs:
# Retry-on-mismatch, cleanup, and the already-verified
# short-circuit, against a fake curl on PATH: no network.
run: bash tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh
- name: fetch-kernel-tarball regression tests
# Bounded curl and no partial tarball after a failed fetch (#1588),
# against a fake curl on PATH: no network.
run: bash tests/fetch-kernel-tarball/run-fetch-kernel-tarball-tests.sh
- name: QEMU busybox fetch regression tests
# Retry on a bad sha256, the cached copy, and a server that answers
# and then stalls (#1588), against local listeners: no network.
run: bash qemu/tests/busybox-fetch.sh
- name: unpack-buildroot regression tests
# Fresh unpack, leave-existing-tree-alone, and every refusal path
# (unpatched selectors, bad tarball, half-present tree, failed
+10 -1
View File
@@ -22,7 +22,16 @@ TB="${1:?usage: fetch-kernel-tarball.sh <destination-path>}"
}
if [ ! -f "$TB" ]; then
echo "== downloading $URL"
curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$URL" -o "$TB"
# Bounded (#1588): a connection that opens and then stalls gives up after
# 30 s below 10 KiB/s and is retried, instead of holding the job until its
# own timeout. The download lands beside $TB and is renamed only when curl
# succeeds, so a failed fetch never leaves a partial tarball that the next
# run would take as present and then reject on its sha256.
tmp="$TB.download.$$"
trap 'rm -f "$tmp"' EXIT
curl --retry 2 --retry-delay 5 --retry-connrefused --connect-timeout 20 \
--speed-limit 10240 --speed-time 30 --max-time 600 -fSL "$URL" -o "$tmp"
mv -f "$tmp" "$TB"
fi
want="$(cat "$SHA_FILE")"
got="$(sha256sum "$TB" | awk '{print $1}')"
+10 -1
View File
@@ -33,11 +33,20 @@ qemu_get_busybox() {
qemu_log "cached busybox failed verification; fetching a clean copy"
fi
# Every attempt is bounded: a server that answers and then stalls would
# otherwise hold the job until its own timeout, with no retry (#1588). Below
# 1 KiB/s for the stall window, or past the total cap, curl gives up and the
# loop moves on. Defaults cap the worst case near 6 minutes; the tests
# shorten them through the environment.
local stall="${BB_FETCH_STALL_SECS:-20}" max="${BB_FETCH_MAX_SECS:-60}"
local delay="${BB_FETCH_RETRY_DELAY:-5}"
local attempt tmp="${BB}.download.$$"
rm -f "$tmp"
for attempt in 1 2 3; do
qemu_log "downloading $BB_URL (attempt $attempt/3)"
if curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$BB_URL" -o "$tmp"; then
if curl --retry 1 --retry-delay "$delay" --retry-connrefused \
--connect-timeout "$stall" --speed-limit 1024 --speed-time "$stall" \
--max-time "$max" -fSL "$BB_URL" -o "$tmp"; then
got="$(sha256sum "$tmp" | awk '{print $1}')"
if [ "$want" = "$got" ]; then
mv -f "$tmp" "$BB"
+50
View File
@@ -69,4 +69,54 @@ cp "$SCRATCH/good" "$SCRATCH/cached/busybox-armv7l"
OUT="$SCRATCH/cached" qemu_get_busybox
test "$(cat "$FAKE_CURL_COUNT")" = 0
# A server that answers and then stalls must not hang the fetch (#1588). This
# runs the real curl against a local listener that sends headers and a few
# bytes, then goes silent; the short limits keep the test fast.
PATH="${PATH//"$FAKE_BIN:"/}"
python3 - "$SCRATCH/stall-port" <<'STALL' &
import os, socket, sys, threading, time
srv = socket.socket()
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", 0))
srv.listen(16)
with open(sys.argv[1] + ".tmp", "w") as f:
f.write(str(srv.getsockname()[1]))
os.rename(sys.argv[1] + ".tmp", sys.argv[1])
held = []
def serve(conn):
try:
conn.recv(4096)
conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 1000000\r\n\r\nstall")
held.append(conn)
time.sleep(600)
except OSError:
pass
while True:
c, _ = srv.accept()
threading.Thread(target=serve, args=(c,), daemon=True).start()
STALL
STALL_PID=$!
trap 'kill "$STALL_PID" 2>/dev/null || true; rm -rf "$SCRATCH"' EXIT
for _ in $(seq 100); do [ -s "$SCRATCH/stall-port" ] && break; sleep 0.1; done
test -s "$SCRATCH/stall-port"
BB_URL="http://127.0.0.1:$(cat "$SCRATCH/stall-port")/busybox"
start=$SECONDS
# The bash -c program expands its own positional arguments.
# shellcheck disable=SC2016
if (OUT="$SCRATCH/stalled" BB_FETCH_STALL_SECS=1 BB_FETCH_MAX_SECS=3 \
BB_FETCH_RETRY_DELAY=1 timeout 60 bash -c \
'set -euo pipefail; source "$1"; QEMU_DIR="$2"; BB_URL="$3"; qemu_get_busybox' \
_ "$TEST_DIR/../lib.sh" "$QEMU_DIR" "$BB_URL") >"$SCRATCH/stall.log" 2>&1; then
echo "FAIL: a stalled download was accepted" >&2
exit 1
fi
elapsed=$((SECONDS - start))
if ! grep -q 'after 3 verified attempts' "$SCRATCH/stall.log"; then
echo "FAIL: a stalled download hung instead of failing (${elapsed}s)" >&2
cat "$SCRATCH/stall.log" >&2
exit 1
fi
test "$elapsed" -lt 45
test ! -e "$SCRATCH/stalled/busybox-armv7l"
echo "busybox fetch tests passed"
@@ -0,0 +1,120 @@
#!/usr/bin/env bash
# Regression tests for build/fetch-kernel-tarball.sh (#1588).
#
# Runs the real script, copied into a fixture dir so its own HERE-relative pin
# lookup finds a pin we control, with a fake curl first on PATH: no network,
# and the outcome of each fetch is exact. Covers:
# - curl is bounded (connect timeout, stall abort, total cap), so a server
# that answers and then stalls cannot hold the job until its timeout;
# - a failed fetch leaves no partial tarball behind, so the next run fetches
# again instead of taking the partial file as present;
# - a good fetch is verified and kept; a present, matching tarball is reused.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
SCRIPT="$HERE/../../build/fetch-kernel-tarball.sh"
[ -f "$SCRIPT" ] || { echo "FAIL: script not found at $SCRIPT"; exit 1; }
KVER="$(sed -n 's/^KVER=//p' "$SCRIPT")"
[ -n "$KVER" ] || { echo "FAIL: no KVER in $SCRIPT"; exit 1; }
FAIL=0
ok() { printf '[PASS] %s\n' "$1"; }
bad() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
setup() { # setup <case-name> -> prints the fixture dir
local dir="$TMP/$1"
mkdir -p "$dir/bin"
cp "$SCRIPT" "$dir/fetch-kernel-tarball.sh"
printf 'good-bytes' | sha256sum | awk '{print $1}' \
> "$dir/linux-$KVER.tar.xz.sha256"
printf '%s' "$dir"
}
# fake_curl <dir> <mode>: records its argv in <dir>/argv and its call count in
# <dir>/count. Modes: good (writes the pinned bytes), partial (writes a few
# bytes, then fails the way curl does on a timeout), never (must not run).
fake_curl() {
local dir="$1" mode="$2"
# These single-quoted lines are the literal source of the fake program.
# shellcheck disable=SC2016
printf '%s\n' '#!/bin/sh' \
'dir=$(dirname "$(dirname "$0")")' \
'printf "%s\n" "$*" > "$dir/argv"' \
'echo x >> "$dir/count"' \
'out=' \
'while [ "$#" -gt 0 ]; do' \
' if [ "$1" = "-o" ]; then shift; out="$1"; fi' \
' shift' \
'done' \
'[ -n "$out" ] || exit 2' \
"case $mode in" \
' good) printf good-bytes > "$out" ;;' \
' partial) printf good > "$out"; exit 28 ;;' \
' *) echo "curl should not have run" >&2; exit 1 ;;' \
'esac' > "$dir/bin/curl"
chmod +x "$dir/bin/curl"
}
run() { # run <dir> <dest> -> output in $OUT, status in $RC
local dir="$1" dest="$2"
OUT="$(PATH="$dir/bin:$PATH" bash "$dir/fetch-kernel-tarball.sh" "$dest" 2>&1)"
RC=$?
}
# --- case A: a good fetch is bounded, verified and kept ---------------------
dir="$(setup case-a)"
fake_curl "$dir" good
run "$dir" "$dir/linux.tar.xz"
if [ "$RC" -eq 0 ] && [ -f "$dir/linux.tar.xz" ] \
&& printf '%s' "$OUT" | grep -q 'tarball sha256 verified'; then
ok "good fetch: verified and kept"
else
bad "good fetch: verified and kept (rc=$RC, out=$OUT)"
fi
for flag in --connect-timeout --speed-limit --speed-time --max-time; do
if grep -q -- "$flag" "$dir/argv"; then
ok "curl carries $flag"
else
bad "curl carries $flag (argv: $(cat "$dir/argv"))"
fi
done
if ls "$dir"/linux.tar.xz.download.* >/dev/null 2>&1; then
bad "good fetch: no download temp file left"
else
ok "good fetch: no download temp file left"
fi
# --- case B: a failed fetch leaves nothing, and the next run fetches again --
dir="$(setup case-b)"
fake_curl "$dir" partial
run "$dir" "$dir/linux.tar.xz"
if [ "$RC" -ne 0 ] && [ ! -e "$dir/linux.tar.xz" ] \
&& ! ls "$dir"/linux.tar.xz.download.* >/dev/null 2>&1; then
ok "failed fetch: no partial tarball left"
else
bad "failed fetch: no partial tarball left (rc=$RC, files: $(ls "$dir"))"
fi
fake_curl "$dir" good
run "$dir" "$dir/linux.tar.xz"
if [ "$RC" -eq 0 ] && [ "$(wc -l < "$dir/count")" -eq 2 ]; then
ok "run after a failed fetch downloads again and passes"
else
bad "run after a failed fetch downloads again and passes (rc=$RC, out=$OUT)"
fi
# --- case C: a present, matching tarball is reused without curl -------------
dir="$(setup case-c)"
printf 'good-bytes' > "$dir/linux.tar.xz"
fake_curl "$dir" never
run "$dir" "$dir/linux.tar.xz"
if [ "$RC" -eq 0 ] && [ ! -e "$dir/count" ]; then
ok "present tarball: reused, curl never invoked"
else
bad "present tarball: reused, curl never invoked (rc=$RC, out=$OUT)"
fi
[ "$FAIL" -eq 0 ] && echo "fetch-kernel-tarball tests passed"
exit "$FAIL"