qemu, build: bound the busybox and kernel fetches
ci / kernel-symbols (push) Successful in 7m23s
ci / bench (pull_request) Successful in 31s
ci / mcdc (pull_request) Successful in 38s
ci / patches-apply (pull_request) Successful in 59s
ci / kernel-symbols (pull_request) Successful in 5m20s
ci / bench (push) Successful in 34s
ci / patches-apply (push) Successful in 1m6s
ci / qemu-tools (push) Successful in 2m37s
ci / prune-artifacts (push) Skipped
ci / kernel-build (push) Skipped
ci / prune-artifacts (pull_request) Skipped
ci / kernel-build (pull_request) Skipped
ci / qemu-tools (pull_request) Successful in 2m46s
ci / test (push) Successful in 40s
ci / mcdc (push) Successful in 33s
ci / test (pull_request) Successful in 43s
ci / quality (push) Successful in 1m56s
ci / badges (push) Skipped
ci / quality (pull_request) Successful in 1m27s
ci / badges (pull_request) Skipped
ci / kernel-symbols (push) Successful in 7m23s
ci / bench (pull_request) Successful in 31s
ci / mcdc (pull_request) Successful in 38s
ci / patches-apply (pull_request) Successful in 59s
ci / kernel-symbols (pull_request) Successful in 5m20s
ci / bench (push) Successful in 34s
ci / patches-apply (push) Successful in 1m6s
ci / qemu-tools (push) Successful in 2m37s
ci / prune-artifacts (push) Skipped
ci / kernel-build (push) Skipped
ci / prune-artifacts (pull_request) Skipped
ci / kernel-build (pull_request) Skipped
ci / qemu-tools (pull_request) Successful in 2m46s
ci / test (push) Successful in 40s
ci / mcdc (push) Successful in 33s
ci / test (pull_request) Successful in 43s
ci / quality (push) Successful in 1m56s
ci / badges (push) Skipped
ci / quality (pull_request) Successful in 1m27s
ci / badges (pull_request) Skipped
qemu_get_busybox and fetch-kernel-tarball.sh ran curl with no connect timeout, stall limit or total cap. A server that answered and then went silent held the flare-edge QEMU UI gate build job until its 30 minute timeout, with no retry (flare-edge #1588, run 7391 job 63953). - qemu/lib.sh: each attempt gives up below 1 KiB/s for 20 s, or after 60 s in all; curl retries once, the 3-attempt loop stays. Limits are overridable through BB_FETCH_* for the tests. - build/fetch-kernel-tarball.sh: 20 s connect timeout, abort below 10 KiB/s for 30 s, 600 s cap. The download lands in a temp file and is renamed on success, so a failed fetch no longer leaves a partial tarball that the next run takes as present and rejects on its sha256. - Tests: busybox-fetch.sh serves a stalling local HTTP endpoint to the real curl (hung for the full 60 s guard before, fails in about 9 s after); new tests/fetch-kernel-tarball suite (bounded argv, no partial file, refetch after a failure, reuse of a matching tarball; 6 of 9 checks red before). Both wired into CI and shellcheck. Refs blueflare/flare-edge#1588 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KerYPVh72hhNgKi3wS5ijG
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
7d18896a0d
commit
44b96074ed
@@ -112,7 +112,7 @@ jobs:
|
||||
qemu/rootfs/etc/warden-lib.sh qemu/rootfs/etc/rc \
|
||||
qemu/rootfs/sbin/init qemu/rootfs/init tests/mk-bootimg/*.sh \
|
||||
tests/fetch-vendor/*.sh tests/fetch-buildroot-tarball/*.sh \
|
||||
tests/unpack-buildroot/*.sh
|
||||
tests/fetch-kernel-tarball/*.sh tests/unpack-buildroot/*.sh
|
||||
- name: qemu init shellcheck regression test
|
||||
run: bash tests/qemu-init/run-init-shellcheck-tests.sh
|
||||
- name: cache apt archives (python3-pil)
|
||||
@@ -175,6 +175,14 @@ jobs:
|
||||
# Retry-on-mismatch, cleanup, and the already-verified
|
||||
# short-circuit, against a fake curl on PATH: no network.
|
||||
run: bash tests/fetch-buildroot-tarball/run-fetch-buildroot-tarball-tests.sh
|
||||
- name: fetch-kernel-tarball regression tests
|
||||
# Bounded curl and no partial tarball after a failed fetch (#1588),
|
||||
# against a fake curl on PATH: no network.
|
||||
run: bash tests/fetch-kernel-tarball/run-fetch-kernel-tarball-tests.sh
|
||||
- name: QEMU busybox fetch regression tests
|
||||
# Retry on a bad sha256, the cached copy, and a server that answers
|
||||
# and then stalls (#1588), against local listeners: no network.
|
||||
run: bash qemu/tests/busybox-fetch.sh
|
||||
- name: unpack-buildroot regression tests
|
||||
# Fresh unpack, leave-existing-tree-alone, and every refusal path
|
||||
# (unpatched selectors, bad tarball, half-present tree, failed
|
||||
|
||||
@@ -22,7 +22,16 @@ TB="${1:?usage: fetch-kernel-tarball.sh <destination-path>}"
|
||||
}
|
||||
if [ ! -f "$TB" ]; then
|
||||
echo "== downloading $URL"
|
||||
curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$URL" -o "$TB"
|
||||
# Bounded (#1588): a connection that opens and then stalls gives up after
|
||||
# 30 s below 10 KiB/s and is retried, instead of holding the job until its
|
||||
# own timeout. The download lands beside $TB and is renamed only when curl
|
||||
# succeeds, so a failed fetch never leaves a partial tarball that the next
|
||||
# run would take as present and then reject on its sha256.
|
||||
tmp="$TB.download.$$"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
curl --retry 2 --retry-delay 5 --retry-connrefused --connect-timeout 20 \
|
||||
--speed-limit 10240 --speed-time 30 --max-time 600 -fSL "$URL" -o "$tmp"
|
||||
mv -f "$tmp" "$TB"
|
||||
fi
|
||||
want="$(cat "$SHA_FILE")"
|
||||
got="$(sha256sum "$TB" | awk '{print $1}')"
|
||||
|
||||
+10
-1
@@ -33,11 +33,20 @@ qemu_get_busybox() {
|
||||
qemu_log "cached busybox failed verification; fetching a clean copy"
|
||||
fi
|
||||
|
||||
# Every attempt is bounded: a server that answers and then stalls would
|
||||
# otherwise hold the job until its own timeout, with no retry (#1588). Below
|
||||
# 1 KiB/s for the stall window, or past the total cap, curl gives up and the
|
||||
# loop moves on. Defaults cap the worst case near 6 minutes; the tests
|
||||
# shorten them through the environment.
|
||||
local stall="${BB_FETCH_STALL_SECS:-20}" max="${BB_FETCH_MAX_SECS:-60}"
|
||||
local delay="${BB_FETCH_RETRY_DELAY:-5}"
|
||||
local attempt tmp="${BB}.download.$$"
|
||||
rm -f "$tmp"
|
||||
for attempt in 1 2 3; do
|
||||
qemu_log "downloading $BB_URL (attempt $attempt/3)"
|
||||
if curl --retry 3 --retry-delay 5 --retry-connrefused -fSL "$BB_URL" -o "$tmp"; then
|
||||
if curl --retry 1 --retry-delay "$delay" --retry-connrefused \
|
||||
--connect-timeout "$stall" --speed-limit 1024 --speed-time "$stall" \
|
||||
--max-time "$max" -fSL "$BB_URL" -o "$tmp"; then
|
||||
got="$(sha256sum "$tmp" | awk '{print $1}')"
|
||||
if [ "$want" = "$got" ]; then
|
||||
mv -f "$tmp" "$BB"
|
||||
|
||||
@@ -69,4 +69,54 @@ cp "$SCRATCH/good" "$SCRATCH/cached/busybox-armv7l"
|
||||
OUT="$SCRATCH/cached" qemu_get_busybox
|
||||
test "$(cat "$FAKE_CURL_COUNT")" = 0
|
||||
|
||||
# A server that answers and then stalls must not hang the fetch (#1588). This
|
||||
# runs the real curl against a local listener that sends headers and a few
|
||||
# bytes, then goes silent; the short limits keep the test fast.
|
||||
PATH="${PATH//"$FAKE_BIN:"/}"
|
||||
python3 - "$SCRATCH/stall-port" <<'STALL' &
|
||||
import os, socket, sys, threading, time
|
||||
srv = socket.socket()
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
srv.bind(("127.0.0.1", 0))
|
||||
srv.listen(16)
|
||||
with open(sys.argv[1] + ".tmp", "w") as f:
|
||||
f.write(str(srv.getsockname()[1]))
|
||||
os.rename(sys.argv[1] + ".tmp", sys.argv[1])
|
||||
held = []
|
||||
def serve(conn):
|
||||
try:
|
||||
conn.recv(4096)
|
||||
conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 1000000\r\n\r\nstall")
|
||||
held.append(conn)
|
||||
time.sleep(600)
|
||||
except OSError:
|
||||
pass
|
||||
while True:
|
||||
c, _ = srv.accept()
|
||||
threading.Thread(target=serve, args=(c,), daemon=True).start()
|
||||
STALL
|
||||
STALL_PID=$!
|
||||
trap 'kill "$STALL_PID" 2>/dev/null || true; rm -rf "$SCRATCH"' EXIT
|
||||
for _ in $(seq 100); do [ -s "$SCRATCH/stall-port" ] && break; sleep 0.1; done
|
||||
test -s "$SCRATCH/stall-port"
|
||||
BB_URL="http://127.0.0.1:$(cat "$SCRATCH/stall-port")/busybox"
|
||||
start=$SECONDS
|
||||
# The bash -c program expands its own positional arguments.
|
||||
# shellcheck disable=SC2016
|
||||
if (OUT="$SCRATCH/stalled" BB_FETCH_STALL_SECS=1 BB_FETCH_MAX_SECS=3 \
|
||||
BB_FETCH_RETRY_DELAY=1 timeout 60 bash -c \
|
||||
'set -euo pipefail; source "$1"; QEMU_DIR="$2"; BB_URL="$3"; qemu_get_busybox' \
|
||||
_ "$TEST_DIR/../lib.sh" "$QEMU_DIR" "$BB_URL") >"$SCRATCH/stall.log" 2>&1; then
|
||||
echo "FAIL: a stalled download was accepted" >&2
|
||||
exit 1
|
||||
fi
|
||||
elapsed=$((SECONDS - start))
|
||||
if ! grep -q 'after 3 verified attempts' "$SCRATCH/stall.log"; then
|
||||
echo "FAIL: a stalled download hung instead of failing (${elapsed}s)" >&2
|
||||
cat "$SCRATCH/stall.log" >&2
|
||||
exit 1
|
||||
fi
|
||||
test "$elapsed" -lt 45
|
||||
test ! -e "$SCRATCH/stalled/busybox-armv7l"
|
||||
|
||||
echo "busybox fetch tests passed"
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regression tests for build/fetch-kernel-tarball.sh (#1588).
|
||||
#
|
||||
# Runs the real script, copied into a fixture dir so its own HERE-relative pin
|
||||
# lookup finds a pin we control, with a fake curl first on PATH: no network,
|
||||
# and the outcome of each fetch is exact. Covers:
|
||||
# - curl is bounded (connect timeout, stall abort, total cap), so a server
|
||||
# that answers and then stalls cannot hold the job until its timeout;
|
||||
# - a failed fetch leaves no partial tarball behind, so the next run fetches
|
||||
# again instead of taking the partial file as present;
|
||||
# - a good fetch is verified and kept; a present, matching tarball is reused.
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||
SCRIPT="$HERE/../../build/fetch-kernel-tarball.sh"
|
||||
[ -f "$SCRIPT" ] || { echo "FAIL: script not found at $SCRIPT"; exit 1; }
|
||||
KVER="$(sed -n 's/^KVER=//p' "$SCRIPT")"
|
||||
[ -n "$KVER" ] || { echo "FAIL: no KVER in $SCRIPT"; exit 1; }
|
||||
|
||||
FAIL=0
|
||||
ok() { printf '[PASS] %s\n' "$1"; }
|
||||
bad() { printf '[FAIL] %s\n' "$1"; FAIL=1; }
|
||||
|
||||
TMP="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
setup() { # setup <case-name> -> prints the fixture dir
|
||||
local dir="$TMP/$1"
|
||||
mkdir -p "$dir/bin"
|
||||
cp "$SCRIPT" "$dir/fetch-kernel-tarball.sh"
|
||||
printf 'good-bytes' | sha256sum | awk '{print $1}' \
|
||||
> "$dir/linux-$KVER.tar.xz.sha256"
|
||||
printf '%s' "$dir"
|
||||
}
|
||||
|
||||
# fake_curl <dir> <mode>: records its argv in <dir>/argv and its call count in
|
||||
# <dir>/count. Modes: good (writes the pinned bytes), partial (writes a few
|
||||
# bytes, then fails the way curl does on a timeout), never (must not run).
|
||||
fake_curl() {
|
||||
local dir="$1" mode="$2"
|
||||
# These single-quoted lines are the literal source of the fake program.
|
||||
# shellcheck disable=SC2016
|
||||
printf '%s\n' '#!/bin/sh' \
|
||||
'dir=$(dirname "$(dirname "$0")")' \
|
||||
'printf "%s\n" "$*" > "$dir/argv"' \
|
||||
'echo x >> "$dir/count"' \
|
||||
'out=' \
|
||||
'while [ "$#" -gt 0 ]; do' \
|
||||
' if [ "$1" = "-o" ]; then shift; out="$1"; fi' \
|
||||
' shift' \
|
||||
'done' \
|
||||
'[ -n "$out" ] || exit 2' \
|
||||
"case $mode in" \
|
||||
' good) printf good-bytes > "$out" ;;' \
|
||||
' partial) printf good > "$out"; exit 28 ;;' \
|
||||
' *) echo "curl should not have run" >&2; exit 1 ;;' \
|
||||
'esac' > "$dir/bin/curl"
|
||||
chmod +x "$dir/bin/curl"
|
||||
}
|
||||
|
||||
run() { # run <dir> <dest> -> output in $OUT, status in $RC
|
||||
local dir="$1" dest="$2"
|
||||
OUT="$(PATH="$dir/bin:$PATH" bash "$dir/fetch-kernel-tarball.sh" "$dest" 2>&1)"
|
||||
RC=$?
|
||||
}
|
||||
|
||||
# --- case A: a good fetch is bounded, verified and kept ---------------------
|
||||
dir="$(setup case-a)"
|
||||
fake_curl "$dir" good
|
||||
run "$dir" "$dir/linux.tar.xz"
|
||||
if [ "$RC" -eq 0 ] && [ -f "$dir/linux.tar.xz" ] \
|
||||
&& printf '%s' "$OUT" | grep -q 'tarball sha256 verified'; then
|
||||
ok "good fetch: verified and kept"
|
||||
else
|
||||
bad "good fetch: verified and kept (rc=$RC, out=$OUT)"
|
||||
fi
|
||||
for flag in --connect-timeout --speed-limit --speed-time --max-time; do
|
||||
if grep -q -- "$flag" "$dir/argv"; then
|
||||
ok "curl carries $flag"
|
||||
else
|
||||
bad "curl carries $flag (argv: $(cat "$dir/argv"))"
|
||||
fi
|
||||
done
|
||||
if ls "$dir"/linux.tar.xz.download.* >/dev/null 2>&1; then
|
||||
bad "good fetch: no download temp file left"
|
||||
else
|
||||
ok "good fetch: no download temp file left"
|
||||
fi
|
||||
|
||||
# --- case B: a failed fetch leaves nothing, and the next run fetches again --
|
||||
dir="$(setup case-b)"
|
||||
fake_curl "$dir" partial
|
||||
run "$dir" "$dir/linux.tar.xz"
|
||||
if [ "$RC" -ne 0 ] && [ ! -e "$dir/linux.tar.xz" ] \
|
||||
&& ! ls "$dir"/linux.tar.xz.download.* >/dev/null 2>&1; then
|
||||
ok "failed fetch: no partial tarball left"
|
||||
else
|
||||
bad "failed fetch: no partial tarball left (rc=$RC, files: $(ls "$dir"))"
|
||||
fi
|
||||
fake_curl "$dir" good
|
||||
run "$dir" "$dir/linux.tar.xz"
|
||||
if [ "$RC" -eq 0 ] && [ "$(wc -l < "$dir/count")" -eq 2 ]; then
|
||||
ok "run after a failed fetch downloads again and passes"
|
||||
else
|
||||
bad "run after a failed fetch downloads again and passes (rc=$RC, out=$OUT)"
|
||||
fi
|
||||
|
||||
# --- case C: a present, matching tarball is reused without curl -------------
|
||||
dir="$(setup case-c)"
|
||||
printf 'good-bytes' > "$dir/linux.tar.xz"
|
||||
fake_curl "$dir" never
|
||||
run "$dir" "$dir/linux.tar.xz"
|
||||
if [ "$RC" -eq 0 ] && [ ! -e "$dir/count" ]; then
|
||||
ok "present tarball: reused, curl never invoked"
|
||||
else
|
||||
bad "present tarball: reused, curl never invoked (rc=$RC, out=$OUT)"
|
||||
fi
|
||||
|
||||
[ "$FAIL" -eq 0 ] && echo "fetch-kernel-tarball tests passed"
|
||||
exit "$FAIL"
|
||||
Reference in New Issue
Block a user