clock diagnostics for issue #3: probe, VM sanity scenario, fix plan
The qemu/ device sim answered the first question off-board: the same kernel family under -M virt gives musl vDSO rate ratio 0.99963 — the generic 6.18 armv7 vDSO is correct, so the board symptom is RV1106 register state (CNTFRQ/CNTVOFF, firmware-owned, secure-world boot chain). - qemu/tests/clockprobe: interval-based musl probe separating RATE error (CNTFRQ) from boot OFFSET (CNTVOFF) — the original single absolute sample cannot distinguish them. - qemu/tests/clock-sanity.sh: VM regression guard asserting the vDSO rate within 1% (PASSES: 1.00026); cross-builds the probe and stages it as payload itself. - kernel/rv1106-enablement/timer/PLAN.md: the DT fix (arm,cpu-registers-not-fw-configured + measured clock-frequency on the board dts) gated on the two bench measurements; c8a3 is currently physically dark, needs hands at the bench. - clockprobe joins the CI test loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018HUayid7W5w7jBdb9Rrj1K
This commit is contained in:
BFE Engineering
co-authored by
Claude Fable 5
Claude Fable 5
parent
ff62991e8d
commit
1d7dec167e
@@ -35,7 +35,7 @@ jobs:
|
||||
run: |
|
||||
set -o pipefail
|
||||
: > /tmp/test.log
|
||||
for d in sim tools/config-lint qemu/rs485-bridge; do
|
||||
for d in sim tools/config-lint qemu/rs485-bridge qemu/tests/clockprobe; do
|
||||
echo "== cargo test in $d ==" | tee -a /tmp/test.log
|
||||
( cd "$d" && cargo test --locked ) 2>&1 | tee -a /tmp/test.log
|
||||
done
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# arch-timer / vDSO clock fix — plan (issue #3)
|
||||
|
||||
Status: DIAGNOSED off-board, fix gated on two bench measurements.
|
||||
|
||||
## Established (2026-08-30, qemu/ device sim)
|
||||
|
||||
The same kernel family under `qemu-system-arm -M virt` gives musl
|
||||
`CLOCK_MONOTONIC`/`_RAW` rate ratio 0.99963 vs `/proc/uptime` (interval
|
||||
measurement, `qemu/tests/clockprobe`). The generic 6.18 armv7 vDSO is
|
||||
therefore CORRECT; the board symptom (musl reads ~12% high, kernel time
|
||||
right) is RV1106-specific. The boot chain runs in the secure world and is
|
||||
closed rkbin — the NS view of the CPU timer registers (CNTFRQ, CNTVOFF) is
|
||||
whatever it left behind, and only the arch-counter path (vDSO,
|
||||
`arch_sys_counter`) trusts them.
|
||||
|
||||
## What the bench must answer (single boot of the `_b` slot)
|
||||
|
||||
Run `clockprobe` (interval mode) on the 6.18 slot:
|
||||
|
||||
1. `ratio_mono` far from 1.0 -> RATE error: CNTFRQ wrong. The true rate =
|
||||
claimed rate (dmesg `arch_timer: cp15 timer running at X MHz`) times the
|
||||
measured ratio.
|
||||
2. `ratio_mono` ~= 1.0 but `abs_ratio` far from 1.0 -> OFFSET error: CNTVOFF
|
||||
left nonzero; rate fine.
|
||||
|
||||
(The original issue measured only one absolute sample, which cannot
|
||||
distinguish these.)
|
||||
|
||||
## The fix (both cases, one DT override)
|
||||
|
||||
Append to the BOARD dts (`rv1106-warden.dts` — never the vendor dtsi) an
|
||||
override on the armv7-timer node:
|
||||
|
||||
arm,cpu-registers-not-fw-configured;
|
||||
clock-frequency = <MEASURED_HZ>;
|
||||
|
||||
The property makes the driver use the physical counter, ignore CNTVOFF, and
|
||||
take the frequency from DT — the documented remedy for firmware that does
|
||||
not configure the CPU timer registers. `MEASURED_HZ` comes from bench
|
||||
answer 1 (do NOT guess; a wrong value makes every clock wrong instead of
|
||||
one path). Ship as an update to the arch/dts patch in `patches/`.
|
||||
|
||||
## Regression guards
|
||||
|
||||
- Off-board: `qemu/tests/clock-sanity.sh` asserts the vDSO rate in the VM
|
||||
(guards the generic path; cannot see board registers).
|
||||
- On-board: re-run `clockprobe` on the patched `_b` slot; both ratios and
|
||||
the absolute ratio must be ~1.0. Record the numbers here and in issue #3.
|
||||
|
||||
## Bench access note
|
||||
|
||||
2026-08-30: c8a3 is physically dark (CP2102 console silent through two
|
||||
remote power cycles; both network paths down) — needs hands at the bench
|
||||
before the measurements can run.
|
||||
Executable
+92
@@ -0,0 +1,92 @@
|
||||
#!/usr/bin/env bash
|
||||
# Clock-sanity scenario (issue #3 regression guard): boot the VM, run the
|
||||
# musl-static clockprobe in the guest, and assert the vDSO monotonic RATE
|
||||
# matches the kernel's /proc/uptime within 1%. Under QEMU -M virt this passes
|
||||
# on the current kernel (measured 0.99963) — a regression here means the
|
||||
# generic vDSO path broke. The RV1106 *board* leg of issue #3 is a separate,
|
||||
# bench-only measurement; this scenario cannot see board-specific CNTFRQ or
|
||||
# CNTVOFF misprogramming.
|
||||
#
|
||||
# FAILS CLOSED on missing prerequisites.
|
||||
#
|
||||
# Usage: clock-sanity.sh <zImage>
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # qemu/tests/
|
||||
QDIR="$(cd "$HERE/.." && pwd)" # qemu/
|
||||
|
||||
ZIMAGE="${1:-}"
|
||||
if [ -z "$ZIMAGE" ] || [ ! -f "$ZIMAGE" ]; then
|
||||
echo "FATAL: usage: $0 <zImage>" >&2
|
||||
exit 1
|
||||
fi
|
||||
command -v qemu-system-arm >/dev/null || {
|
||||
echo "FATAL: qemu-system-arm not on PATH — see qemu/README.md" >&2
|
||||
exit 1
|
||||
}
|
||||
command -v arm-linux-gnueabihf-gcc >/dev/null || {
|
||||
echo "FATAL: arm-linux-gnueabihf-gcc needed to cross-build the probe" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Build the probe for the guest (static musl armv7) and stage it as payload.
|
||||
( cd "$HERE/clockprobe" && \
|
||||
CARGO_TARGET_ARMV7_UNKNOWN_LINUX_MUSLEABIHF_LINKER=arm-linux-gnueabihf-gcc \
|
||||
cargo build -q --release --target armv7-unknown-linux-musleabihf )
|
||||
install -m 0755 "$HERE/clockprobe/target/armv7-unknown-linux-musleabihf/release/clockprobe" \
|
||||
"$QDIR/payload/clockprobe"
|
||||
|
||||
WORK="$(mktemp -d /tmp/wqc.XXXXXX)"
|
||||
QEMU_PID=""
|
||||
cleanup() {
|
||||
if [ -n "$QEMU_PID" ]; then kill "$QEMU_PID" 2>/dev/null || true; fi
|
||||
rm -rf "$WORK"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
bash "$QDIR/mkinitramfs.sh"
|
||||
bash "$QDIR/mkimage.sh"
|
||||
|
||||
for _attempt in 1 2 3; do
|
||||
PORT=$((22000 + RANDOM % 20000))
|
||||
: > "$WORK/console.log"
|
||||
{ sleep 40; printf '/usr/bin/clockprobe\n'; sleep 16; printf 'poweroff -f\n'; sleep 8; } | \
|
||||
bash "$QDIR/run.sh" --kernel "$ZIMAGE" --shell \
|
||||
--ssh-port "$PORT" --http-port $((PORT + 1)) --api-port $((PORT + 2)) \
|
||||
> "$WORK/console.log" 2>&1 &
|
||||
QEMU_PID=$!
|
||||
sleep 3
|
||||
kill -0 "$QEMU_PID" 2>/dev/null && break
|
||||
if grep -aq 'Could not set up host forwarding' "$WORK/console.log"; then
|
||||
echo "== hostfwd port collision on base $PORT — retrying"
|
||||
QEMU_PID=""
|
||||
continue
|
||||
fi
|
||||
echo "FATAL: VM died at launch:" >&2
|
||||
tail -20 "$WORK/console.log" >&2
|
||||
exit 1
|
||||
done
|
||||
if [ -z "$QEMU_PID" ] || ! kill -0 "$QEMU_PID" 2>/dev/null; then
|
||||
echo "FATAL: could not launch the VM after 3 port attempts" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
wait "$QEMU_PID" || true
|
||||
grep -a 'CLOCKPROBE' "$WORK/console.log" || {
|
||||
echo "FATAL: probe never ran; console tail:" >&2
|
||||
tail -25 "$WORK/console.log" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
python3 - "$WORK/console.log" <<'EOF'
|
||||
import re, sys
|
||||
text = open(sys.argv[1], errors="replace").read()
|
||||
m = re.search(r"ratio_mono=([0-9.]+) ratio_raw=([0-9.]+)", text)
|
||||
if not m:
|
||||
sys.exit("FATAL: no ratio line in console output")
|
||||
mono, raw = float(m.group(1)), float(m.group(2))
|
||||
ok = abs(mono - 1.0) < 0.01 and abs(raw - 1.0) < 0.01
|
||||
print(f"clock-sanity: ratio_mono={mono} ratio_raw={raw} -> {'PASS' if ok else 'FAIL'}")
|
||||
sys.exit(0 if ok else 1)
|
||||
EOF
|
||||
echo "CLOCK-SANITY-PASS"
|
||||
Generated
+16
@@ -0,0 +1,16 @@
|
||||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
|
||||
[[package]]
|
||||
name = "clockprobe"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "libc"
|
||||
version = "0.2.189"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
|
||||
@@ -0,0 +1,12 @@
|
||||
[package]
|
||||
name = "clockprobe"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "Interval-based monotonic-clock rate probe: compares musl/vDSO CLOCK_MONOTONIC(+_RAW) against /proc/uptime so rate errors and boot-time offsets (CNTVOFF) separate cleanly. Diagnostic for issue #3."
|
||||
license = "GPL-2.0-only"
|
||||
|
||||
[dependencies]
|
||||
libc = "0.2"
|
||||
|
||||
[profile.release]
|
||||
strip = true
|
||||
@@ -0,0 +1,25 @@
|
||||
// Differential clock probe for warden-sdk issue #3: compares the musl/vDSO
|
||||
// CLOCK_MONOTONIC rate against the kernel's own /proc/uptime over a fixed
|
||||
// interval, so boot-time offsets cancel and only the RATE ratio remains.
|
||||
use std::{fs, thread, time::Duration};
|
||||
|
||||
fn mono(clock: libc::clockid_t) -> f64 {
|
||||
let mut ts = libc::timespec { tv_sec: 0, tv_nsec: 0 };
|
||||
unsafe { libc::clock_gettime(clock, &mut ts) };
|
||||
ts.tv_sec as f64 + ts.tv_nsec as f64 / 1e9
|
||||
}
|
||||
|
||||
fn uptime() -> f64 {
|
||||
fs::read_to_string("/proc/uptime").unwrap()
|
||||
.split_whitespace().next().unwrap().parse().unwrap()
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let interval = 10.0;
|
||||
let (m0, r0, u0) = (mono(libc::CLOCK_MONOTONIC), mono(libc::CLOCK_MONOTONIC_RAW), uptime());
|
||||
thread::sleep(Duration::from_secs_f64(interval));
|
||||
let (m1, r1, u1) = (mono(libc::CLOCK_MONOTONIC), mono(libc::CLOCK_MONOTONIC_RAW), uptime());
|
||||
let (dm, dr, du) = (m1 - m0, r1 - r0, u1 - u0);
|
||||
println!("CLOCKPROBE monotonic={dm:.4} raw={dr:.4} uptime={du:.4} ratio_mono={:.5} ratio_raw={:.5}", dm / du, dr / du);
|
||||
println!("CLOCKPROBE abs monotonic={m1:.2} uptime={u1:.2} abs_ratio={:.5}", m1 / u1);
|
||||
}
|
||||
Reference in New Issue
Block a user