ci: self-hosted code quality grade and badge

A new quality job runs the linter battery (clippy, cppcheck, shellcheck,
ruff, lizard, jscpd, cargo-audit per crate, scc for the LOC denominator)
and feeds tools/quality/score.py: findings convert to SQALE remediation
minutes, debt ratio grades A-F on SonarQube's published grid, and a
separate worst-of security axis can only drag the overall grade down.
The job uploads the full quality.json breakdown and fails when the
security grade is worse than C.

The badges job now renders all four SVGs in-runner with anybadge
(shields hex palette); the previous img.shields.io curls were the one
external-service dependency left in the pipeline. quality.svg is seeded
at the current locally-computed grade (A, debt ratio 0.42%).

Also fixes the two ruff findings the battery surfaced in flowgen.py.
This commit is contained in:
BFE Engineering
2026-08-31 16:18:38 -06:00
parent e73996a53b
commit 249c311953
6 changed files with 357 additions and 13 deletions
+23
View File
@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="100" height="20">
<linearGradient id="b" x2="0" y2="100%">
<stop offset="0" stop-color="#bbb" stop-opacity=".1"/>
<stop offset="1" stop-opacity=".1"/>
</linearGradient>
<mask id="anybadge_1">
<rect width="100" height="20" rx="3" fill="#fff"/>
</mask>
<g mask="url(#anybadge_1)">
<path fill="#555" d="M0 0h83v20H0z"/>
<path fill="#4c1" d="M83 0h17v20H83z"/>
<path fill="url(#b)" d="M0 0h100v20H0z"/>
</g>
<g fill="#fff" text-anchor="middle" font-family="DejaVu Sans,Verdana,Geneva,sans-serif" font-size="11">
<text x="42.5" y="15" fill="#010101" fill-opacity=".3">code quality</text>
<text x="41.5" y="14">code quality</text>
</g>
<g fill="#fff" text-anchor="middle" font-family="DejaVu Sans,Verdana,Geneva,sans-serif" font-size="11">
<text x="92.5" y="15" fill="#010101" fill-opacity=".3">A</text>
<text x="91.5" y="14">A</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.0 KiB

+83 -9
View File
@@ -257,8 +257,71 @@ jobs:
# active bound, this is the backstop. # active bound, this is the backstop.
retention-days: 5 retention-days: 5
quality:
# Self-hosted Codacy-style grade: a linter battery feeds
# tools/quality/score.py (SQALE debt ratio + a separate security axis;
# thresholds documented in the script). No external assessment service;
# the badge is rendered and committed by the badges job.
runs-on: ubuntu-latest
timeout-minutes: 20
outputs:
grade: ${{ steps.score.outputs.grade }}
color: ${{ steps.score.outputs.color }}
steps:
- uses: actions/checkout@v4
- name: install analyzers
run: |
sudo apt-get update -qq && sudo apt-get install -y -qq cppcheck shellcheck
pip install --quiet lizard ruff
sudo npm install --silent -g jscpd
curl -fsSL -o /tmp/scc.tar.gz \
https://github.com/boyter/scc/releases/download/v3.6.0/scc_Linux_x86_64.tar.gz
sudo tar -C /usr/local/bin -xzf /tmp/scc.tar.gz scc
rustup component add clippy
- uses: taiki-e/install-action@b6ff580856c41316412a0b9b60540fbc6f8c82cc # v2.86.7
with:
tool: cargo-audit
- name: collect linter outputs
run: |
Q="$RUNNER_TEMP/qual"; mkdir -p "$Q"
scc --format json \
--exclude-dir .git,target,patches,kernel,docs/workflows,.github/badges \
. > "$Q/scc.json"
: > "$Q/clippy.jsonl"
for d in sim tools/config-lint qemu/rs485-bridge qemu/tests/clockprobe; do
( cd "$d" && cargo clippy --locked --all-targets --message-format=json \
2>/dev/null >> "$Q/clippy.jsonl" )
( cd "$d" && cargo audit --json -q > "$Q/audit-$(basename "$d").json" )
done
shellcheck -f json1 qemu/*.sh qemu/tests/*.sh build/*.sh \
qemu/rootfs/init qemu/rootfs/etc/rc qemu/rootfs/sbin/init \
> "$Q/shellcheck.json" || true
cppcheck --enable=warning,style,performance,portability --inline-suppr \
--xml drivers/ 2> "$Q/cppcheck.xml"
lizard -C 10 --csv sim/src qemu/rs485-bridge/src tools/config-lint/src \
drivers/ tools/flowgen.py qemu/tests/clockprobe/src > "$Q/lizard.csv"
ruff check --output-format=json tools/ qemu/ > "$Q/ruff.json" || true
jscpd --silent --reporters json --output "$Q" \
--pattern '**/*.{rs,c,h,sh,py}' \
--ignore '**/target/**,**/patches/**,**/kernel/**' .
- name: score
id: score
run: |
python3 tools/quality/score.py "$RUNNER_TEMP/qual" \
--out "$RUNNER_TEMP/qual/quality.json"
J="$RUNNER_TEMP/qual/quality.json"
echo "grade=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['grade'])" "$J")" >> "$GITHUB_OUTPUT"
echo "color=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['badge_color'])" "$J")" >> "$GITHUB_OUTPUT"
- name: security gate
run: python3 tools/quality/score.py "$RUNNER_TEMP/qual" --gate-security C
- uses: actions/upload-artifact@v4
with:
name: quality-report
path: ${{ runner.temp }}/qual/quality.json
retention-days: 30
badges: badges:
needs: [test] needs: [test, quality]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15 timeout-minutes: 15
@@ -266,26 +329,37 @@ jobs:
contents: write contents: write
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: install cloc - name: install tools
run: sudo apt-get update -qq && sudo apt-get install -y -qq cloc run: |
sudo apt-get update -qq && sudo apt-get install -y -qq cloc
pip install --quiet anybadge
- name: render badges - name: render badges
# Rendered locally with anybadge: the committed SVG must not depend
# on any external service, at view time or at render time.
env: env:
PASSED: ${{ needs.test.outputs.passed }} PASSED: ${{ needs.test.outputs.passed }}
COVERAGE: ${{ needs.test.outputs.coverage }} COVERAGE: ${{ needs.test.outputs.coverage }}
GRADE: ${{ needs.quality.outputs.grade }}
QCOLOR: ${{ needs.quality.outputs.color }}
run: | run: |
mkdir -p .github/badges mkdir -p .github/badges
loc=$(cloc --quiet --json --exclude-dir=target,build,build-target,patches,data,docs . \ loc=$(cloc --quiet --json --exclude-dir=target,build,build-target,patches,data,docs . \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["SUM"]["code"])') | python3 -c 'import sys,json; print(json.load(sys.stdin)["SUM"]["code"])')
col=orange; [ "${COVERAGE:-0}" -ge 60 ] && col=yellow; [ "${COVERAGE:-0}" -ge 80 ] && col=brightgreen col='#fe7d37'; [ "${COVERAGE:-0}" -ge 60 ] && col='#dfb317'; [ "${COVERAGE:-0}" -ge 80 ] && col='#4c1'
curl -fsSL "https://img.shields.io/badge/lines%20of%20code-${loc}-blue" -o .github/badges/loc.svg anybadge --overwrite --label="lines of code" --value="$loc" --color='#007ec6' \
curl -fsSL "https://img.shields.io/badge/tests-${PASSED}%20passing-brightgreen" -o .github/badges/tests.svg --file=.github/badges/loc.svg
curl -fsSL "https://img.shields.io/badge/coverage-${COVERAGE}%25-${col}" -o .github/badges/coverage.svg anybadge --overwrite --label=tests --value="${PASSED} passing" --color='#4c1' \
--file=.github/badges/tests.svg
anybadge --overwrite --label=coverage --value="${COVERAGE}%" --color="$col" \
--file=.github/badges/coverage.svg
anybadge --overwrite --label="code quality" --value="$GRADE" --color="$QCOLOR" \
--file=.github/badges/quality.svg
- name: commit badges - name: commit badges
run: | run: |
git config user.name "github-actions[bot]" git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com" git config user.email "github-actions[bot]@users.noreply.github.com"
git add .github/badges/loc.svg .github/badges/tests.svg .github/badges/coverage.svg git add .github/badges/*.svg
if ! git diff --cached --quiet; then if ! git diff --cached --quiet; then
git commit -m "ci: update loc/tests/coverage badges [skip ci]" git commit -m "ci: update badges [skip ci]"
git push git push
fi fi
+1
View File
@@ -4,6 +4,7 @@
![Lines of code](.github/badges/loc.svg) ![Lines of code](.github/badges/loc.svg)
![Tests](.github/badges/tests.svg) ![Tests](.github/badges/tests.svg)
![Coverage](.github/badges/coverage.svg) ![Coverage](.github/badges/coverage.svg)
![Code quality](.github/badges/quality.svg)
A modern, open development environment for the **Luckfox Pico 86 Panel** A modern, open development environment for the **Luckfox Pico 86 Panel**
(Rockchip RV1106), replacing the vendor SDK, and honest about what runs on (Rockchip RV1106), replacing the vendor SDK, and honest about what runs on
+6 -2
View File
@@ -14,6 +14,7 @@ execute code on private infrastructure (ADR-0007).
| `bench` | ubuntu-latest | Smoke-runs the sim + rs485-bridge micro-benchmarks; emits ns/op trend JSON. | | `bench` | ubuntu-latest | Smoke-runs the sim + rs485-bridge micro-benchmarks; emits ns/op trend JSON. |
| `patches-apply` | ubuntu-latest | Fetches pristine linux-6.18.46 (cached, sha256-verified) and applies `patches/*` in order. | | `patches-apply` | ubuntu-latest | Fetches pristine linux-6.18.46 (cached, sha256-verified) and applies `patches/*` in order. |
| `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`; builds the initramfs (pinned busybox) and the A/B disk image. | | `qemu-tools` | ubuntu-latest | shellcheck on `qemu/**.sh`; builds the initramfs (pinned busybox) and the A/B disk image. |
| `quality` | ubuntu-latest | Codacy-style grade computed in-pipeline: clippy, cppcheck, shellcheck, ruff, lizard, jscpd, cargo-audit feed `tools/quality/score.py` (SQALE debt ratio + a separate worst-of security axis; SonarQube's published thresholds). Uploads `quality.json`; fails if the security grade is worse than C. |
| `kernel-build` | ubuntu-latest, **dispatch-only** | apt-installs the cross toolchain + qemu, `build/build-kernel.sh` -> `zImage` + `rv1106-warden.dtb`, QEMU `-M virt` boot smoke (fail-closed), artifact upload (best-effort). Trigger: `gh workflow run ci.yml`. | | `kernel-build` | ubuntu-latest, **dispatch-only** | apt-installs the cross toolchain + qemu, `build/build-kernel.sh` -> `zImage` + `rv1106-warden.dtb`, QEMU `-M virt` boot smoke (fail-closed), artifact upload (best-effort). Trigger: `gh workflow run ci.yml`. |
| `prune-artifacts` | ubuntu-latest, dispatch-only | Deletes `kernel-rv1106` artifacts beyond the newest 3. | | `prune-artifacts` | ubuntu-latest, dispatch-only | Deletes `kernel-rv1106` artifacts beyond the newest 3. |
| `badges` | ubuntu-latest | Renders loc/tests/coverage shields on push to `main` (`[skip ci]` + `paths-ignore` loop guard). | | `badges` | ubuntu-latest | Renders loc/tests/coverage shields on push to `main` (`[skip ci]` + `paths-ignore` loop guard). |
@@ -28,5 +29,8 @@ deployment log, not here.
## Badges ## Badges
Static shields SVGs are committed by the `badges` job. The GitHub-native SVGs are rendered in-runner with anybadge and committed by the `badges`
`ci.yml` status badge works live regardless. job: no external badge or assessment service at render or view time. The
quality letter comes from the `quality` job; `tools/quality/score.py`
documents the scoring model and thresholds. The GitHub-native `ci.yml`
status badge works live regardless.
Regular → Executable
+2 -2
View File
@@ -150,8 +150,8 @@ def main():
os.makedirs(OUT, exist_ok=True) os.makedirs(OUT, exist_ok=True)
index = ["# Workflow Flowcharts", "", index = ["# Workflow Flowcharts", "",
"Generated by `tools/flowgen.py` from the modelled decision paths.", "Generated by `tools/flowgen.py` from the modelled decision paths.",
"Each is an outcome-first flowchart of a workflow the SDK tests, with its" ("Each is an outcome-first flowchart of a workflow the SDK tests,"
" benchmark or MC/DC metric.", ""] " with its benchmark or MC/DC metric."), ""]
for w in WORKFLOWS: for w in WORKFLOWS:
body = (f"# {w['title']}\n\n" body = (f"# {w['title']}\n\n"
f"> **Outcome tested:** {w['outcome']}\n\n" f"> **Outcome tested:** {w['outcome']}\n\n"
+242
View File
@@ -0,0 +1,242 @@
#!/usr/bin/env python3
"""Self-hosted code-quality grade: SQALE debt ratio plus a security axis.
Reads the linter outputs collected by the CI quality job from one directory
and prints a letter grade. No external service is involved at any point;
every threshold below traces to a published number (SonarQube's default
30 min/line development cost and its maintainability grid, which
Code Climate/Qlty publish almost verbatim).
score.py <dir> [--gate-security GRADE] [--out quality.json]
Required files in <dir> (fail-closed: a missing file is an error, so a
broken collection step can never inflate the grade):
scc.json scc --format json (LOC denominator)
clippy.jsonl cargo clippy JSON messages, one per line
shellcheck.json shellcheck -f json1
cppcheck.xml cppcheck --xml (v2)
lizard.csv lizard --csv
jscpd-report.json jscpd --reporters json
audit-*.json cargo audit --json, one per crate
Exit 0 normally; 1 when --gate-security is given and the security grade is
worse; 2 on missing/unparseable input.
"""
import argparse
import csv
import glob
import json
import os
import sys
import xml.etree.ElementTree as ET
# Remediation minutes per finding severity (SQALE-style constants).
MINUTES = {"critical": 60, "major": 20, "minor": 5}
DUP_CLONE_MINUTES = 30
DEV_COST_PER_LINE = 30 # SonarQube's documented default.
# Languages that count as code for the LOC denominator.
CODE_LANGS = {"Rust", "C", "C Header", "Shell", "Python", "BASH", "Bourne Shell"}
GRADES = [(0.05, "A"), (0.10, "B"), (0.20, "C"), (0.50, "D"), (9e9, "F")]
GRADE_ORDER = "ABCDF"
# Shields palette hex codes (anybadge rejects the shields color names).
BADGE_COLORS = {"A": "#4c1", "B": "#97ca00", "C": "#dfb317", "D": "#fe7d37", "F": "#e05d44"}
def die(msg):
print(f"FATAL: {msg}", file=sys.stderr)
sys.exit(2)
def need(path):
if not os.path.exists(path):
die(f"missing required input {path}")
return path
def load_json(path):
with open(need(path)) as f:
return json.load(f)
def grade_from_ratio(ratio):
for cap, letter in GRADES:
if ratio < cap:
return letter
return "F"
def parse_scc(path):
langs = load_json(path)
loc = sum(l["Code"] for l in langs if l["Name"] in CODE_LANGS)
if loc <= 0:
die("scc reports zero code lines")
return loc
def parse_clippy(path, add):
# Raw `cargo clippy --message-format=json` stream: one JSON object per
# line, most of them build bookkeeping. Only lint diagnostics (messages
# carrying a code) count.
with open(need(path)) as f:
lines = f.read().splitlines()
for line in lines:
line = line.strip()
if not line:
continue
try:
m = json.loads(line)
except ValueError:
die(f"unparseable clippy line: {line[:80]}")
if m.get("reason") != "compiler-message":
continue
msg = m.get("message") or {}
if not msg.get("code"):
continue
level = msg.get("level")
if level == "error":
add("clippy", "critical")
elif level == "warning":
add("clippy", "major")
def parse_shellcheck(path, add):
data = load_json(path)
for c in data.get("comments", []):
level = c.get("level")
sev = {"error": "critical", "warning": "major"}.get(level, "minor")
add("shellcheck", sev)
def parse_cppcheck(path, add):
root = ET.parse(need(path)).getroot()
for e in root.iter("error"):
sev = e.get("severity")
if sev == "information":
continue
mapped = {"error": "critical", "warning": "major"}.get(sev, "minor")
add("cppcheck", mapped)
def parse_lizard(path, add):
# CSV columns: nloc, ccn, tokens, params, length, location, path, name, ...
with open(need(path)) as f:
rows = list(csv.reader(f))
for row in rows:
if len(row) < 2 or not row[1].isdigit():
continue
ccn = int(row[1])
if ccn > 20:
add("complexity", "critical")
elif ccn > 15:
add("complexity", "major")
elif ccn > 10:
add("complexity", "minor", minutes=20)
def parse_ruff(path, add, security):
for f in load_json(path):
code = f.get("code") or ""
if code.startswith("S"):
add("ruff", "major")
security["findings"] += 1
elif code.startswith(("E9", "F")):
add("ruff", "major")
else:
add("ruff", "minor")
def parse_jscpd(path):
stats = load_json(path)["statistics"]["total"]
return int(stats["clones"]), float(stats["percentage"])
def parse_audits(pattern, security):
paths = glob.glob(pattern)
if not paths:
die(f"no cargo-audit outputs match {pattern}")
for p in paths:
d = load_json(p)
security["vulns"] += int(d["vulnerabilities"]["count"])
security["warnings"] += sum(len(v) for v in d.get("warnings", {}).values())
def security_grade(sec):
if sec["vulns"] >= 2:
return "D"
if sec["vulns"] == 1 or sec["findings"] > 0:
return "C"
if sec["warnings"] > 0:
return "B"
return "A"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("dir")
ap.add_argument("--gate-security", choices=list(GRADE_ORDER), default=None)
ap.add_argument("--out", default=None)
args = ap.parse_args()
d = args.dir
counts = {}
minutes = [0.0]
def add(tool, sev, minutes_each=None, **kw):
counts.setdefault(tool, {}).setdefault(sev, [0, 0.0])
m = kw.get("minutes", minutes_each)
if m is None:
m = MINUTES[sev]
counts[tool][sev][0] += 1
counts[tool][sev][1] += m
minutes[0] += m
security = {"vulns": 0, "warnings": 0, "findings": 0}
loc = parse_scc(os.path.join(d, "scc.json"))
parse_clippy(os.path.join(d, "clippy.jsonl"), add)
parse_shellcheck(os.path.join(d, "shellcheck.json"), add)
parse_cppcheck(os.path.join(d, "cppcheck.xml"), add)
parse_lizard(os.path.join(d, "lizard.csv"), add)
parse_ruff(os.path.join(d, "ruff.json"), add, security)
clones, dup_pct = parse_jscpd(os.path.join(d, "jscpd-report.json"))
for _ in range(clones):
add("duplication", "minor", minutes=DUP_CLONE_MINUTES)
parse_audits(os.path.join(d, "audit-*.json"), security)
ratio = minutes[0] / (loc * DEV_COST_PER_LINE)
maint = grade_from_ratio(ratio)
sec = security_grade(security)
overall = max(maint, sec, key=GRADE_ORDER.index)
report = {
"grade": overall,
"maintainability": {"grade": maint, "debt_ratio_pct": round(ratio * 100, 3),
"remediation_minutes": round(minutes[0], 1), "code_lines": loc},
"security": {"grade": sec, **security},
"duplication_pct": round(dup_pct, 2),
"findings": {t: {s: {"count": v[0], "minutes": v[1]} for s, v in sevs.items()}
for t, sevs in counts.items()},
"badge_color": BADGE_COLORS[overall],
}
if args.out:
with open(args.out, "w") as f:
json.dump(report, f, indent=1)
print(f"code quality: {overall} "
f"(maintainability {maint}, debt ratio {ratio * 100:.2f}%, "
f"security {sec}, duplication {dup_pct:.1f}%)")
for tool, sevs in sorted(counts.items()):
line = ", ".join(f"{s}={v[0]}" for s, v in sorted(sevs.items()))
print(f" {tool}: {line}")
if args.gate_security and GRADE_ORDER.index(sec) > GRADE_ORDER.index(args.gate_security):
print(f"FAIL: security grade {sec} is worse than the {args.gate_security} gate",
file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())