Add a full-OS .afw bundle and strict CI gate

Replace the release-tree producer with a board-aware schema-2 manifest,
fixed-order ustar packing, optional Ed25519 signing, and a checker that
rejects a wrong board or arch, a missing or oversized image, a bootloader
member under preserve, a hash or size mismatch, and a traversal member.
Document the format and add offline fixtures for every refusal.

Co-Authored-By: DeepSeek Harness <noreply@deepseek.com>
This commit is contained in:
2026-11-04 14:31:35 -06:00
co-authored by DeepSeek Harness
parent 997771ab5e
commit 553f6a04fb
8 changed files with 1593 additions and 57 deletions
+2
View File
@@ -28,6 +28,8 @@ jobs:
run: rustup component add rustfmt clippy
- name: Image safety gate
run: sh ci/check-image-safety.sh
- name: .afw format tests
run: python3 ci/test-afw.py
- name: Formatting
run: cargo fmt --check
- name: Last-attempt badge checks
+2
View File
@@ -17,6 +17,8 @@ jobs:
fetch-depth: 0
- name: Image safety gate
run: sh ci/check-image-safety.sh
- name: .afw format tests
run: python3 ci/test-afw.py
- name: Build release candidate
run: cargo build --release --offline --locked
- name: Select candidate version
+141
View File
@@ -0,0 +1,141 @@
# SPDX-License-Identifier: MIT
"""Board profiles used by the .afw producer and the CI checker.
An .afw is valid for exactly one board family. The arch and bootloader policy
come from this table, never from the caller alone, so a bundle cannot claim a
board it was not built for. Only cv1835 has a complete, buildable profile in
this release; the other Antminer families are listed so a wrong-board bundle
is rejected by name instead of being accepted as "unknown".
"""
BOARDS = {
"cv1835": {
"arch": "arm64",
"bootloader_policy": "preserve",
"buildable": True,
"format_checks": ["fit", "arm64-image", "fdt", "cpio-gzip"],
"partitions": [
{
"name": "p1",
"device": "/dev/mmcblk0p1",
"offset": 0,
"size": 16 * 1024 * 1024,
"secure": False,
"boot_critical": True,
"write_order": 1,
"image": "p1",
},
{
"name": "p3",
"device": "/dev/mmcblk0p3",
"offset": 0,
"size": 32 * 1024 * 1024,
"secure": True,
"boot_critical": False,
"write_order": 0,
"image": None,
},
{
"name": "boot0",
"device": "/dev/mmcblk0boot0",
"offset": 0,
"size": 4 * 1024 * 1024,
"secure": True,
"boot_critical": True,
"write_order": 0,
"image": None,
},
{
"name": "boot1",
"device": "/dev/mmcblk0boot1",
"offset": 0,
"size": 4 * 1024 * 1024,
"secure": True,
"boot_critical": True,
"write_order": 0,
"image": None,
},
],
"images": [
{
"name": "kernel",
"source": "Image",
"role": "kernel",
"format": "arm64-image",
"partition": None,
"install": False,
"exact_bytes": None,
},
{
"name": "dtb",
"source": "cv-alkane.dtb",
"role": "dtb",
"format": "fdt",
"partition": None,
"install": False,
"exact_bytes": None,
},
{
"name": "rootfs",
"source": "rootfs.cpio.gz",
"role": "rootfs",
"format": "cpio-gzip",
"partition": None,
"install": False,
"exact_bytes": None,
},
{
"name": "p1",
"source": "alkane-cv1835-p1.fit",
"role": "boot",
"format": "fit",
"partition": "p1",
"install": True,
"exact_bytes": 16 * 1024 * 1024,
},
],
},
# Antminer families that the seven-board release gate still lists. They are
# known, so a wrong-board bundle is refused by identity, but they do not
# have a buildable .afw profile in this release.
"zynq-7010": {"arch": "armv7", "bootloader_policy": "preserve", "buildable": False},
"zynq-s17": {"arch": "armv7", "bootloader_policy": "include", "buildable": False},
"a113d": {"arch": "armv7", "bootloader_policy": "preserve", "buildable": False},
"a113d-s21pro": {"arch": "armv7", "bootloader_policy": "preserve", "buildable": False},
"am335x-bb": {"arch": "armv7", "bootloader_policy": "include", "buildable": False},
}
SAFE_IMAGE_FORMATS = {
"fit",
"arm64-image",
"fdt",
"cpio-gzip",
"uimage",
"raw",
}
SAFE_IMAGE_ROLES = {
"kernel",
"dtb",
"rootfs",
"bootloader",
"boot",
"component",
}
def board(name):
"""Return the profile for a board family, or raise KeyError."""
return BOARDS[name]
def known_board(name):
return name in BOARDS
def expect_arch(name):
return BOARDS[name]["arch"]
def expect_policy(name):
return BOARDS[name]["bootloader_policy"]
+383
View File
@@ -0,0 +1,383 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: MIT
"""Generate an .afw manifest and stage its payload members.
The output is a full OS install bundle for exactly one board family: the
manifest names every image, its size and SHA-256, the partition plan, the
bootloader policy, the network recovery path and a liveness marker. Payload
members are staged under <staging>/payload/ in manifest order.
This script does not create the tar. ci/make-afw.sh packs the staged members
in the fixed order manifest.json, optional manifest.json.sig, then payload.
ci/check-afw.py verifies the finished bundle before it is published.
"""
import argparse
import datetime
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import afw_boards
SCHEMA = 2
P1_LOAD = 0x100200000
GOLDEN_LBA_BYTES = 0x300000 * 512
REVISION = 40
def die(message):
print(f"afw_manifest: error: {message}", file=sys.stderr)
raise SystemExit(1)
def sha256_bytes(data):
return hashlib.sha256(data).hexdigest()
def sha256_file(path):
digest = hashlib.sha256()
with path.open("rb") as source:
for chunk in iter(lambda: source.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def git_revision(path):
try:
return subprocess.check_output(
["git", "-C", str(path), "rev-parse", "HEAD"],
text=True,
stderr=subprocess.DEVNULL,
).strip()
except (OSError, subprocess.CalledProcessError):
return None
def check_revision(name, value):
if value is None or len(value) != REVISION:
die(f"{name} must be a 40-character git revision, got {value!r}")
if any(character not in "0123456789abcdef" for character in value):
die(f"{name} must be lowercase hexadecimal")
return value
def build_date(value):
if value is None:
return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
try:
moment = datetime.datetime.fromtimestamp(int(value), datetime.timezone.utc)
except (ValueError, OSError):
die("SOURCE_DATE_EPOCH must be an integer Unix timestamp")
return moment.strftime("%Y-%m-%dT%H:%M:%SZ")
def magic_ok(fmt, path):
with path.open("rb") as source:
data = source.read() if fmt == "fit" else source.read(4096)
if fmt == "fit":
if not data.startswith(b"\xd0\x0d\xfe\xed"):
return "does not start with the FIT/FDT magic d00dfeed"
if b"config@cv1835_miner_emmc" not in data:
return "FIT is missing config@cv1835_miner_emmc"
if b"signature@" in data:
return "FIT carries signature@ nodes, which this board cannot boot"
elif fmt == "fdt":
if not data.startswith(b"\xd0\x0d\xfe\xed"):
return "does not start with the FDT magic d00dfeed"
elif fmt == "arm64-image":
if len(data) < 64 or data[0x38:0x3C] != b"ARMd":
return "does not carry the arm64 Image ARMd header magic"
elif fmt == "cpio-gzip":
if not data.startswith(b"\x1f\x8b"):
return "is not gzip-compressed"
elif fmt == "uimage":
if not data.startswith(b"\x27\x05\x19\x56"):
return "is not a legacy uImage"
return None
def pad_to(source, destination, exact_bytes):
size = source.stat().st_size
if size > exact_bytes:
die(f"{source.name} is {size} bytes, larger than its {exact_bytes}-byte partition")
destination.parent.mkdir(parents=True, exist_ok=True)
with source.open("rb") as reader, destination.open("wb") as writer:
shutil.copyfileobj(reader, writer)
if size < exact_bytes:
writer.write(b"\0" * (exact_bytes - size))
if destination.stat().st_size != exact_bytes:
die(f"{destination.name} did not reach the exact {exact_bytes}-byte size")
def load_plan(path, profile, board):
plan = json.loads(Path(path).read_text())
if plan.get("board") != board:
die(f"plan board {plan.get('board')!r} does not match --board {board!r}")
if plan.get("arch") != profile["arch"]:
die(f"plan arch {plan.get('arch')!r} does not match {profile['arch']!r}")
policy = plan.get("bootloader_policy", profile["bootloader_policy"])
if policy != profile["bootloader_policy"]:
die(f"plan bootloader_policy {policy!r} does not match {profile['bootloader_policy']!r}")
partitions = plan.get("partitions")
images = plan.get("images")
if not isinstance(partitions, list) or not isinstance(images, list):
die("plan must contain partitions and images lists")
return partitions, images
def signing_key(path):
raw = Path(path).read_bytes()
if len(raw) == 64:
try:
raw = bytes.fromhex(raw.decode("ascii").strip())
except (ValueError, UnicodeDecodeError):
die("signing key file must be 32 raw bytes or 64 hexadecimal characters")
if len(raw) != 32:
die("signing key file must be 32 raw bytes or 64 hexadecimal characters")
try:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
except ImportError:
die("Ed25519 signing needs the Python cryptography package")
private = Ed25519PrivateKey.from_private_bytes(raw)
public = private.public_key().public_bytes(
serialization.Encoding.Raw, serialization.PublicFormat.Raw
)
return private, public.hex()
def build(args):
board = args.board
if not afw_boards.known_board(board):
die(f"unknown board family {board!r}")
profile = afw_boards.board(board)
if not profile["buildable"]:
die(f"board {board!r} has no buildable .afw profile in this release")
input_dir = Path(args.input)
if not input_dir.is_dir():
die(f"input directory does not exist: {input_dir}")
plan_document = None
if args.plan:
plan_document = json.loads(Path(args.plan).read_text())
partitions, image_specs = load_plan(args.plan, profile, board)
else:
partitions = profile["partitions"]
image_specs = profile["images"]
staging = Path(args.staging) if args.staging else Path(tempfile.mkdtemp(prefix="afw-manifest."))
payload_dir = staging / "payload"
payload_dir.mkdir(parents=True, exist_ok=True)
images = []
for spec in image_specs:
source_name = spec.get("source", spec.get("file"))
if not source_name:
die(f"image {spec.get('name')!r} has no source file")
source = input_dir / source_name
if not source.is_file():
die(f"required image is missing: {source}")
name = spec["name"]
fmt = spec.get("format", "raw")
if fmt not in afw_boards.SAFE_IMAGE_FORMATS:
die(f"image {name!r} has unsupported format {fmt!r}")
role = spec.get("role", "component")
if role not in afw_boards.SAFE_IMAGE_ROLES:
die(f"image {name!r} has unsupported role {role!r}")
target = payload_dir / source.name
exact = spec.get("exact_bytes")
if exact is not None:
pad_to(source, target, int(exact))
else:
shutil.copyfile(source, target)
problem = magic_ok(fmt, target)
if problem:
die(f"image {name!r} ({target.name}) {problem}")
size = target.stat().st_size
images.append(
{
"name": name,
"file": f"payload/{target.name}",
"role": role,
"format": fmt,
"size": size,
"sha256": sha256_file(target),
"partition": spec.get("partition"),
"install": bool(spec.get("install", False)),
}
)
names = {image["name"] for image in images}
for partition in partitions:
image_name = partition.get("image")
if image_name is not None and image_name not in names:
die(f"partition {partition.get('name')!r} references missing image {image_name!r}")
if not partition.get("secure", False):
size = int(partition.get("size", 0))
if size <= 0:
die(f"partition {partition.get('name')!r} has no positive size")
if partition.get("write_order", 0) > 0 and image_name is None:
die(f"installable partition {partition.get('name')!r} has no image")
device = partition.get("device", "")
if not device.startswith("/dev/"):
die(f"partition {partition.get('name')!r} has an invalid device path")
if "boot0" in device and not partition.get("secure", False):
die("boot0 must be declared secure and must never be a write target")
if "boot0" in device and partition.get("write_order", 0) > 0:
die("boot0 must not be in the write order")
for image in images:
partition_name = image.get("partition")
if partition_name is None:
continue
matches = [item for item in partitions if item.get("name") == partition_name]
if len(matches) != 1:
die(f"image {image['name']!r} targets unknown partition {partition_name!r}")
partition = matches[0]
if partition.get("secure", False):
die(f"image {image['name']!r} targets secure partition {partition_name!r}")
if image["size"] > int(partition["size"]):
die(
f"image {image['name']!r} is {image['size']} bytes, larger than "
f"partition {partition_name!r} ({partition['size']} bytes)"
)
exact = image.get("exact_bytes", partition.get("exact_bytes"))
if exact is not None and image["size"] != int(exact):
die(f"image {image['name']!r} must be exactly {exact} bytes")
installs = [image for image in images if image["install"]]
if not installs:
die("bundle has no installable image")
if board == "cv1835":
if len(installs) != 1 or installs[0]["name"] != "p1":
die("cv1835 must install exactly one image, p1")
if installs[0]["size"] != 16 * 1024 * 1024:
die("cv1835 p1 must be the 16 MiB padded FIT")
policy = profile["bootloader_policy"]
if policy == "preserve" and any(image["role"] == "bootloader" for image in images):
die("bootloader_policy is preserve but the bundle carries a bootloader image")
if policy == "include" and not any(image["role"] == "bootloader" for image in images):
die("bootloader_policy is include but the bundle has no bootloader image")
sdk_revision = check_revision(
"sdk_revision", args.sdk_revision or (git_revision(args.sdk_dir) if args.sdk_dir else None)
)
os_revision = check_revision(
"os_revision", args.os_revision or (git_revision(args.os_dir) if args.os_dir else None)
)
signature = {
"type": "unsigned",
"member": None,
"public_key_hex": None,
}
signer = None
if args.signing_key:
signer, public_hex = signing_key(args.signing_key)
signature = {
"type": "ed25519",
"member": "manifest.json.sig",
"public_key_hex": public_hex,
}
manifest = {
"schema": SCHEMA,
"format": "afw",
"board": board,
"arch": profile["arch"],
"version": args.version,
"build_date": build_date(os.environ.get("SOURCE_DATE_EPOCH")),
"sdk_revision": sdk_revision,
"os_revision": os_revision,
"payload_type": "os",
"bootloader_policy": policy,
"signature": signature,
"partitions": [
{
"name": item["name"],
"device": item["device"],
"offset": int(item.get("offset", 0)),
"size": int(item["size"]),
"secure": bool(item.get("secure", False)),
"boot_critical": bool(item.get("boot_critical", False)),
"write_order": int(item.get("write_order", 0)),
"image": item.get("image"),
}
for item in partitions
],
"images": images,
"boot_switch": {
"method": "partition-overwrite"
if board == "cv1835"
else "board-specific",
"detail": "Writing /dev/mmcblk0p1 replaces the FIT. The vendor boot0, "
"BL2, BL31 and U-Boot chain is preserved and never written."
if board == "cv1835"
else "See the board plan.",
},
"preflight": {
"required_commands": ["dd", "md5sum", "sync"],
"forbidden_devices": ["/dev/mmcblk0", "/dev/mmcblk0boot0", "/dev/mmcblk0boot1"],
"min_free_bytes": 32 * 1024 * 1024,
"source_os": ["stock-bitmain", "vnish", "braiins", "ouros", "alkane"],
},
"recovery": {
"network": "afw restore --host <ip> --held-backup <file>",
"held_partition": "p1",
"golden_lba_bytes": GOLDEN_LBA_BYTES if board == "cv1835" else None,
},
"marker": (plan_document or {}).get("marker") or args.marker,
"kernel_version": (plan_document or {}).get("kernel_version")
or args.kernel_version
or ("6.18.48" if board == "cv1835" else "unknown"),
}
if args.notes:
manifest["notes"] = args.notes
manifest_path = staging / "manifest.json"
manifest_bytes = (json.dumps(manifest, indent=2) + "\n").encode()
manifest_path.write_bytes(manifest_bytes)
if signer is not None:
signature_bytes = signer.sign(manifest_bytes)
(staging / "manifest.json.sig").write_bytes(signature_bytes)
(staging / "manifest.pub").write_text(signature["public_key_hex"] + "\n")
print(str(manifest_path))
return staging, manifest_path
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--board", required=True)
parser.add_argument("--version", required=True)
parser.add_argument("--input", required=True, help="SDK board output directory")
parser.add_argument("--staging", help="staging directory for manifest.json and payload/")
parser.add_argument("--plan", help="plan JSON emitted by alkane-sdk ci/check-plans.py")
parser.add_argument("--sdk-revision")
parser.add_argument("--os-revision")
parser.add_argument("--sdk-dir")
parser.add_argument("--os-dir", default=".")
parser.add_argument("--signing-key", help="raw 32-byte or 64-hex Ed25519 seed file")
parser.add_argument("--kernel-version")
parser.add_argument("--marker", default="ALKANE")
parser.add_argument("--notes")
args = parser.parse_args()
try:
build(args)
except SystemExit:
raise
except (OSError, ValueError, KeyError) as error:
die(str(error))
if __name__ == "__main__":
main()
+446
View File
@@ -0,0 +1,446 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: MIT
"""Verify an .afw OS install bundle before it can be published or installed.
The gate is deliberately strict and offline. It reads the tar itself instead of
trusting GNU tar, checks the fixed member order, rejects path traversal and
non-regular members, verifies every payload size and SHA-256 against the
manifest, checks the board/arch and bootloader policy, checks each image against
its partition ceiling, and refuses a signed bundle it cannot verify.
Exit codes: 0 pass, 1 failure, 2 usage error.
"""
import argparse
import hashlib
import json
from pathlib import Path
import sys
import afw_boards
BLOCK = 512
MAX_MANIFEST = 64 * 1024
MAX_ARCHIVE = 512 * 1024 * 1024
class AfwError(Exception):
pass
def fail(message):
raise AfwError(message)
def parse_octal(field, name):
text = field.split(b"\0", 1)[0].strip()
if not text:
return 0
if any(byte >= 0x80 for byte in text):
fail(f"{name}: base-256 tar numbers are not part of this format")
try:
return int(text, 8)
except ValueError:
fail(f"{name}: invalid octal field")
def safe_member_name(name):
if not name or len(name) > 255:
fail(f"unsafe member name length: {name!r}")
if name.startswith("/") or name.startswith("./"):
fail(f"absolute or ./-prefixed member is not allowed: {name!r}")
if "\\" in name or "\0" in name:
fail(f"member name contains a forbidden character: {name!r}")
parts = name.split("/")
if any(part in ("", ".", "..") for part in parts):
fail(f"path traversal or empty component in member name: {name!r}")
if any(ord(character) < 0x20 for character in name):
fail(f"control character in member name: {name!r}")
return name
def read_tar(path):
try:
size = path.stat().st_size
except OSError as error:
fail(f"cannot read {path}: {error}")
if size == 0 or size > MAX_ARCHIVE:
fail(f"archive size {size} is outside the accepted range")
if size % BLOCK:
fail(f"archive length {size} is not a multiple of {BLOCK} (truncated tar)")
data = path.read_bytes()
members = []
offset = 0
zero_blocks = 0
while offset < len(data):
block = data[offset : offset + BLOCK]
offset += BLOCK
if block == b"\0" * BLOCK:
zero_blocks += 1
if zero_blocks == 2:
break
continue
if zero_blocks:
fail("non-zero data appears between end-of-archive blocks")
if block[257:262] != b"ustar":
fail("member header is not POSIX ustar")
stored = parse_octal(block[148:156], "tar checksum")
header = bytearray(block)
header[148:156] = b" " * 8
if sum(header) != stored:
fail("tar header checksum mismatch")
name = block[0:100].split(b"\0", 1)[0].decode("utf-8", "strict")
prefix = block[345:500].split(b"\0", 1)[0].decode("utf-8", "strict")
if prefix:
name = f"{prefix}/{name}"
safe_member_name(name)
typeflag = block[156:157]
if typeflag not in (b"\0", b"0"):
fail(f"member {name!r} is not a regular file (typeflag {typeflag!r})")
length = parse_octal(block[124:136], f"size of {name}")
if length > MAX_ARCHIVE:
fail(f"member {name!r} is too large")
padded = (length + BLOCK - 1) // BLOCK * BLOCK
if offset + padded > len(data):
fail(f"member {name!r} runs past the end of the archive (truncated tar)")
payload = data[offset : offset + length]
offset += padded
members.append((name, payload))
if zero_blocks < 2:
fail("archive has no two-block end marker (truncated tar)")
if any(data[offset:]):
fail("non-zero trailing bytes after the end-of-archive marker")
if not members:
fail("archive has no members")
names = [name for name, _ in members]
if len(names) != len(set(names)):
fail("archive has duplicate member names")
return members
def require(condition, message):
if not condition:
fail(message)
def is_hex(value, length=64):
return (
isinstance(value, str)
and len(value) == length
and all(character in "0123456789abcdef" for character in value)
)
def check_format(fmt, payload, name):
if fmt == "fit":
require(payload.startswith(b"\xd0\x0d\xfe\xed"), f"{name} is not a FIT")
require(b"config@cv1835_miner_emmc" in payload, f"{name} has no default CV config")
require(b"signature@" not in payload, f"{name} carries signature@ nodes")
elif fmt == "fdt":
require(payload.startswith(b"\xd0\x0d\xfe\xed"), f"{name} is not an FDT")
elif fmt == "arm64-image":
require(len(payload) >= 64 and payload[0x38:0x3C] == b"ARMd", f"{name} is not an arm64 Image")
elif fmt == "cpio-gzip":
require(payload.startswith(b"\x1f\x8b"), f"{name} is not gzip-compressed")
elif fmt == "uimage":
require(payload.startswith(b"\x27\x05\x19\x56"), f"{name} is not a legacy uImage")
def verify_signature(manifest_bytes, manifest, signature_member, public_key):
signature = manifest.get("signature")
require(isinstance(signature, dict), "manifest signature is not an object")
kind = signature.get("type")
if kind == "unsigned":
require(signature_member is None, "unsigned manifest carries a signature member")
require(signature.get("member") is None, "unsigned manifest names a signature member")
return "unsigned"
require(kind == "ed25519", f"unsupported signature type {kind!r}")
require(signature.get("member") == "manifest.json.sig", "signature member name is wrong")
require(signature_member is not None, "ed25519 manifest has no signature member")
require(len(signature_member) == 64, "manifest.json.sig must be exactly 64 bytes")
key_hex = public_key or signature.get("public_key_hex")
require(is_hex(key_hex), "ed25519 manifest needs a 32-byte public key")
if public_key and signature.get("public_key_hex") not in (None, public_key):
fail("--public-key differs from the manifest public_key_hex")
try:
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
except ImportError:
fail("Ed25519 verification needs the Python cryptography package")
try:
Ed25519PublicKey.from_public_bytes(bytes.fromhex(key_hex)).verify(
signature_member, manifest_bytes
)
except InvalidSignature:
fail("manifest.json.sig does not verify against the public key")
return "ed25519"
def validate(path, allow_unsigned=False, public_key=None, expect_board=None, plan_path=None):
members = read_tar(Path(path))
if members[0][0] != "manifest.json":
fail("first member must be manifest.json")
manifest_bytes = members[0][1]
if len(manifest_bytes) > MAX_MANIFEST:
fail(f"manifest.json is {len(manifest_bytes)} bytes, over the 64 KiB cap")
try:
manifest = json.loads(manifest_bytes)
except (ValueError, UnicodeDecodeError) as error:
fail(f"manifest.json is not valid JSON: {error}")
require(isinstance(manifest, dict), "manifest.json must be an object")
required = (
"schema",
"format",
"board",
"arch",
"version",
"build_date",
"sdk_revision",
"os_revision",
"payload_type",
"bootloader_policy",
"signature",
"partitions",
"images",
"boot_switch",
"preflight",
"recovery",
"marker",
"kernel_version",
)
for key in required:
require(key in manifest, f"manifest is missing {key!r}")
require(manifest["schema"] == 2, "only .afw schema 2 is accepted")
require(manifest["format"] == "afw", "manifest format is not afw")
require(manifest["payload_type"] == "os", "payload_type must be os")
board = manifest["board"]
require(afw_boards.known_board(board), f"unknown board family {board!r}")
profile = afw_boards.board(board)
require(manifest["arch"] == profile["arch"], f"board {board} must use {profile['arch']}")
require(
manifest["bootloader_policy"] == profile["bootloader_policy"],
f"board {board} requires bootloader_policy {profile['bootloader_policy']}",
)
require(
manifest["bootloader_policy"] in ("include", "preserve"),
"bootloader_policy must be include or preserve",
)
if expect_board is not None:
require(board == expect_board, f"bundle is for {board}, expected {expect_board}")
for field in ("sdk_revision", "os_revision"):
require(is_hex(manifest[field], 40), f"{field} must be a 40-character revision")
require(isinstance(manifest["version"], str) and manifest["version"], "version is empty")
require(isinstance(manifest["build_date"], str) and manifest["build_date"], "build_date is empty")
require(isinstance(manifest["marker"], str) and manifest["marker"], "marker is empty")
require(
isinstance(manifest["kernel_version"], str) and manifest["kernel_version"],
"kernel_version is empty",
)
for field in ("boot_switch", "preflight", "recovery"):
require(isinstance(manifest[field], dict), f"{field} must be an object")
images = manifest["images"]
partitions = manifest["partitions"]
require(isinstance(images, list) and images, "images must be a non-empty list")
require(isinstance(partitions, list) and partitions, "partitions must be a non-empty list")
image_names = set()
install_images = []
for image in images:
require(isinstance(image, dict), "image entry is not an object")
for key in ("name", "file", "role", "format", "size", "sha256", "install"):
require(key in image, f"image entry is missing {key!r}")
name = image["name"]
require(isinstance(name, str) and name and name not in image_names, f"duplicate image {name!r}")
image_names.add(name)
require(image["role"] in afw_boards.SAFE_IMAGE_ROLES, f"image {name} has an unknown role")
require(image["format"] in afw_boards.SAFE_IMAGE_FORMATS, f"image {name} has an unknown format")
require(isinstance(image["size"], int) and image["size"] > 0, f"image {name} has no size")
require(is_hex(image["sha256"]), f"image {name} has no sha256")
require(isinstance(image["install"], bool), f"image {name} install flag is not a bool")
safe_member_name(image["file"])
require(image["file"].startswith("payload/"), f"image {name} is outside payload/")
require(image["file"] not in ("payload/manifest.json",), f"image {name} reuses a reserved name")
if image["install"]:
require(image.get("partition"), f"installable image {name} has no partition")
install_images.append(image)
require(install_images, "bundle has no installable image")
partition_names = set()
for partition in partitions:
require(isinstance(partition, dict), "partition entry is not an object")
for key in ("name", "device", "offset", "size", "secure", "boot_critical", "write_order"):
require(key in partition, f"partition entry is missing {key!r}")
name = partition["name"]
require(isinstance(name, str) and name and name not in partition_names, f"duplicate partition {name!r}")
partition_names.add(name)
device = partition["device"]
require(isinstance(device, str) and device.startswith("/dev/"), f"partition {name} has a bad device")
require(isinstance(partition["size"], int) and partition["size"] > 0, f"partition {name} has no size")
require(partition["offset"] >= 0, f"partition {name} has a negative offset")
require(isinstance(partition["secure"], bool), f"partition {name} secure is not a bool")
require(isinstance(partition["boot_critical"], bool), f"partition {name} boot_critical is not a bool")
require(isinstance(partition["write_order"], int) and partition["write_order"] >= 0, f"partition {name} write_order")
if "boot0" in device:
require(partition["secure"], "boot0 must be declared secure")
require(partition["write_order"] == 0, "boot0 must not be in the write order")
require(partition.get("image") is None, "boot0 must not carry an install image")
image_name = partition.get("image")
if image_name is not None:
require(image_name in image_names, f"partition {name} names missing image {image_name!r}")
for image in images:
if image.get("partition") is None:
continue
matches = [item for item in partitions if item["name"] == image["partition"]]
require(len(matches) == 1, f"image {image['name']} names an unknown partition")
partition = matches[0]
require(not partition["secure"], f"image {image['name']} targets a secure partition")
require(
image["size"] <= partition["size"],
f"image {image['name']} ({image['size']} B) exceeds partition "
f"{partition['name']} ({partition['size']} B)",
)
if board == "cv1835":
require(len(install_images) == 1 and install_images[0]["name"] == "p1", "cv1835 installs only p1")
p1 = install_images[0]
require(p1["format"] == "fit", "cv1835 p1 must be a FIT")
require(p1["size"] == 16 * 1024 * 1024, "cv1835 p1 must be the 16 MiB padded FIT")
p1_parts = [item for item in partitions if item["name"] == "p1"]
require(len(p1_parts) == 1, "cv1835 manifest must plan p1")
require(p1_parts[0]["size"] == 16 * 1024 * 1024, "cv1835 p1 partition must be 16 MiB")
require(p1_parts[0]["device"] == "/dev/mmcblk0p1", "cv1835 p1 device is wrong")
require(not p1_parts[0]["secure"], "cv1835 p1 must not be secure")
require(p1_parts[0]["write_order"] == 1, "cv1835 p1 must be written once")
if manifest["bootloader_policy"] == "preserve":
require(
not any(image["role"] == "bootloader" for image in images),
"bootloader_policy is preserve but the bundle carries a bootloader image",
)
else:
require(
any(image["role"] == "bootloader" for image in images),
"bootloader_policy is include but the bundle has no bootloader image",
)
signature = manifest["signature"]
require(isinstance(signature, dict), "signature must be an object")
kind = signature.get("type")
if kind == "unsigned":
if not allow_unsigned:
fail("bundle is unsigned; pass --allow-unsigned for bench-only use")
elif kind != "ed25519":
fail(f"unsupported signature type {kind!r}")
expected = [image["file"] for image in images]
signature_member = None
body = members[1:]
if signature.get("member") is not None:
require(len(body) >= 1 and body[0][0] == "manifest.json.sig", "signature member must be second")
signature_member = body[0][1]
body = body[1:]
verify_signature(manifest_bytes, manifest, signature_member, public_key)
actual = [name for name, _ in body]
require(actual == expected, "payload members are missing, extra, or out of manifest order")
for (name, payload), image in zip(body, images):
require(len(payload) == image["size"], f"size mismatch for {name}")
digest = hashlib.sha256(payload).hexdigest()
require(digest == image["sha256"], f"sha256 mismatch for {name}")
check_format(image["format"], payload, name)
summary = {
"file": str(path),
"board": board,
"arch": manifest["arch"],
"version": manifest["version"],
"bootloader_policy": manifest["bootloader_policy"],
"signature": kind,
"marker": manifest["marker"],
"kernel_version": manifest["kernel_version"],
"members": [name for name, _ in members],
"payloads": [
{"name": image["name"], "file": image["file"], "size": image["size"], "sha256": image["sha256"]}
for image in images
],
"install": [
{
"image": image["name"],
"partition": image["partition"],
"size": image["size"],
}
for image in install_images
],
}
if plan_path:
plan = json.loads(Path(plan_path).read_text())
require(plan.get("board") == board, "plan board differs from manifest board")
require(plan.get("arch") == manifest["arch"], "plan arch differs from manifest arch")
require(
plan.get("bootloader_policy") == manifest["bootloader_policy"],
"plan bootloader policy differs from manifest",
)
plan_parts = {item["name"]: item for item in plan.get("partitions", [])}
for partition in partitions:
if partition["name"] not in plan_parts:
continue
require(
int(plan_parts[partition["name"]]["size"]) == partition["size"],
f"plan size for {partition['name']} differs from the manifest",
)
return summary
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("afw")
parser.add_argument("--allow-unsigned", action="store_true")
parser.add_argument("--public-key", help="Ed25519 public key hex, or a file containing it")
parser.add_argument("--expect-board")
parser.add_argument("--plan", help="plan JSON to cross-check")
parser.add_argument("--json", action="store_true")
parser.add_argument("--quiet", action="store_true")
args = parser.parse_args()
public_key = args.public_key
if public_key and len(public_key) != 64:
try:
public_key = Path(public_key).read_text().strip()
except OSError as error:
parser.error(f"cannot read public key: {error}")
try:
summary = validate(
Path(args.afw),
allow_unsigned=args.allow_unsigned,
public_key=public_key,
expect_board=args.expect_board,
plan_path=args.plan,
)
except AfwError as error:
print(f"afw: FAIL {args.afw}: {error}", file=sys.stderr)
return 1
except (OSError, ValueError) as error:
print(f"afw: FAIL {args.afw}: {error}", file=sys.stderr)
return 1
if args.json:
print(json.dumps(summary, indent=2))
elif not args.quiet:
print(f"afw: OK {args.afw}")
print(
f" board {summary['board']} arch {summary['arch']} version {summary['version']} "
f"bootloader {summary['bootloader_policy']} signature {summary['signature']}"
)
print(f" marker {summary['marker']} kernel {summary['kernel_version']}")
for item in summary["payloads"]:
print(f" payload {item['name']}: {item['size']} B sha256 {item['sha256']}")
for item in summary["install"]:
print(f" install {item['image']} -> {item['partition']} ({item['size']} B)")
return 0
if __name__ == "__main__":
sys.exit(main())
Regular → Executable
+131 -57
View File
@@ -1,68 +1,142 @@
#!/bin/sh
# Build an .afw firmware package for one control board and architecture.
#!/usr/bin/env bash
# Build a full-OS .afw install bundle for one board family.
#
# Modelled on the Warden `.wfw` bundle (scada/flare-edge/docs/firmware-format.md):
# one file, a strict POSIX ustar tar, a JSON manifest first, then the payload,
# with the payload's size and sha256 in the manifest so the device can verify
# before it touches flash.
# A .afw is a strict POSIX ustar with members in this exact order:
# manifest.json
# manifest.json.sig (only for an Ed25519-signed bundle)
# payload/<image>... (in manifest.images order)
#
# Differences from .wfw, both deliberate:
# The manifest carries the board, arch, SDK/OS revisions, every image's size
# and SHA-256, the partition plan, bootloader_policy, the network recovery path
# and a liveness marker. ci/check-afw.py verifies the finished bundle before
# this script returns success. There is no default signing key: unsigned is
# allowed only with an explicit --allow-unsigned, and that is bench-only.
#
# * the payload is COMPRESSED (`payload.tar.gz`) because these images ship to
# boards over metered links, and a firmware bundle is mostly redundant;
# * the manifest carries `board` and `arch`, and both go in the file name, so a
# bundle can never be mistaken for another board family. Flashing the wrong
# family is how boards are lost.
#
# Signature: `.wfw` carries an Ed25519 detached signature. We have no release
# key yet, so `.afw` currently ships `members=2` and says so in the manifest
# (`signature: "unsigned"`). Do not claim integrity beyond the sha256 until a
# key exists.
#
# usage: make-afw.sh BOARD ARCH VERSION INPUT_DIR [OUTPUT_DIR]
# e.g. make-afw.sh am335x-bb armv7 0.1.0 /tmp/release
set -eu
# usage:
# ci/make-afw.sh --board cv1835 --version 0.1.0 \
# --input build/out/cv1835 --sdk-dir ../alkane-sdk \
# --sdk-revision "$(git -C ../alkane-sdk rev-parse HEAD)" \
# --os-revision "$(git rev-parse HEAD)" \
# --output dist [--plan plan.json] [--signing-key FILE] [--allow-unsigned]
set -euo pipefail
BOARD=${1:?board family, e.g. am335x-bb}
ARCH=${2:?architecture, e.g. armv7 or aarch64}
VERSION=${3:?version, e.g. 0.1.0}
INPUT=${4:?directory holding the release files}
OUTPUT=${5:-dist}
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
BOARD=
VERSION=
INPUT=
OUTPUT=dist
PLAN=
SIGNING_KEY=
SDK_DIR=
SDK_REVISION=
OS_REVISION=
KERNEL_VERSION=
ALLOW_UNSIGNED=0
usage() {
sed -n '2,30p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
exit "${1:-1}"
}
while [ "$#" -gt 0 ]; do
case "$1" in
--board) BOARD=${2:?--board needs a value}; shift 2 ;;
--version) VERSION=${2:?--version needs a value}; shift 2 ;;
--input) INPUT=${2:?--input needs a value}; shift 2 ;;
--output) OUTPUT=${2:?--output needs a value}; shift 2 ;;
--plan) PLAN=${2:?--plan needs a value}; shift 2 ;;
--signing-key) SIGNING_KEY=${2:?--signing-key needs a value}; shift 2 ;;
--sdk-dir) SDK_DIR=${2:?--sdk-dir needs a value}; shift 2 ;;
--sdk-revision) SDK_REVISION=${2:?--sdk-revision needs a value}; shift 2 ;;
--os-revision) OS_REVISION=${2:?--os-revision needs a value}; shift 2 ;;
--kernel-version) KERNEL_VERSION=${2:?--kernel-version needs a value}; shift 2 ;;
--allow-unsigned) ALLOW_UNSIGNED=1; shift ;;
-h|--help) usage 0 ;;
*) echo "make-afw: unknown argument: $1" >&2; usage 2 ;;
esac
done
[ -n "$BOARD" ] || { echo "make-afw: --board is required" >&2; usage 2; }
[ -n "$VERSION" ] || { echo "make-afw: --version is required" >&2; usage 2; }
[ -n "$INPUT" ] || { echo "make-afw: --input is required" >&2; usage 2; }
[ -d "$INPUT" ] || { echo "make-afw: input directory not found: $INPUT" >&2; exit 1; }
if [ -z "$SDK_REVISION" ] && [ -n "$SDK_DIR" ]; then
SDK_REVISION=$(git -C "$SDK_DIR" rev-parse HEAD)
fi
if [ -z "$OS_REVISION" ]; then
OS_REVISION=$(git -C "$ROOT" rev-parse HEAD)
fi
[ -n "$SDK_REVISION" ] || { echo "make-afw: --sdk-revision or --sdk-dir is required" >&2; exit 1; }
mkdir -p "$OUTPUT"
WORK=$(mktemp -d)
OUTPUT="$(cd "$OUTPUT" && pwd)"
WORK=$(mktemp -d "${TMPDIR:-/tmp}/make-afw.XXXXXX")
trap 'rm -rf "$WORK"' EXIT
# The payload is a tar of the release tree, gzipped at maximum.
tar --format=ustar --owner=0 --group=0 --numeric-owner \
--create --file "$WORK/payload.tar" -C "$INPUT" .
gzip -9 -n -c "$WORK/payload.tar" > "$WORK/payload.tar.gz"
PAYLOAD_SIZE=$(wc -c < "$WORK/payload.tar.gz")
PAYLOAD_SHA=$(sha256sum "$WORK/payload.tar.gz" | cut -d' ' -f1)
BUILD_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
cat > "$WORK/manifest.json" <<MANIFEST
{
"schema": 1,
"format": "afw",
"board": "$BOARD",
"arch": "$ARCH",
"version": "$VERSION",
"build_date": "$BUILD_DATE",
"compression": "gzip",
"signature": "unsigned",
"payload": {
"file": "payload.tar.gz",
"size": $PAYLOAD_SIZE,
"sha256": "$PAYLOAD_SHA"
}
}
MANIFEST
MANIFEST_ARGS=(
--board "$BOARD"
--version "$VERSION"
--input "$INPUT"
--staging "$WORK"
--sdk-revision "$SDK_REVISION"
--os-revision "$OS_REVISION"
--os-dir "$ROOT"
)
[ -n "$PLAN" ] && MANIFEST_ARGS+=(--plan "$PLAN")
[ -n "$SIGNING_KEY" ] && MANIFEST_ARGS+=(--signing-key "$SIGNING_KEY")
[ -n "$KERNEL_VERSION" ] && MANIFEST_ARGS+=(--kernel-version "$KERNEL_VERSION")
python3 "$ROOT/ci/afw_manifest.py" "${MANIFEST_ARGS[@]}" >/dev/null
ARCH=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["arch"])' "$WORK/manifest.json")
NAME="alkane-os-$VERSION-$BOARD-$ARCH.afw"
(cd "$WORK" && tar --format=ustar --owner=0 --group=0 --numeric-owner \
--create --file "$OUTPUT/$NAME" manifest.json payload.tar.gz)
MEMBERS="$WORK/members.txt"
python3 - "$WORK" >"$MEMBERS" <<'PY'
import json
import sys
from pathlib import Path
stage = Path(sys.argv[1])
manifest = json.loads((stage / "manifest.json").read_text())
members = ["manifest.json"]
if manifest["signature"]["member"]:
members.append(manifest["signature"]["member"])
members.extend(image["file"] for image in manifest["images"])
for member in members:
if not (stage / member).is_file():
raise SystemExit(f"staged member is missing: {member}")
print("\n".join(members))
PY
MTIME=()
if [ -n "${SOURCE_DATE_EPOCH:-}" ]; then
MTIME=(--mtime="@${SOURCE_DATE_EPOCH}")
fi
tar --format=ustar --owner=0 --group=0 --numeric-owner "${MTIME[@]}" \
--create --file "$OUTPUT/$NAME" -C "$WORK" -T "$MEMBERS"
GOT=$(tar -tf "$OUTPUT/$NAME")
WANT=$(cat "$MEMBERS")
if [ "$GOT" != "$WANT" ]; then
echo "make-afw: FATAL: member order is wrong" >&2
printf 'expected:\n%s\nactual:\n%s\n' "$WANT" "$GOT" >&2
rm -f "$OUTPUT/$NAME"
exit 1
fi
CHECK=(--expect-board "$BOARD")
if [ -f "$WORK/manifest.pub" ]; then
CHECK+=(--public-key "$(cat "$WORK/manifest.pub")")
elif [ "$ALLOW_UNSIGNED" -eq 1 ]; then
CHECK+=(--allow-unsigned)
else
echo "make-afw: unsigned bundle needs --allow-unsigned (bench-only)" >&2
rm -f "$OUTPUT/$NAME"
exit 1
fi
python3 "$ROOT/ci/check-afw.py" "$OUTPUT/$NAME" "${CHECK[@]}"
printf '%s\n' "$OUTPUT/$NAME"
printf 'board %s\narch %s\npayload %s bytes\nsha256 %s\n' \
"$BOARD" "$ARCH" "$PAYLOAD_SIZE" "$PAYLOAD_SHA"
Executable
+342
View File
@@ -0,0 +1,342 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: MIT
"""Offline tests for the .afw producer and the CI brick-prevention gate."""
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tarfile
import tempfile
import unittest
HERE = Path(__file__).resolve().parent
ROOT = HERE.parent
MAKE = ROOT / "ci" / "make-afw.sh"
CHECK = ROOT / "ci" / "check-afw.py"
REVISION = "0" * 40
P1 = 16 * 1024 * 1024
def run(command, **kwargs):
return subprocess.run(command, text=True, capture_output=True, **kwargs)
def fixture_input(directory):
directory.mkdir(parents=True, exist_ok=True)
image = bytearray(4096)
image[0x38:0x3C] = b"ARMd"
image[0x08:0x10] = (0x200000).to_bytes(8, "little")
(directory / "Image").write_bytes(image)
dtb = bytearray(1024)
dtb[0:4] = b"\xd0\x0d\xfe\xed"
(directory / "cv-alkane.dtb").write_bytes(dtb)
(directory / "rootfs.cpio.gz").write_bytes(b"\x1f\x8b" + b"rootfs" + b"\0" * 64)
fit = bytearray(P1)
fit[0:4] = b"\xd0\x0d\xfe\xed"
marker = b"config@cv1835_miner_emmc"
fit[0x100 : 0x100 + len(marker)] = marker
(directory / "alkane-cv1835-p1.fit").write_bytes(fit)
return directory
def build_afw(work, input_dir, name="good.afw", extra=()):
output = work / "dist"
result = run(
[
"bash",
str(MAKE),
"--board",
"cv1835",
"--version",
"0.1.0",
"--input",
str(input_dir),
"--output",
str(output),
"--sdk-revision",
REVISION,
"--os-revision",
REVISION,
"--allow-unsigned",
*extra,
]
)
if result.returncode != 0:
raise AssertionError(f"producer failed:\n{result.stdout}\n{result.stderr}")
files = list(output.glob("*.afw"))
if len(files) != 1:
raise AssertionError(f"expected one bundle, found {files}")
return files[0]
def pack_members(directory, members, output):
listing = output.with_suffix(".list")
listing.write_text("\n".join(members) + "\n")
result = run(
[
"tar",
"--format=ustar",
"--owner=0",
"--group=0",
"--numeric-owner",
"--create",
"--file",
str(output),
"-C",
str(directory),
"-T",
str(listing),
]
)
if result.returncode != 0:
raise AssertionError(result.stderr)
listing.unlink()
return output
def rewrite_bundle(source, destination, edit_manifest=None, flip_payload=None):
"""Unpack a bundle, edit the manifest or payload, and repack it strictly."""
work = destination.parent / (destination.name + ".unpack")
if work.exists():
shutil.rmtree(work)
work.mkdir()
with tarfile.open(source, "r:") as archive:
names = archive.getnames()
archive.extractall(work)
manifest_path = work / "manifest.json"
manifest = json.loads(manifest_path.read_text())
if edit_manifest is not None:
edit_manifest(manifest, work)
manifest_path.write_text(json.dumps(manifest, indent=2) + "\n")
members = ["manifest.json"]
if manifest["signature"]["member"]:
members.append(manifest["signature"]["member"])
members.extend(image["file"] for image in manifest["images"])
if flip_payload:
for member in members:
if member in ("manifest.json", "manifest.json.sig"):
continue
path = work / member
if path.is_file():
data = bytearray(path.read_bytes())
if data:
data[len(data) // 2] ^= 0x01
path.write_bytes(data)
break
pack_members(work, members, destination)
shutil.rmtree(work)
return destination
class AfwTests(unittest.TestCase):
def setUp(self):
self.temp = Path(tempfile.mkdtemp(prefix="afw-test."))
self.input = fixture_input(self.temp / "sdk-out")
def tearDown(self):
shutil.rmtree(self.temp, ignore_errors=True)
def check(self, path, *extra):
return run([sys.executable, str(CHECK), str(path), *extra])
def test_good_bundle_passes_and_reports_install(self):
bundle = build_afw(self.temp, self.input)
result = self.check(bundle, "--allow-unsigned", "--expect-board", "cv1835")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("afw: OK", result.stdout)
self.assertIn("install p1 -> p1", result.stdout)
result = self.check(bundle, "--allow-unsigned", "--json")
self.assertEqual(result.returncode, 0, result.stderr)
summary = json.loads(result.stdout)
self.assertEqual(summary["board"], "cv1835")
self.assertEqual(summary["arch"], "arm64")
self.assertEqual(summary["install"][0]["size"], P1)
def test_unsigned_bundle_needs_the_flag(self):
bundle = build_afw(self.temp, self.input)
result = self.check(bundle)
self.assertNotEqual(result.returncode, 0)
self.assertIn("unsigned", result.stderr)
def test_one_byte_tampered_payload_fails(self):
bundle = build_afw(self.temp, self.input)
tampered = rewrite_bundle(bundle, self.temp / "tampered.afw", flip_payload=True)
result = self.check(tampered, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("sha256 mismatch", result.stderr)
def test_wrong_board_is_refused(self):
bundle = build_afw(self.temp, self.input)
result = self.check(bundle, "--allow-unsigned", "--expect-board", "zynq-7010")
self.assertNotEqual(result.returncode, 0)
self.assertIn("expected zynq-7010", result.stderr)
def test_manifest_cannot_claim_a_different_arch(self):
bundle = build_afw(self.temp, self.input)
def edit(manifest, _work):
manifest["arch"] = "armv7"
wrong = rewrite_bundle(bundle, self.temp / "wrong-arch.afw", edit_manifest=edit)
result = self.check(wrong, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("must use arm64", result.stderr)
def test_truncated_tar_fails(self):
bundle = build_afw(self.temp, self.input)
truncated = self.temp / "truncated.afw"
truncated.write_bytes(bundle.read_bytes()[:100])
result = self.check(truncated, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("truncated", result.stderr)
def test_traversal_member_fails(self):
stage = self.temp / "traversal"
stage.mkdir()
(stage / "manifest.json").write_text("{}\n")
(stage / "payload").mkdir()
(stage / "payload" / "p1.fit").write_bytes(b"\xd0\x0d\xfe\xed")
bundle = self.temp / "traversal.afw"
import io
with tarfile.open(bundle, "w", format=tarfile.USTAR_FORMAT) as archive:
for name, data in (
("manifest.json", b"{}\n"),
("payload/../escape", b"x"),
):
info = tarfile.TarInfo(name)
info.size = len(data)
archive.addfile(info, io.BytesIO(data))
result = self.check(bundle, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("traversal", result.stderr)
def test_oversize_fit_is_refused_by_producer(self):
fit = self.input / "alkane-cv1835-p1.fit"
data = fit.read_bytes()
fit.write_bytes(data + b"\0")
result = run(
[
"bash",
str(MAKE),
"--board",
"cv1835",
"--version",
"0.1.0",
"--input",
str(self.input),
"--output",
str(self.temp / "dist"),
"--sdk-revision",
REVISION,
"--os-revision",
REVISION,
"--allow-unsigned",
]
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("larger than", result.stderr)
def test_oversize_fit_is_refused_by_checker(self):
bundle = build_afw(self.temp, self.input)
def edit(manifest, work):
manifest["images"][-1]["size"] = P1 + 1
manifest["images"][-1]["sha256"] = "0" * 64
path = work / manifest["images"][-1]["file"]
path.write_bytes(path.read_bytes() + b"\0")
oversize = rewrite_bundle(bundle, self.temp / "oversize.afw", edit_manifest=edit)
result = self.check(oversize, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("exceeds partition", result.stderr)
def test_bootloader_member_under_preserve_is_refused(self):
bundle = build_afw(self.temp, self.input)
def edit(manifest, work):
manifest["images"].append(
{
"name": "u-boot",
"file": "payload/u-boot.img",
"role": "bootloader",
"format": "uimage",
"size": 4,
"sha256": "0" * 64,
"partition": None,
"install": False,
}
)
(work / "payload" / "u-boot.img").write_bytes(b"\x27\x05\x19\x56")
wrong = rewrite_bundle(bundle, self.temp / "bootloader.afw", edit_manifest=edit)
result = self.check(wrong, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("preserve", result.stderr)
def test_hash_mismatch_fails(self):
bundle = build_afw(self.temp, self.input)
def edit(manifest, _work):
manifest["images"][0]["sha256"] = "1" * 64
wrong = rewrite_bundle(bundle, self.temp / "hash.afw", edit_manifest=edit)
result = self.check(wrong, "--allow-unsigned")
self.assertNotEqual(result.returncode, 0)
self.assertIn("sha256 mismatch", result.stderr)
def test_missing_image_fails_producer(self):
(self.input / "Image").unlink()
result = run(
[
"bash",
str(MAKE),
"--board",
"cv1835",
"--version",
"0.1.0",
"--input",
str(self.input),
"--output",
str(self.temp / "dist"),
"--sdk-revision",
REVISION,
"--os-revision",
REVISION,
"--allow-unsigned",
]
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("required image is missing", result.stderr)
def test_unknown_board_fails(self):
result = run(
[
"bash",
str(MAKE),
"--board",
"whatsminer-h6",
"--version",
"0.1.0",
"--input",
str(self.input),
"--output",
str(self.temp / "dist"),
"--sdk-revision",
REVISION,
"--os-revision",
REVISION,
"--allow-unsigned",
]
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("unknown board family", result.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)
+146
View File
@@ -0,0 +1,146 @@
# .afw OS install bundle format
Status: schema 2. Released only by `alkane-os`. One `.afw` is valid for exactly
one board family.
An `.afw` is a full OS install bundle: kernel, device tree, rootfs, the
bootable image(s), the partition plan, the boot switch, the preflight rules and
the network recovery path. It is not an application package.
## Container
A strict POSIX ustar archive. No compression, no directory entries, no leading
`./`, no GNU long-name or pax members, and no links. Members are, in this exact
order:
1. `manifest.json`
2. `manifest.json.sig` (only for an Ed25519-signed bundle)
3. Each payload member named by `manifest.images[].file`, in manifest order.
Every member is a regular file. The manifest is at most 64 KiB. The verifier
reads the tar itself: it checks the header checksum, rejects path traversal,
absolute paths, duplicate names, non-regular members and truncated archives,
and requires the two-block end marker. It verifies the manifest and every
payload hash before any consumer may write to a board.
## Manifest
Required members:
| Field | Meaning |
|---|---|
| `schema` | `2` |
| `format` | `afw` |
| `board` | One board family, for example `cv1835` |
| `arch` | `arm64` or `armv7`; must match the board profile |
| `version` | Release version |
| `build_date` | UTC ISO-8601 |
| `sdk_revision` | 40-character SDK git revision |
| `os_revision` | 40-character alkane-os git revision |
| `payload_type` | `os` |
| `bootloader_policy` | `include` or `preserve` |
| `signature` | Signature object, see below |
| `partitions` | Partition plan, see below |
| `images` | Payload images, see below |
| `boot_switch` | How the preserved or written boot path selects the new OS |
| `preflight` | Required commands, forbidden devices, free space, source OS list |
| `recovery` | Network recovery command, held partition, golden copy location |
| `marker` | String served by the installed OS, used by `afw verify-live` |
| `kernel_version` | Kernel release the installed OS must report |
`notes` is optional. Unknown keys are ignored by this schema so a future field
does not break old readers.
### Images
Each image entry has `name`, `file`, `role`, `format`, `size`, `sha256`,
`partition` (or null) and `install` (bool). Roles are `kernel`, `dtb`,
`rootfs`, `bootloader`, `boot` and `component`. Formats are `fit`,
`arm64-image`, `fdt`, `cpio-gzip`, `uimage` and `raw`.
An image with `install: true` must name a partition and is the only kind the
installer writes. Component images (kernel, dtb, rootfs) are carried for
inspection and for boards whose installer composes them; for CV1835 they are
also the inputs of the p1 FIT.
`bootloader_policy: preserve` means no image with role `bootloader` may be
present. `include` means exactly one must be present. The toolkit refuses a
bundle whose policy does not match its board module.
### Partitions
Each partition entry has `name`, `device`, `offset`, `size`, `secure`,
`boot_critical`, `write_order` and `image` (or null). Secure partitions and
`boot0` are declared so preflight can refuse them, never as write targets.
`write_order` is 0 for a region that is not written and a positive order for
an install target.
For CV1835 the plan is:
| Region | Device | Size | Secure | Written |
|---|---|---|---|---|
| `p1` | `/dev/mmcblk0p1` | 16 MiB | no | yes, order 1 |
| `p3` | `/dev/mmcblk0p3` | 32 MiB | yes | no |
| `boot0` | `/dev/mmcblk0boot0` | 4 MiB | yes | no |
| `boot1` | `/dev/mmcblk0boot1` | 4 MiB | yes | no |
The p1 payload is the 16 MiB padded FIT. A FIT larger than the partition is a
build failure. A wrong or missing FIT is a boot failure, not a write failure,
as long as the held p1 backup is present.
## Signatures
`signature` is always an object:
```json
{"type": "unsigned", "member": null, "public_key_hex": null}
```
A signed bundle uses:
```json
{"type": "ed25519", "member": "manifest.json.sig",
"public_key_hex": "<64 hex characters>"}
```
`manifest.json.sig` is the 64-byte raw Ed25519 signature over the exact
`manifest.json` bytes, and is the second member. The key is a 32-byte raw seed
or 64 hex characters in a file named by the release environment; no key value
is stored in this repository. Adding a signature changes only the member set,
not the schema, so unsigned bundles remain readable.
Site use requires a signed bundle. The bench allows an unsigned bundle only
with `--allow-unsigned`, which prints a warning and the payload SHA-256.
## Build and check
The producer stages the manifest and payload, pads the CV p1 FIT to exactly
16 MiB, and packs the members. The checker is the release gate. Both are
offline and use only the Python standard library, except that Ed25519 signing
and verification use the `cryptography` package.
```sh
alkane-os/ci/make-afw.sh \
--board cv1835 --version 0.1.0 \
--input alkane-sdk/build/out/cv1835 \
--sdk-revision "$(git -C alkane-sdk rev-parse HEAD)" \
--os-revision "$(git -C alkane-os rev-parse HEAD)" \
--output alkane-os/dist --allow-unsigned
python3 alkane-os/ci/check-afw.py \
alkane-os/dist/alkane-os-0.1.0-cv1835-arm64.afw \
--allow-unsigned --expect-board cv1835
```
The checker fails on a wrong board or arch, a missing image, an image larger
than its partition, a bootloader member under `preserve`, a hash or size
mismatch, a path-traversal member, a truncated tar, a signed bundle without a
valid signature, or an unsigned bundle without the explicit flag.
## Install rule
The toolkit verifies the whole bundle before opening a transport. It then
preflights the board, holds the current p1 over the network, writes only the
declared install partition, reads the whole partition back, and compares the
hash before it reboots. No install step needs a serial console, an SD card, a
U-Boot prompt, JTAG or a button.