bfe-core1106-sdk/kernel/rv1106-enablement/npu/OPEN-NPU-PLAN.md

Open NPU to 100% — plan, feasibility, and the honest ceiling

Goal as stated: take the RV1106 NPU to "100% — open source, verified on hardware." Bottom line up front: two very different things live under that sentence, and only one of them is reachable soon.

1. Open, on-hardware-verified kernel driver (/dev/dri/cardN binds, answers a version-query ioctl on our self-built Linux 6.18.46). Achievable now, low-single-digit engineer-days, license-clean GPL forward-port. This is the PORT-PLAN.md work in this directory — a real, bounded milestone (M6).
2. Open userspace that runs a real model (an open compiler/encoder emitting a valid regcmd stream this driver can submit, no closed librknnrt, no .rknn blob). Not achievable on any near horizon. It is a from-scratch, per-SoC register reverse-engineering project with zero existing public prior art for RV1106/RV1103 — the least-covered tier of the entire RKNPU family.

This document scopes both, states the most ambitious end state that is actually achievable, and defines the concrete first milestone toward open compute. PORT-PLAN.md remains the authoritative, file-by-file plan for item (1); this document is the strategic wrapper and the item-(2) reality check. Read both.

---

1. Brutally honest feasibility

1.1 "Port the GPL kernel driver" — TRACTABLE (do it)

• The vendor driver (flare-edge/sdk/sysdrv/source/kernel/drivers/rknpu/, v0.9.2, DRIVER_DATE 20230825) is SPDX GPL-2.0 at the file level, MODULE_LICENSE("GPL v2") (rknpu_drv.c:2176), authored by Rockchip (Felix Zeng), mirrored at github.com/airockchip/rknpu. Legitimately open, cleanly forward-portable.
• It is a single multi-SoC codebase that already targets RV1106 natively: of_match entry { .compatible = "rockchip,rv1106-rknpu", .data = &rv1106_rknpu_config } (rknpu_drv.c:196-198) + a dedicated config struct (rknpu_drv.c:143-160: dma_mask=32, pc_data_amount_scale=2, pc_task_number_bits=16, pc_task_status_offset=0x3c, single-core irqs/resets arrays, nbuf_phyaddr=0/nbuf_size=0).
• The DT node is already fully specified in the base tree, only disabled — npu@ff660000, reg=<0xff660000 0x10000>, GIC_SPI 109, clocks ACLK_RKNN/HCLK_RKNN, assigned-clock-rates=<420000000>, resets SRST_A_RKNN/SRST_H_RKNN, status="disabled" (rv1106.dtsi:1127-1138). One board-DTS override (&npu { status="okay"; }) enables it.
• No IOMMU on this board ("rknpu iommu device-tree entry not found!, using non-iommu mode" — CMA/contiguous DMA only), no power-domains (single-rail), no OPP table (fixed clock). This shrinks the port: the multi-domain/DVFS/ thermal code is all provably dead for RV1106's DT.
• The one real build blocker is four vendor-only downstream headers (soc/rockchip/rockchip_{iommu,opp_select,system_monitor,ipa}.h) absent from mainline 6.18 — all four back dead-code call sites for this SoC, fixable with small local compat-shim stubs (same pattern already used for clk-rv1106.c's panic_notifier_list move). Full delta in PORT-PLAN.md §2.4.

Verdict: real, bounded, worth doing. Same class as the RGA port. Ends at a clean probe + RKNPU_GET_DRV_VERSION/RKNPU_GET_HW_VERSION ioctl round-trip on hardware — and stops there, because of §1.2.

1.2 "Open userspace that runs a model" — HARD → effectively BLOCKED for RV1106

The kernel driver's entire hardware contract is: power/clock/reset, DMA/GEM buffer management, then it drops a pointer + length to a userspace-authored regcmd blob into four registers and pulses "go," then fields one IRQ. It never inspects the regcmd contents. (Traced through rknpu_job.c:266-364, rknpu_job_subcore_commit_pc().)

The four PC registers the driver programs per task (offsets into the 0xFF660000 block, from rknpu_ioctl.h:21-39 used via REG_WRITE in rknpu_job.c):

Offset	Name	Meaning
0x00/0x04	VERSION / VERSION_NUM	read-only HW version
0x08	PC_OP_EN	pulse 1→0 kicks the job; 0x1=slave mode before commit
0x10	PC_DATA_ADDR	device address of the regcmd buffer
0x14	PC_DATA_AMOUNT	(regcfg_amount + 4 + scale-1)/scale - 1
0x20/0x24/0x28/0x2c	INT_MASK / INT_CLEAR / INT_STATUS / INT_RAW_STATUS	interrupt handshake
0x30	PC_TASK_CONTROL	((0x6|pingpong)<<pc_task_number_bits)|task_number (the 0x6 is an undecoded fixed mode field)
0x34	PC_DMA_BASE_ADDR	task_base_addr

This is the entire openly-documented register map — the wrapper/sequencer only. None of it describes the compute engine (convolution/pooling/activation config, feature/weight DMA descriptors). Those live inside the opaque regcmd buffer. The op-code semantics of that stream:

• Have no public register-level documentation anywhere. TRM Part 2 does not exist publicly — confirmed by exhausting Rockchip doc mirrors and confirmed directly by a Luckfox engineer that Luckfox itself was never given one. Even Rockchip's own 90-page ROCKIVA SDK guide is pure userspace-API reference with zero register offsets. This is a deliberate architectural gate, not a hard-to-find doc.
• Only the closed RKNN-Toolkit2 compiler emits valid regcmd streams. The on-device closed librknnrt parses .rknn blobs and issues the same RKNPU_MEM_*/RKNPU_SUBMIT ioctls. DRM_IOCTL_RKNPU_SUBMIT technically accepts a raw command buffer directly — nothing requires librknnrt — but since the buffer format is closed, that ioctl is not a "write your own kernel" door.

1.3 Why there is no shortcut through the existing open efforts

Open effort	Chip(s)	Reaches RV1106?
Mainline accel/rocket (merged 2025-07, in our research/linux-6.18.46/)	RK3588 only, as shipped	No. Kconfig: depends on (ARCH_ROCKCHIP && ARM64). RV1106 is 32-bit arch/arm — excluded before the register question. of_match is hardcoded "rockchip,rk3588-rknn-core".
Mesa Teflon (TFLite delegate)	Only etnaviv or rocket backends	No. Entirely downstream of Rocket — its Rockchip reach is Rocket's reach: RK3588 only.
gahingwoo/linux-rk3576-npu + charsiu (active, days-old as of 2026-08)	RK3576	No — wrong generation. MobileNetV1 end-to-end + a Llama-3.2-1B INT4 open runtime (charsiu, opens /dev/accel/accel0, emits its own register stream) reportedly work byte-exact — but on RK3576's NPU, a newer generation than RV1106's. Unmerged (LKML "[PATCH v9] accel/rocket: RK3576" in review; unverifiable by second source — Anubis-walled).
RK3568/RK3566 Armbian fork	RK3568	No. Built by byte-level diff against captured vendor command streams; multi-task still imperfect; unmerged.
phhusson/rknpu-reverse-engineering	RK3588 only, dormant since 2024-03, no license	No. Exploratory strings/strace/GEM-dump notes; no compiler, no encoder, no runtime. ~1 person-year got RK3588 to "structures identified."
RV1106/RV1103	—	Zero hits of any kind. No driver, no fork, no RE writeup, no captured traces. Least-covered tier in the family.

RV1106 sits in the RKNPU2 "v1" family (RK3566/68, RV1109/26, RV1103/06 — INT8-only, quantize=8 mandatory). RK3576/RK3588 are a different, newer generation. Precedent (RK3568 needing byte-level RE off RK3588) shows this is a bespoke per-SoC RE project even between adjacent generations, not a recompile.

1.4 The most ambitious achievable end state

Ranked by realism:

• Tier A — ACHIEVABLE NOW (commit to this): open GPL kernel driver, statically built into our 6.18.46, /dev/dri/cardN binds, /proc/rknpu/load live for the Monitor page, RKNPU_GET_HW_VERSION/RKNPU_GET_DRV_VERSION verified on hardware. No blob shipped. This is item (1); it is the honest "100% open" for the driver.
• Tier B — ACHIEVABLE AS A RESEARCH SPIKE, bounded, low-value: a single hand-written op (one conv or one matmul) executed via a regcmd captured from the closed stack and replayed through our open DRM_IOCTL_RKNPU_SUBMIT, byte-for-byte, with an open host tool — proving the open submit path can drive real compute. This does not require understanding the regcmd ISA; it requires capturing one. It is the concrete first milestone toward open compute (§3, M-NPU-2) and the only way to get an on-hardware compute number without the RE project.
• Tier C — NOT ACHIEVABLE on any near horizon: an open compiler/encoder that emits novel regcmd streams for arbitrary models. This is the ~person-year+ from-scratch RE project (§3, M-NPU-3+). We should scope it, not staff it, unless a strategic reason to own open NPU compute emerges.

Honest recommendation: ship Tier A. Do Tier B only as a time-boxed spike if a real non-visual inference use case (audio/RS-485/touch-gesture classifier — the only sensible NPU use here; no camera exists on this board) is actually wanted. Treat Tier C as closed-vendor-runtime-only for the foreseeable future. This matches the standing PROVENANCE.md policy: we do not ship the closed runtime blob; the open path is a from-scratch regcmd encoder tracked separately, never a vendored binary — and until that encoder exists, open NPU compute does not exist for us.

---

2. Key technical facts (for the implementer)

Register / memory map (TRM Part1 — Part2 does not exist publicly)

Block	Base	Size
NPU core	0xFF660000	64KB (reg=<0xff660000 0x10000>)
NPU_GRF	0xFF018000	32KB
NPU_SGRF	0xFF074000	8KB
NPU_CRU	0xFF3B6000	8KB
NPU_CBUF (scratch SRAM)	0xFF680000	256KB (unused: config sets nbuf=0)

IRQ GIC_SPI 109 LEVEL_HIGH, node ff660000.npu. Clocks ACLK_RKNN/HCLK_RKNN off the CRU matrix; only measured clock point anywhere is NPU 500MHz (Rockchip internal power-test xlsx, "typical IPC workload" corner). A commented-out board-DTS override would set 700MHz — never applied.

UAPI (the seam an open userspace must target)

Two memory-manager paths, Kconfig-selected. RV1106's shipping defconfig picks CONFIG_ROCKCHIP_RKNPU_DMA_HEAP=y (/dev/rknpu misc device, bare IOCTL_RKNPU_* codes, magic 'r') — but PORT-PLAN.md deliberately chooses DRM_GEM for the 6.18 port (/dev/dri/cardN, DRM_IOCTL_RKNPU_*, DRM_RENDER_ALLOW), because M4 already pulls DRM in for the panel and DRM_GEM is the vendor default. Same struct payloads either way. Six ioctls: RKNPU_ACTION, RKNPU_SUBMIT, RKNPU_MEM_{CREATE,MAP,DESTROY,SYNC} (rknpu_ioctl.h:288-322).

struct rknpu_task {                 // rknpu_ioctl.h:218-228
    __u32 flags, op_idx, enable_mask;
    __u32 int_mask, int_clear, int_status;
    __u32 regcfg_amount, regcfg_offset;
    __u64 regcmd_addr;              // <-- points at the OPAQUE compute program
} __packed;
struct rknpu_submit {               // rknpu_ioctl.h:260-274
    __u32 flags, timeout, task_start, task_number, task_counter;
    __s32 priority;
    __u64 task_obj_addr, regcfg_obj_addr, task_base_addr, user_data;
    __u32 core_mask; __s32 fence_fd;
    struct rknpu_subcore_task subcore_task[5];
};

An open userspace must produce: (a) the struct rknpu_task[] list — fully inferable from the kernel driver alone, trivial; and (b) the regcmd byte stream each task points to — NOT inferable from the driver or any published TRM. (b) is the whole problem.

/proc/rknpu — safety (hardware-confirmed)

• /proc/rknpu/load — safe (50 clean reads verified); the Monitor page polls it.
• /proc/rknpu/freq, /power — exposed, unverified.
• /proc/rknpu/volt — CONFIRMED SIGSEGV (null deref — no regulator wired). Never poll.

Provenance / policy

PROVENANCE.md: the kernel driver is portable GPL; the closed piece is the userspace RKNN runtime + regcmd format (a blob). Per directive we do not ship that blob. CAPABILITIES-AUDIT.md's Remaining-blocks table rates the NPU "open driver VERIFIED; compute deferred" until an open encoder exists.

URLs

• Vendor driver: github.com/airockchip/rknpu, github.com/rockchip-linux/rknpu
• Mainline Rocket: drivers/accel/rocket/ (RK3588-only); blog: blog.tomeuvizoso.net/2025/07/rockchip-npu-update-6-we-are-in-mainline.html
• Teflon: docs.mesa3d.org/teflon.html; NLnet: nlnet.nl/project/Rockchip-NPU-driver/
• RK3576 RE: github.com/gahingwoo/{linux-rk3576-npu,charsiu,kiln} (active, unmerged)
• RK3588 RE: github.com/phhusson/rknpu-reverse-engineering (dormant, no license)
• Official closed docs (RV1106/RV1103 quick-start PDFs, API-only): github.com/airockchip/rknn-toolkit2 doc/

---

3. Milestones

M-NPU-1 — Open kernel driver, verified on hardware (ACHIEVABLE — the real deliverable)

Follows PORT-PLAN.md in full. Summary of the gates:

1. Build: CONFIG_ROCKCHIP_RKNPU=y + _DRM_GEM=y + _DEBUG_FS/_PROC_FS; compile rknpu_{drv,job,gem,reset,iommu,debugger}.o clean into built-in.a. Prove the four §2.4 compat shims (compat/soc/rockchip/rockchip_{iommu,opp_select, system_monitor,ipa}.h) against real 6.18 headers — this is where they get verified, not just inspected.
2. DT: board override &npu { status="okay"; }; dtc -W clean; re-check the "place the fdt high" DTB-placement fix (bigger built-in.a).
3. Boot via the warden-c8a3 _b-slot one-shot loop (never touches _a, auto-reverts on hang): dmesg | grep -i rknpu shows clean probe — clock/reset/ IRQ acquired, no panic, no -EPROBE_DEFER stall.
4. Node: /dev/dri/cardN (or renderD1xx) appears — classic DRM node, not /dev/accel/.
5. Verify ioctl: host-buildable C program opens the node, issues DRM_IOCTL_RKNPU_ACTION {.flags=RKNPU_GET_DRV_VERSION}, checks value decodes to 0.9.2; RKNPU_GET_HW_VERSION returns something plausible. Exercises full dispatch → power-get/put → clock/reset with no regcmd dependency.
6. Capture the durable delta as a patch series in this dir (compat shims + DT fragment + config fragment); vendor source stays in research/linux-6.18.46/.

Effort: low-single-digit engineer-days. Risk: a surprise vendor-only symbol not surfaced by the read-through; DTB placement. This is "NPU driver = 100% open, verified." Done here for the driver half.

M-NPU-2 — First open compute: capture-and-replay one op (SPIKE, only if wanted)

The concrete first milestone toward open compute, and the honest Tier-B ceiling. It does not require decoding the ISA.

1. On a unit with the closed stack available (build host / a dev panel with the closed librknnrt + a trivial single-conv or single-matmul .rknn), capture the regcmd buffer(s) the runtime DMAs in — via strace of the RKNPU_MEM_*/ RKNPU_SUBMIT ioctls + GEM_FLINK/GEM_OPEN buffer dumps (phhusson's method), plus the input/output tensor buffers.
2. Write an open host tool that: allocates the same GEM buffers via RKNPU_MEM_CREATE, writes the captured regcmd bytes + captured input tensor, builds the struct rknpu_task[] (fully open, §2), and submits via DRM_IOCTL_RKNPU_SUBMIT on our M-NPU-1 driver.
3. Verify: output tensor matches the closed runtime's output byte-for-byte on hardware, and matches a CPU/NEON reference for the same op. Record the measured job-submission latency (ioctl+DMA+blocking-IRQ) — a number that exists nowhere today and settles the "NPU vs NEON dispatch overhead" argument.

Deliverable: proof the open submit path drives real compute + first latency number. Not a general runtime — the regcmd is a fixed captured blob for one op shape. Value: low; do only as a time-boxed research spike behind a real use case. Never ship a captured Rockchip regcmd as a product artifact (it is their compiler's output) — this stays in research/.

M-NPU-3+ — Open regcmd encoder (SCOPE ONLY — do not staff without a strategic reason)

The Tier-C from-scratch RE project. Method (the only demonstrated one): differentially decode captured regcmd streams across many hand-built minimal ONNX models (isolate one op/param at a time), reconstruct the compute-engine register semantics, build an encoder for a first op, then grow the op set. Precedent cost: ~1 person-year got RK3588 to "exploratory, no compiler"; RK3568 needed byte-level diffing and still isn't merged; RV1106 starts from literal zero. Track here if ever begun; otherwise this is the documented reason open NPU compute is deferred.

---

4. Risks / open questions

• The core risk is misframing. "The driver is open source" must not be sold as "the NPU is open-source-usable." M-NPU-1 delivers an open driver that can submit jobs; it delivers no open way to produce a valid job. Guard this in every status line.
• Additional vendor-only symbols beyond the four headers may surface only at compile — M-NPU-1 gate 1 is the real test.
• SKU unconfirmed (G2 128MB / 0.5-TOPS vs G3 256MB / 1.0-TOPS; datasheet Rev 2.0 retracted the split to "both 1.0 TOPS"). Determines the compute ceiling if M-NPU-2/3 are ever pursued; needs an on-hardware chip-ID/TOPS register read.
• No open regcmd project for RV1106 exists (re-confirm periodically — community moves fast, esp. the gahingwoo RK3576 work; but it is the wrong generation).
• No NPU use case is currently established for this board (no camera; RGA2 + RKIVE already cover the 2D/CNN-shaped image tasks better — see npu-graphics-feasibility.md). Absent a concrete non-visual classifier need, M-NPU-2 and beyond have no pull, and M-NPU-1 alone (driver binds, load graph works, no blob) is the correct stopping point.

---

Cross-refs: PORT-PLAN.md (authoritative file-by-file kernel port), ../../docs/npu-graphics-feasibility.md, ../CAPABILITIES-AUDIT.md, ../PROVENANCE.md, ../DRIVER-PARITY.md:41, ../REMAINING-PORTS.md §6.