fold in external review (N-5): decode-rate corrected to unknown/plan-at-5-tok/s, cache semantics fixed (hits not re-prefill), KV headroom, feral-cats 7-slot gotcha + engineering debt, five-strategy table, benchmark gate replaces the N-4 lean, TCO

Review source: GPT-5.6 pass shared by jhestyr 2026-07-17. Accept/pushback record in NOTES N-5.

[ezra]

NOTES.md

@@ -16,8 +16,11 @@ cached-in · $3/M in · $15/M out). Repricing the adventure game's real build to
 
 - **1,147M cache-read** + 18M uncached-in + 3.5M out → **≈ $451 at K3 rates** (±30%, session-level
   granularity) vs ≈ $1,307 at Claude list prices. Actual marginal cost: $0 (subscription).
-- The load-bearing fact: agentic coding is **~99% context re-reads, ~1% output**. Cached-input price
-  (cloud) / prefill speed + KV-cache persistence (local) dominate everything.
+- The load-bearing fact: agentic coding is **~99% cache reads, ~1% output** — cache-*dominated*.
+  (Corrected per N-5: cache reads are **hits**, not reprocessing. Local first-order requirements =
+  KV-cache persistence + hit rate + capacity; raw prefill bandwidth matters on the misses only.
+  Operational gotcha: harnesses that mutate the prompt *prefix* silently break llama.cpp/vLLM prefix
+  caching — a slower box that keeps a 100K-token cache warm beats a faster box that keeps evicting it.)
 - K3 itself is never local: ~1.4TB weights at 4-bit. "Local like that" = best open model that fits a box.
 
 ## N-2 (2026-07-17) — The open-model landscape, mid-2026
@@ -46,7 +49,7 @@ mid-2026; SK Hynix sold out through 2026, fabs pivoted to HBM). Consequences:
 - RTX Pro 6000 Blackwell 96GB: Nvidia list jumped 55% in 16 months to $13,250; street/Max-Q hunts lower.
 - Relief not expected before **2027**. Waiting is a legitimate strategy; re-price at buy time.
 
-## N-4 (2026-07-17) — Decision lean (not a decision)
+## N-4 (2026-07-17) — Decision lean (not a decision) — ⚠️ SUPERSEDED BY N-5
 
 Ezra's recommendation if/when the itch turns real: **the workhorse** (Milan + 512GB DDR4 refurb + one
 RTX Pro 6000 Max-Q, ~$15–19k) over the 8×3090 rig — a smarter model that's slower wastes fewer turns
@@ -55,3 +58,38 @@ second card slot) without replacing bones. Integration shape: headless Debian, l
 OpenAI-compatible endpoint on the tailnet; per [[mill-is-home-for-workloads]] it's a Mill annex, not a
 new species. Sensible trigger to revisit: **after 2026-07-27** (K3 weights + first distills) or when
 DDR5 unclenches.
+
+**Superseded 2026-07-17 (same day) by N-5:** the workhorse's decode-rate assumption didn't survive
+external review. Lean is now benchmark-gated; see N-5.
+
+## N-5 (2026-07-17) — External review (GPT-5.6 via jhestyr's friend) — what survived, what didn't
+
+jhestyr shared BUILDS.md out (repo made public for it) and brought back a GPT-5.6-thinking review.
+Graded on merits; most of it holds. **Accepted corrections** (BUILDS.md updated to match):
+
+1. **Workhorse decode rate was wrong.** My "15–25 tok/s" for Qwen3-Coder-480B silently assumed
+   CPU-executed experts. Naive vLLM/llama.cpp offload ships ~11GB of active expert weights over PCIe
+   (~25GB/s) *per token* → **2–3 tok/s ceiling**. KTransformers-style CPU-expert execution could reach
+   low teens against 205GB/s DDR4 — but its fast paths want AVX-512/AMX, and **Milan is AVX2** (native
+   AVX2 support still maturing as of Feb 2026). Honest number: **unknown; plan at ~5 tok/s** until
+   benchmarked on this exact combo. The workhorse is a hypothesis, not a validated build.
+2. **Cache semantics were backward** (fixed in N-1): cache reads = hits, not re-prefill. Persistence/
+   hit-rate/capacity are first-order; prefill bandwidth is the miss path. Corollary gotcha: Claude
+   Code-style harnesses can mutate the prompt prefix and silently kill prefix-cache reuse.
+3. **KV-cache headroom ignored:** Qwen3-480B at 100K context ≈ 24GiB BF16 (12 FP8) of KV — the 96GB
+   card really offers ~60–75GB for weights, pushing MORE experts to RAM. Compounds #1.
+4. **Feral cats under-engineered:** ROMED8-2T has **7** x16 slots — my table put 8 GPUs on it with no
+   bifurcation plan, lane map, Above-4G/MMIO validation, or power engineering. Also: 3090s DO have
+   pairwise NVLink (no 8-way fabric) — my "no NVLink" line was too flat.
+5. **Capability tiers not decision-grade:** "Sonnet-4-era" conflates full-precision benchmarks with an
+   aggressive quant on an exotic execution path. The metric that matters: **successful agent turns per
+   hour** on OUR repos. And test the **80B-A3B first** (≈96% quality) — if it holds, one GPU suffices.
+6. **Economics:** subscription makes marginal Claude cost $0 — the box is justified by privacy/control/
+   unlimited background tokens, not savings. Add TCO: ~$0.15/kWh → 700W continuous ≈ $920/yr.
+   (My pushback: our duty cycle isn't continuous; idle ~100–150W. Real number is a few hundred $/yr.)
+
+**Pushback recorded (minor):** the 2–3 tok/s "upper bound" assumes uniformly distributed expert
+selection with zero locality; hot-expert pinning skews that upward in practice — but direction stands
+and the burden of proof is ours. **New lean (replaces N-4):** *benchmark before buying* — the friend's
+strongest frame: **one RTX Pro 6000 running a strong 80–120B local model, with cloud escalation for
+hard turns, is more defensible than either full build.** Benchmark gate lives in BUILDS.md.