fold in external review (N-5): decode-rate corrected to unknown/plan-at-5-tok/s, cache semantics fixed (hits not re-prefill), KV headroom, feral-cats 7-slot gotcha + engineering debt, five-strategy table, benchmark gate replaces the N-4 lean, TCO

Review source: GPT-5.6 pass shared by jhestyr 2026-07-17. Accept/pushback record in NOTES N-5.

[ezra]
This commit is contained in:
2026-07-17 11:24:07 -04:00
parent c46ff918a2
commit ad15ab5200
3 changed files with 117 additions and 20 deletions
+9 -3
View File
@@ -15,9 +15,15 @@ re-derive it: two build shapes, the models each runs, honest capability tiers, a
that will invalidate the numbers over time.
The trigger was real data: repricing the lassiter-creek-adventure-game's build (10 sessions, Jul 517,
from local transcripts) showed **1.15B cache-read tokens vs 3.5M output** — agentic coding is ~99%
re-reading context. That shape drives every hardware conclusion here (prefill/memory-bandwidth is the
bottleneck, not generation speed).
from local transcripts) showed **1.15B cache-read tokens vs 3.5M output** — agentic coding is
cache-dominated. The hardware conclusion that follows (corrected by external review, NOTES N-5): the
first-order requirements are **KV-cache persistence, hit rate, and capacity** — cache reads are *hits*,
not reprocessing; raw prefill bandwidth matters on the misses, not on the 99%.
**Status 2026-07-17 (late):** externally reviewed (jhestyr's friend via GPT-5.6 — NOTES N-5). Two
original numbers did not survive: the workhorse decode-rate estimate and the feral-cats slot count.
**Any purchase is now gated on the benchmark plan in BUILDS.md** — the workhorse is a hypothesis, not
a validated build.
Standing constraints from jhestyr: **no Mac** ("not interested in the mac approach"). Not that fancy a
man. Any coffee is coffee as long as it's not burnt.