9. Sustained, Not Peak🔗

Every accelerator quotes a peak number — the multiply rate if every multiplier ran every cycle. Real workloads never see it. The honest question, the one this chapter answers with measurements rather than adjectives, is: how much of the peak does the machine actually sustain on real work, and how does that — placed and routed — compare to the machines it must compete with?

9.1. The scalar tax: ~23% of peak, measured🔗

Run a real attention block — the matmuls of $QK^top$ and $cdot V$, and the softmax between — on the single-element semantics, and count cycles against useful multiply-accumulates. The result is sobering and clarifying at once: 32 useful MACs in 138 cycles — ~23% of peak. The matmuls themselves sustain only ~40% (every MAC waits on an operand load), and the softmax burns 39% of all cycles doing no matmul at all. Peak, for real attention on a scalar core, overstates reality by ~4×.

Two architectural moves close most of that gap, and each is measurable. Feeding operands through a dataflow fabric instead of loading them per-MAC lifts the matmul; hardware transcendentals instead of a software exp collapse the softmax. Re-measured on the same block: 23% → 30% → 44%.

9.2. The systolic answer: ~90%, measured🔗

The deeper fix is organisational. Wire the MAC elements into a systolic array — weights held, activations flowing one axis, partial sums the other — and in steady state every element does a MAC every cycle. We simulate the wavefront cycle-accurately (counting active elements per cycle, not asserting the textbook formula), then fold in the element's real per-cycle issue overhead.

namespace Perf

/-- Elements doing a MAC at cycle `t`: element (i,j) is active while
`i+j ≤ t < i+j+K`. -/
def activePEs (N K t : Nat) : Nat :=
  (List.range N).foldl (fun acc i =>
    acc + ((List.range N).filter (fun j => i + j ≤ t ∧ t < i + j + K)).length) 0

/-- Measured useful MAC-cycles over the whole run (sum over cycles). -/
def macCycles (N K : Nat) : Nat :=
  (List.range (K + 2 * N + 2)).foldl (fun acc t => acc + activePEs N K t) 0

/-- Measured wall-clock: cycles for which the array is busy. -/
def wall (N K : Nat) : Nat :=
  ((List.range (K + 2 * N + 2)).filter (fun t => activePEs N K t > 0)).length

/-- Array utilization, per-mille, straight from the simulation. -/
def arrayUtil (N K : Nat) : Nat := macCycles N K * 1000 / (N * N * wall N K)

/-- Sustained, folding in the element's mac-loop overhead (U MACs per U+1
cycles): a programmable element gives up almost nothing. -/
def sustained (N K U : Nat) : Nat := arrayUtil N K * U / (U + 1)

The simulation really does the full $N^2K$ useful MACs, and the sustained figures are machine-checked facts, not claims:

theorem sim_exact : macCycles 8 64 = 8 * 8 * 64 := by⊢ macCycles 8 64 = 8 * 8 * 64 native_decideAll goals completed! 🐙

theorem util_8x8   : sustained 8 256 16 = 892 := by⊢ sustained 8 256 16 = 892 native_decideAll goals completed! 🐙   -- 89.2%
theorem util_16x16 : sustained 16 1024 16 = 913 := by⊢ sustained 16 1024 16 = 913 native_decideAll goals completed! 🐙 -- 91.3%

end Perf

So a systolic tile of these elements sustains ~85–91% — against the scalar core's 23–44%. Crucially, the element's own overhead costs only the last few percent: a programmable, Turing-complete element gives up almost nothing versus a fixed-function cell. Utilization is GPU-class (FlashAttention-3 on an H100 sustains ~75%).

9.3. The density, in routed silicon🔗

Utilization is half the story; density is the other. Placed and routed on an open 130 nm process, at a utilization target where the earlier thirty-two-register element failed to route at all, the numbers are:

The full element: 0.159 mm², thirty-two registers — failed global routing; its register-file crossbar was the congestion wall.
The Nano core: 0.0525 mm² (eight registers), then 0.0271 mm² (four registers, memory off-tile) — both routing cleanly. ~6× denser, still Turing-complete.
A systolic tile of 64 MAC elements: routed, 61 MAC/mm².

One measurement corrected a guess worth stating plainly: at ~1,600 cells per element, the 16×16 multiplier dominates — it is ~94% of a fixed-function cell and ~60% of the programmable Nano core. The multiplier is an irreducible floor, the same in every design. Which means a programmable Nano element is only ~1.65× larger than a fixed-function one — the price of full generality and machine-checked correctness is a small constant, not an order of magnitude.

9.4. Where that leaves the MAC🔗

Combining measured density and measured utilization, and projecting the routed 130 nm figures to a 16 nm node, the systolic tile lands at roughly 1–12 TOPS/mm² depending on datatype — within a small single-digit multiple of a modern GPU's local tensor-core density, at full utilization. The journey of this project's numbers tells the real story:

on peak throughput, the starting element was ~60,000× behind a datacenter GPU;
on sustained, routed density, the Nano systolic tile is ~1.5–3× behind a GPU's local MAC density — and the programmable, verifiable version costs only ~1.65× more than a fixed-function one.

The two orders of magnitude the peak number hid have been closed to a small constant — measured, placed, and routed. What no competitor can put beside its throughput figure, this one carries throughout: a machine-checked proof that the multiply-accumulate is correct. That is the datasheet as a theorem, priced in silicon.