Dataflow, executed · 02
Why the systolic array feeds itself diagonally
Systolic dataflow
Every explainer draws numbers entering a grid on a diagonal and moves on. The diagonal is not a convention someone chose — it is the only arrival schedule that works, and you can watch it fall out of the timing rules.
Watch the left-hand matrix, not the array. The
entries that leave on cycle t are exactly the anti-diagonal
m + k = t. The staircase sweeping across A is
the skew.
That is the part usually left as an illustration. A grid appears, values slide in on a diagonal, and the viewer is expected to accept the shape. But the shape is forced, and seeing where the values come from is what makes it obvious — which is why the input matrix is on screen at all.
The three rules that produce it
Take Y = A · B with A streaming and
B stationary. The processing element at row k,
column j holds the weight B[k][j] for the
whole run. Two things then move, in different directions:
Activations travel rightwards, one cell per cycle, so
A[m][k] entering row k reaches column
j at cycle m + k + j. Partial sums travel
downwards, one cell per cycle, so the running total for
output Y[m][j] is at row k at the same cycle
m + k + j.
Those two expressions are equal on purpose. The multiply at
PE(k, j) only produces a correct result if the activation
and the running sum that belong together arrive on the same cycle, and
the only way to arrange that is to launch row k of the
stream k cycles late. Skew row 0 by anything, or launch the
rows together, and every multiply meets the wrong partial sum. The
diagonal is not aesthetic; it is a rendezvous condition.
What the array actually buys
The second thing a static diagram cannot show is why anyone builds this.
A naive matrix multiply re-reads a weight for every output it
contributes to: M × K × N reads, 27 for the
product in the video. The array reads each weight once —
K × N, so 9 — and then lets every activation in the
stream flow past it.
| Quantity | Rule | In the video |
|---|---|---|
| Weight reads from SRAM | K × N | 9 |
| Naive GEMM weight reads | M × K × N | 27 |
| Activation arrives at PE(k, j) | m + k + j | — |
| Output leaves column j | m + K + j | — |
| Total cycles | M + K + N - 2, plus drain | 8 |
The ratio is the point, not the constant: reads fall by a factor of
M, the length of the activation stream. That is why real
arrays are large and fed for a long time — a 256×256 unit
amortises its weight load across thousands of rows, and the memory
system stops being the thing that decides throughput.
Nothing here is drawn
Every tile in the animation is placed by the timing rules above, and
each output is asserted against A @ B as it leaves the
bottom of the array. A scene with the skew wrong does not render a
convincing picture of a broken schedule — it raises.
git clone https://github.com/SciMigo/straightedge
cd straightedge && python3 -m pip install -e '.[render]'
cd examples/systolic_array && manim -qm scene.py SystolicArray
Change A or B at the top of the file and the
weights, the arrival times, the result matrix and the read counts all
follow. The one thing you cannot do is make it show an answer that is
not the product.
A note on what the picture had to lose
The array in the video has no self-loops, though a real weight-stationary cell holds its weight across cycles and every accurate block diagram shows that. At the size a reader actually watches this — a card in a feed, 200 pixels wide — each extra stroke costs more legibility than the fact is worth, and the arrows between cells are what make the picture read as a systolic array at a glance. Worth stating rather than hiding: a simplification you cannot see is indistinguishable from an error.