The pipeline bubble 1F1B doesn't fix

The usual sentence about 1F1B is that it reduces the bubble. It is repeated in blog posts, in slide decks, and in the framing of most diagrams of the two schedules — and it is wrong in a way that matters, because it points at the wrong reason to adopt it.

What the simulation measures

Neither chart above is drawn by hand. Every operation is placed at the first cycle its dependencies allow: forward m on stage p waits for forward m on stage p-1; backward m waits for backward m on stage p+1, or for its own forward if it is on the last stage. The only difference between the two runs is the order each stage is told to work through its own queue.

Why the bubble cannot move

The bubble is the cost of filling and draining a pipeline, and it is fixed by the shape of the dependency graph rather than by the order work is issued. The last stage cannot start until the first forward has walked all the way down to it; the first stage cannot finish until the last backward has walked all the way back up. That round trip is P-1 stages of latency at each end, against M microbatches of useful work — the familiar (P-1)/M.

1F1B reorders operations within a stage. Reordering a queue cannot shorten the critical path through a graph it does not change, so the finish time is identical. The animation makes this checkable rather than assertable: if you can find a cycle where one chart is ahead of the other, the claim is false.

Why the memory does move

A forward pass has to keep its activations until the matching backward consumes them. That interval — forward finishes, backward begins — is the entire story, and the two schedules stretch it very differently.

GPipe runs every forward before any backward, so microbatch 0's activations sit in memory while all five later forwards run. At the peak the cluster is holding almost every microbatch's activations at once. 1F1B starts backwards the moment the pipeline is full, so each stage carries roughly P sets instead of M, and the gauge stays flat instead of climbing to a hump.

The trade this actually decides

Here is why the "1F1B shrinks the bubble" framing is worse than merely inaccurate. The bubble is (P-1)/M, so the lever you reach for is more microbatches. Under GPipe, every microbatch you add to shrink the bubble also adds a set of activations to the peak. The two goals fight.

1F1B decouples them. Its peak is set by the pipeline depth, not the microbatch count, so you can raise M until the bubble is negligible and the memory high-water mark does not follow. That is the reason to adopt it, and it is invisible in a diagram that draws only the boxes and not what is being held while the boxes run.

So what does shrink the bubble?

Splitting each device's work into several non-contiguous chunks of layers — the interleaved schedule in Megatron-LM — genuinely does, at the cost of more cross-device traffic. That is the next post in this series, and it gets the same treatment: simulated, measured, and animated from the simulation rather than illustrated alongside it.

Run it yourself

The scene refuses to render if its own claim stops being true. The makespans are asserted equal at import time, so an edit that made 1F1B finish earlier would raise instead of producing a convincing video of something false.

git clone https://github.com/SciMigo/straightedge
cd straightedge && python3 -m pip install -e '.[render]'
cd examples/pipeline_schedules && manim -qm scene.py PipelineSchedules

Change P and M at the top of the file and the charts, the gauges, and the numbers in the closing line all follow — they are read from the simulation, not typed into the scene.