Diffusion LLMs Are Fast and Interesting!

I got curious about diffusion language models after watching Lisa Li’s talk on controlling language models. Until then, I associated diffusion with images, not text, reasoning, or control. Her talk brought back a question I had stopped asking. Does a language model really need to produce text from left to right, one token at a time?

One of my research colleagues used to work on generating or imputing single-cell Hi-C contact maps using a diffusion-based approach as well. I would never have thought this would work!

This is my attempt to make sense of the alternative.

I will call them DLLMs, short for diffusion language models or diffusion large language models. The naming is still loose. You will also see diffusion-based LLMs, large language diffusion models, and masked diffusion language models. The idea is simpler than the names: an autoregressive model writes token by token. A diffusion model starts with a corrupted sequence and cleans it up in rounds.

The core shift

As of March 2026, autoregressive LLMs are still the default. Their training objective is simple, we understand their scaling behavior fairly well, and the tools around them all assume next-token prediction. When most people say “LLM,” this is what they mean.

Formally, an autoregressive language model factorizes a sequence like this:

P(x_{1:T}) = \prod_{t=1}^{T} P(x_t \mid x_{<t})

and it is usually trained with next-token cross-entropy:

\mathcal{L} = -\sum_{t=1}^{T} \log P_\theta(x_t \mid x_{<t})

At inference time, the model reads the current prefix, predicts the next token, appends it, and repeats until it stops. The recipe works absurdly well. It may be one of the most consequential simplifications in modern machine learning.

Its limits are structural, though. Token 500 cannot arrive before token 499. The model uses left context, not future context. Early mistakes can leak into everything that follows. Rewriting the middle of a sequence is possible, but clumsy. Autoregressive models are still extremely good. These are simply the places where diffusion has a real case to make.

In machine learning, “diffusion” usually means two processes. The forward process corrupts data bit by bit. The reverse process learns to recover clean data from those corrupted versions. This is intuitive for images because pixels occupy a continuous space and Gaussian noise makes sense. Text is more annoying. Tokens are discrete symbols, not coordinates. You cannot add 0.3 units of “banana” to a word and call the result noisy.

That mismatch is why text diffusion took longer to become convincing than image diffusion. Language is fragile. One changed token can flip a meaning, break syntax, or ruin a program. Order matters locally for grammar and across the whole sequence for coherence and long-range dependencies. A text diffusion model has to recover text without wrecking any of it.

How text diffusion works

Most DLLMs take one of two routes. With continuous diffusion, the model maps text into a continuous representation, often an embedding space, runs diffusion there, then maps it back to tokens. This was the basic approach behind work such as Diffusion-LM. It keeps some of the mathematical convenience of image diffusion and gives researchers useful ways to steer output. But getting back to discrete text is hard. Early work in this family was often more useful for experiments than for ordinary language generation.

The other route is discrete diffusion, which corrupts tokens directly. A common version uses an absorbing state such as [MASK]. The forward process replaces clean tokens with [MASK]. The reverse process predicts the missing tokens and gradually unmasks the sequence. It naturally connects to masked language modeling, denoising, infilling, and editing. That is a big reason the approach has drawn so much attention.

A useful mental model:

Autoregressive LLM: writes the answer from left to right.
Diffusion LLM: starts from a rough, corrupted draft and repeatedly revises the whole thing.

This sounds like a small difference. It is not. A diffusion model can update many positions at once, so it behaves more like an editor revising a draft than someone typing into an empty document.

Suppose the final sentence is:

diffusion models may change how we generate code

A masked diffusion process might begin here:

[MASK] [MASK] [MASK] [MASK] [MASK] [MASK] [MASK] [MASK]

Then it could fill easier positions first:

diffusion [MASK] may [MASK] how we [MASK] code

Then refine further:

diffusion models may change how we generate code

Some methods stop there. Others revisit uncertain positions for a few more steps. The details vary, but the important part does not: the model can predict many tokens in parallel during each refinement step. That is where the speed story comes from. It also creates most of the tradeoffs.

The research path is easier to see in hindsight. D3PM established a way to use diffusion in discrete state spaces. Diffusion-LM and related work showed that language diffusion was not a category error. Plaid took likelihood and scaling more seriously. SEDD improved discrete-diffusion training, and MDLM showed that masked diffusion could go farther than many people expected. When LLaDA argued that an 8B diffusion model could compete with strong autoregressive baselines on several tasks, the question changed from “can this work at all?” to “where does it work best?”

Where it looks strongest

The strongest case for DLLMs is not that they beat autoregressive models everywhere. They do not. Their way of generating text fits a different set of jobs.

Parallel decoding gets the headlines. If a model can revise many positions at once and get good enough in a few steps, latency can fall, particularly for long outputs. This is also where the hype gets slippery. Speed depends on output length, the number of denoising steps, hardware, batch size, and whether the job is generation, editing, or completion. Some speedups are real. Some are benchmark theater.

Editing and infilling are a more natural fit. If the job is filling a missing span, rewriting a paragraph, updating a code block, or revising a constrained sentence, masked diffusion is on home ground. The model already treats a sequence as partly known and partly in need of repair.

Bidirectional context is another practical advantage. Since denoising happens over the whole sequence, the model can use the text on both sides when deciding what belongs in one position. That matters for infilling, structured output, and work where global consistency matters more than streaming tokens.

Controllability is what made Lisa Li’s talk stick with me. Diffusion models give you more chances to intervene because the output changes over several steps instead of being committed one token at a time. Researchers, especially with continuous variants, have shown ways to steer output toward syntax, sentiment, or a desired structure. That does not make diffusion automatically controllable. It does give you more places to take hold.

This is why diffusion keeps coming up in code editing and other structured work. A model that can revise the whole output may do better when a task needs global planning instead of irreversible left-to-right commitment.

Evaluation is awkward. No single metric settles the argument. Language quality still matters, and perplexity has not gone away, but it does not compare every diffusion formulation cleanly. Results on instruction following, coding, infilling, summarization, and reasoning tasks often say more. So do latency, throughput, and the number of steps needed to reach acceptable quality. The useful tests are often not chat benchmarks. They are tasks that actually need whole-sequence revision.

Why it is still hard

Diffusion does not remove a bottleneck and set everything free. It trades one set of limits for another.

Next-token prediction is simple, general, and battle-tested. Diffusion training has fewer settled conventions. You need to care about corruption schedules, parameterization, loss functions, denoising samplers, and how the model updates tokens. That gives researchers more room to try things. It also makes the field messier.

Inference is iterative too. Diffusion avoids strict left-to-right decoding, but it still needs several denoising steps. It wins only when parallelism pays for those extra passes. Long-form coherence is still hard. Revising a whole sequence is powerful, but it does not magically keep a long answer consistent across many positions and updates.

The ecosystem is another constraint. Serving stacks, benchmark habits, UX expectations, and infrastructure tricks such as KV-cache-heavy optimization all grew around autoregressive assumptions. Diffusion models are not starting from a blank slate. They are trying to break into a world tuned for something else.

“DLLMs are the future” is too blunt a frame. We will probably get a mix:

autoregressive models for the broad default case
diffusion models for editing, infilling, controllable refinement, and some low-latency long-output workloads
hybrid systems that borrow from both

DLLMs offer a different picture of language generation. Instead of a model typing one token after another, picture one drafting and revising the text again and again. It feels more like editing and planning than autocomplete. That is compelling. It is also not yet a reason to throw out the architecture that got us here.

The core shift

How text diffusion works

Where it looks strongest

Why it is still hard

References

Foundational diffusion

Diffusion for text and categorical data

Recent DLLM progress

Products and research systems