Almost every chatbot you use writes one token at a time, left to right. A diffusion language model works differently: it starts with a block of blank or masked positions and fills many of them in parallel, refining the text over a few passes. The promise is the same quality at much higher speed.
The idea got a boost on October 30, 2026, when DiffuSpace released Clover-35B-A3B-Preview as open weights under an Apache license. This guide explains how diffusion language models work in plain English, what Clover claims (and what's still unproven), and whether it's worth your attention.
QUICK ANSWER
A diffusion language model generates text by starting from masked placeholders and filling many tokens in parallel over several refinement steps, instead of predicting one token at a time. That can make it much faster. Clover-35B-A3B-Preview, released October 30, 2026 under an Apache license, is a 35-billion-parameter mixture-of-experts diffusion model with about 3 billion active parameters and a 128K context window.
Autoregressive vs diffusion language models
Models like GPT, Claude, Gemini and Llama are autoregressive: they predict the next token, add it to the text, and repeat. It works brilliantly, but it's inherently sequential. A 500-token answer needs roughly 500 steps.
Diffusion models borrowed their core idea from image generators like Stable Diffusion, which start with noise and gradually clean it up. For text, the usual approach is masking: the model sees a sequence where some or all positions are hidden, and it learns to predict the hidden tokens using context on both sides. At generation time it can reveal many tokens per step.
The 2025 research paper "Large Language Diffusion Models" (LLaDA) helped show that this could work at large scale, and the University of Hong Kong NLP group's Dream 7B (April 2025) pushed open diffusion models further. Clover's team says it grew out of the Dream project.
How block diffusion works
Pure diffusion over a whole long answer is awkward, so Clover uses block diffusion, a hybrid. According to the Clover model card:
- Text is generated in blocks (the default is 32 tokens).
- Each block starts as masked tokens. The model predicts all positions and fills in the ones it is confident about (default threshold 0.95) in parallel.
- If none pass the threshold, it fills the single most confident position, so it always makes progress.
- Once a block is resolved, it becomes part of the fixed prefix, and the next block starts.
Inside a block, tokens can "see" each other in both directions. Across blocks, the model still works left to right. You get some of the speed of parallel generation while keeping the structure that makes long, coherent answers possible.
Clover-35B: what was released
DiffuSpace announced the model on the Hugging Face blog on October 30, 2026. Key facts from the release and model card:
| Spec | Clover-35B-A3B-Preview |
|---|---|
| Type | Text-only mixture-of-experts diffusion language model |
| Parameters | 35B total, about 3B active per token |
| Base | Derived from Qwen3.6-35B-A3B |
| Context | 128K (131,072 tokens, prompt plus response) |
| Licence | Apache 2.0, open weights |
| Runs with | Hugging Face Transformers, SGLang, llama.cpp |
| Memory | About 69 GB for BF16 weights, plus inference state |
The claims (from DiffuSpace)
DiffuSpace reports 73.6 on SWE-bench Verified, 83.8 on GPQA Diamond and speeds of about 680 tokens per second on its internal agent workloads on a single NVIDIA GB300 GPU, and 515 tokens per second on an H200. It says Clover beats its similarly sized autoregressive base, Qwen3.6-35B-A3B, on several benchmarks while generating faster. These are the developer's own numbers; independent tests had not been published when we checked, so treat them as promising rather than proven.
Why speed matters (and what it doesn't fix)
Faster generation matters most for AI agents and coding tools, which make many model calls in a row. If each step returns in a fraction of the time, a multi-step task finishes much sooner and costs less compute. It also makes local, private AI more practical.
What it doesn't fix: hallucinations, safety, and hardware needs. A 69 GB model is not a laptop model in full precision, although llama.cpp support suggests quantised versions for smaller machines may follow. Compare that with Recursive Flint, which targets laptops, or open models like Mistral Large 4.
Should you try it?
- Developers and researchers: yes, if you have a large GPU or cloud access. Its Apache licence allows commercial use.
- Everyday users: not yet. There's no polished consumer app; wait for hosted demos or quantised builds.
Diffusion vs autoregressive: pros and cons
| Autoregressive (GPT-style) | Diffusion (block diffusion) | |
|---|---|---|
| How it writes | One token at a time, left to right | Many tokens per step inside a block |
| Speed | Limited by the number of tokens | Potentially much faster per answer |
| Context use | Looks only backwards | Looks both ways within a block |
| Maturity | Years of tooling, tuning and safety work | Newer; fewer tools and independent tests |
| Ecosystem | Supported by nearly every app and API | Growing: Transformers, SGLang, llama.cpp for Clover |
The honest summary: diffusion models are not a replacement for ChatGPT, Claude or Gemini today. They are a serious alternative architecture that could make future AI cheaper and faster, especially for agents and code. Watch for independent benchmarks, quantised builds that run on consumer GPUs, and hosted APIs. Those three signals will tell you when it's time to switch real work over.
Common misconceptions
- "Diffusion means images." The technique started in image generation, but here it is applied to text tokens.
- "Parallel means lower quality." Not necessarily; the model only fills positions it is confident about and refines the rest.
- "Open weights means tiny." Clover is open but still needs data-centre-class memory in full precision.
Step by step: check if your prompts fit a 128K model
Clover's 128K window covers your prompt and the answer together. Before you test any model with long documents, estimate the size with our free AI Token Counter. It runs entirely in your browser.
- Paste your prompt, document or chat history into the counter.
- Read the estimated tokens. The tool averages common rules of thumb, so expect it to be within about 10 to 15% for typical English.
- Check "Will it fit?" against the 128K tier, and leave room for the response.
- Enter a price per million tokens if you're using a hosted API, to estimate cost.
- Trim or split long inputs if you're close to the limit.
Planning a bigger budget? The AI API Cost Calculator compares monthly costs across usage levels.
Sources and further reading
- Large Language Diffusion Models (arXiv)
- Dream 7B (HKU NLP Group)
- Diffuspace/Clover-35B-A3B-Preview (Hugging Face)
- Clover-35B-A3B-Preview: A New Way for Intelligence (Hugging Face)
FAQ
What is a diffusion language model in simple terms?
It's an AI text model that starts with masked placeholders and fills in many words in parallel over a few refinement steps, instead of writing one token at a time.
Are diffusion language models faster than ChatGPT-style models?
They can be, because they generate many tokens per step. DiffuSpace reports large speedups for Clover, but those are the developer's own measurements.
Is Clover-35B free to use?
Yes. The weights are open under an Apache license on Hugging Face and GitHub. You need your own hardware or cloud GPU to run it.
Can I run Clover-35B on my laptop?
Not in full precision. The BF16 weights need about 69 GB of GPU memory. llama.cpp support may make smaller quantised versions possible.
Who made Clover-35B?
DiffuSpace, a team that grew out of the University of Hong Kong NLP group behind the Dream 7B diffusion model.
Comments
Post a Comment