Discrete Diffusion Reading Group

Image classification is well studied, yet it stays hard under high uncertainty, such as when inputs are corrupted or training data are limited. Standard classifiers learn to predict a class label directly from an image, which can underperform in these regimes. This work introduces Discrete Diffusion Classification Modeling (DiDiCM), a framework that uses a diffusion process to model the posterior distribution over class labels given the input image. DiDiCM can run the diffusion over class probabilities or over discrete class labels, letting you trade compute against memory. Across a broad empirical study, DiDiCM beats standard classifiers: a few diffusion iterations reach higher accuracy on ImageNet than the baselines, and the gains grow as the task gets harder. The authors release their code.

Diffusion has drawn wide attention in language modeling, yet continuous diffusion has looked less scalable than discrete approaches. This work challenges that belief. The authors revisit Plaid, a likelihood-based continuous diffusion language model, and construct RePlaid by aligning its architecture with modern discrete diffusion language models. In this unified setting they establish the first scaling law for continuous models that rivals discrete ones: RePlaid sits within a 20x compute gap of autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime. Against recent continuous models on OpenWebText, RePlaid reaches a new state-of-the-art perplexity bound of 22.1 with better generation quality. The paper also offers theory for why likelihood-based training helps. Optimizing the noise schedule to minimize the variance of the ELBO naturally yields linear cross-entropy over time, which spreads denoising difficulty evenly without any case-specific time reparameterization. Optimizing token embeddings via likelihood creates structured geometries and drives most of the likelihood gain. Together these results suggest that continuous diffusion, when trained via likelihood, is a competitive and scalable alternative to discrete diffusion.

Nemotron-Labs-Diffusion is a tri-mode language model that unifies three decoding strategies in a single architecture: autoregressive (AR), diffusion, and self-speculation. Autoregressive models decode one token at a time, which underuses hardware at low batch sizes, while diffusion models decode many tokens per forward pass but usually trail AR in accuracy and need far more data to catch up. This work trains a single model with a joint AR-diffusion objective so it can switch modes to sustain high throughput across deployment settings and concurrency levels. The study makes three points. First, the AR and diffusion objectives are complementary: diffusion improves lookahead planning, while AR provides the left-to-right priors of natural language. Second, in self-speculation mode the diffusion path drafts tokens while the AR path verifies them, beating multi-token prediction in both acceptance rate and real-device efficiency. Third, a speed-of-light analysis shows diffusion can produce up to 76.5% more tokens per forward pass than self-speculation under an optimal sampler. The released family spans 3B, 8B, and 14B parameters with base, instruct, and vision-language variants. Nemotron-Labs-Diffusion-8B decodes six times more tokens per forward than Qwen3-8B at comparable accuracy, which translates to four times higher throughput on SPEED-Bench with SGLang on a GB200 GPU.