MinerU-Diffusion Kills Autoregressive OCR: 3.2x Faster Document Parsing Revealed

What if the biggest bottleneck in document AI wasn't model size—but the way your model reads?

Every developer who's battled production OCR pipelines knows the nightmare: autoregressive decoding crawling left-to-right, token by token, while your GPU sits embarrassingly underutilized. Page layouts explode context windows. Formulas collapse into gibberish. Tables? Don't even ask. You've tried every trick—batching, speculative decoding, KV-cache gymnastics—but the fundamental problem remains. Autoregressive generation is inherently sequential, and sequential means slow.

Now imagine ripping out that bottleneck entirely. No more token-by-token chains. No more left-to-right prison. What if your OCR model could reconstruct document content like an artist refining a sketch—starting rough, then progressively sharpening details in parallel across entire blocks?

That's exactly what MinerU-Diffusion delivers. Born from OpenDataLab's relentless pursuit of efficient document intelligence, this 2.5B parameter framework reframes document OCR as inverse rendering via diffusion decoding—and the results are nothing short of explosive. Up to 3.26x throughput speedup. 99.9% relative accuracy at 2.12x faster inference. A fundamentally different paradigm that top ML engineers are already racing to productionize.

Ready to discover why autoregressive OCR is living on borrowed time? Let's decode the secret.

What is MinerU-Diffusion?

MinerU-Diffusion is a diffusion-based framework for document OCR developed by OpenDataLab, the same research collective behind the wildly popular MinerU document extraction ecosystem. Released in March 2026 (V1-0320), this 2.5B parameter vision-language model represents a radical architectural departure from conventional approaches.

Instead of generating text tokens sequentially like GPT-style autoregressive models, MinerU-Diffusion treats document understanding as inverse rendering—reconstructing structured content from visual observations through iterative denoising. The model learns to predict masked token positions in parallel, conditioned on document images, progressively refining its outputs across multiple diffusion steps.

The core innovation? Block-level parallel diffusion decoding. Rather than maintaining strict left-to-right causality, MinerU-Diffusion organizes tokens into blocks that can be refined bidirectionally within each block, while preserving coarse autoregressive structure across blocks. This hybrid approach shatters the sequential barrier without sacrificing structural coherence.

MinerU-Diffusion is trending now because it solves the exact pain point plaguing production document AI: the accuracy-speed tradeoff doesn't have to exist. Traditional methods force you to choose between fast but brittle heuristic parsers, or accurate but glacially slow autoregressive VLMs. By leveraging diffusion's inherent parallelizability, MinerU-Diffusion occupies a previously impossible position in the Pareto frontier—simultaneously faster and more robust.

The model builds upon foundations from Qwen2-VL for visual encoding, incorporates architectural insights from LLaDA and Block Diffusion research, and ships with production-ready inference engines including Hugging Face Transformers, SGLang, and a custom Nano-DVLM adaptation for single-GPU deployment.

Key Features That Rewrite the Rules

🔥 Block-Wise Parallel Diffusion Decoding The headline feature transforms inference economics. By grouping tokens into blocks and applying bidirectional attention within each block, MinerU-Diffusion eliminates the O(n) serial dependency chain of autoregressive models. Blocks refine in parallel while maintaining cross-block causality—think of it as structured brainstorming rather than rigid dictation.

⚡ Uncertainty-Driven Curriculum Learning During training, the model learns with a sophisticated masking strategy that progressively increases difficulty. Easier tokens get resolved first, building confidence for harder predictions. This curriculum approach directly translates to more reliable outputs on complex documents with degraded scans, unusual fonts, or dense mathematical notation.

🎯 Flexible Accuracy-Throughput Trade-off Unlike fixed-speed models, MinerU-Diffusion exposes threshold controls that let you dial your operating point. Need maximum accuracy? Crank up denoising steps. Need blazing speed for batch processing? Reduce steps with graceful degradation. The performance curve is your choice, not the model's dictation.

🧩 Layout-Aware Structured Output Four specialized prompt types handle distinct document elements: Layout Detection (bounding boxes with rotation), Text Recognition (plain OCR), Formula Recognition (LaTeX output), and Table Recognition (OTSL format). No more post-processing nightmares trying to reconstruct structure from flat text dumps.

🚀 Multi-Engine Production Deployment Ships with three inference backends: Hugging Face for flexibility, Nano-DVLM for optimized single-GPU throughput, and SGLang for distributed serving. The nano_dvlm engine particularly shines for resource-constrained deployments, adapted from the nano-vLLM project.

📐 Variable Resolution Visual Encoding Handles native resolutions from 4 to 2048 image tokens, automatically adapting to document complexity. No forced resizing that destroys fine-grained table structures or micro-text in academic papers.

Use Cases Where MinerU-Diffusion Dominates

1. High-Volume Document Processing Pipelines

Legal firms, insurance companies, and government agencies process millions of pages monthly. Traditional autoregressive VLMs create impossible bottlenecks—either pay for GPU clusters that sit idle during serial generation, or accept 10-second-per-page latency. MinerU-Diffusion's 3x speedup transforms unit economics, enabling real-time processing on single A100s instead of multi-GPU orchestration.

2. Mathematical and Scientific Publication Parsing

Academic PDFs with dense formulas, multi-line equations, and complex tabular data expose the brittleness of conventional OCR. MinerU-Diffusion's formula recognition prompt outputs clean LaTeX, while its block-wise processing preserves spatial relationships that autoregressive models garble when context windows overflow.

3. Mobile and Edge Document Scanning

The Nano-DVLM engine enables deployment on single-GPU or even high-end consumer hardware. Scan-to-structured-data pipelines on edge devices become feasible—critical for healthcare point-of-care systems, field inspection apps, or privacy-sensitive document processing where cloud transmission is prohibited.

4. Real-Time Interactive Document Assistants

Chat-with-your-document products demand sub-second response times. MinerU-Diffusion's parallel decoding, combined with SGLang serving infrastructure, enables streaming layout analysis and content extraction that keeps pace with conversational interfaces. No more loading spinners while the model "thinks" through each token.

5. Legacy Document Digitization at Scale

Degraded scans, mixed fonts, skewed pages, and handwritten annotations break heuristic-based OCR. Diffusion's iterative refinement naturally handles uncertainty, progressively correcting misreads rather than committing irreversibly at each step like autoregressive decoders.

Step-by-Step Installation & Setup Guide

Let's get MinerU-Diffusion running locally. The setup prioritizes CUDA 12.8 compatibility for optimal performance.

Environment Creation

# Create dedicated conda environment
conda create -n dmineru python=3.12 -y
conda activate dmineru

# Upgrade pip for modern wheel handling
pip install --upgrade pip

Core Dependencies

# Install PyTorch with CUDA 12.8 support
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

# Transformers with remote code trust (required for custom architecture)
pip install "transformers>=4.52.1"

# Flash Attention 2 for memory-efficient attention (critical for long sequences)
# Download prebuilt wheel matching your CUDA/PyTorch/Python combo
wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3/flash_attn-2.8.3+cu12torch2.8cxx11abiFALSE-cp312-cp312-linux_x86_64.whl
pip install flash_attn-2.8.3+cu12torch2.8cxx11abiFALSE-cp312-cp312-linux_x86_64.whl

# Install remaining dependencies from project requirements
pip install -r requirements.txt

Critical Notes:

• flash-attn==2.8.3 must exactly match your CUDA, compiler, and PyTorch versions. If the prebuilt wheel fails, build from source or locate a compatible wheel.
• The root requirements.txt covers HF inference, Nano-DVLM engine, and SGLang client paths—but not the SGLang server binary itself. Deploy SGLang separately if needed.

Model Weights Download

# Set local model path
MODEL_PATH=/path/to/MinerU-Diffusion-V1-0320-2.5B

# Download from Hugging Face (recommended)
# Visit: https://huggingface.co/opendatalab/MinerU-Diffusion-V1-0320-2.5B
# Or use huggingface-cli: huggingface-cli download opendatalab/MinerU-Diffusion-V1-0320-2.5B

# Alternative: ModelScope mirror for China-based users
# https://modelscope.cn/models/OpenDataLab/MinerU-Diffusion-V1-0320-2.5B

Quick Verification

# Test with HF engine
ENGINE=hf \
MODEL_PATH=/path/to/MinerU-Diffusion-model \
IMAGE_PATH=/path/to/test-image.png \
bash scripts/run_inference.sh

REAL Code Examples from the Repository

Example 1: Full Transformers Inference Pipeline

This is the canonical Python implementation for running MinerU-Diffusion via Hugging Face Transformers, extracted directly from the repository documentation:

import torch
from transformers import AutoModel, AutoProcessor, AutoTokenizer

# Model identifier—use official checkpoint or local path
model_id = "Niujunbo2002/MinerU-Diffusion-V1-0320-2.5B"
image_path = "path/to/page.png"

# Load tokenizer with remote code trust for custom tokens
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Processor handles image preprocessing and prompt templating
processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
    use_fast=False,  # Disable fast tokenizer for compatibility with custom diffusion tokens
)

# Load model in bfloat16 for memory efficiency, with optimized CPU→GPU transfer
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,  # Shard loading to prevent OOM on consumer hardware
).eval().to("cuda")

# Construct multimodal conversation with system prompt and image
messages = [
    {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path},
            {"type": "text", "text": "\nText Recognition:"},  # Prompt type triggers specific output format
        ],
    },
]

# Apply chat template to generate model-ready prompt string
prompt_text = processor.apply_chat_template(messages, add_generation_prompt=True)
if isinstance(prompt_text, tuple):
    prompt_text = prompt_text[0]  # Unpack if processor returns tuple

# Process inputs: image tokenization + text encoding with truncation
inputs = processor(
    images=[image_path],
    text=prompt_text,
    truncation=True,
    max_length=4096,  # Cap sequence length for memory management
    return_tensors="pt",
)

# Move tensors to GPU with appropriate dtypes
input_ids = inputs["input_ids"].to(torch.long).to("cuda")
pixel_values = inputs["pixel_values"].to(torch.bfloat16).to("cuda")

# Handle variable-resolution image grid (Qwen2-VL heritage)
image_grid_thw = inputs.get("image_grid_thw")
if image_grid_thw is not None:
    image_grid_thw = image_grid_thw.to(torch.long).to("cuda")

# Generate with diffusion-specific parameters
with torch.no_grad():
    generate_outputs = model.generate(
        pixel_values=pixel_values,
        image_grid_thw=image_grid_thw,
        input_ids=input_ids,
        mask_token_id=tokenizer.convert_tokens_to_ids("<|MASK|>"),  # Special token for diffusion
        denoising_steps=32,  # Number of diffusion iterations—trade accuracy for speed
        gen_length=1024,     # Maximum output sequence length
        block_length=32,     # Tokens per parallel block—core parallelism parameter
        temperature=1.0,     # Sampling randomness (1.0 = balanced)
        remasking_strategy="low_confidence_dynamic",  # Retokenize least confident predictions
        dynamic_threshold=0.95,  # Confidence cutoff for early token finalization
        tokenizer=tokenizer,
        stopping_criteria=["<|endoftext|>", "<|im_end|>"],  # Stop sequences
    )

# Decode output, handling both tuple and tensor returns
output_ids = generate_outputs[0] if isinstance(generate_outputs, tuple) else generate_outputs
text = tokenizer.decode(output_ids[0], skip_special_tokens=False)

# Clean stop tokens from final output
for stop in ("<|endoftext|>", "<|im_end|>"):
    text = text.split(stop, 1)[0]

print(text.strip())

Key insight: The mask_token_id, denoising_steps, block_length, and remasking_strategy parameters are diffusion-native controls with no autoregressive equivalent. The low_confidence_dynamic remasking specifically targets uncertain predictions for refinement—impossible in left-to-right generation where each token is frozen after output.

Example 2: Shell-Based HF Engine Inference

For production scripting and CI/CD integration:

cd /path/to/MinerU-Diffusion
ENGINE=hf \
MODEL_PATH=/path/to/MinerU-Diffusion-model \
IMAGE_PATH=/path/to/input-image.png \
bash scripts/run_inference.sh

This wrapper handles environment validation, engine dispatch, and output formatting. The ENGINE=hf selector routes through the engines/hf/runner.py implementation with automatic mixed precision and batching optimizations.

Example 3: Nano-DVLM Engine for Single-GPU Optimization

cd /path/to/MinerU-Diffusion
ENGINE=nano_dvlm \
MODEL_PATH=/path/to/MinerU-Diffusion-model \
IMAGE_PATH=/path/to/input-image.png \
bash scripts/run_inference.sh

The nano_dvlm engine, adapted from Nano-vLLM, implements custom CUDA kernels and memory scheduling specifically optimized for diffusion decoding patterns. Expect 15-25% additional throughput versus standard HF inference on single-GPU setups.

Example 4: SGLang Server Deployment

For distributed serving and OpenAI-compatible APIs:

# Terminal 1: Start server
cd /path/to/MinerU-Diffusion
MODEL_PATH=/path/to/MinerU-Diffusion-model \
bash scripts/run_sglang_server.sh

# Terminal 2: Send request
ENGINE=sglang \
MODEL_PATH=/path/to/MinerU-Diffusion-model \
IMAGE_PATH=/path/to/input-image.png \
SGLANG_SERVER_URL=http://127.0.0.1:31002/v1/chat/completions \
bash scripts/run_inference.sh

SGLang's RadixAttention and continuous batching dramatically improve throughput under concurrent load. The OpenAI-compatible endpoint enables drop-in replacement for existing VLM pipelines.

Example 5: End-to-End Document Parsing

The production-grade two-stage pipeline:

cd /path/to/MinerU-Diffusion
MODEL_PATH=/path/to/MinerU-Diffusion-model \
IMAGE_PATH=/path/to/input-page.png \
OUTPUT_PATH=/path/to/output.md \
BLOCKS_JSON_PATH=/path/to/output-blocks.json \
SAVE_LAYOUT_IMAGE=1 \
LAYOUT_IMAGE_PATH=/path/to/output-layout.png \
bash scripts/run_end2end.sh

This executes: (1) layout detection with bounding box extraction, (2) per-block content recognition with prompt type selection, and (3) markdown assembly with optional visualization. Environment variables like KEEP_PARATEXT=1 preserve headers/footers, while VERBOSE=1 enables debugging output.

Advanced Usage & Best Practices

Tune denoising_steps for your latency budget. The default 32 steps balances quality and speed, but production workloads can reduce to 16-20 steps for 2x faster inference with ~98% relative accuracy. Conversely, increase to 64 for maximum fidelity on critical documents.

Leverage dynamic_threshold for early stopping. Setting dynamic_threshold=0.99 finalizes high-confidence tokens earlier, reducing effective computation. Combine with remasking_strategy="low_confidence_dynamic" to focus refinement where uncertainty persists.

Batch layout detection before content extraction. The end-to-end pipeline's two-stage design isn't just organizational—it's computational. Layout detection at reduced resolution (1036×1036) is cheap, enabling intelligent crop selection that minimizes expensive full-resolution passes.

Monitor block_length for hardware alignment. The default 32 tokens per block suits A100/H100 architectures. For consumer GPUs with smaller SRAM, reduce to 16-24 to prevent memory thrashing. For H100 with large L2 cache, experiment with 64 for additional parallelism.

Use bfloat16 consistently. MinerU-Diffusion's training included bfloat16 mixed precision. Full float32 offers marginal accuracy gains with 2x memory overhead—rarely worthwhile. float16 risks overflow in diffusion's iterative refinement.

Comparison with Alternatives

Why MinerU-Diffusion wins: It occupies the accuracy-speed Pareto frontier that alternatives cannot reach. Heuristic OCR is faster but breaks on complexity. Autoregressive VLMs are accurate but sequentially bottlenecked. Only diffusion decoding achieves both.

FAQ

Q: Is MinerU-Diffusion a drop-in replacement for existing MinerU pipelines? A: Yes, with adaptation. The output formats (markdown, JSON blocks) align with MinerU ecosystem conventions. Migration primarily involves swapping inference backends and adjusting for diffusion-specific parameters like denoising_steps.

Q: What hardware minimum do I need? A: Single A100 40GB runs full inference comfortably. With Nano-DVLM engine and reduced gen_length, A10G or even RTX 4090 (24GB) handles moderate documents. CPU-only inference is not currently supported.

Q: How does block-wise diffusion preserve reading order? A: The structured block-attention mask enforces causal attention across blocks while allowing bidirectional refinement within blocks. Coarse autoregressive structure at block boundaries maintains sequence, while internal parallelism accelerates generation.

Q: Can I fine-tune on my document domain? A: Training code release is on the roadmap (V2). Currently, prompt engineering with the four specialized prompt types (Layout, Text, Formula, Table) provides substantial domain adaptation.

Q: How does this compare to speculative decoding for speedup? A: Speculative decoding accelerates autoregressive models by ~2x with draft models, but adds complexity and draft-model overhead. MinerU-Diffusion's 3.2x speedup is intrinsic to the architecture—no auxiliary models, no acceptance-rate limitations.

Q: Is commercial use permitted? A: Yes, MIT License allows commercial deployment. Note upstream dependencies (PyTorch, Transformers, SGLang) have their own license terms.

Q: What document languages are supported? A: Training emphasized multilingual academic and technical documents. Performance is strongest on English, Chinese, and major European languages. Right-to-left scripts are supported but less extensively validated.

Conclusion

Autoregressive OCR had a good run. For years, we accepted sequential token generation as an immutable law—trading patience for accuracy, throwing GPUs at a fundamentally serial problem. MinerU-Diffusion exposes that tradeoff as false.

By reframing document understanding as inverse rendering through parallel diffusion decoding, OpenDataLab has delivered something genuinely disruptive: a 2.5B parameter model that runs 3.2x faster than autoregressive equivalents while preserving 99.9% relative accuracy at practical operating points. The block-wise architecture, uncertainty-driven refinement, and multi-engine deployment options make this immediately production-viable—not a research curiosity.

If you're building document AI pipelines, maintaining legacy OCR infrastructure, or simply tired of watching your GPUs idle through token-by-token generation, the path forward is clear. The autoregressive era is ending. The diffusion era is beginning.

Star the repository, download the model, and benchmark against your current pipeline. The numbers will speak for themselves. Your users—and your cloud bill—will thank you.

👉 Get MinerU-Diffusion on GitHub

Built with ❤️ by OpenDataLab. Powered by diffusion, not patience.