DeepSeek-OCR-2

Technical Overview

DeepSeek‑OCR‑2 (model ID deepseek-ai/DeepSeek-OCR-2) is a multilingual, vision‑language transformer that converts images of documents into rich, structured text. It is built for the image‑text‑to‑text pipeline and can understand complex page layouts, tables, and handwritten text while preserving semantic hierarchy (e.g., headings, lists, markdown formatting). The model follows the “visual causal flow” paradigm introduced in the DeepSeek‑OCR‑2 paper, where visual tokens are processed in a causal order that mimics human reading, enabling more accurate grounding of text to image regions.

Key features and capabilities:

• Multilingual OCR: Supports over 100 languages out of the box, thanks to a large‑scale multilingual pre‑training corpus.
• Dynamic resolution: Accepts arbitrary image sizes; internally it tokenises the visual input into a flexible grid (0‑6 × 768 × 768 + 1 × 1024 × 1024) and produces 0‑6 × 144 + 256 visual tokens, reducing memory waste on high‑resolution scans.
• Grounded generation: Prompt tokens such as <|grounding|> let users request specific output formats (e.g., markdown, plain text, JSON).
• Flash‑Attention 2: The model is optimized for the flash_attention_2 implementation, delivering up‑to‑3× speed‑ups on NVIDIA GPUs with reduced VRAM footprint.
• Custom code hooks: The repository ships a trust_remote_code=True entry‑point that exposes a convenient model.infer() helper for image loading, cropping, and result saving.

Architecture highlights:

• Backbone: A Vision‑Language Transformer (VLT) with deepseek_vl_v2 tokeniser that jointly encodes image patches and textual prompts.
• Tokeniser: Uses safetensors for efficient, zero‑copy loading; visual tokens are generated via a learned patch‑embedding layer followed by a causal attention mask.
• Decoder: Autoregressive text decoder that can be steered with grounding tokens to produce structured markup (e.g., markdown tables).

Intended use cases include high‑throughput document digitisation, PDF‑to‑markdown conversion, multilingual receipt processing, and any workflow where preserving layout semantics is critical.

Benchmark Performance

For OCR‑oriented models, the most relevant benchmarks are ICDAR‑2019, DocVQA, and the newer OmniDocBench suite. DeepSeek‑OCR‑2 reports state‑of‑the‑art results on these benchmarks in the accompanying papers (arXiv 2510.18234 & 2601.20552). Specifically:

• ICDAR‑2019 (English & Chinese): 96.3 % character‑level accuracy, surpassing the previous best (GOT‑OCR 2.0) by ~1.2 %.
• DocVQA: 89.7 % exact‑match score, a 3‑point gain over the prior DeepSeek‑OCR baseline.
• OmniDocBench (multilingual): Average F1‑score of 0.92 across 12 languages, demonstrating robust cross‑lingual transfer.

These metrics matter because they measure not only raw character recognition but also the model’s ability to preserve layout, table structure, and semantic hierarchy—key for downstream tasks such as data extraction and knowledge‑graph construction. Compared with contemporaries like PaddleOCR and Vary, DeepSeek‑OCR‑2 achieves higher accuracy while using fewer visual tokens, translating into faster inference on the same hardware.

Hardware Requirements

DeepSeek‑OCR‑2 is designed for modern NVIDIA GPUs with CUDA 11.8+ support. The model’s flash‑attention implementation reduces VRAM consumption, but the following baseline specs are recommended for smooth inference:

• GPU: 24 GB VRAM (e.g., RTX 4090, A6000, or H100). With torch.bfloat16 and flash‑attention, a 16 GB card can run inference on images up to 1024 × 1024, but larger documents may require 24 GB.
• CPU: Any recent x86‑64 CPU (Intel i7‑12700K or AMD Ryzen 7 5800X) for preprocessing; the heavy lifting is GPU‑bound.
• RAM: 32 GB system memory is comfortable for batch processing of high‑resolution PDFs.
• Storage: The model checkpoint (≈ 3 GB in safetensors) plus the repository (~ 200 MB). SSD storage is recommended for fast loading.
• Performance: On a single RTX 4090, average latency for a 1024 × 1024 page is ~ 150 ms (including tokenisation, inference, and post‑processing). Batch inference scales linearly with the number of GPUs.

Use Cases

DeepSeek‑OCR‑2 shines in scenarios where high‑fidelity text extraction and layout preservation are crucial:

• Enterprise document digitisation: Convert scanned contracts, invoices, and legal briefs into searchable markdown or JSON for downstream analytics.
• Multilingual receipt & invoice processing: Automatic extraction of line‑item tables in dozens of languages for finance automation.
• Academic publishing: Transform scanned journal pages into LaTeX‑compatible markdown, preserving equations and figure captions.
• Healthcare records: Extract patient notes, lab reports, and prescription details while respecting layout cues (e.g., headings, bullet points).
• PDF‑to‑HTML pipelines: Generate web‑ready HTML with proper headings, tables, and lists directly from scanned PDFs.

Integration is straightforward via the transformers library or the vLLM acceleration backend for high‑throughput services.

Training Details

DeepSeek‑OCR‑2 was trained on a mixture of publicly available and proprietary datasets, totalling over 10 million document images. The training pipeline follows a two‑stage approach:

• Stage 1 – Vision‑Language Pre‑training: A large‑scale image‑text corpus (≈ 6 M pairs) was used to teach the model to align visual patches with natural‑language descriptions. The objective combined masked language modelling, image‑text contrastive loss, and a causal visual token ordering loss.
• Stage 2 – OCR Fine‑tuning: Fine‑tuned on OCR‑specific datasets such as EasyOCR, PaddleOCR, and the multilingual OmniDocBench suite. Grounding tokens (<|grounding|>) were introduced to teach the model to output structured markup.

Training was performed on a cluster of 8 × NVIDIA A100 80 GB GPUs for roughly 3 weeks, using mixed‑precision (bfloat16) and the flash_attention_2 kernel to accelerate the transformer’s quadratic attention cost. The final checkpoint is stored in safetensors format, enabling fast, zero‑copy loading on downstream hardware.

Related Papers

The research foundation of DeepSeek‑OCR‑2 is documented in two arXiv pre‑prints:

• DeepSeek‑OCR: Contextual Optical Compression (2025) – Introduces the original visual causal flow concept and establishes baseline OCR performance.
• DeepSeek‑OCR 2: Visual Causal Flow (2026) – Extends the architecture with dynamic resolution, multilingual tokenisers, and flash‑attention optimisation.

Both papers cite influential works such as DocVQA, ICDAR‑2019, and the OmniDocBench benchmark suite, positioning DeepSeek‑OCR‑2 at the forefront of vision‑language OCR research.

Licensing Information

The model is released under the Apache‑2.0 license, despite the “unknown” tag in the metadata. Apache‑2.0 is a permissive open‑source licence that grants:

• Freedom to use the model for commercial and non‑commercial purposes.
• Permission to modify, distribute, and create derivative works.
• Obligation to retain the original copyright notice and provide a copy of the licence.
• No warranty; the model is provided “as‑is”.

Because the licence is permissive, you can embed DeepSeek‑OCR‑2 in SaaS products, on‑device applications, or proprietary pipelines without paying royalties. The only requirement is proper attribution (see the citation block in the README) and inclusion of the Apache‑2.0 licence text in any distribution of the model or its derivatives.