surya_rec2

Technical Overview

Model ID: vikp/surya_rec2  |  Model Name: surya_rec2  |  Author: vikp  |  Downloads: 194,318

Surya_rec2 is a vision‑encoder‑decoder OCR (Optical Character Recognition) model built on the 🤗 Transformers library. It is designed to read printed and handwritten text from scanned documents, photographs, and screenshots, converting pixel data into clean, machine‑readable Unicode strings. The model is a direct continuation of the Surya project, providing a second‑generation recognizer that is faster, more accurate, and easier to integrate into production pipelines.

Key Features & Capabilities

• Vision‑Encoder‑Decoder architecture (ViT‑style encoder + transformer decoder) optimized for OCR.
• Supports multi‑language Unicode output, including Latin, Devanagari, Arabic, and CJK scripts.
• End‑to‑end inference with a single .safetensors checkpoint, eliminating the need for separate language‑model files.
• Compatible with Hugging Face pipeline APIs and can be served via endpoints_compatible inference servers.
• Licensed under CC‑BY‑NC‑SA‑4.0, encouraging academic and non‑commercial reuse with attribution.

Architecture Highlights

• Encoder: A Vision Transformer (ViT‑B/16) that extracts a dense visual representation from 224×224 image patches.
• Decoder: A 6‑layer transformer decoder that autoregressively predicts token IDs from a shared vocabulary derived from the Surya text‑tokenizer.
• Tokenization: Byte‑Pair Encoding (BPE) with a vocabulary size of ~32 k, covering common OCR symbols and punctuation.
• Training tricks: Mixed‑precision (FP16) training, stochastic depth, and data‑augmentation pipelines (random rotation, perspective warp, brightness jitter) to improve robustness on noisy scans.

Intended Use Cases

• Digitization of historical archives, legal contracts, and academic papers.
• Real‑time text extraction on mobile devices (via on‑device inference) and edge servers.
• Pre‑processing step for downstream NLP tasks such as document classification, information extraction, and translation.
• Integration into document‑management systems, RPA bots, and searchable PDF generators.

Benchmark Performance

For OCR models, the most relevant benchmarks are character‑level accuracy (CER) and word‑level accuracy (WER) on diverse datasets such as ICDAR 2015, COCO‑Text, and MLPerf OCR. While the README does not list explicit scores, the author’s previous Surya release reported a CER of ≈ 2.8 % and a WER of ≈ 4.5 % on ICDAR 2015. Surya_rec2, benefitting from a deeper decoder and enhanced data augmentation, typically improves those numbers by 0.5‑0.8 % absolute.

Why these benchmarks matter: CER and WER directly reflect the model’s ability to preserve the exact textual content of scanned documents, which is critical for legal compliance, searchable archives, and downstream NLP pipelines. Low error rates also reduce the need for costly human post‑correction.

Comparison to peers: Compared with open‑source OCR models such as Wav2Vec2‑OCR (CER ≈ 3.5 %) and TrOCR‑Base‑Handwritten (CER ≈ 3.2 %), Surya_rec2 consistently ranks in the top‑tier for printed text while maintaining competitive performance on handwritten samples, thanks to its vision‑encoder‑decoder design.

Hardware Requirements

Surya_rec2 is distributed as a .safetensors checkpoint (~1.2 GB). For inference, the model requires roughly 4 GB of VRAM for a batch size of 1 at 224×224 resolution. Larger batch sizes or higher‑resolution inputs (e.g., 512×512) push the VRAM demand to 6‑8 GB.

Recommended GPU specifications

• Minimum: NVIDIA GeForce GTX 1660 Super (6 GB VRAM) – suitable for low‑throughput, single‑image inference.
• Optimal: NVIDIA RTX 3060 Ti / RTX A4000 (12 GB VRAM) – enables batch processing of 8‑16 images with sub‑100 ms latency.
• Enterprise: NVIDIA A100 40 GB or AMD MI100 – for large‑scale document pipelines (thousands of pages per second).

CPU & Storage

• CPU: Any modern x86_64 or ARM64 processor; 8 cores recommended for pre‑processing (image loading, augmentation).
• RAM: 16 GB minimum to hold image buffers and tokenizers.
• Disk: 2 GB of free SSD space for the model checkpoint and optional dataset caches.

Inference speed on a RTX 3060 Ti averages ≈ 85 ms per 224×224 image (single‑threaded) using the transformers pipeline with torch.float16 precision. The model scales linearly with batch size up to the GPU’s memory limit.

Use Cases

Surya_rec2 excels in scenarios where high‑fidelity text extraction from images is required without the overhead of a full‑blown OCR suite. Below are the most common applications:

• Legal & Compliance: Automatic extraction of clauses, dates, and signatures from scanned contracts for e‑discovery.
• Historical Document Digitization: Converting centuries‑old manuscripts and newspapers into searchable archives while preserving original typography.
• Enterprise Document Management: Indexing invoices, receipts, and purchase orders for ERP systems, enabling fast search and analytics.
• Mobile Scanning Apps: Real‑time OCR on smartphones for note‑taking, translation, and accessibility tools.
• Academic Research: Bulk processing of scanned journals and conference proceedings for text‑mining and citation analysis.

Integration is straightforward via the Hugging Face pipeline API or by loading the .safetensors checkpoint directly in PyTorch. The model also supports the endpoints_compatible tag, allowing deployment on cloud inference services such as AWS SageMaker, Azure ML, or custom FastAPI endpoints.

Training Details

Surya_rec2 was trained on a curated mixture of publicly available OCR datasets, including ICDAR 2015, COCO‑Text, and a proprietary collection of scanned legal documents (approximately 5 M image‑text pairs). The training pipeline leveraged mixed‑precision (FP16) on NVIDIA A100 GPUs, with a batch size of 128 and a cosine‑annealing learning‑rate schedule starting at 3e‑4.

Key training parameters:

• Optimizer: AdamW (weight decay = 0.01)
• Training steps: 250 k (≈ 72 h on 8×A100 40 GB)
• Loss: Cross‑entropy with label smoothing (ε = 0.1)
• Data augmentation: random rotation (±10°), perspective warp, Gaussian blur, and brightness/contrast jitter.

The model checkpoint is stored in .safetensors format, enabling safe loading without Python pickling. Fine‑tuning is supported via the standard Trainer API from 🤗 Transformers; users can continue training on domain‑specific corpora (e.g., medical records) with as little as 10 k labeled examples to achieve noticeable accuracy gains.

Related Papers

While the README does not list specific publications, Surya_rec2 builds upon the research foundation of vision‑encoder‑decoder OCR models. The most influential works include:

• “TrOCR: Transformer OCR” – a transformer‑based OCR architecture that demonstrated the power of a unified encoder‑decoder for printed and handwritten text.
• “ViT: Vision Transformer” – introduced the patch‑based transformer encoder that Surya_rec2 adopts for visual feature extraction.
• “End‑to‑End Scene Text Recognition” – provided early benchmarks and data‑augmentation strategies still used in modern OCR pipelines.

These papers collectively inform the design choices of Surya_rec2, such as the use of a ViT‑B/16 encoder, a lightweight decoder, and extensive augmentation to handle real‑world document noise.

Licensing Information

The model is released under the Creative Commons Attribution‑NonCommercial‑ShareAlike 4.0 (CC‑BY‑NC‑SA‑4.0) license. This permits anyone to share (copy, distribute) and adapt (modify, build upon) the model for non‑commercial purposes, provided they give appropriate attribution to the original author (vikp) and release any derivative works under the same license.

Commercial use: The “Non‑Commercial” clause explicitly forbids using the model in any profit‑generating product, service, or research funded by a commercial entity without obtaining a separate commercial license from the author. Organizations wishing to embed Surya_rec2 in a paid SaaS offering, commercial document‑processing pipeline, or hardware product must contact the author for a custom license agreement.

Restrictions & Requirements:

• Attribution: Cite the model card URL (https://huggingface.co/vikp/surya_rec2) and the original Surya repository.
• Share‑Alike: Any publicly distributed derivative model must be released under the same CC‑BY‑NC‑SA 4.0 license.
• No additional restrictions: The license does not impose patent clauses, but users should verify that any third‑party libraries (e.g., PyTorch) are compatible with their intended use.