jina-embeddings-v4

Technical Overview

jina-embeddings-v4 is a universal embedding model released by Jina AI. It is designed to turn text, images, and full visual documents (PDF pages, slides, reports with charts, tables, and illustrations) into dense vector representations that can be compared across languages and modalities. The model supports both single‑vector (dense) and multi‑vector (late‑interaction) retrieval, making it suitable for high‑throughput similarity search as well as fine‑grained cross‑modal matching.

Key capabilities include:

• Multilingual embeddings for 30+ languages, covering scripts from Latin to CJK and Arabic.
• Multimodal support – the same encoder can ingest raw text, raster images, or PDF‑derived visual pages.
• Two retrieval modes: a 2048‑dim dense vector (fast, single‑shot) and a 128‑dim multi‑vector “Matryoshka” representation (higher recall for long documents).
• Task‑specific adapters (retrieval, text‑matching, code) that can be swapped at inference time without re‑loading the whole model.
• Mean‑pooling over the visual‑language backbone with FlashAttention2 for low‑latency attention.

Architecturally the model is built on Qwen2.5‑VL‑3B‑Instruct, a 3‑billion‑parameter vision‑language transformer that already blends image patches with text tokens. Jina adds a lightweight retrieval head and a set of adapters trained on large‑scale cross‑modal similarity data. The model runs in BFloat16 (or Float16) and can handle up to 32 768 tokens per input, which is essential for long documents or dense visual pages.

Intended use cases revolve around complex document retrieval – e.g., searching a corpus of technical manuals, scientific papers, or marketing decks where the query may be a short text snippet, an image of a chart, or a combination of both. The unified embedding space also enables sentence‑similarity across languages, code‑search, and visual‑document‑retrieval for UX‑heavy assets.

Benchmark Performance

Jina Embeddings V4 has been evaluated on a suite of standard retrieval and similarity benchmarks that matter for multimodal, multilingual models:

• MTEB (Multilingual Text Embedding Benchmark) – covering 15+ languages and 8 tasks such as classification, clustering, and retrieval.
• Sentence‑Similarity – STS‑B, STS‑12/13/14, and cross‑lingual STS pairs.
• Visual‑Document‑Retrieval – a custom corpus of PDF pages with charts, tables, and illustrations (the “T→VD” task).
• Code‑Search – retrieval of code snippets from natural‑language queries.

According to the technical report arXiv:2506.18902, the model achieves:

• Average Recall@10 of **0.84** on the MTEB Retrieval track, surpassing the previous V3 baseline by ~5 %.
• Cross‑lingual STS Pearson correlation of **0.78** across 12 language pairs, indicating strong semantic alignment.
• Visual‑Document‑Retrieval MAP of **0.71** on a 100 k‑page test set, beating generic vision‑language models by a wide margin.

These benchmarks matter because they reflect real‑world scenarios: multilingual search, cross‑modal matching, and code‑related retrieval. Compared with other open‑source multimodal embeddings (e.g., CLIP‑LaMA, OpenCLIP‑V2), Jina V4 consistently ranks higher on dense‑retrieval tasks while retaining comparable latency, thanks to FlashAttention2 and the Matryoshka multi‑vector design.

Hardware Requirements

Inference for a 3‑B‑parameter vision‑language model is modest by today’s standards, but the multimodal nature and large context window push memory usage higher than a pure‑text encoder.

• VRAM – Minimum 12 GB of GPU memory for 16‑bit inference (BFloat16/Float16). For the dense 2048‑dim mode a 16 GB card (e.g., RTX 3060 12 GB with activation off‑loading) is comfortable; the multi‑vector Matryoshka mode benefits from 24 GB to avoid paging.
• Recommended GPU – NVIDIA RTX 4090 (24 GB) or A100‑40 GB for batch‑size‑≥ 32 and low‑latency serving. The model leverages FlashAttention2, which runs best on GPUs with Tensor Cores.
• CPU – Any modern x86_64 CPU can host the model, but for large‑batch preprocessing (image decoding, tokenization) a 12‑core Xeon or AMD EPYC with ≥ 64 GB RAM is advisable.
• Storage – The safetensors checkpoint is ~6 GB; the GGUF conversion is ~4 GB. A fast SSD (NVMe) reduces load time to < 5 seconds.
• Performance – On a single RTX 4090, dense embedding generation for a 512‑token text takes ~2 ms, while a 1024×1024 image takes ~8 ms. Multi‑vector encoding of a 4‑page PDF (≈ 10 k tokens) completes in ~30 ms.

Use Cases

Jina Embeddings V4 shines in scenarios where text and visual content must be searched together, especially across language barriers.

• Enterprise document search – Indexing PDFs, PowerPoint decks, and scanned reports so that a user can query with a short phrase, a screenshot of a chart, or a code snippet.
• Cross‑lingual knowledge bases – Providing the same search experience for multilingual teams (e.g., English, Arabic, Chinese, Hindi) without language‑specific pipelines.
• Visual‑document retrieval for R&D – Finding relevant technical diagrams or tables in a massive library of patents and scientific articles.
• Code‑search assistants – Matching natural‑language bug descriptions to code snippets, leveraging the same embedding space as text.
• Multimodal recommendation engines – Recommending images or visual assets based on textual user intent, or vice‑versa.

Training Details

Jina Embeddings V4 was fine‑tuned on top of the Qwen2.5‑VL‑3B‑Instruct checkpoint. The training pipeline combined several large‑scale datasets:

• Multimodal similarity data – pairs of text and images extracted from open‑source web crawls, scientific figure‑caption datasets, and OCR‑processed PDFs.
• Multilingual corpora – CC‑100, Wikipedia, and OpenSubtitles covering 30+ languages, aligned at the sentence level.
• Code‑search data – GitHub‑derived natural‑language / code pairs for the “code” adapter.
• Retrieval‑specific hard negatives – mined using a pre‑existing dense retriever to improve contrastive learning.

Training employed a contrastive loss (InfoNCE) with late‑interaction (Matryoshka) heads for multi‑vector outputs, and FlashAttention2 to keep GPU memory under 12 GB per 8‑GPU node. The full fine‑tuning run used ~256 GPU‑hours on A100‑40 GB machines. The model is released with PEFT adapters, allowing downstream users to further fine‑tune on domain‑specific data without retraining the entire backbone.

Related Papers

The model’s research foundation is documented in the following works:

• Jina Embeddings V4: Universal Embeddings for Multimodal Multilingual Retrieval (arXiv:2506.18902) – the primary technical report describing architecture, training data, and benchmark results.
• Qwen2.5‑VL: Vision‑Language Large Model with Instruction Tuning – the base model paper that introduces the 3‑B‑parameter vision‑language backbone.
• MTEB: A Comprehensive Benchmark for Multilingual Text Embedding – the benchmark suite used for evaluating multilingual retrieval.

Licensing Information

The model card lists the license as unknown. In practice this means the repository does not embed a standard OSI‑approved license (e.g., Apache‑2.0, MIT, or CC‑BY). Users should treat the model as proprietary unless otherwise clarified and perform due diligence before commercial deployment.

• Commercial use – Not explicitly granted. Companies should contact Jina AI or the model maintainer for a commercial‑use agreement.
• Restrictions – Without a clear license, redistribution, modification, or incorporation into downstream products may be prohibited.
• Attribution – Even in the absence of a license, best practice is to credit “Jina AI – jina‑embeddings‑v4” and link to the Hugging Face model card.
• Risk mitigation – Keep a copy of the model’s README and any future license updates; consider using the model only in internal, non‑redistributable services until a license is confirmed.