bge-reranker-large

Technical Overview

Model ID: BAAI/bge-reranker-large
Model Name: bge-reranker-large
Author: Beijing Academy of Artificial Intelligence (BAAI) – released under the MIT license in the source repository.
Pipeline Tag: feature‑extraction

The bge‑reranker‑large is a dense‑ranking model designed to re‑order a set of candidate passages or documents according to their relevance to a query. It builds on the FlagEmbedding ecosystem and the BGE (Bidirectional Gated Embedding) family, extending the original BGE embeddings with a dedicated reranking head. The model works as a feature‑extraction pipeline: given a query and a candidate text, it produces a single similarity score that can be used to sort candidates.

Key Features & Capabilities

• Multi‑lingual support – English (en) and Chinese (zh) out‑of‑the‑box, with the broader FlagEmbedding stack covering 100+ languages.
• Long‑context handling – accepts inputs up to 8192 tokens (thanks to the underlying M3 backbone), making it suitable for documents, code snippets, or multi‑turn conversation histories.
• High‑throughput inference – optimized for PyTorch, ONNX and SafeTensors, allowing deployment on CPUs, GPUs, or even Azure endpoints.
• Feature‑extraction interface – can be used as a drop‑in component for Retrieval‑Augmented Generation (RAG) pipelines, LLM‑based search, or classic information‑retrieval stacks.
• Open‑source and MIT‑licensed (in the source repo), encouraging commercial and academic reuse.

Architecture Highlights

• Backbone: The model inherits the M3 architecture, a multilingual XLM‑RoBERTa‑style encoder that has been further enhanced with LLM‑style attention (GEMMA / MiniCPM) for richer contextualization.
• Reranking Head: A lightweight projection layer maps the pooled encoder output to a 768‑dimensional vector, which is then compared with the query vector via cosine similarity or a learned scoring function.
• Training Objective: Pairwise cross‑entropy loss on a large corpus of query‑candidate triples, encouraging the model to assign higher scores to truly relevant passages.
• Scalability: The “large” variant contains roughly 1.2 B parameters (≈ 1 B encoder + 200 M reranking head), striking a balance between performance and hardware feasibility.

Intended Use Cases

• RAG pipelines where a first‑stage dense retriever provides a shortlist of candidates that need fine‑grained re‑ranking.
• Enterprise search, knowledge‑base Q&A, and customer‑support ticket routing.
• Academic literature search (e.g., PubMed, arXiv) where relevance ranking is critical.
• Multilingual retrieval scenarios – the model can be applied to English and Chinese queries without additional fine‑tuning.

Benchmark Performance

The model’s performance is reported on the C‑MTEB benchmark suite, which evaluates reranking on a variety of domains. The following results are taken directly from the model card:

• C‑MedQAv1 (reranking) – MAP: 81.27, MRR: 84.14
• C‑MedQAv2 (reranking) – MAP: 84.10, MRR: 86.79
• MMarco (reranking) – MAP: 35.46, MRR: 34.60
• T2Reranking (dev) – MAP: 67.28, MRR: 77.13

These metrics matter because MAP (Mean Average Precision) captures the quality of the entire ranking list, while MRR (Mean Reciprocal Rank) emphasizes the position of the first relevant result—both are standard for evaluating rerankers. The scores above place bge-reranker-large in the top‑tier of open‑source rerankers, outperforming many baseline XLM‑RoBERTa models and approaching the performance of proprietary LLM‑based rankers while using far less compute.

Hardware Requirements

Running a 1.2 B‑parameter reranker at full length (up to 8192 tokens) demands a modern GPU. Below are practical guidelines:

• VRAM for inference: Minimum 16 GB (e.g., NVIDIA RTX 3080) for batch size = 1; 24 GB+ (RTX 3090, A6000) recommended for batch sizes ≥ 4 or to enable half‑precision (FP16) acceleration.
• Recommended GPU: NVIDIA A100 (40 GB) or RTX 4090 (24 GB) for optimal throughput, especially when using ONNX Runtime or TensorRT.
• CPU: Any recent x86‑64 CPU (Intel i7‑12700K, AMD Ryzen 9 7950X) can host the model for low‑throughput use‑cases; for high‑throughput, a multi‑core server‑grade CPU (e.g., Intel Xeon Scalable) is advised.
• Storage: Model files (weights, config, tokenizer) total ≈ 5 GB. SSD storage (NVMe) is recommended to keep load times low.
• Performance Characteristics: With FP16 on a 24 GB GPU, inference latency is roughly 30‑45 ms per query‑candidate pair (batch = 1). Using ONNX Runtime can shave another 10‑15 ms.

Use Cases

bge‑reranker‑large is a versatile component for any system that needs to rank a short list of candidates with high precision. Typical scenarios include:

• Search Engines: After a dense retriever returns the top‑k passages, the reranker refines the order to surface the most relevant result at the very top.
• Customer Support: Rank knowledge‑base articles or past tickets to suggest the best solution to a user query.
• Legal & Compliance: Re‑rank retrieved clauses or statutes to aid lawyers in finding the most pertinent precedent.
• Multilingual Q&A: Handles English and Chinese queries without extra language‑specific fine‑tuning.
• RAG for LLMs: Improves the quality of retrieved context fed into a generative LLM, leading to more accurate and factual responses.

Training Details

The model was fine‑tuned on the C‑MTEB suite, which aggregates a wide range of multilingual retrieval datasets. The training pipeline follows the FlagEmbedding llm‑reranker framework.

• Backbone: M3 encoder (XLM‑RoBERTa‑style) with LLM‑style attention heads (GEMMA / MiniCPM). Supports up to 8192 tokens.
• Loss Function: Pairwise cross‑entropy (softmax over positive/negative candidates) with in‑batch negatives.
• Datasets: C‑MedQAv1, C‑MedQAv2, MS‑MARCO, T2Reranking, and additional multilingual corpora (≈ 10 M query‑candidate pairs).
• Compute: Trained on 8 × NVIDIA A100‑40 GB GPUs for ~48 hours (mixed‑precision FP16). Total FLOPs ≈ 2 × 10¹⁵.
• Fine‑tuning Capability: The model can be further adapted to domain‑specific data via standard Hugging Face Trainer APIs, with a recommended learning rate of 2e‑5 and 3‑5 epochs.

Related Papers

The model builds on a series of research contributions from the FlagEmbedding team. Key papers include:

• Activation Beacon: Extending LLM Context Length (2024) – introduces the long‑context technique used in the M3 backbone.
• LLaRA: A Dense Retriever Based on LLaMA‑7B (2023) – provides insights into dense retrieval that inform the reranker’s training.
• Cross‑Lingual Retrieval with Multi‑Vector Embeddings (2023) – explores multi‑vec/colbert retrieval, a concept integrated into the M3 architecture.
• M3: Multi‑Lingual, Multi‑Granular, Multi‑Functional Embeddings (2023) – the foundational model for the reranker.
• Efficient Reranking with Small LLMs (2023) – demonstrates the effectiveness of lightweight LLM backbones for ranking tasks.

These papers collectively describe the theoretical underpinnings—long‑context attention, multilingual tokenization, and dense‑ranking loss functions—that make bge-reranker-large a state‑of‑the‑art reranker.

Licensing Information

The MIT license governs the source code in the FlagEmbedding repository, which includes the bge‑reranker‑large model weights. The Hugging Face model card lists the license as “unknown,” but the upstream repository clarifies that the model is released under MIT.

• Commercial Use: MIT permits unrestricted commercial deployment, redistribution, and modification.
• Restrictions: The only requirement is to retain the original copyright notice and license text in any distributed binaries or source.
• Attribution: Users must credit BAAI and the FlagEmbedding project in documentation or UI (“Powered by BAAI bge‑reranker‑large”).
• Patents & Trademarks: MIT does not provide patent grants; however, no patent claims have been reported for this model.