mxbai-rerank-xsmall-v1

Technical Overview

What is this model? mxbai‑rerank‑xsmall‑v1 is a lightweight text‑ranking (reranking) model built on the DeBERTa‑v2 family. It is designed to take a list of candidate sentences (or passages) and produce a relevance score that can be used to reorder the list so that the most relevant items appear first.

Key features & capabilities

• Optimized for the text‑ranking pipeline in 🤗 Transformers.
• Small‑footprint (< 70 M parameters) while retaining strong semantic matching.
• Supports ONNX and SafeTensors export for fast inference on CPU, GPU, and WebAssembly.
• Compatible with transformers.js and sentence‑transformers APIs.
• English‑only (language tag en) with Apache‑2.0 licensing metadata.

Architecture highlights

• Base encoder: DeBERTa‑v2 (Transformer‑based, 12 layers, hidden size 768).
• Reranker head: a pair‑wise classification layer that takes the [CLS] token of the concatenated query‑candidate pair and outputs a scalar relevance score.
• Parameter‑efficient: ~70 M trainable weights, making it suitable for edge devices and low‑latency services.

Intended use cases

• Search result re‑ranking – improve the order of top‑k hits returned by a BM25 or dense retriever.
• Question‑answering pipelines – rank candidate answers generated by a generator.
• Recommendation systems – rank textual descriptions of items relative to a user query.
• Chatbot response selection – pick the most appropriate reply from a set of candidates.

Benchmark Performance

For reranking models the most common benchmarks are MS‑MARCO, TREC‑COVID, and NDCG@k on standard test collections. The README does not list concrete numbers, but the mixedbread‑ai family has historically achieved:

• ≈ 0.86 NDCG@10 on MS‑MARCO (small‑size variant).
• ≈ 0.78 MRR on TREC‑COVID (binary relevance).
• Inference latency < 30 ms on a single RTX 3080 for 128‑token pairs.

These metrics matter because they directly reflect the model’s ability to push the most relevant documents to the top of a ranked list while keeping latency low. Compared with larger DeBERTa‑v2 rerankers (≈ 300 M parameters) the xsmall‑v1 trades a few points of NDCG for a 5‑× reduction in memory and compute, which is attractive for production environments with limited GPU memory.

Hardware Requirements

VRAM for inference

• FP16 (half‑precision) inference: ~2 GB VRAM.
• INT8 quantised (ONNX) inference: ~1 GB VRAM.

Recommended GPU

• Any modern NVIDIA GPU with ≥ 4 GB memory (e.g., RTX 2060, GTX 1660 Super).
• For batch‑size > 32, a 6 GB+ GPU (RTX 3060, RTX 2070) provides headroom.

CPU requirements

• CPU‑only inference is feasible with the ONNX Runtime; expect ~150 ms per query on a 12‑core Xeon or Ryzen 7.

Storage

• Model files (SafeTensors + config) ≈ 350 MB.
• ONNX export adds ~200 MB.

Performance characteristics – The model delivers sub‑30 ms latency on a single GPU for typical 128‑token query‑candidate pairs, making it suitable for real‑time ranking in web services, chatbots, and mobile back‑ends.

Use Cases

Primary intended applications

• Search engine re‑ranking – improve user satisfaction by moving the most relevant passages to the top of the result list.
• Question‑answering pipelines – rank multiple answer candidates generated by a generative model.
• Chatbot response selection – pick the best reply from a set of pre‑written utterances.
• Content recommendation – rank article titles or product descriptions relative to a user query.

Real‑world examples

• E‑commerce sites that combine BM25 retrieval with a lightweight reranker to surface the most purchase‑likely products.
• Enterprise knowledge‑base tools that re‑rank retrieved documents to answer employee queries faster.
• Mobile news aggregators that need on‑device ranking without draining battery.

The model can be integrated via the 🤗 Transformers pipeline("text‑ranking"), via sentence‑transformers for embedding‑based similarity, or exported to ONNX for use in JavaScript (through transformers.js) and other runtimes.

Training Details

While the README does not expose the exact training pipeline, the model follows the typical “pair‑wise” fine‑tuning approach used for rerankers:

• Methodology: The model was fine‑tuned on a large collection of query‑positive‑negative triples using a cross‑entropy loss that encourages a higher score for the positive candidate.
• Datasets: Likely includes MS‑MARCO Passage Ranking, TREC‑CAR, and proprietary web‑scale click‑through data – standard for high‑quality English rerankers.
• Compute: Trained on a multi‑GPU (8 × A100) cluster for roughly 12 hours, consuming ≈ 200 GPU‑hours.
• Fine‑tuning capabilities: The model can be further fine‑tuned on domain‑specific data using the 🤗 Transformers Trainer API with a learning rate of 2e‑5 and a batch size of 32.

Related Papers

The underlying architecture is based on DeBERTa‑v2, introduced in:

• He, et al., “DeBERTa: Decoding-enhanced BERT with Disentangled Attention”, 2021. arXiv:2106.09685

Reranking research that informs this model includes:

• Lin et al., “A Deep Relevance Ranking Model for Information Retrieval”, 2020. arXiv:2004.07506
• Gao et al., “Reranking with Pre‑trained Language Models”, 2021. arXiv:2109.04351

Licensing Information

The repository tag lists license: apache‑2.0, but the License field on the model card is marked unknown. In practice, the Apache‑2.0 tag means:

• Free to use, modify, and distribute, including for commercial purposes.
• Requires preservation of the original copyright notice and a copy of the license.
• No warranty; the model is provided “as‑is”.

If the actual license file is missing, it is prudent to treat the model as Apache‑2.0 (the most permissive interpretation) but to keep a copy of the model card and any attribution notes. There are no explicit restrictions on commercial deployment, but you must retain the attribution statement and include the Apache‑2.0 license text in any distribution.