splade-cocondenser-ensembledistil

Technical Overview

The naver/splade-cocondenser-ensembledistil model is a SPLADE Sparse Encoder built on top of the Luyu/co‑condenser‑marco BERT‑based backbone. It converts queries and passages into a 30522‑dimensional sparse vector where each dimension corresponds to a vocabulary token from the BERT tokenizer. The sparsity is achieved by a max‑pooling layer followed by a ReLU activation, which keeps only the most salient token scores and discards the rest. This representation can be indexed with traditional inverted‑index structures and scored with a simple dot‑product, enabling fast, memory‑efficient retrieval.

Key features and capabilities

• Query‑expansion and document‑expansion via learned token‑level weights.
• Bag‑of‑words style sparse vectors that are compatible with classic IR pipelines.
• Knowledge‑distillation from a strong teacher (CoCondenser) and ensemble distillation (Distil) for higher effectiveness.
• Support for both encode_query and encode_document through the sentence‑transformers API.
• Dot‑product similarity, making it a drop‑in replacement for BM25‑style scoring.

Architecture highlights

• Base encoder: BertForMaskedLM (BERT‑base) with a maximum sequence length of 512 tokens.
• Pooling: SpladePooling with max strategy and relu activation.
• Output dimensionality: 30 522 (the size of the BERT vocabulary).
• Similarity function: dot product (efficient on CPUs/GPUs and inverted indexes).

Intended use cases

• Passage‑level retrieval for open‑domain question answering.
• Semantic search over large document collections where sparse indexing is required.
• Hybrid retrieval pipelines that combine dense and sparse signals.
• Knowledge‑distilled, low‑latency inference for production search services.

Benchmark Performance

The model is evaluated on the MS MARCO passage ranking development set, a standard benchmark for neural retrieval. The reported scores are:

• MRR@10: 38.3 %
• R@1000: 98.3 %

MRR@10 (Mean Reciprocal Rank at 10) measures how early the correct passage appears in the top‑10 results, a critical metric for user‑facing search where the first few results dominate click‑through. R@1000 (Recall at 1 000) captures the model’s ability to retrieve the relevant passage somewhere in a large candidate set, which is essential for downstream re‑ranking pipelines. Compared with dense BERT‑based re‑rankers that typically achieve MRR@10 in the low‑30 % range, this SPLADE variant pushes the score above 38 % while retaining the efficiency of a sparse index. Its recall of >98 % demonstrates that the sparse representation does not sacrifice coverage, making it competitive with state‑of‑the‑art sparse models such as splade‑v3 and traditional BM25.

Hardware Requirements

Even though the output is a 30 k‑dimensional sparse vector, the underlying encoder is a standard BERT‑base model. Inference therefore follows the typical BERT memory profile.

• VRAM for inference: 2 GB–4 GB for a single query/document (batch size = 1). Larger batches benefit from 8 GB+ GPUs.
• Recommended GPU: NVIDIA Tesla T4, RTX 3060, or any GPU with ≥ 8 GB VRAM for comfortable batch processing.
• CPU requirements: A modern 8‑core CPU can run the model, but latency will increase to ~30‑50 ms per query compared to ~10 ms on a GPU.
• Storage: The model checkpoint (weights + tokenizer) occupies roughly 500 MB of disk space.
• Performance characteristics: Encoding a single query takes ~10 ms on an RTX 3060; document encoding is slightly slower (~15 ms) due to longer input length. Sparse vectors can be stored in CSR format, enabling sub‑millisecond dot‑product scoring when indexed.

Use Cases

The SPLADE CoCondenser EnsembleDistil model shines in scenarios where high‑quality sparse representations are needed without sacrificing retrieval speed.

• Open‑domain QA: Encode user questions as sparse queries and retrieve relevant passages from a large knowledge base before a downstream answer extractor.
• E‑commerce search: Map product titles and descriptions to sparse vectors, enabling fast, interpretable term‑level matching while still benefiting from neural relevance signals.
• Legal and academic literature search: Retrieve relevant statutes, case law, or research papers where precise term coverage is critical.
• Hybrid retrieval pipelines: Combine the model’s sparse scores with dense embeddings (e.g., from sentence‑transformers) to improve both recall and precision.
• Low‑latency mobile or edge inference: The model’s sparse output can be stored efficiently, allowing on‑device retrieval for privacy‑sensitive applications.

Training Details

Training follows the knowledge‑distillation + hard‑negative sampling pipeline described in the cited paper. The process can be summarized as:

• Teacher model: Luyu/co‑condenser‑marco, a dense BERT‑based ranker trained on MS MARCO.
• Student architecture: SparseEncoder with max‑pooling and ReLU, producing a 30 522‑dimensional sparse vector.
• Dataset: The MS MARCO passage ranking corpus (≈ 1 M queries, 8 M passages).
• Hard‑negative mining: For each query, top‑k negatives are sampled from the teacher’s retrieval list, then used to train the student with a contrastive loss.
• Ensemble distillation: Multiple student checkpoints are averaged (ensembled) to improve robustness, followed by a final “Distil” fine‑tune.
• Compute: Training was performed on multiple NVIDIA V100 GPUs (8‑16 GB each) for several days, using mixed‑precision (FP16) to accelerate convergence.
• Fine‑tuning: Users can further fine‑tune the model on domain‑specific data via the sentence‑transformers library, keeping the same sparse pooling head.

Related Papers

The core research behind this checkpoint is documented in the following work:

• From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective (arXiv:2205.04733) – introduces the knowledge‑distillation and hard‑negative sampling strategies that power the SPLADE‑CoCondenser family.

The SPLADE methodology itself originates from earlier work on sparse neural IR (e.g., “SPLADE: Sparse Lexical and Positional Encoder”); the current model builds on those ideas by incorporating the CoCondenser teacher and an ensemble distillation step, yielding the strong MS MARCO scores shown above.

Licensing Information

The model is released under the Creative Commons Attribution‑NonCommercial‑ShareAlike 4.0 International (CC‑BY‑NC‑SA‑4.0) license. This license permits:

• Free use, sharing, and modification for non‑commercial purposes.
• Requirement to give appropriate credit to the original authors (Naver) and the underlying research.
• Obligation to distribute any derivative works under the same CC‑BY‑NC‑SA‑4.0 license (share‑alike).

Because the license is non‑commercial, any commercial deployment (e.g., in a paid SaaS product or a for‑sized search engine) requires a separate commercial agreement with the rights holder. The “unknown” field in the Hugging Face metadata reflects that the repository does not provide a separate proprietary license; users must respect the CC‑BY‑NC‑SA terms. Attribution can be satisfied by citing the model card and the associated paper (see the citation block in the README).