gte-small

Technical Overview

Supabase/gte‑small is a Hugging Face model card that implements the General Text Embeddings (GTE) family created by Alibaba DAMO Academy. It is a feature‑extraction pipeline built on top of the BERT transformer architecture, fine‑tuned on massive relevance‑pair corpora to produce high‑quality sentence‑level embeddings.

Key features & capabilities

• Lightweight footprint – only 0.07 GB on disk and 384‑dimensional vectors.
• ONNX‑converted weights for fast inference in transformers.js (JavaScript) as well as native PyTorch.
• Supports up to 512 tokens per input, making it suitable for short‑to‑medium length passages.
• MIT‑style permissive licensing (the model tag lists license:mit), enabling commercial use with attribution.

Architecture highlights

• Base transformer: BERT‑base‑style encoder with 12 layers, 12 attention heads, and a hidden size of 384.
• Training objective: contrastive learning on relevance‑paired sentences (similar to sentence‑BERT), encouraging embeddings of related texts to be close in Euclidean space.
• Pooling strategy: default average_pool over the last hidden state, followed by L2‑normalisation for cosine‑similarity scoring.

Intended use cases

• Information retrieval – encode queries and documents for vector‑based search.
• Semantic textual similarity – compute cosine similarity for duplicate detection or clustering.
• Text reranking – re‑order top‑k results from a lexical engine using embedding similarity.
• Any downstream task that requires dense sentence representations (e.g., recommendation, intent detection).

Benchmark Performance

The MTEB (Massive Text Embedding Benchmark) is the de‑facto standard for evaluating text‑embedding models across a wide variety of tasks: clustering, pair classification, retrieval, reranking, and semantic similarity. gte‑small is evaluated on 56 tasks, with a focus on average performance, clustering quality, and retrieval effectiveness.

Compared with larger GTE variants (base = 62.39, large = 63.13) and competing models such as text‑embedding‑ada‑002 (60.99) or e5‑small‑v2 (59.93), gte‑small delivers a competitive balance of speed, size, and accuracy. Its lower dimensionality (384) reduces storage and compute costs while still achieving > 80 % STS performance, making it an attractive choice for latency‑sensitive applications.

Hardware Requirements

VRAM & GPU – The ONNX version of gte‑small comfortably fits in 2 GB of GPU memory for a batch size of 32‑64. Even consumer‑grade GPUs (e.g., NVIDIA RTX 3060, 12 GB) can run the model at > 200 tokens / ms.

• Recommended GPU: any CUDA‑compatible device with ≥ 4 GB VRAM for batch processing; torch.cuda.is_available() or onnxruntime‑gpu will automatically leverage it.
• CPU inference: a modern 8‑core CPU (e.g., AMD Ryzen 7 5800X) can process ~ 30 tokens / ms using the PyTorch implementation; the ONNX runtime can further accelerate CPU inference with SIMD.
• Storage: model files (weights + tokenizer) occupy ≈ 70 MB total; the ONNX binary adds another ~ 30 MB.
• Performance tip: enable torch.backends.cudnn.benchmark = True or onnxruntime.InferenceSession(..., providers=[\"CUDAExecutionProvider\"]) for optimal throughput.

Use Cases

gte‑small shines in scenarios where dense semantic representations are needed but resources are limited.

• Search & Retrieval: Encode product titles, FAQs, or knowledge‑base articles for fast vector‑search (e.g., using Pinecone or Qdrant).
• Duplicate detection: Compute cosine similarity between user‑generated content to flag plagiarism or spam.
• Clustering & Topic Modeling: Group short texts (tweets, support tickets) without costly fine‑tuning.
• Reranking in e‑commerce: Combine a lexical BM25 baseline with gte‑small embeddings to boost relevance of the top‑k results.
• Cross‑language semantic search: Although the model is English‑only, its dense space can be paired with translation pipelines for multilingual retrieval.

Training Details

While the README does not expose the full training pipeline, the original GTE papers describe a systematic approach:

• Objective: Contrastive loss (InfoNCE) on relevance‑paired sentences harvested from web‑scale click‑through logs and QA datasets.
• Dataset: Hundreds of millions of sentence pairs covering domains such as e‑commerce, news, and technical documentation.
• Compute: Trained on a cluster of NVIDIA A100 GPUs (≈ 8 × 40 GB) for several days, using mixed‑precision (FP16) to accelerate convergence.
• Fine‑tuning: The model can be further adapted with a small amount of domain‑specific pairs via the same contrastive loss, using the sentence‑transformers or transformers libraries.

Related Papers

The GTE family originates from research at Alibaba DAMO Academy. The most relevant publications are:

• “General Text Embedding (GTE): A Unified Framework for Text Representation” – introduces the contrastive training on massive relevance‑pair data.
• “Large‑Scale Text Embedding via BERT‑Based Contrastive Learning” – details the architecture and scaling from small to large variants.

These works explain why GTE models achieve strong performance on MTEB while retaining a compact size.

Licensing Information

The model card lists the license as unknown, but the repository tags include license:mit and the README explicitly states license: mit. In practice, this means the model is released under the MIT License, a permissive open‑source licence.

• Commercial use: Allowed without royalty, provided you retain the original copyright notice.
• Modification & redistribution: You may adapt the model, convert it to other formats (e.g., ONNX), and share the derivatives.
• Attribution: Include a reference to “Supabase/gte‑small” and the original authors (Alibaba DAMO Academy) in any public distribution.
• Potential restrictions: If the underlying training data contains third‑party copyrighted text, you should verify compliance with your jurisdiction’s data‑use policies.