instructor-large

Technical Overview

The Instructor‑Large model, hosted under the Hugging Face identifier hkunlp/instructor‑large, is a high‑capacity sentence‑transformer built on the T5‑base architecture (≈770 M parameters). It is designed to generate dense, semantically‑rich embeddings for English text, enabling a broad range of downstream tasks such as similarity search, clustering, classification, and retrieval‑augmented generation. Unlike vanilla sentence‑transformers, Instructor‑Large incorporates a prompt‑guided training regime that teaches the model to “listen” to a natural‑language instruction (e.g., “summarize”, “find similar”) and produce embeddings that are aligned with the intended downstream objective.

Key Features & Capabilities

• Supports sentence‑similarity pipelines out‑of‑the‑box.
• Multi‑task training on benchmarks such as MS‑MARCO, Natural Questions, FEVER, Hotpot‑QA, and the MTEB suite.
• Fine‑grained control via textual prompts, allowing a single model to serve both retrieval and classification needs.
• Optimized for PyTorch and the sentence‑transformers library, making integration with existing pipelines straightforward.
• Built on the robust t5 encoder‑decoder backbone, inheriting strong language understanding and generation abilities.

Architecture Highlights

• Encoder‑decoder transformer with 12 layers, 12 heads, and a hidden size of 768.
• Pre‑trained on the T5‑base checkpoint, then further trained with contrastive and instruction‑following objectives.
• Uses a pooling strategy that combines the [CLS] token with a learned weighted mean of token embeddings to produce a 768‑dimensional sentence vector.

Intended Use Cases

• Semantic search over large document collections (e.g., knowledge bases, FAQs).
• Duplicate‑question detection and reranking in community forums.
• Zero‑shot text classification by embedding both labels and inputs.
• Clustering of research papers, news articles, or product reviews.
• Prompt‑driven retrieval‑augmented generation where the instruction tailors the embedding space.

Benchmark Performance

Instructor‑Large has been evaluated on the MTEB benchmark suite, covering classification, retrieval, clustering, and semantic textual similarity (STS). The most notable results include:

• Amazon Polarity Classification – Accuracy ≈ 91.5 %, F1 ≈ 91.5 %.
• Amazon Counterfactual Classification – Accuracy ≈ 88.1 %, F1 ≈ 83.3 %.
• ArguAna Retrieval – MAP@10 ≈ 47.9 %, NDCG@10 ≈ 57.0 %.
• BIOSSES STS – Cosine‑Similarity Spearman ≈ 84.4 %.
• AskUbuntu Duplicate‑Question Reranking – MAP ≈ 64.3 %, MRR ≈ 76.4 %.

These benchmarks matter because they reflect real‑world scenarios: classification accuracy shows how well the embeddings separate semantic classes; retrieval metrics (MAP, NDCG, MRR) gauge the model’s ability to rank relevant documents; clustering V‑measure indicates the quality of unsupervised grouping; and STS Spearman correlation measures fine‑grained similarity judgment. Compared with earlier Instructor‑Base (≈400 M params) and other sentence‑transformers such as all‑mpnet‑base‑v2, Instructor‑Large consistently outperforms on retrieval‑heavy tasks while maintaining competitive classification scores, thanks to its larger capacity and instruction‑aware training.

Hardware Requirements

Inference with Instructor‑Large is memory‑intensive due to its 770 M‑parameter T5 backbone. Typical VRAM consumption for a single sentence batch (≈32 tokens) is around 6–7 GB on a modern GPU. For higher throughput (larger batch sizes or longer sequences) you should allocate 12 GB + of VRAM.

• Recommended GPU: NVIDIA RTX 3080/3090, A6000, or any GPU with ≥12 GB VRAM. For production‑scale serving, consider A100 40 GB or V100 32 GB.
• CPU: 8‑core Xeon or AMD EPYC with ≥16 GB RAM for preprocessing and tokenization.
• Storage: Model checkpoint size ≈ 3 GB (compressed). Keep an additional 2 GB for tokenizers and auxiliary files.
• Performance: On an RTX 3080, single‑sentence embedding latency is ~30 ms; batch‑size‑64 latency drops to ~150 ms, enabling ~600 queries / second in a multi‑threaded server.

Use Cases

Instructor‑Large shines in scenarios where a single model must adapt to multiple semantic tasks via prompts. Typical real‑world applications include:

• Enterprise Knowledge‑Base Search: Encode internal documents and queries with task‑specific prompts (“find policy documents”) to retrieve the most relevant pages.
• Customer‑Support Ticket Routing: Embed tickets and department labels; a similarity search routes tickets to the correct team.
• Academic Literature Clustering: Group arXiv papers by topic without manual labeling, useful for research assistants.
• Duplicate‑Question Detection in Forums: Rerank candidate duplicates using the model’s instruction‑aware embeddings.
• Zero‑Shot Text Classification: Encode label descriptions (“spam”, “ham”) and compare to input embeddings for rapid, label‑agnostic classification.

Training Details

Instructor‑Large was trained in two stages:

• Base Pre‑training: Initialized from the publicly available t5‑base checkpoint (770 M parameters) trained on the C4 corpus.
• Instruction‑Tuned Fine‑Tuning: A multi‑task contrastive objective using a mixture of datasets:
  • MS‑MARCO (passage ranking)
  • Natural Questions (question answering)
  • FEVER (fact verification)
  • Hotpot‑QA (multi‑hop QA)
  • MTEB classification, retrieval, and clustering tasks

Training was performed on a cluster of 8 × NVIDIA A100 40 GB GPUs for roughly 2 weeks, employing mixed‑precision (FP16) and a batch size of 256. The loss combined a standard contrastive (InfoNCE) term with a prompt‑consistency regularizer, encouraging the model to align embeddings with the provided instruction.

Fine‑tuning on downstream data is straightforward: replace the final pooling layer with a task‑specific head or continue contrastive training on a domain‑specific corpus. The sentence‑transformers library provides ready‑made scripts for this purpose.

Related Papers

The model builds on the INSTRUCTOR: A Multi‑Task Instruction‑Tuned Sentence Embedding Model (arXiv 2212.09741) paper, which introduced the prompt‑guided contrastive training paradigm. The underlying T5 architecture is described in “Exploring the Limits of Transfer Learning with a Unified Text‑to‑Text Transformer” (Raffel et al., 2020). Additional evaluation methodology follows the MTEB benchmark suite (MTEB 2022) and the BEIR information‑retrieval benchmark.

Licensing Information

The model is released under the Apache 2.0 license, as indicated in the README. This permissive license permits:

• Free use, modification, and distribution of the model weights and code.
• Commercial deployment in products or services without royalty payments.
• Inclusion in proprietary software, provided you retain the original copyright notice and license text.

There are no “unknown” restrictions; the Apache 2.0 terms are clear. You must include a copy of the license and a notice of any modifications you make. The license also provides an explicit patent‑grant, protecting downstream users from patent litigation related to the model’s implementation.