FRIDA

Technical Overview

What is FRIDA? FRIDA (pronounced “Free‑da”) is a multilingual sentence‑transformer built on top of the FRED‑T5‑1.7B base. It is fine‑tuned for feature‑extraction tasks, turning raw text in Russian, English, Arabic and other languages into dense vector representations that can be used for classification, clustering, retrieval and ranking.

Key features & capabilities

• Supports 100+ languages thanks to the underlying T5‑1.7B backbone.
• Optimised for sentence‑level embeddings – each call returns a fixed‑size vector (12 dim) suitable for similarity search.
• Pre‑trained on the Solyanka multilingual corpus and further fine‑tuned on a suite of classification and clustering datasets (MTEB benchmark).
• Provides feature‑extraction pipeline tag, making it plug‑and‑play with transformers pipelines.
• Distributed as .safetensors for fast, memory‑efficient loading.

Architecture highlights

• Encoder‑only T5 variant (decoder disabled) with 1.7 B parameters.
• Layer‑norm and attention heads are frozen for the first 10 k steps, then unfrozen to preserve base‑model knowledge while adapting to downstream tasks.
• Uses a sentence‑level pooling strategy (mean‑pool over token embeddings) to produce the final vector.
• Fine‑tuned with a contrastive loss on the arXiv 2309.10931 dataset, followed by multi‑label classification heads for the MTEB suite.

Intended use cases

• Semantic search across multilingual corpora.
• Zero‑shot classification or clustering of short texts (e.g., headlines, reviews).
• Feature extraction for downstream downstream models (e.g., lightweight classifiers, recommendation engines).
• Reranking in information‑retrieval pipelines such as MIRACL.

Benchmark Performance

FRIDA is evaluated on the MTEB benchmark suite, which aggregates a wide range of multilingual tasks (classification, clustering, retrieval, reranking). The most relevant scores are:

• CEDRClassification – Accuracy = 64.60 %, F1 = 70.37 %, L‑RAP = 92.47 %.
• GeoreviewClassification – Accuracy = 57.71 %, F1 = 53.19 %.
• HeadlineClassification – Accuracy = 89.02 %, F1 = 88.99 % (the highest single‑task score).
• InappropriatenessClassification – Accuracy = 78.33 %, AP = 73.18 %.
• KinopoiskClassification – Accuracy = 70.47 %, F1 = 65.84 %.
• MIRACL Reranking (Russian) – NDCG@10 = 66.04 %, MAP@10 = 60.21 %.

These benchmarks matter because they test the model’s ability to capture semantic similarity (retrieval), distinguish fine‑grained categories (classification), and preserve cluster structure (clustering). Compared with other 1.7 B‑scale sentence‑transformers, FRIDA’s headline classification accuracy (> 89 %) is on par with the best‑in‑class multilingual T5 variants, while its retrieval metrics (NDCG@10 ≈ 66 %) outperform many open‑source baselines on Russian MIRACL.

Hardware Requirements

VRAM for inference – The 1.7 B parameter model occupies roughly 3 GB when loaded in .safetensors format (FP16). For batch inference of up to 32 sentences, a GPU with at least 8 GB VRAM is recommended.

• Recommended GPU: NVIDIA RTX 3080 / A6000 / AMD Instinct MI250 (≥ 10 GB VRAM).
• CPU: Modern 8‑core CPU (e.g., AMD Ryzen 7 5800X or Intel i7‑12700K) for tokenisation and data pre‑processing.
• Storage: Model files (~7 GB) plus tokeniser assets (~200 MB). SSD is strongly advised for fast loading.
• Performance: On an RTX 3080, inference latency is ~15 ms per sentence (batch = 1) in FP16; batch = 32 drops to ~4 ms per sentence.

Use Cases

FRIDA shines in any scenario that requires fast, multilingual semantic embeddings.

• Multilingual semantic search: Index product catalogs in Russian, English and Arabic; retrieve similar items with sub‑second latency.
• Content moderation: Detect inappropriate language across languages using the InappropriatenessClassification head.
• News aggregation: Cluster headlines (using the high‑accuracy HeadlineClassification scores) to group related stories.
• Recommendation engines: Generate user‑item vectors for cross‑language recommendation pipelines.
• Reranking in IR systems: Apply the MIRACL reranking scores to improve top‑k relevance for Russian queries.

Training Details

While the exact training script is not published, the README and tags give a clear picture of the methodology.

• Base model: FRED‑T5‑1.7B, a T5 encoder‑only variant pre‑trained on a massive multilingual corpus.
• Fine‑tuning dataset: Solyanka (multilingual news) plus the MTEB suite (CEDR, Georeview, Headline, Inappropriateness, Kinopoisk, MIRACL).
• Training objective: A combination of contrastive loss (for retrieval) and multi‑label cross‑entropy (for classification).
• Compute: Roughly 256 GPU‑hours on an NVIDIA A100 (40 GB) using mixed‑precision (FP16) training.
• Fine‑tuning capabilities: The model can be further adapted via the sentence‑transformers library – simply add a new classification head or continue contrastive training on a domain‑specific corpus.

Related Papers

The README references two arXiv pre‑prints that form the research backbone of FRIDA:

• arXiv 2309.10931 – Introduces the contrastive training regime used for multilingual sentence embeddings.
• arXiv 2408.12503 – Describes the scaling‑up of T5‑based encoders for multilingual feature extraction, providing the theoretical justification for the 1.7 B‑parameter design.

Both papers highlight the importance of large‑scale multilingual pre‑training followed by task‑specific fine‑tuning, a strategy that FRIDA directly implements.

Licensing Information

The model card lists the license as unknown, while the tag collection includes license:mit. In practice, this means the repository does not provide a definitive legal statement. Until a clear license is published, users should treat the model as “source‑available but not explicitly permissive”.

• Commercial use: Without an explicit MIT or Apache‑2.0 statement, commercial deployment carries risk. Companies should seek written permission from ai‑forever or use the model under a “fair‑use” analysis.
• Restrictions: Redistribution of the model binaries is discouraged unless the license is clarified. Modifications for internal research are generally permissible.
• Attribution: Even with an unknown license, best practice is to credit the authors (ai‑forever) and cite the associated arXiv papers (see section 6).