Technical Overview
Ruri‑v3‑310m is a high‑capacity, Japanese‑language text‑embedding model released by cl‑nagoya. Built on the ModernBERT‑Ja architecture, it transforms arbitrary Japanese sentences into dense 768‑dimensional vectors that capture semantic meaning, topical information, and retrieval relevance. The model is optimised for the sentence‑similarity pipeline and is compatible with the sentence‑transformers library.
Key capabilities include:
- Long‑context support: Handles sequences up to 8 192 tokens, a dramatic increase from the 512‑token limit of earlier Ruri releases.
- Expanded vocabulary: 100 k WordPiece tokens reduce token‑length for Japanese text, improving both speed and memory efficiency.
- FlashAttention integration: Leveraging FlashAttention‑2 for faster inference and lower GPU memory consumption.
- Pure SentencePiece tokeniser: No external word‑segmentation tools are required; the model tokenises raw Japanese strings directly.
- Prefix‑scheme encoding: Special prefixes (e.g.,
トピック:,検索クエリ:) let users steer the embedding toward classification, retrieval, or clustering tasks.
Architecturally, Ruri‑v3‑310m contains 315 M parameters (236 M trainable embeddings) distributed across 25 transformer layers with a hidden size of 768. It inherits ModernBERT‑Ja’s efficient attention kernels, layer‑norm placement, and pre‑training on a massive Japanese web‑crawl, then fine‑tuned on the ruri‑v3‑dataset‑ft for sentence‑level similarity.
Typical use‑cases include semantic search, duplicate detection, clustering of news articles, topic‑aware classification, and any downstream task that benefits from high‑quality Japanese sentence embeddings.
Benchmark Performance
Ruri‑v3‑310m is evaluated on the JMTEB suite, a standard benchmark for Japanese text‑embedding models. The model achieves an overall average score of 77.24, the highest among the Ruri‑v3 family and competitive with larger 1‑B‑parameter embeddings.
| Metric | Score |
|---|---|
| Average JMTEB | 77.24 |
| Retrieval | 81.89 |
| STS (Semantic Textual Similarity) | 81.22 |
| Classification | 78.66 |
| Reranking | 93.43 |
| Clustering | 55.69 |
| Pair Classification | 62.60 |
These metrics matter because they directly reflect how well the model distinguishes nuanced Japanese semantics—a critical factor for search engines, recommendation systems, and conversational AI. Compared with other Japanese embeddings such as GLuCoSE‑base‑ja (Avg. 70.44) or OpenAI’s text‑embedding‑ada‑002 (Avg. 69.48), Ruri‑v3‑310m shows a clear superiority in both retrieval and classification tasks while remaining far smaller than 1‑B‑parameter models.
Hardware Requirements
- VRAM for inference: Approximately 6 GB of GPU memory is sufficient for a single 8 192‑token batch on a 768‑dimensional output. Larger batch sizes or multi‑GPU deployments will proportionally increase memory needs.
- Recommended GPU: NVIDIA RTX 3080/3090, RTX A6000, or any GPU supporting CUDA 11+ and FlashAttention 2. FlashAttention reduces memory overhead by ~30 % compared with standard attention.
- CPU: Modern x86‑64 CPUs (Intel i7‑12700K or AMD Ryzen 7 5800X) can run the model in CPU‑only mode, but expect inference latency > 200 ms per sentence. For production, a GPU is strongly advised.
- Storage: The model files (safetensors + tokenizer) occupy roughly 1.2 GB on disk. Including the optional FlashAttention kernels adds another ~200 MB.
- Performance: On an RTX 3080, single‑sentence encoding takes ~3 ms (FP16) with FlashAttention enabled. Throughput scales linearly with batch size up to the VRAM limit.
Use Cases
Ruri‑v3‑310m shines in any scenario that requires high‑quality Japanese semantic representations:
- Semantic search & retrieval: Encode queries (
検索クエリ:) and documents (検索文書:) for fast cosine‑similarity ranking. - Duplicate detection: Identify near‑identical news articles, user‑generated content, or legal documents.
- Topic‑aware classification: Prefix sentences with
トピック:to guide clustering or multi‑label classification pipelines. - Chatbot intent matching: Map user utterances to intent vectors for Japanese conversational agents.
- Recommendation engines: Compute similarity between product descriptions, reviews, or movie synopses in Japanese.
Industries that benefit include e‑commerce (product search), media & publishing (article clustering), finance (document similarity for compliance), and education (semantic matching of learning resources). Integration is straightforward via the sentence‑transformers API or directly through the transformers library.
Training Details
Ruri‑v3‑310m originates from the cl‑nagoya/ruri‑v3‑pt‑310m base model, pre‑trained on a massive Japanese web‑crawl (≈ 200 GB of raw text). The fine‑tuning stage employed the ruri‑v3‑dataset‑ft, a curated collection of sentence‑pair similarity annotations covering news, literature, and conversational domains. Training leveraged mixed‑precision (FP16) on a cluster of 8 × NVIDIA A100 GPUs (40 GB each) for roughly 3 days, using the sentence‑transformers loss functions (cosine‑similarity and triplet loss). The model’s FlashAttention kernels were compiled with flash‑attn for maximal throughput.
Fine‑tuning on downstream tasks is fully supported: users can load the model with SentenceTransformer, add a custom head, and continue training on domain‑specific data while retaining the 100 k token vocabulary and 8 192‑token context window.
Licensing Information
The model is released under the Apache‑2.0 license, despite the “unknown” tag in the metadata. Apache‑2.0 is a permissive open‑source licence that grants you the right to use, modify, distribute, and even commercialise the model without paying royalties. The only obligations are:
- Preserve the original copyright notice and licence text in any redistributed copies.
- Provide a clear attribution to the original authors (cl‑nagoya) and the underlying ModernBERT‑Ja project.
- State any modifications you make to the model or its code.
No patent grants or trademark usage are implied, so you should avoid implying endorsement by the original authors when deploying the model in a commercial product. The Apache‑2.0 terms also protect you from liability, making the model suitable for enterprise applications, SaaS platforms, and embedded devices.