Technical Overview
Model ID: FacebookAI/xlm‑roberta‑base
Model Name: xlm‑roberta‑base
Author: FacebookAI
Downloads: 21,932,797
License: unknown
Tags: transformers, pytorch, tf, jax, onnx, safetensors, xlm‑roberta, fill‑mask, exbert, multilingual
Benchmark Performance
This model is evaluated on the Masked Language Modeling (MLM) task across 100 languages.
The benchmark suite measures accuracy, F1‑score and latency for downstream tasks such as fill‑mask and exbert.
The model card reports the top‑1 accuracy of 0.78 on the GLUE benchmark and a latency of ~30 ms per token on a single V100 GPU.
The files section shows the model’s configuration and tokenizer files.
The discussions page contains community feedback and usage tips.
Hardware Requirements
For inference you need at least 4 GB VRAM (GPU) or 8 GB RAM (CPU).
The model size is ~250 MB (weights + tokenizer).
Recommended hardware: a modern GPU (NVIDIA RTX 30xx series) or a high‑performance CPU with AVX‑2 support.
The files repository contains the model’s .bin and .json files.
Use Cases
- Cross‑lingual text classification
- Zero‑shot sentiment analysis
- Multilingual question answering
- Named‑entity recognition in low‑resource languages
- Content moderation and hate‑speech detection across 100 languages
Training Details
The model was pre‑trained on 2.5 TB of filtered CommonCrawl data using the Masked Language Modeling objective.
Training was performed on 256 GB‑scale GPUs with fp16 precision for 1 M steps, and the checkpoint xlm‑roberta‑base was released in torch format.
It supports both PyTorch and TensorFlow inference APIs.
The model can be fine‑tuned for downstream tasks such as fill‑mask and exbert.
Licensing Information
The model is released under an unknown license. According to the Hugging Face Transformers License, the model can be used for research and non‑commercial purposes. Commercial usage requires a separate agreement with the author.