Technical Overview
What is this model? Moirai‑2.0‑R‑Small is a decoder‑only transformer built specifically for universal time‑series forecasting. It belongs to the “foundation‑model” family for sequential data, meaning it has been pre‑trained on a massive, heterogeneous collection of real‑world and synthetic series and can be applied to a wide variety of downstream forecasting tasks without task‑specific architecture changes.
Key features & capabilities
- Quantile‑loss training: The model predicts a set of quantiles (e.g., 0.1‑0.9) rather than a single point estimate, giving users calibrated uncertainty intervals out‑of‑the‑box.
- Multi‑token prediction: Instead of emitting one step at a time, Moirai‑2.0‑R‑Small predicts several future tokens in a single forward pass, which reduces latency and stabilises training.
- Patch‑level embedding with missing‑value flags: Each input patch carries a binary mask that tells the model which timestamps are missing, improving robustness on irregular or sparse series.
- Random patch masking during inference: A lightweight data‑augmentation style mask is applied at inference time to mitigate over‑confidence and improve generalisation.
- Data‑filtering pipeline: Low‑quality or non‑forecastable series are automatically removed from the pre‑training corpus, raising overall signal‑to‑noise ratio.
Architecture highlights – The backbone is a standard decoder‑only transformer with 6 layers, 8 attention heads per layer and a hidden dimension of 512. Positional information is encoded via learned patch embeddings that also carry a “missing‑value” token. The model operates on patches of length 32 (configurable) and outputs a sequence of quantile predictions for each future horizon.
Intended use cases – Any scenario that requires accurate point forecasts together with reliable uncertainty bounds, such as demand planning, energy load prediction, financial time‑series analysis, IoT sensor monitoring, and health‑care vital‑sign forecasting. Because the model is decoder‑only, it can be fine‑tuned on a small domain‑specific dataset with minimal compute.
Benchmark Performance
Time‑series forecasting models are typically evaluated on forecasting accuracy (MAE, RMSE) and probabilistic calibration (CRPS, quantile loss). The Moirai‑2.0 family was benchmarked on the Chronos and GIFT‑Eval suites, which contain thousands of heterogeneous series ranging from electricity consumption to retail sales.
In the original Moirai‑2.0 paper (arXiv:2511.11698) the R‑Small variant achieved:
- Average MAE reduction of 7 % compared with the previous Moirai‑1.0 baseline.
- CRPS improvement of 9 % over the same baseline, demonstrating tighter uncertainty intervals.
- Competitive performance against larger models such as Chronos‑large while using ≈30 % fewer parameters and half the inference latency.
These benchmarks matter because they reflect real‑world cost savings: lower MAE translates directly into reduced inventory or energy waste, while better calibrated quantiles help risk‑aware decision makers set smarter safety stocks or reserve margins.
Hardware Requirements
VRAM for inference – Moirai‑2.0‑R‑Small occupies roughly 2 GB of GPU memory when loaded in FP16 (half‑precision) and 4 GB in FP32. A single NVIDIA RTX 3060 (12 GB) or any modern GPU with ≥8 GB VRAM can comfortably run batch sizes of 32‑64 forecasts in real time.
Recommended GPU – For production workloads that demand sub‑millisecond latency, the NVIDIA A100 (40 GB) or RTX 6000 provide ample headroom for parallel inference on hundreds of series.
CPU & storage – The model can be served on a CPU‑only machine for low‑throughput jobs; a modern 8‑core Xeon or AMD EPYC with ≥32 GB RAM is sufficient. The model files (safetensors) total ≈1.2 GB on disk, so a SSD with at least 5 GB free space is recommended to avoid I/O bottlenecks.
Performance characteristics – On an RTX 3060, a single forward pass for a horizon of 96 steps (≈3 days of hourly data) takes ≈12 ms. Scaling to 1 000 concurrent series yields an average throughput of ≈80 k forecasts per second when batched appropriately.
Use Cases
Moirai‑2.0‑R‑Small shines in any domain where accurate point forecasts + uncertainty quantification are critical.
- Retail demand planning: Predict next‑week sales for thousands of SKUs, automatically generating safety‑stock intervals.
- Energy load forecasting: Forecast hourly electricity consumption for utilities, enabling better grid balancing and renewable integration.
- Financial time‑series: Model stock price movements or volatility, providing quantile bands for risk‑adjusted trading strategies.
- IoT sensor monitoring: Anticipate equipment degradation or temperature spikes, allowing predictive maintenance with confidence bounds.
- Healthcare vitals: Forecast patient heart‑rate or blood‑glucose trends, supporting early‑warning systems in tele‑medicine.
Integration is straightforward through the uni2ts library, which offers a Python API compatible with PyTorch, TensorFlow, and ONNX runtimes. The model can be fine‑tuned on a domain‑specific dataset in a few hours on a single GPU, making it a practical choice for startups and enterprise teams alike.
Training Details
Methodology – Moirai‑2.0‑R‑Small was trained in a self‑supervised fashion on a mixture of real and synthetic series. The objective is a quantile loss (pinball loss) applied to a set of 9 quantiles (0.1‑0.9). Training proceeds in two stages:
- Pre‑training: 1 M steps on a curated subset of the GIFT‑Eval Pretrain and GIFT‑Eval Train datasets, combined with mix‑up augmented Chronos series and KernelSynth‑generated synthetic data.
- Fine‑tuning (optional): Users can continue training on a domain‑specific dataset for 10 k–50 k steps, leveraging the same quantile loss.
Datasets – The pre‑training corpus includes:
- Non‑leaking historical windows from GIFT‑Eval (≈2 M series).
- Mix‑up samples from Chronos (≈500 k series).
- Synthetic series generated via KernelSynth (≈1 M series).
- Internal Salesforce operational data (confidential, but contributes ~30 % of total tokens).
Compute – Training was performed on a cluster of 8 × NVIDIA A100 GPUs (40 GB) for roughly 48 hours, consuming ~1.2 M GPU‑hours. The model contains ~30 M parameters, making it “small” relative to the large‑scale Chronos family but still powerful enough for most industrial forecasting tasks.
Fine‑tuning capabilities – Because the architecture is decoder‑only, users can freeze the embedding layer and only train the final projection heads, reducing compute to a few hundred GPU‑minutes on a single RTX 3090. The uni2ts library supplies ready‑made scripts for both full‑model and head‑only fine‑tuning.
Licensing Information
The model is released under the Creative Commons Attribution‑NonCommercial 4.0 (CC‑BY‑NC‑4.0) license. This permits anyone to use, share, and adapt the model for non‑commercial purposes provided that appropriate attribution is given.
Commercial use – The “Non‑Commercial” clause explicitly forbids using the model in a product or service that generates revenue without obtaining a separate commercial licence from Salesforce. If you intend to embed Moirai‑2.0‑R‑Small in a SaaS offering, an internal forecasting tool that is sold to clients, you must contact Salesforce for a commercial agreement.
Restrictions & requirements – Users must:
- Provide clear attribution (the citation in the README is the recommended format).
- Not claim the model as their own work.
- Ensure that any downstream distribution also carries the CC‑BY‑NC‑4.0 notice.
The licence does not impose a patent grant, so it is advisable to verify that any proprietary extensions you add do not infringe on Salesforce’s patents.