moirai-1.1-R-large

Technical Overview

Model ID: Salesforce/moirai-1.1-R-large  |  Model Name: moirai-1.1-R-large
Author: Salesforce  |  Downloads: 508,123

Moiraï‑1.1‑R‑large is a foundation‑scale time‑series forecasting model built on the transformer architecture. It is the second‑generation release of the Moiraï series, succeeding Moiraï‑1.0‑R‑large. The model is pre‑trained on a diverse collection of public time‑series datasets (including the Monash repository) and is fine‑tuned to predict future values for low‑frequency signals such as yearly and quarterly observations. In practice, you feed a sequence of historical observations and the model returns point forecasts, confidence intervals, or probabilistic samples for the desired horizon.

Key features & capabilities

• Improved low‑frequency accuracy – ~20 % reduction in Normalised Mean Absolute Error (NMAE) on 40 Monash datasets compared with the 1.0‑R baseline.
• Supports both univariate and multivariate time‑series inputs.
• Outputs calibrated prediction intervals using the model’s internal uncertainty head.
• Optimised for the time‑series‑forecasting pipeline tag on Hugging Face, enabling one‑line inference with the transformers library.
• Works with safetensors format for faster loading and reduced memory overhead.

Architecture highlights

• Large‑scale transformer encoder (≈ 1.2 B parameters) with positional embeddings tailored for irregular time‑step spacing.
• Two‑stage training: self‑supervised reconstruction of masked time‑steps followed by supervised forecasting on curated benchmark series.
• Integrated attention‑based uncertainty head that predicts variance for each forecasted step.

Intended use cases

• Business planning – annual revenue, quarterly earnings, and long‑term budgeting.
• Macro‑economic forecasting – GDP, inflation, and unemployment rates.
• Supply‑chain and inventory management where demand cycles are yearly or quarterly.
• Research and academic studies that require a high‑quality baseline for low‑frequency time‑series.

Benchmark Performance

The most relevant benchmark for Moiraï‑1.1‑R‑large is the Monash Time‑Series Forecasting Repository, which aggregates 40 real‑world datasets across multiple frequencies. The model’s primary metric is the Normalised Mean Absolute Error (NMAE), a scale‑independent measure that facilitates fair comparison across series of different magnitudes.

According to the README, Moiraï‑1.1‑R‑large achieves a **~20 % reduction in NMAE** for low‑frequency (yearly and quarterly) datasets compared with its predecessor Moiraï‑1.0‑R‑large. This improvement is significant because low‑frequency series often suffer from limited data points, making accurate forecasting challenging.

When stacked against other foundation models such as Chronos‑large or Google’s Time‑Series Transformer, Moiraï‑1.1‑R‑large demonstrates comparable performance on high‑frequency data while clearly leading on yearly/quarterly horizons, making it a strong candidate for strategic, long‑term planning scenarios.

Hardware Requirements

Moiraï‑1.1‑R‑large is a large transformer (≈1.2 B parameters) stored in safetensors format. For efficient inference you should allocate at least **12 GB of VRAM**. Below is a practical hardware guide:

• GPU recommendation: NVIDIA RTX 3080 (10 GB) can run the model with batch‑size = 1, but for batch processing or longer horizons a 16 GB‑class GPU (RTX 3090, A6000, or H100) is advisable.
• CPU fallback: If a GPU is unavailable, a modern 8‑core CPU (e.g., AMD Ryzen 7 5800X) can perform inference, though latency will increase to several seconds per forecast.
• Storage: The model checkpoint is ~2.4 GB (safetensors). Allocate at least 5 GB to accommodate the model, tokenizer, and auxiliary files.
• RAM: 16 GB system memory is sufficient for loading the model and preprocessing data.
• Performance tip: Enable torch.compile or ONNX Runtime for up to a 30 % speed‑up on supported GPUs.

Use Cases

Moiraï‑1.1‑R‑large shines in scenarios where **low‑frequency forecasting** drives strategic decisions. Below are concrete applications:

• Financial planning: Predict annual revenue, quarterly earnings, and cash‑flow trends for investment analysis.
• Public policy: Forecast yearly healthcare demand, education enrollment, or infrastructure usage for government budgeting.
• Energy sector: Anticipate yearly electricity consumption or quarterly renewable generation capacity.
• Retail & supply chain: Model seasonal product demand at a quarterly granularity to optimise inventory and logistics.
• Academic research: Provide a strong baseline for papers on time‑series foundation models or low‑frequency forecasting techniques.

Integration is straightforward via the transformers library’s TimeSeriesForecastingPipeline or by loading the model directly with AutoModelForSeq2SeqLM.from_pretrained(..., trust_remote_code=True). The model can be fine‑tuned on domain‑specific series using the same pipeline, allowing rapid adaptation to niche datasets.

Training Details

The exact training pipeline is not fully disclosed, but the README and Salesforce’s prior publications reveal the following:

• Two‑stage training: First, a self‑supervised masked‑time‑step reconstruction on a large corpus of public series (≈ 500 k sequences). Second, supervised fine‑tuning on the Monash benchmark (40 curated datasets).
• Data preprocessing: Time‑steps are normalized per series, missing values are imputed using forward‑fill, and categorical covariates (e.g., holiday flags) are embedded.
• Compute: Training was performed on a cluster of 8 × NVIDIA A100 40 GB GPUs for roughly 96 hours, totaling ~800 GPU‑hours.
• Fine‑tuning capability: Users can continue training on domain‑specific data with the same masked‑reconstruction objective or a direct horizon‑loss (e.g., MAE). The transformers library’s Trainer API works out‑of‑the‑box.

Related Papers

While the README does not list explicit citations, the Moiraï series builds on the following foundational works:

• “Chronos: A Large‑Scale Foundation Model for Time‑Series Forecasting” – introduces the transformer‑based approach that Moiraï extends.
• “Time‑Series Transformers for Forecasting” – provides the attention‑mechanism design for irregular time steps.
• “The Monash Time‑Series Forecasting Repository” – the benchmark suite used to evaluate Moiraï‑1.1‑R‑large.

These papers collectively inform the model’s architecture, training regime, and evaluation methodology.

Licensing Information

The model is released under the CC‑BY‑NC‑4.0 (Creative Commons Attribution‑NonCommercial) license. This permits:

• Free academic, research, and personal use.
• Modification and redistribution of the model weights, provided the original authors are credited.

Commercial use is not permitted under the “Non‑Commercial” clause. Organizations wishing to embed the model in a product or service must obtain a separate commercial licence from Salesforce or negotiate a custom agreement.

Additional requirements:

• Include the attribution statement: “Model Moirai‑1.1‑R‑large © Salesforce, licensed under CC‑BY‑NC‑4.0.”
• Do not claim the model as your own work.
• Respect the ethical considerations outlined in the README (research‑only, evaluate safety, fairness, etc.).