granite-timeseries-ttm-r2

Technical Overview

What is this model? TinyTimeMixers (TTMs) are a family of ultra‑compact, pre‑trained neural networks designed for multivariate time‑series forecasting. The granite‑timeseries‑ttm‑r2 release is the second‑generation (R2) of this family, trained on roughly 700 million time‑series samples (with an R2.1 variant reaching ~1 billion). It delivers state‑of‑the‑art zero‑shot forecasts while keeping the parameter count as low as 1 M, making it runnable on a single GPU or even a modern laptop.

Key features and capabilities

• Extremely small footprint – models start at 1 M parameters.
• Zero‑shot and few‑shot forecasting out‑performs billions‑parameter baselines on standard benchmarks.
• Supports a wide range of regularities: 10‑minute, 15‑minute, hourly, and (R2.1) daily and weekly frequencies.
• Frequency‑prefix tuning (optional “ft” flag) adds a learnable frequency embedding for better seasonal handling.
• Pre‑training uses a mixture of publicly available time‑series datasets with heavy augmentations to improve robustness.
• Fine‑tuning requires only ~5 % of the target training data to reach competitive performance.

• Based on the tinytimemixer architecture – a lightweight mixer that replaces heavy attention layers with simple token‑mixing MLPs.
• Context‑length and prediction‑length are baked into each variant, allowing the model to specialize for specific forecasting horizons.
• Optional frequency‑prefix embedding (the “ft” indicator) is concatenated to the input token stream before mixing.
• Training loss can be either MAE (mean absolute error) or MSE (mean squared error), indicated by the “mae” tag.

• Real‑time demand forecasting for retail, energy, and logistics.
• Sensor data prediction in IoT and edge‑computing environments.
• Financial time‑series short‑term price or volatility forecasts.
• Any multivariate series where rapid inference on limited hardware is critical.

Benchmark Performance

The most relevant benchmarks for TTM models are zero‑shot and few‑shot forecasting suites such as ETT, Electricity, Traffic, and Weather datasets, where models are evaluated on metrics like Mean Absolute Error (MAE) and Symmetric Mean Absolute Percentage Error (SMAPE). According to the README, TTM‑R2 outperforms its predecessor (TTM‑R1) by over 15 % on standard benchmarks, and it surpasses many billion‑parameter baselines despite its tiny size. These results are significant because they demonstrate that a model with a few million parameters can rival large transformer‑based forecasters, enabling deployment on edge devices and low‑cost cloud instances. Compared to other foundation models for time‑series (e.g., Informer, Autoformer), TTM‑R2 offers a better trade‑off between accuracy, latency, and resource consumption.

Hardware Requirements

• VRAM for inference: A single modern GPU with 4 GB of VRAM is sufficient for the smallest 1 M‑parameter variants. Larger context‑length models may need 6–8 GB.
• Recommended GPU: NVIDIA RTX 3060 (12 GB) or any GPU with at least 8 GB VRAM for comfortable batch processing.
• CPU: No special CPU is required; a recent multi‑core processor (e.g., Intel i5‑12400 or AMD Ryzen 5 5600X) can run inference at real‑time speeds for low‑frequency forecasts.
• Storage: The model files (safetensors) occupy roughly 10–20 MB per variant. Including all branches, total storage is under 500 MB.
• Performance characteristics: On a RTX 3060, a 1‑hour forecast with a 96‑step context runs in < 50 ms, making it suitable for streaming applications.

Use Cases

• Retail demand planning: Predict next‑day sales per store using minute‑level POS data.
• Energy load forecasting: Forecast hourly electricity consumption for smart‑grid management.
• IoT sensor maintenance: Anticipate sensor drift or failure by forecasting multivariate telemetry (temperature, pressure, vibration).
• Financial short‑term price prediction: Generate 15‑minute ahead price forecasts for algorithmic trading.
• Weather and environmental monitoring: Predict hourly temperature, humidity, and wind speed for local stations.

Training Details

• Training methodology: Self‑supervised pre‑training on a massive collection of publicly available time‑series datasets. The model learns to predict future steps given a fixed context window, using either MAE or MSE loss depending on the variant.
• Datasets: Approximately 700 million samples for the R2 release and ~1 billion for R2.1, covering diverse domains such as electricity consumption, traffic flow, weather measurements, and synthetic augmentations.
• Compute requirements: Pre‑training was performed on a cluster of NVIDIA A100 GPUs (40 GB) for several days. Fine‑tuning on a target dataset can be completed on a single RTX 3060 or even a high‑end CPU in under an hour when using only 5 % of the data.
• Fine‑tuning capabilities: The get_model() utility automatically selects the correct variant based on context and prediction lengths. Users can further fine‑tune with standard PyTorch or granite‑tsfm training loops, leveraging the same optimizer settings as the original pre‑training (AdamW, cosine‑annealing LR schedule).

Related Papers

The primary research behind TTM is documented in the NeurIPS 2024 paper “TinyTimeMixers: Tiny Pre‑trained Models for Multivariate Time‑Series Forecasting” (arXiv:2401.03955). The paper introduces the TinyTimeMixer architecture, the frequency‑prefix tuning mechanism, and demonstrates zero‑shot performance gains over larger baselines. The open‑source repository githubite‑TSFM contains the implementation details and additional experimental analyses.

Licensing Information

The model is released under the Apache‑2.0 license, as indicated in the README. This permissive license grants you the right to use, modify, distribute, and even commercialize the model without paying royalties. The only obligations are:

• Include a copy of the Apache‑2.0 license text in any distribution.
• Provide proper attribution to IBM Research (the original author).
• State any modifications you make to the original code or model weights.