Technical Overview
Sundial‑base‑128m is a generative time‑series foundation model released by thuml. It is a decoder‑only Transformer that treats a univariate or multivariate time series as a sequence of patch tokens. By learning an auto‑regressive representation of these tokens, the model can generate a distribution of future values rather than a single deterministic point forecast. In practice this means you can obtain the mean forecast, any quantile, confidence intervals, or even full sample trajectories with a single forward pass.
Key features & capabilities
- Zero‑shot point and probabilistic forecasting – no fine‑tuning required for many downstream tasks.
- Supports arbitrary look‑back and prediction horizons (e.g., 2880‑step look‑back, 96‑step forecast in the quick‑start).
- Generates multiple plausible futures in parallel (e.g., 20 samples in < 1 s on an Apple M1 Pro).
- Built‑in TimeFlow Loss that learns a per‑token probability distribution, enabling flow‑matching generation.
- Optimized inference with FlashAttention, KV‑cache, and shared‑condition tricks for long horizons.
Architecture highlights
- Patch‑based tokenisation – continuous values are grouped into fixed‑size patches and embedded into vectors.
- Decoder‑only Transformer with ~128 M parameters (≈1 trillion training time‑points).
- TimeFlow loss function that parameterises each token’s conditional distribution, allowing the model to sample from a learned stochastic process.
- Speed‑up modules (FlashAttention, KV‑cache) that keep latency low even for long look‑backs (e.g., 2880 steps).
Intended use cases
- Industrial IoT sensor forecasting (temperature, vibration, power usage).
- Financial time‑series prediction (stock prices, volatility, order‑book dynamics).
- Demand planning and inventory management for retail and e‑commerce.
- Energy load forecasting for smart grids.
- Any domain where probabilistic, multi‑modal future trajectories are valuable.
Benchmark Performance
Sundial‑base‑128m has been evaluated on three widely‑cited time‑series benchmarks:
- GIFT‑Eval – achieved 1st place on the MASE metric (Mean Absolute Scaled Error).
- Time‑Series‑Library – secured 1st place on both zero‑shot MSE (Mean Squared Error) and MAE (Mean Absolute Error).
- FEV Leaderboard – competitive scores across multiple forecasting horizons.
These benchmarks are important because they test both deterministic accuracy (MSE/MAE) and scaled error (MASE) across heterogeneous datasets, ensuring the model generalises well to unseen series. Compared with contemporary foundation models such as Chronos and Informer, Sundial’s generative approach yields lower error while also providing full predictive distributions, a capability many point‑forecast‑only models lack.
Hardware Requirements
VRAM & GPU
- For inference on a single 96‑step forecast with a single sample, a GPU with ≥ 8 GB VRAM (e.g., RTX 3060) is sufficient.
- Generating 20 samples in parallel benefits from ≥ 16 GB VRAM (e.g., RTX 3090, A100‑40G) to keep latency under 1 s.
CPU
- On CPU‑only hardware (Apple M1 Pro, 16 GB RAM) the model runs at ~250 ms for a 672‑step look‑back / 16‑step forecast.
- FlashAttention and KV‑cache optimisations reduce latency on modern CPUs and GPUs.
Storage
- Model checkpoint size ≈ 500 MB (safetensors format).
- Additional space for tokeniser and example notebooks (~200 MB).
Use Cases
Sundial‑base‑128m shines in any scenario that benefits from fast, zero‑shot forecasting with uncertainty quantification.
- Predictive maintenance – forecast sensor drift to schedule repairs before failure.
- Energy management – predict short‑term load to balance renewable generation.
- Retail demand forecasting – generate multiple demand scenarios for inventory optimisation.
- Financial risk modelling – simulate price paths for VaR or stress‑testing.
- Smart‑city traffic flow – anticipate congestion and adjust signal timing in real‑time.
Integration is straightforward via the transformers library, and the model can be embedded in pipelines such as Apache IoTDB (as of Aug 2025) or any Python‑based data‑science stack.
Training Details
Methodology
- Decoder‑only Transformer trained with a flow‑matching objective (TimeFlow Loss) that learns a conditional distribution for each token.
- Auto‑regressive tokenisation: raw series are split into fixed‑size patches, embedded, and fed into the Transformer.
Datasets
- thuml/UTSD – a large‑scale univariate time‑series dataset.
- Salesforce/lotsa_data – heterogeneous multivariate series from various domains.
- autogluon/chronos_datasets – the benchmark suite used in the Chronos paper.
Compute
- Training on ~1 trillion time‑points, equivalent to several months of GPU time on an A100‑40G cluster.
- Mixed‑precision (FP16) and gradient checkpointing were employed to fit the 128 M‑parameter model in GPU memory.
Fine‑tuning
- Model can be fine‑tuned on domain‑specific series using the same
AutoModelForCausalLMAPI. - Because the architecture is decoder‑only, standard language‑model fine‑tuning scripts (e.g., Hugging Face
Trainer) work out‑of‑the‑box.
Licensing Information
The model card lists the license as “unknown”, but the README states an Apache‑2.0 license for the code and weights. Under Apache‑2.0 you may:
- Use the model for commercial and non‑commercial purposes.
- Modify, redistribute, and create derivative works.
- Include the model in SaaS offerings, embedded devices, or cloud services.
Key requirements are:
- Provide a copy of the Apache‑2.0 license text in any distribution.
- Retain attribution to the original authors (thuml) and the paper(s) cited.
- State any modifications you make.
If the “unknown” tag persists on the Hugging Face hub, users should verify the license file in the repository before commercial deployment.