phi-4

Technical Overview

What is this model? phi‑4 is a 14‑billion‑parameter dense decoder‑only Transformer released by Microsoft Research in December 2024. It is designed as a “small but capable” language model that excels in reasoning‑heavy tasks while fitting comfortably on modern GPU hardware. The model accepts plain‑text prompts, with a strong preference for chat‑style instruction input, and returns fluent, context‑aware generated text.

Key features & capabilities

• 16 K token context window – enables long‑form reasoning, code reviews, and multi‑turn conversations without truncation.
• Hybrid training data: synthetic textbook‑style examples, filtered public‑domain documents, academic books, and high‑quality Q&A pairs.
• Direct Preference Optimization (DPO) combined with supervised fine‑tuning for robust instruction following and safety.
• Specialised sub‑tokens for math, programming, and logical reasoning, delivering strong performance on GSM‑8K, MMLU, HumanEval and similar benchmarks.
• Optimised for latency‑bound and memory‑constrained environments – runs efficiently on a single 48 GB GPU when quantised to 4‑bit or 8‑bit.

Architecture highlights

• 14 B parameters, dense decoder‑only Transformer with rotary positional embeddings.
• 16 K token context length, achieved via a sliding‑window attention implementation that scales linearly with token count.
• Trained on 9.8 T tokens (≈ 6 TB of raw text) over 21 days using 1 920 NVIDIA H100‑80G GPUs.
• Model weights are released as safetensors for safe, zero‑copy loading.

Intended use cases

• Research prototyping – a high‑quality baseline for exploring instruction tuning, retrieval‑augmented generation, and low‑latency inference.
• Chat‑based assistants that require logical reasoning, math solving, or code generation within tight response‑time budgets.
• Edge‑oriented deployments where GPU memory is limited (e.g., 24‑48 GB GPUs) but high‑quality output is still required.

Benchmark Performance

Benchmarks that matter for a 14 B instruction‑following model include:

• MMLU (Massive Multitask Language Understanding) – measures broad knowledge across 57 subjects.
• GSM‑8K – evaluates multi‑step math reasoning.
• HumanEval – tests code generation and functional correctness.
• OpenAI‑Evals (Chat, Summarisation, Reasoning) – assess instruction adherence and safety.

According to the Phi‑4 Technical Report, phi‑4 surpasses its predecessor Phi‑3 on all three core benchmarks:

• MMLU: 71.4 % (vs. 68.9 % for Phi‑3).
• GSM‑8K: 84.2 % (vs. 80.5 %).
• HumanEval: 54.3 % pass@1 (vs. 49.1 %).

These improvements stem from the synthetic “textbook‑like” data and the DPO alignment stage, which together boost logical consistency and factuality. Compared with other 14 B models such as LLaMA‑2‑13B, phi‑4 consistently ranks in the top‑5 on the same benchmarks while offering a larger context window and stronger safety alignment.

Hardware Requirements

VRAM for inference – The full‑precision (FP16) checkpoint occupies ~28 GB. With 4‑bit quantisation it drops to ~7 GB, and 8‑bit quantisation to ~14 GB, making it feasible on a single 24‑48 GB GPU.

Recommended GPU specifications

• Minimum: NVIDIA RTX 4090 (24 GB) with 8‑bit quantisation.
• Optimal: NVIDIA H100‑80G or A100‑80G (80 GB) for FP16 or BF16 inference, enabling the full 16 K context without off‑loading.
• Multi‑GPU setups (e.g., 2 × A100‑80G) can be used for batch inference or to run the model at higher precision.

CPU & storage

• CPU is not a bottleneck for inference; a modern 8‑core Xeon or AMD EPYC works fine.
• Model files (weights + tokenizer) total ~30 GB on disk. SSD storage is recommended for fast loading.
• During inference, the model streams tokens; a 1 Gbps network is sufficient for remote deployment.

Performance characteristics – On a single H100‑80G, phi‑4 can generate ~200 tokens/second (FP16) for a 16 K context, achieving sub‑100 ms latency for typical 50‑token responses. Quantised 4‑bit inference on an RTX 4090 yields ~120 tokens/second with a small quality trade‑off.

Use Cases

Primary intended applications

• Interactive chatbots that need to reason about math, code, or scientific concepts.
• Code‑assistant tools that generate, refactor, or explain snippets in Python, JavaScript, C++, etc.
• Educational platforms that provide step‑by‑step problem solving for STEM subjects.
• Low‑latency inference services (e.g., on‑device assistants, edge servers) where memory budget is limited.

Real‑world examples

• University tutoring bots that solve calculus problems and explain each step.
• IDE plugins that suggest bug‑fixes and write unit tests on the fly.
• Customer‑support agents that can safely handle technical queries while staying within a 100 ms response window.

Industries & domains

• EdTech – adaptive learning and homework help.
• Software development – code generation, documentation, and review.
• Finance & analytics – quick data‑driven insights without large‑scale GPU farms.

Integration possibilities – The model can be loaded via the transformers library, served with text‑generation‑inference, or wrapped in an OpenAI‑compatible API endpoint for seamless integration into existing pipelines.

Training Details

Methodology – Training proceeded in two phases:

1. Base pre‑training: 14 B dense decoder trained on 9.8 T tokens for 21 days using 1 920 NVIDIA H100‑80G GPUs. The data mix comprised:
   • Filtered public‑domain web text (≈ 70 %).
   • Synthetic “textbook‑like” passages targeting math, coding, and common‑sense reasoning (≈ 20 %).
   • Acquired academic books and curated Q&A datasets (≈ 10 %).
2. Alignment phase: Supervised fine‑tuning on high‑quality chat logs followed by Direct Preference Optimization (DPO) to improve truthfulness, honesty, and helpfulness.

Compute budget – The 21‑day run consumed roughly 1.6 M GPU‑hours (≈ 12 PF‑days) on the H100 cluster, reflecting a high‑throughput mixed‑precision pipeline (FP16/BF16).

Fine‑tuning capabilities – The model is released with a safetensors checkpoint that can be further fine‑tuned using the transformers library. Users can apply LoRA, QLoRA, or full‑parameter fine‑tuning for domain‑specific adaptation while preserving the original safety alignment.

Related Papers

The core research behind phi‑4 is documented in the following open‑access papers:

• Phi‑4 Technical Report (arXiv:2412.08905) – Describes the model architecture, training methodology, and benchmark results.
• “Direct Preference Optimization” (DPO) – a recent alignment technique that underpins the model’s instruction‑following behaviour (see arXiv:2305.18290 for the original formulation).
• “Synthetic Textbook Generation for LLMs” – outlines the creation of high‑quality synthetic data used to boost reasoning performance (arXiv:2308.12345).

These works collectively inform phi‑4’s design choices, from the dense decoder‑only backbone to the safety‑first alignment pipeline.

Licensing Information

The model is released under the MIT License. This permissive licence grants:

• Free commercial and non‑commercial use.
• The right to modify, distribute, and sublicense the model and its derivatives.
• No warranty or liability from the authors.

Attribution – When redistributing the model (or a derivative), you must retain the original copyright notice and license text. A typical attribution line could be:

© 2024 Microsoft Research. Licensed under the MIT License.

Restrictions – The MIT licence imposes no explicit restrictions on usage, but you must still comply with applicable laws (e.g., privacy, export controls). The model’s data cutoff (June 2024) means it may not contain the most recent events; users should verify factual accuracy for high‑risk applications.