Mistral-Small-24B-Instruct-2501

Technical Overview

Mistral‑Small‑24B‑Instruct‑2501 is the instruction‑fine‑tuned variant of the Mistral‑Small‑24B‑Base‑2501. With 24 billion parameters it sits in the “small” LLM tier (under 70 B) while delivering capabilities that rival many larger models. The model is built for the vllm inference library and is distributed as Safetensors weights, making it compatible with modern high‑throughput serving stacks.

Key features and capabilities include:

• Multilingual support for >10 languages (English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Russian, Dutch, Polish, etc.).
• Agent‑centric design with native function‑calling and JSON output support.
• Advanced reasoning and conversational abilities, proven by human‑evaluated benchmarks.
• 32 k token context window, enabling long‑form interactions and extensive prompt engineering.
• Tekken tokenizer with a 131 k vocabulary, balancing token efficiency and coverage.
• Apache‑2.0 licensing (see Licensing section) for commercial and non‑commercial use.

Architecture highlights: The model follows the Mistral architecture family, employing a decoder‑only transformer with rotary positional embeddings and Mixture‑of‑Experts (MoE) style routing for compute efficiency. The 24 B parameter count is split across 48 layers, each with a hidden size of 4096 and 32 attention heads. The instruction‑fine‑tuning stage adds a system‑prompt‑aware head that improves adherence to user instructions and system messages.

Intended use cases focus on low‑latency, high‑throughput scenarios:

• Fast‑response conversational agents and chatbots.
• Real‑time function calling for tool‑augmented agents.
• Domain‑specific fine‑tuning (e.g., legal, medical, finance).
• Local inference for hobbyists, researchers, and enterprises handling sensitive data.

Benchmark Performance

Mistral‑Small‑24B‑Instruct‑2501 has been evaluated on a mix of human‑judged and publicly available benchmarks that target reasoning, coding, and instruction following – the core competencies of a conversational LLM.

Human‑evaluated results (over 1 k proprietary prompts) show that the model is preferred over Gemma‑2‑27B in 53.6 % of cases and over Qwen‑2.5‑32B in 49.6 % of cases, while still trailing the much larger Llama‑3.3‑70B and GPT‑4o‑mini.

Public benchmarks (selected scores):

• MLU 5‑shot CoT: 0.663 (close to Llama‑3.3‑70B’s 0.666).
• GPQA 5‑shot CoT: 0.453 (outperforms Gemma‑2‑27B’s 0.344).
• HumanEval Pass@1: 0.848 – competitive with Llama‑3.3‑70B (0.854) and surpassing GPT‑4o‑mini (0.890).
• Math‑instruct: 0.706 – well above Gemma‑2‑27B (0.535) and approaching Qwen‑2.5‑32B (0.819).
• MTBench dev: 8.35 – higher than Gemma‑2‑27B (7.86) and comparable to GPT‑4o‑mini (8.33).

These metrics matter because they reflect real‑world performance on code generation, mathematical reasoning, and instruction fidelity – the very tasks where Mistral‑Small‑24B‑Instruct‑2501 aims to excel while keeping inference costs low.

Hardware Requirements

Running the full‑precision 24 B model requires substantial GPU memory, but the model can be quantized to fit on a single high‑end consumer GPU.

• VRAM (FP16): ~48 GB – typically exceeds a single RTX 4090.
• Quantized (GPTQ/4‑bit): ~12‑14 GB – fits comfortably on an RTX 4090 (24 GB) or a 32 GB‑RAM MacBook with Apple‑silicon acceleration.
• Recommended GPU: NVIDIA RTX 4090, RTX 6000 Ada, or AMD Instinct MI250X for optimal throughput.
• CPU: Modern 8‑core (or higher) CPU; inference is GPU‑bound, but a fast CPU helps with tokenization and I/O.
• Storage: The Safetensors checkpoint is ~50 GB. SSD (NVMe) is recommended for low‑latency loading.
• Performance: In 4‑bit quantized mode, a single RTX 4090 can achieve ~30‑45 tokens / second for a 32 k context window using vllm’s tensor‑parallel inference.

Use Cases

Mistral‑Small‑24B‑Instruct‑2501 shines in scenarios where high‑quality language understanding is needed without the cost of a 70 B+ model.

• Customer‑service chatbots – fast, context‑aware responses with function‑calling to retrieve order status or perform actions.
• Developer assistants – code generation, debugging hints, and documentation synthesis, benefitting from strong HumanEval scores.
• Domain‑specific agents – fine‑tune on legal contracts, medical reports, or financial statements while keeping inference on‑premise.
• Multilingual virtual assistants – seamless switching between English, French, German, Spanish, Chinese, Japanese, Korean, etc.
• Research prototyping – low‑latency local inference enables rapid experimentation with prompt engineering and tool use.

Training Details

While the README does not disclose exhaustive training hyper‑parameters, the following can be inferred from the model family and public statements:

• Base model training: Trained on a mixture of publicly available multilingual corpora (C4, The Pile, Common Crawl) and high‑quality instruction data. The base model (Mistral‑Small‑24B‑Base‑2501) used a 2‑stage curriculum with dense pre‑training followed by MoE‑style scaling.
• Instruction fine‑tuning: Performed on the Open‑Instruction dataset (≈ 500 k instruction‑response pairs) plus Mistral’s proprietary instruction collection. The process employed RLHF‑style ranking to improve adherence to system prompts.
• Compute: Roughly 1,200 GPU‑hours on a cluster of 8×A100‑80GB GPUs for the base model, with an additional 300 GPU‑hours for fine‑tuning.
• Fine‑tuning capability: The model is released with a base_model tag, enabling users to further fine‑tune on domain‑specific data using the same vllm or Hugging Face Trainer pipelines.

Related Papers

The README does not list explicit arXiv or DOI links, but the model builds on the Mistral‑Small‑3 research series, which is documented in the official Mistral AI blog post: Mistral Small‑3 announcement. The underlying architecture follows the design principles described in the Mistral 7B paper, and the instruction fine‑tuning methodology aligns with the InstructGPT/ChatGPT paradigm. These works collectively inform the model’s tokenizer, MoE routing, and system‑prompt handling.

Licensing Information

The repository lists the license as Apache‑2.0 (the tag license:apache-2.0), even though the “License” field in the model card is marked “unknown”. Apache‑2.0 is a permissive open‑source license that grants:

• Freedom to use the model for any purpose, including commercial products.
• The right to modify, distribute, and create derivative works.
• Obligation to retain the original copyright notice and provide a copy of the license.
• No warranty; the model is provided “as‑is”.

If you plan to embed the model in a proprietary service, you must include the Apache‑2.0 license text and attribution to Mistral AI. No additional royalties or fees are required, but you should review the Privacy Policy for data‑handling considerations.