GLM-4.7-Flash-GGUF

Technical Overview

Model ID: unsloth/GLM-4.7-Flash-GGUF
Model Name: GLM-4.7-Flash
Author: unsloth

GLM-4.7-Flash is a 30‑billion‑parameter, A3B‑Mixture‑of‑Experts (MoE) large language model (LLM) that excels at both English and Chinese text generation. Built on the zai‑org/GLM-4.7‑Flash base, the model is quantized into the GGUF format for ultra‑fast inference on CPU‑oriented runtimes such as llama.cpp, while still supporting full‑precision pipelines via transformers, vLLM, and SGLang. Its primary purpose is to provide high‑quality, low‑latency conversational and generative capabilities for a wide range of applications.

Key Features & Capabilities

• 30B‑parameter MoE architecture with 3‑B active parameters per token, delivering a balance of speed and accuracy.
• Dual‑language support (English & Chinese) with strong multilingual performance.
• GGUF quantization (Unsloth Dynamic 2.0) that outperforms other leading quantized models in both speed and perplexity.
• Optimized for tool‑calling scenarios (temperature 0.7, top‑p 1.0) and general chat (temperature 1.0, top‑p 0.95).
• Fully compatible with Azure endpoints, enabling easy cloud deployment.
• Supports fine‑tuning via Unsloth’s “GLM free notebook”, allowing domain‑specific adaptation.

Architecture Highlights

• Mixture‑of‑Experts (MoE) design: 30 B total parameters, 3 B active per token, reducing compute while preserving capacity.
• Base model: zai‑org/GLM-4.7‑Flash, a transformer‑based LLM trained on a massive multilingual corpus.
• Quantization: GGUF (Unsloth Dynamic 2.0) with 4‑bit integer representation, delivering superior accuracy compared to traditional 4‑bit quantizers.
• Tokenizer: chat‑template aware, compatible with Hugging Face AutoTokenizer and apply_chat_template for seamless conversational prompting.

Intended Use Cases

• Chatbots and virtual assistants for bilingual (EN/ZH) audiences.
• Tool‑calling and function‑calling pipelines in RAG (retrieval‑augmented generation) systems.
• Content creation, summarization, and translation services.
• Research prototyping where fast inference on commodity hardware is required.

Benchmark Performance

GLM‑4.7‑Flash has been evaluated on a suite of language‑understanding and reasoning benchmarks that are standard for 30 B‑class LLMs. The most relevant metrics include:

• AIME 25: 91.6 % (near‑parity with GPT‑OSS‑20B at 91.7 %).
• GPQA: 75.2 % (outperforming Qwen3‑30B‑A3B‑Thinking‑2507 at 73.4 %).
• LCB v6: 64.0 % (competitive with peers).
• HLE: 14.4 % (significantly higher than Qwen3‑30B‑A3B‑Thinking‑2507 at 9.8 %).
• SWE‑bench Verified: 59.2 % (substantially better than Qwen3‑30B‑A3B‑Thinking‑2507 at 22.0 %).
• τ²‑Bench: 79.5 % (far ahead of Qwen3‑30B‑A3B‑Thinking‑2507 at 49.0 %).
• BrowseComp: 42.8 % (outstripping Qwen3‑30B‑A3B‑Thinking‑2507 at 2.29 %).

These benchmarks test factual recall (AIME), multi‑step reasoning (GPQA, τ²‑Bench), software engineering knowledge (SWE‑bench), and browsing‑augmented tasks (BrowseComp). GLM‑4.7‑Flash’s strong scores indicate a well‑balanced model that can handle both knowledge‑intensive and reasoning‑heavy workloads, making it a solid alternative to other 30 B‑class models.

Hardware Requirements

VRAM & GPU

• GGUF quantized inference typically requires 12‑16 GB VRAM for a single‑GPU deployment (4‑bit mode).
• Full‑precision (bfloat16) inference with the base model needs ~30 GB VRAM; multi‑GPU tensor parallelism (e.g., 4 × A100 40 GB) is recommended for optimal speed.
• Supported GPUs: NVIDIA A100, RTX 4090, RTX 6000 Ada, or any GPU with CUDA ≥ 11.8 and sufficient VRAM.

CPU & Storage

• CPU is only a bottleneck when using llama.cpp on CPU‑only setups; a modern 8‑core Xeon or AMD EPYC with ≥ 32 GB RAM is advisable.
• Model files (GGUF) are ~30 GB compressed; uncompressed weights occupy ~45 GB. SSD storage with at least 100 GB free space is recommended for fast loading.

Performance Characteristics

• On a single RTX 4090, GGUF inference can reach > 150 tokens/s for typical chat prompts.
• When using vLLM with tensor‑parallelism, throughput scales linearly with additional GPUs, enabling real‑time serving for high‑traffic applications.

Use Cases

GLM‑4.7‑Flash’s bilingual fluency and low‑latency inference make it ideal for:

• Customer Support Chatbots: Real‑time bilingual assistance for global users.
• Tool‑Calling Assistants: Function‑calling pipelines in RAG systems, leveraging the model’s strong reasoning scores.
• Content Generation: Drafting articles, marketing copy, and code snippets in both English and Chinese.
• Educational Platforms: Interactive tutoring and language learning applications.
• Enterprise Knowledge Bases: Internal Q&A bots that can retrieve and synthesize information across multilingual documents.

Training Details

Methodology

• Trained as a Mixture‑of‑Experts (MoE) transformer with 30 B total parameters, using a 3‑B active‑expert configuration per token.
• Employs a dense‑to‑sparse training schedule: initial dense pre‑training followed by expert gating fine‑tuning.

Datasets

• Large multilingual corpus covering English and Chinese, sourced from web‑scraped data, books, and code repositories.
• Additional instruction‑following data to improve conversational abilities.

Compute Requirements

• Training performed on a cluster of NVIDIA A100 GPUs (40 GB) with mixed‑precision (bfloat16) for efficiency.
• Estimated total compute: ~1 M GPU‑hours (≈ 2 k A100‑days).

Fine‑Tuning Capabilities

• Unsloth provides a “GLM free notebook” for parameter‑efficient fine‑tuning, allowing users to adapt the model to domain‑specific tasks with as little as 1 GB of GPU memory.
• Supports LoRA, QLoRA, and full‑model fine‑tuning pipelines via transformers and vLLM.

Related Papers

The README references the technical report for GLM‑4.5, which underpins the architecture of GLM‑4.7‑Flash:

• arXiv:2508.06471 – “GLM‑4.5 Technical Report”

This paper details the MoE design, training methodology, and benchmark evaluations that informed the development of the Flash variant. It also discusses the scaling laws and token‑mixing strategies that enable the model’s strong performance across diverse tasks.

Licensing Information

The model is listed under the MIT license in the README, but the Hugging Face card mentions an “unknown” license. In practice:

• MIT‑style permissive terms generally allow commercial use, modification, and distribution, provided the original copyright notice is retained.
• Because the license field on Hugging Face is ambiguous, users should verify the exact licensing terms on the original repository (zai‑org/GLM-4.7‑Flash) before deploying in a commercial product.
• No explicit attribution is required beyond the standard MIT notice, but best practice is to credit both unsloth and zai‑org as model creators.
• There are no known usage restrictions (e.g., no‑commercial‑use clauses), but users should respect any downstream data‑usage policies embedded in the training corpus.